History log of /5.5.2/forestdb/utils/ (Results 1 - 25 of 92)
Revision (<<< Hide revision tags) (Show revision tags >>>)Date Author Comments
(<<< Hide modified files)
(Show modified files >>>)
effd487220-Sep-2017 Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>

MB-26165:[utils/debug.cc] Use ucontext_t instead of ucontext.

In recent glibc versions the struct ucontext was renamed to
ucontext_t. The details can be found at [1].

The change

MB-26165:[utils/debug.cc] Use ucontext_t instead of ucontext.

In recent glibc versions the struct ucontext was renamed to
ucontext_t. The details can be found at [1].

The change is compatible with older glibc versions because the typedef
for the struct has been called ucontext_t for a long while. So it will
be picked up instead of the struct tag.

[1] https://sourceware.org/bugzilla/show_bug.cgi?id=21457

Change-Id: I6096e1576a973e8bcb33cad3a7cc94c2009d390d
Reviewed-on: http://review.couchbase.org/83579
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>

show more ...

Revision tags: v1.2
acc133d706-May-2016 abhinavdangeti <abhinav@couchbase.com>

Extend usecase tests to cover multiple file scenario

===== Avg Latencies (10H, 5F, 4RW, 4RO - Seperate Pool test) - 2214662 samples (ms) ===

Percenti

Extend usecase tests to cover multiple file scenario

===== Avg Latencies (10H, 5F, 4RW, 4RO - Seperate Pool test) - 2214662 samples (ms) ===

Percentile
Median 95th 99th Std Dev Histogram of samples

set 0.002 0.016 0.188 0.061 ▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
commit 0.150 0.251 1.127 1.072 ▁█▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
in-mem snapshot 2.414 5.208 6.142 1.599 ▁▆▇▆▇█▇▇▇▇▇▆▆▆▇▆▆▆▆▆▆▆▆▆▆▅▆▅▅▄▄▄
clone snapshot 0.002 0.005 0.008 0.007 ▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
iterator-init 0.006 0.012 0.018 0.005 ▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
iterator-seek 0.007 0.012 0.018 0.005 ▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
iterator-get 0.001 0.002 0.003 0.004 ▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
iterator-close 0.002 0.004 0.007 0.004 ▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
0 ms 5
=======================================================================================

Change-Id: I7cc1e212fa9cb74248b3bb0fef0ddc3f471954b5
Reviewed-on: http://review.couchbase.org/63801
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

e984c92f13-Apr-2016 Sundar Sridharan <sundar.sridharan@gmail.com>

Enable HANG_ON_CRASH for segfault+assertion debugging

Change-Id: Icd0eca89fe341c9d6bfd1e943b493e31ade15712
Reviewed-on: http://review.couchbase.org/62761
Reviewed-by: abhinav dangeti

Enable HANG_ON_CRASH for segfault+assertion debugging

Change-Id: Icd0eca89fe341c9d6bfd1e943b493e31ade15712
Reviewed-on: http://review.couchbase.org/62761
Reviewed-by: abhinav dangeti <abhinav@couchbase.com>
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>

show more ...

d18182a113-Apr-2016 Chiyoung Seo <chiyoung.seo@gmail.com>

Remove verbose log messages from utils/debug

Change-Id: Ic576039ce01bcd78d163a049ef6fbe8a8ece1199
Reviewed-on: http://review.couchbase.org/62766
Reviewed-by: Jung-Sang Ahn <jungsang.

Remove verbose log messages from utils/debug

Change-Id: Ic576039ce01bcd78d163a049ef6fbe8a8ece1199
Reviewed-on: http://review.couchbase.org/62766
Reviewed-by: Jung-Sang Ahn <jungsang.ahn@gmail.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

b84960c205-Apr-2016 Jung-Sang Ahn <jungsang.ahn@gmail.com>

Log error messages through stderr if log callback is not given

- If error log callback function is not given, print error messages
through stderr.

- _fdb_restore_wal() may repor

Log error messages through stderr if log callback is not given

- If error log callback function is not given, print error messages
through stderr.

- _fdb_restore_wal() may report some false alarms due to circular
block reuse, we temporarily disable it.

Change-Id: I501c05e4926b4c94d47ef5ebc63de5692565652f
Reviewed-on: http://review.couchbase.org/62399
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

b050253012-Apr-2016 Sundar Sridharan <sundar.sridharan@gmail.com>

fix build breakage for 32-bit linux (use %eip vs %rip)

Change-Id: I4f3e8de2a1468b07b07e06c5a24a8f691ec69d93
Reviewed-on: http://review.couchbase.org/62696
Tested-by: buildbot <build@

fix build breakage for 32-bit linux (use %eip vs %rip)

Change-Id: I4f3e8de2a1468b07b07e06c5a24a8f691ec69d93
Reviewed-on: http://review.couchbase.org/62696
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

6d85230b11-Apr-2016 abhinavdangeti <abhinav@couchbase.com>

MB-19152: Generate backtrace when linked with platform directory

Backtrace has a lot of dependencies that platform and
tlm cater to. Removing backtrace support from standalone
forest

MB-19152: Generate backtrace when linked with platform directory

Backtrace has a lot of dependencies that platform and
tlm cater to. Removing backtrace support from standalone
forestdb, to avoid compilation errors seen on windows:

Linking CXX executable forestdb_dump.exe
backtrace.cc.obj : error LNK2019: unresolved external symbol
__imp__SymGetModuleInfo64@16 reference in function
"void __cdecl describe_address(char *, unsigned int, void *)"
(?describe_address@@YAXPADIPAX@Z)
...

Change-Id: I7f860f8130f131754603fc5a6113b1c653fc979e
Reviewed-on: http://review.couchbase.org/62671
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: buildbot <build@couchbase.com>

show more ...

90aea09511-Apr-2016 Sundar Sridharan <sundar.sridharan@gmail.com>

Backtrace forestdb segfaults with co-operative signal handling

When forestdb is invoked from golang segmentation faults in
forestdb have no line numbers or backtraces printed out.
Th

Backtrace forestdb segfaults with co-operative signal handling

When forestdb is invoked from golang segmentation faults in
forestdb have no line numbers or backtraces printed out.
This change installs signal handler for SIGSEGV on linux.
If segfault is in forestdb code, invoke breakpad,
else call golang's signal handler to ensure proper functioning.
Follows rules by Ian at https://golang.org/src/os/signal/doc.go

Change-Id: Ie50d4c40d7e3afecbd3c182362ab42d556573964
Reviewed-on: http://review.couchbase.org/62589
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

b95738ed23-Mar-2016 abhinavdangeti <abhinav@couchbase.com>

MB-18837: BreakPAD & Backtrace Support with ForestDB

+ fdb_assert() will print the backtrace.
+ Integrating breakpad support for Windows and Linux,
generates minidump only in case

MB-18837: BreakPAD & Backtrace Support with ForestDB

+ fdb_assert() will print the backtrace.
+ Integrating breakpad support for Windows and Linux,
generates minidump only in case of fdb_assert.

Change-Id: Ic6fc0cb796fb8acbfe78018605fc595cdfcb3a67
Reviewed-on: http://review.couchbase.org/61897
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

Revision tags: v1.1, v1.0
42b318ce11-Jan-2016 Trond Norbye <trond.norbye@gmail.com>

FreeBSD may use CLOCK_MONOTONIC

Change-Id: If89e22bde25d0f6fb02237d9716ff7ee5070f314

bc44457d23-Oct-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

MB-16608 Support cross-platform encryption options.

ForestDB uses the CommonCrypto library to perform file level encryption,
but that library is specific to Apple products (iOS and OS X)

MB-16608 Support cross-platform encryption options.

ForestDB uses the CommonCrypto library to perform file level encryption,
but that library is specific to Apple products (iOS and OS X).
To address it, this change provides an option to use other cross-platform
libraries (OpenSSL and LibTomCrypt).

_ENCRYPTION macro can be optionally passed to specify which crypto library
is used to support database encryption and set to 'commoncrypto' on iOS and
OS X, 'openssl' for OpenSSL, or 'libtomcrypt' for LibTomCrypt.

For example,

% cmake -D_ENCRYPTION=commoncrypto path_to_build_directory

Change-Id: I740eddfc4cf0ea59ef630879d872b978ad362574

show more ...

ac637aba06-Nov-2015 Sundar Sridharan <sundar.sridharan@gmail.com>

Provide latency timings of forestdb APIs

Currently, this change provides min, max, and avg latency of
fdb_set, fdb_get, fdb_commit, fdb_snapshot_open, and fdb_compact APIs.

late

Provide latency timings of forestdb APIs

Currently, this change provides min, max, and avg latency of
fdb_set, fdb_get, fdb_commit, fdb_snapshot_open, and fdb_compact APIs.

latency(us) min avg max num_samples
sets : 1 19 78430 100000
gets : 1 2 23 2000
commits : 45 68 1179 10000
snapshots: 3 2852 9868 2000
compact : 343546 1086552 1932497 6

Change-Id: If3e25e7cebb0df00a172b39e983e23f4cc7e7a75

show more ...

062d3cb926-Oct-2015 Tommie McAfee <tommie@couchbase.com>

Move timed api's to util directory

Having timing api's will allow other test components
to make use of timed calls for performance testing

Change-Id: Ibd213a17da8bd20ce537d2a238

Move timed api's to util directory

Having timing api's will allow other test components
to make use of timed calls for performance testing

Change-Id: Ibd213a17da8bd20ce537d2a2381554245bcf288c

show more ...

4690a73f27-Sep-2015 Jens Alfke <jens@couchbase.com>

MB-11620: Database encryption

Encryption is specified by setting the fdb_config.encryption_key field
when opening a database. The key's algorithm field should be set to
a supported a

MB-11620: Database encryption

Encryption is specified by setting the fdb_config.encryption_key field
when opening a database. The key's algorithm field should be set to
a supported algorithm, and the raw key data copied into the bytes field.

The only algorithm specified yet is AES-256, using the Encrypted Salt-
Sector Initialization Value (ESSIV) algorithm for per-block encryption.
It's only implemented for Apple platforms in this commit, using the
built-in CommonCrypto library. Additional platforms can be supported by
adding to utils/crypto_primitives.h.

An alternate algorithm FDB_ENCRYPTION_BOGUS is for testing purposes
only. It provides no real security; it's not much more sophisticated
than ROT13.

Change-Id: I1188799b04923b22adca0fdb89ed5d31691dc238

show more ...

874fd6ce22-Sep-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

Merge remote-tracking branch 'origin/sherlock'

* origin/sherlock:
MB-16263: Add background bcache flushing capability
MB-16263: Track & flush immutable dirty buffer blocks

Merge remote-tracking branch 'origin/sherlock'

* origin/sherlock:
MB-16263: Add background bcache flushing capability
MB-16263: Track & flush immutable dirty buffer blocks

Conflicts:
CMakeLists.txt
src/forestdb.cc
tests/functional/CMakeLists.txt

Change-Id: Iaf37bd65d96de926becf7c916058ac731fff374c

show more ...


4cdca91e22-Sep-2015 Sundar Sridharan <sundar.sridharan@gmail.com>

MB-16263: Add background bcache flushing capability

New background flusher threads will iterate over all files
and flush any immutable dirty blocks in a loop sleeping when
there are

MB-16263: Add background bcache flushing capability

New background flusher threads will iterate over all files
and flush any immutable dirty blocks in a loop sleeping when
there are no dirty immutable blocks. Keeps I/O utilized frees
more buffer cache blocks for front end end threads

Change-Id: Ib55c8c6cea43caa7f5f2635d35826c7d5be7f022

show more ...

fdf4483d03-Sep-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

Merge remote-tracking branch 'origin/sherlock' into master-old

* origin/sherlock:
MB-16216: Fix fdb_open failure on 32bit systems
[BP]:MB-16109 Relax the assertion check on snaps

Merge remote-tracking branch 'origin/sherlock' into master-old

* origin/sherlock:
MB-16216: Fix fdb_open failure on 32bit systems
[BP]:MB-16109 Relax the assertion check on snapshot_reader tests
CBD-1638: filename conflict in mvcc_ & fdb_ functional test
CBD-1638: Replace use of 'dd' with custom func for windows

Conflicts:
src/filemgr.h

Change-Id: If86959b44c6dfdb7af6fdd804cd52768086d228f

show more ...


1e5b052502-Sep-2015 Sundar Sridharan <sundar.sridharan@gmail.com>

MB-16216: Fix fdb_open failure on 32bit systems

get_memory_size()'s 32 bit arithmetic causes overflow
resulting in fdb_init() failing buffer_cache size validation
with FDB_RESULT_TOO

MB-16216: Fix fdb_open failure on 32bit systems

get_memory_size()'s 32 bit arithmetic causes overflow
resulting in fdb_init() failing buffer_cache size validation
with FDB_RESULT_TOO_BIG_BUFFER_CACHE.
Fix is to use 64 bit arithmetic.

Change-Id: Iaa9be80a957a103710da596469d4fa15327b4832

show more ...

2eac781525-Aug-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

Merge remote-tracking branch 'origin/sherlock'

* origin/sherlock:
MB-16154: Add new API to trigger a callback upon a fatal error
Revert "Spawn gdb to get backtrace of all threads

Merge remote-tracking branch 'origin/sherlock'

* origin/sherlock:
MB-16154: Add new API to trigger a callback upon a fatal error
Revert "Spawn gdb to get backtrace of all threads on assert (Linux only)"

Change-Id: If4109cac438d96fe7a28520b5ad1a2773fb119c6

show more ...


2406d68125-Aug-2015 Dave Rigby <daver@couchbase.com>

MB-16154: Add new API to trigger a callback upon a fatal error

Add a new API to forestdb: fdb_set_fatal_error_callback(). This allows
users of forestdb to optionally specify a callback t

MB-16154: Add new API to trigger a callback upon a fatal error

Add a new API to forestdb: fdb_set_fatal_error_callback(). This allows
users of forestdb to optionally specify a callback to be invoked in
the event of a fatal error inside forestdb; before forestdb raises a
SIGABRT.

This replaces the previous manditory dumping of thread backtraces,
which is not suitable for all users of forestdb (it was added for GSI,
but kv-engine doesn't want or need it as it uses Google Breakpad to
catch all fatal errors.

Change-Id: I5408b597e4d1073ab78153ccb4f3d2091bb8160e

show more ...

2822739b25-Aug-2015 Dave Rigby <daver@couchbase.com>

Revert "Spawn gdb to get backtrace of all threads on assert (Linux only)"

A more robust crash catching mechanism (using Breakpad) will replace
it.

This reverts commit bda6f4266c

Revert "Spawn gdb to get backtrace of all threads on assert (Linux only)"

A more robust crash catching mechanism (using Breakpad) will replace
it.

This reverts commit bda6f4266cc36ca3b387e410d9add1b91484f5ba.

show more ...

76e1613807-Aug-2015 Jim Walker <jim@couchbase.com>

Move forestDB to use high-performance CRC32C (from platform if available).

Note: if libplatform is not available, e.g. not a couchbase build
then crc32c is not enabled, the in-built util

Move forestDB to use high-performance CRC32C (from platform if available).

Note: if libplatform is not available, e.g. not a couchbase build
then crc32c is not enabled, the in-built utils/crc32.cc is used.

This changes the underlying hashing function used for integrity
checking of ForestDB data.

When creating a new file, CRC32C is selected as the CRC and all data
is thus stored with a CRC32C derived checksum. All CRCs stored in
a ForestDB file are written using the same CRC function.

When reading a file the CRC32 mode is detected as part of reading
the header block, we check the header block CRC against each CRC mode
and use the one which produces a match (or report checksum fail).
This allows files written before this commit to be read by builds containing
this commit. The existing file is then maintained with the detected CRC,
i.e. new documents added to a CRC32 file will be written with CRC32, not
CRC32C.

A new flag is also added to the ForestDB open API that allows code to explicitly
request the 'legacy' CRC mode. This is primarily so we can programtically
perform upgrade tests. The flag is only valid for new files or legacy CRC files.
Opening a CRC32C file with the new flag results in FDB_RESULT_INVALID_ARGS.

Compaction now performs CRC upgrades.

Compacting a file always writes the new file with CRC32C. Thus
compaction will effectively upgrade files that are using CRC32.

Compaction meta-files are also written using crc32c, but read using
all known modes removing the need for changing the stored version number.

This patch removes the adler32 code which was unused and would complicate
upgrades further if ever enabled.

This patch removes 'crc_test.cc' which never made any assertions.
- Note that crc32c is covered by IETF test vectors in the platform test suite.

This patch adds 'crc_migration.cc' which is a 'google-test' program that covers
the functionality in this commit.

Change-Id: I81a2ef481cb81feaccbbf9312be55328b76603c5

show more ...

17cd38f914-Aug-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

Merge remote-tracking branch 'origin/sherlock'

* origin/sherlock:
MB-16060: Do not free buffer cache blocks twice
MB-16038 Use intentional sleep instead of grabbing mutex for thr

Merge remote-tracking branch 'origin/sherlock'

* origin/sherlock:
MB-16060: Do not free buffer cache blocks twice
MB-16038 Use intentional sleep instead of grabbing mutex for throttling writer
MB-16038 Don't load KVS header for snapshot handle
MB-16038 Don't grab the file manager lock in restoring the WAL for snapshot
MB-16038 Avoid grabbing the file manager lock upon creating KVS info instance
[BP] MB-16038 Reduce the filemgr lock contention in fdb_open()
MB-16038 Support the lazy cleanup of a file's buffer cache blocks

Change-Id: Ic396d15231e0511ccd92335e3613b14294ced62f

show more ...


3155b82d13-Aug-2015 Jung-Sang Ahn <jungsang.ahn@gmail.com>

MB-16038 Use intentional sleep instead of grabbing mutex for throttling writer

- Since it is hard to control writer speed using mutex, we use sleep instead.

Change-Id: I51a6af7077fe

MB-16038 Use intentional sleep instead of grabbing mutex for throttling writer

- Since it is hard to control writer speed using mutex, we use sleep instead.

Change-Id: I51a6af7077fe56e74b5e0976993f36e25b6abd1c

show more ...

2f21d56707-Aug-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

Merge remote-tracking branch 'origin/sherlock'

* origin/sherlock:
MB-15908 Support lazy deletion of old DB file
Relax error checking for e2e_concurrency_scan_test
Spawn gdb

Merge remote-tracking branch 'origin/sherlock'

* origin/sherlock:
MB-15908 Support lazy deletion of old DB file
Relax error checking for e2e_concurrency_scan_test
Spawn gdb to get backtrace of all threads on assert (Linux only)

Change-Id: I38e7ac5ca6dab8febb1d2d6d3f6c1e9a4456d66a

show more ...


1234