History log of /6.6.0/forestdb/ (Results 1 - 25 of 1308)
Revision (<<< Hide revision tags) (Show revision tags >>>)Date Author Comments
(<<< Hide modified files)
(Show modified files >>>)
5836cfc626-Feb-2020 sduvuru <sduvuru@gmail.com>

MB-37955: Forestdb does not return erroron read failure

When a corruption is detected, forestdb does not pass the error all the way up.
The fix is to handle the error returned and pass i

MB-37955: Forestdb does not return erroron read failure

When a corruption is detected, forestdb does not pass the error all the way up.
The fix is to handle the error returned and pass it up.

Change-Id: I123f59d4c0f2010bd6a34c6a6b596c13c28cc83e
Reviewed-on: http://review.couchbase.org/122826
Tested-by: Build Bot <build@couchbase.com>
Well-Formed: Wayne Siu <wayne@couchbase.com>
Well-Formed: Build Bot <build@couchbase.com>
Reviewed-by: Srinath Duvuru <srinath.duvuru@couchbase.com>

show more ...

4c3b2f9b08-Oct-2019 sduvuru <sduvuru@gmail.com>

MB-36385: Forestdb API does not return error to user when write fails

Return the error up the stack.

Change-Id: I2913d530d36b33fca700d63000f6759231373d60
Reviewed-on: http://rev

MB-36385: Forestdb API does not return error to user when write fails

Return the error up the stack.

Change-Id: I2913d530d36b33fca700d63000f6759231373d60
Reviewed-on: http://review.couchbase.org/116298
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Srinath Duvuru <srinath.duvuru@couchbase.com>

show more ...

3599026410-Oct-2019 sduvuru <sduvuru@gmail.com>

MB-36342: Intermittent Rebalance Failure

The fix for MB-34678 added cheeck to get the memory limit for a containerized
environment. The value is obtained from /sys/fs/cgroup/memory/memo

MB-36342: Intermittent Rebalance Failure

The fix for MB-34678 added cheeck to get the memory limit for a containerized
environment. The value is obtained from /sys/fs/cgroup/memory/memory.limit_in_bytes
The value when not set is initialized to 9223372036854771712 (max number of 4k blocks
represented by a 64 bit int). But, it looks like in some case, it may be set to
18446744073709551615 (max unsined int). As the value was being read as an int it was
interpreted as -1. The ram size was set incorrectly. The fix is to read it as unsigned int.

Change-Id: Ifd85ad90eb65f9c382438bb2b41cdc9c67c3105c
Reviewed-on: http://review.couchbase.org/116189
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Srinath Duvuru <srinath.duvuru@couchbase.com>

show more ...

7965de3317-Sep-2019 sduvuru <sduvuru@gmail.com>

MB-34678: Forestdb does not calculate max memory correctly in
containerized environments

In a containerized environment, the memory usage is obtained from
"/sys/fs/cgroup/m

MB-34678: Forestdb does not calculate max memory correctly in
containerized environments

In a containerized environment, the memory usage is obtained from
"/sys/fs/cgroup/memory/memory.limit_in_bytes".
If that value is not initialized, it is set to a large number
and in that case the max memory of the system is to be used.

Change-Id: I8859ef2e8ad5e6bc4ce93de33718cf99763ec7c5
Reviewed-on: http://review.couchbase.org/115645
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Srinath Duvuru <srinath.duvuru@couchbase.com>

show more ...

605833d501-Oct-2019 sduvuru <sduvuru@gmail.com>

MB-36248: Forestdb reserve bitmap revision number not set correctly

The stale blocks used by block reuse are managed using two bitmaps.
One is the current bit map and the other is the re

MB-36248: Forestdb reserve bitmap revision number not set correctly

The stale blocks used by block reuse are managed using two bitmaps.
One is the current bit map and the other is the reserve bitmap. The
reserve bitmap is switced with the main bitmap when mainbitmap is
exhausted. The bitmaps have revision numebrs associated with them and
must be set correctly for the block reuse to work. The revision number
of the reserve bit map will be one more than the main bit map's version.
In the case the bitamps are read from the file on disk, the version
number must be initialized correctly. The reserve bitmap's revision
number was not set correcty and the block resue failed.

The fix is to set the revision number correctly. Also, there is a
corruption issue with cached btree nodes when block reuse is used.
The cached blocks remain in cache even after the blocks are reused.
The fix is to disable the b-tree node cache.

Change-Id: I348046d7dd8ff6920d16ce78797128304762ad5c
Reviewed-on: http://review.couchbase.org/115646
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Srinath Duvuru <srinath.duvuru@couchbase.com>

show more ...

6428a5c422-Aug-2019 sduvuru <sduvuru@gmail.com>

MB-35666: Forestdb: Renaming a compacted file can lead to the file
getting deleted

When a file is compacted the older file is deleted when there are
no references to it. This causes

MB-35666: Forestdb: Renaming a compacted file can lead to the file
getting deleted

When a file is compacted the older file is deleted when there are
no references to it. This causes an issue when the compacted file is
renamed to the original file and then backed up. When the backed up
file is opened, the original file gets deleted as the name of the
previous file for the compacted file matches the renamed file.

The fix is to not remove the old file if it is in a different dirrectory.

Change-Id: I7d3fc397768483976567774f79c49882847aba18
Reviewed-on: http://review.couchbase.org/113762
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Srinath Duvuru <srinath.duvuru@couchbase.com>

show more ...

94812b1f15-May-2019 sduvuru <sduvuru@gmail.com>

MB-34158: Segfault when getting reusable blocks

* Memory allocation failure is not handled in fdb_gather_stale_blocks.
Space allocation is doubled as needed.
In a system with memory

MB-34158: Segfault when getting reusable blocks

* Memory allocation failure is not handled in fdb_gather_stale_blocks.
Space allocation is doubled as needed.
In a system with memory pressure the allocation can fail and can
cause a crash and corruption.
Check and handle the memory allocation failure cases.

Change-Id: Ia8987cd71c04691b332a00be894c35e0a548a7f9
Reviewed-on: http://review.couchbase.org/109372
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Srinath Duvuru <srinath.duvuru@couchbase.com>

show more ...

68a432f410-May-2019 Chris Hillery <ceej@couchbase.com>

Add note regarding 'master' and 'cb-master' git branches to README

Change-Id: I3c39db71e0cb3455ee6ab740c3474a5ae87bd25a
Reviewed-on: http://review.couchbase.org/108990
Reviewed-by: S

Add note regarding 'master' and 'cb-master' git branches to README

Change-Id: I3c39db71e0cb3455ee6ab740c3474a5ae87bd25a
Reviewed-on: http://review.couchbase.org/108990
Reviewed-by: Srinath Duvuru <srinath.duvuru@couchbase.com>
Tested-by: Chris Hillery <ceej@couchbase.com>

show more ...

5623660309-Mar-2018 sduvuru <sduvuru@gmail.com>

MB-28014: Index node crashing with FDB error

When ForestDB encounters a corrupted buffer, currently it just asserts and crashes.
When indexer restarts, it again encounters the same corru

MB-28014: Index node crashing with FDB error

When ForestDB encounters a corrupted buffer, currently it just asserts and crashes.
When indexer restarts, it again encounters the same corrupted buffer and crashes again.
The indexer then has no way of determining the specific index file (forestDB file)
which is corrupt and disable the specific index.

This fix will handle corruption that is detected in the following way.
If buffer (on disk or on the buffer pool) is found to be corrupt,
the current dbheader will be marked as invalid and an error, FDB_RECOVERABLE_ERR,
will be returned. When the file is reopened, forestdb will move to the first valid
dbheader skipping the DBheaders that were marked as invalid. If the forestdb file open
encounters "num_keeping_headers", a config parameter, invalid dbheaders it will return
FDB_NONRECOVERABLE_ERR, a nonrecoverable error. The indexer will now be able to
determine the index is not recoverable and invalidate it so that it can be rebuilt.

The fix also has two functional tests which corrupt index and data pages and recovers
up to num_keeping_headers configuration value.

Change-Id: I181abface1ffe2e926597dccb883bb818b31d63a
Reviewed-on: http://review.couchbase.org/91777
Reviewed-by: Sundar Sridharan <sundar@couchbase.com>
Tested-by: Sundar Sridharan <sundar@couchbase.com>

show more ...

99e2edb228-Oct-2016 abhinavdangeti <abhinav@couchbase.com>

MB-26364: Warnings in forestdb on Mac during build
MB-21503: Replace OSSpinLock with os_unfair_lock for OSX>=10.12

OSSpinLock has been deprecated in OSX 10.12 onwards (Sierra),
repla

MB-26364: Warnings in forestdb on Mac during build
MB-21503: Replace OSSpinLock with os_unfair_lock for OSX>=10.12

OSSpinLock has been deprecated in OSX 10.12 onwards (Sierra),
replacing it with the suggestion from the compiler.

Removed the SPIN_INITIALIZER check in filemgr_shutdown as it is not needed and casues failures.

Reviewed-on: http://review.couchbase.org/69301
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>
Change-Id: I4642be7b1302fb209326ee85796b9cf5a23d8cdf
Reviewed-on: http://review.couchbase.org/87701
Reviewed-by: Sundar Sridharan <sundar@couchbase.com>
Tested-by: Sundar Sridharan <sundar@couchbase.com>

show more ...

a54fc06a27-Jan-2018 sduvuru <sduvuru@gmail.com>

MB-28038: [FM] Attempting to write to a buffer still in use

Original fix was in MB-27005
sb_bmp_is_writable() was reporting a block is not writable. This is because the
test bid >= l

MB-28038: [FM] Attempting to write to a buffer still in use

Original fix was in MB-27005
sb_bmp_is_writable() was reporting a block is not writable. This is because the
test bid >= last_commit fails. This is due to the last_writable_bmp_revnum
getting set in all cases, even when last_commit was set to the end of file.
This causes the a bitmap version mismatch in the check for a writable block.
This fix is a work around to avoid the failure. By setting the last_writable_bmp_revnum
to current bitmap version only when last_commit is not set to end of file.

Change-Id: I4ac68b381599f9ebf87badb3fbb6e350203b8a8e
Reviewed-on: http://review.couchbase.org/88454
Well-Formed: Build Bot <build@couchbase.com>
Reviewed-by: Sundar Sridharan <sundar@couchbase.com>
Tested-by: Sundar Sridharan <sundar@couchbase.com>
Reviewed-on: http://review.couchbase.org/88549

show more ...

04f1e58b08-Dec-2017 sduvuru <sduvuru@gmail.com>

MB-26978: Unit test failed with "fdb_anomaly_test........*** Exception

Compaction opens the new file, switches to it, closes and unlinks
upon success. However when compaction fails after

MB-26978: Unit test failed with "fdb_anomaly_test........*** Exception

Compaction opens the new file, switches to it, closes and unlinks
upon success. However when compaction fails after the switch, the old file is
not closed. An attempt to shutdown forestdb, destroys the compactor
mutex and then fails with file busy error. The next attempt to open the
old file does not initialize the compactor mutex as forest db was not
shutdown. When an attempt to signal a condition for that mutex is made,
the signal call hangs as the associated mutex is destroyed.

The fix is to close the old file upon a failure.

The tests for compaction failure in anomaly tests are also updated to
look for the status of all the calls and validate them.

Change-Id: Ifed47c06442c18c295d493a54d266c37cb02cf71
Reviewed-on: http://review.couchbase.org/86676
Well-Formed: Build Bot <build@couchbase.com>
Reviewed-by: Sundar Sridharan <sundar@couchbase.com>
Tested-by: Srinath Duvuru <sduvuru@gmail.com>

show more ...

a7644acc08-Nov-2017 sduvuru <sduvuru@gmail.com>

MB-26434: After a failed compaction, the original file cannot be re-compacted

This fixes a regression caused by a change to update the next file after
compaction recovery.
https://gi

MB-26434: After a failed compaction, the original file cannot be re-compacted

This fixes a regression caused by a change to update the next file after
compaction recovery.
https://github.com/couchbase/forestdb/blob/spock/src/forestdb.cc#L2194

The fix is to not have the old filename and new filename in the handle.
Recovery of the file has made sure the correct file is now used based
on whether the compaction succeeded or failed.

Added more logging to follow the compaction recovery activity.
Added test cases to verify the re-compaction works after recovery.

Change-Id: I1ec74abb8c23000c64366a0e0e0ff1cf03e823c9
Reviewed-on: http://review.couchbase.org/85266
Reviewed-by: Sundar Sridharan <sundar@couchbase.com>
Tested-by: Srinath Duvuru <sduvuru@gmail.com>

show more ...

effd487220-Sep-2017 Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>

MB-26165:[utils/debug.cc] Use ucontext_t instead of ucontext.

In recent glibc versions the struct ucontext was renamed to
ucontext_t. The details can be found at [1].

The change

MB-26165:[utils/debug.cc] Use ucontext_t instead of ucontext.

In recent glibc versions the struct ucontext was renamed to
ucontext_t. The details can be found at [1].

The change is compatible with older glibc versions because the typedef
for the struct has been called ucontext_t for a long while. So it will
be picked up instead of the struct tag.

[1] https://sourceware.org/bugzilla/show_bug.cgi?id=21457

Change-Id: I6096e1576a973e8bcb33cad3a7cc94c2009d390d
Reviewed-on: http://review.couchbase.org/83579
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>

show more ...

9c4bf80323-Jun-2017 Jung-Sang Ahn <jungsang.ahn@gmail.com>

MB-24963: Add ISO 8601 timestamp for ForestDB

* Now every ForestDB log is prefixed with ISO 8601 timestamp.

Change-Id: I5138238d380e60e6cd3fea96ad56078e1f10fa0b
Reviewed-on: htt

MB-24963: Add ISO 8601 timestamp for ForestDB

* Now every ForestDB log is prefixed with ISO 8601 timestamp.

Change-Id: I5138238d380e60e6cd3fea96ad56078e1f10fa0b
Reviewed-on: http://review.couchbase.org/79924
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
Tested-by: Jung-Sang Ahn <jungsang.ahn@gmail.com>

show more ...

a3fc160b28-Jun-2017 Jung-Sang Ahn <jungsang.ahn@gmail.com>

[BP] MB-22718: Support validity check of old format files

* Added one more routine that checks whether opened file name is the
same with the given file name. If file name is different, i

[BP] MB-22718: Support validity check of old format files

* Added one more routine that checks whether opened file name is the
same with the given file name. If file name is different, it means that
previous compaction already succeeded. We can use it for validity check
of old format files that do not include SUCCESSFULLY_COMPACTED flag.

Change-Id: I7820c3bc1272b3bff0d106995fdaf110182102b6
Reviewed-on: http://review.couchbase.org/80123
Reviewed-by: Jung-Sang Ahn <jungsang.ahn@gmail.com>
Tested-by: Jung-Sang Ahn <jungsang.ahn@gmail.com>

show more ...

f9852de722-Jun-2017 Jung-Sang Ahn <jungsang.ahn@gmail.com>

[BP] MB-22718: Do sort by compaction number in fdb_validate_files() API

* Lexicographical order does not guarantee the correct order of file,
for example, fdb.10 is prior than fdb.5.

[BP] MB-22718: Do sort by compaction number in fdb_validate_files() API

* Lexicographical order does not guarantee the correct order of file,
for example, fdb.10 is prior than fdb.5.

* Extract the compaction number at the end of each file name, and then
sort them in a numerical order.

Change-Id: Ic72afe5bff5dec5473453a16b72567196764877c
Reviewed-on: http://review.couchbase.org/79902
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
Tested-by: Jung-Sang Ahn <jungsang.ahn@gmail.com>

show more ...

b6e47bcd22-Jun-2017 Jung-Sang Ahn <jungsang.ahn@gmail.com>

[BP] MB-22718: Add an API to find last valid file

* Added a new flag to DB header, indicates whether or not compaction
has been done successfully.

* Added a new API to check val

[BP] MB-22718: Add an API to find last valid file

* Added a new flag to DB header, indicates whether or not compaction
has been done successfully.

* Added a new API to check validity of the given series of files, using
above flag.

* It returns the last valid index number of the given file name array.

* All given files should belong to the same series of logical DB
instance.

Change-Id: Id6fd51cb3186e614f6c76837512374292a8877dd
Reviewed-on: http://review.couchbase.org/79901
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
Tested-by: Jung-Sang Ahn <jungsang.ahn@gmail.com>

show more ...

664c0dd505-Jan-2017 Sundar Sridharan <sundar.sridharan@gmail.com>

[BP] MB-22046: forestdb_dump --key must work on deleted keys too

Don't dump full header if a specific key or specific kv store
name is given. This is to aid testing code which is just

[BP] MB-22046: forestdb_dump --key must work on deleted keys too

Don't dump full header if a specific key or specific kv store
name is given. This is to aid testing code which is just
examining if a given key made it to the storage end.
Search by meta if search by key fails so that deleted keys
can be point searched too.

Change-Id: I460f663d5d7eab3ac89f814c12fb1be3c78c11ed
Reviewed-on: http://review.couchbase.org/79133
Reviewed-by: Jung-Sang Ahn <jungsang.ahn@gmail.com>
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>

show more ...

5827abea23-Mar-2017 Jung-Sang Ahn <jungsang.ahn@gmail.com>

MB-23416: Handle memory allocation failure during compaction

* The sorting window or document write batch in compaction logic are
big memory segments, thus allocating them may fail in th

MB-23416: Handle memory allocation failure during compaction

* The sorting window or document write batch in compaction logic are
big memory segments, thus allocating them may fail in the environment
with small RAM, such as mobile devices.

* We retry allocation with reduced size, and then return failure after
a few more attempts.

Change-Id: I16d04c8ff6314a47dfbf8f71b27e2729c1d3d6b4
Reviewed-on: http://review.couchbase.org/75630
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
Tested-by: Jung-Sang Ahn <jungsang.ahn@gmail.com>

show more ...

cc8ad99118-Apr-2017 Jung-Sang Ahn <jungsang.ahn@gmail.com>

MB-23835: Propagate allocation failure in docio_init()

* The failure of the allocation of aligned memory causes various kinds
of crashes in DocIO layer.

* Set 'handle->readbuffe

MB-23835: Propagate allocation failure in docio_init()

* The failure of the allocation of aligned memory causes various kinds
of crashes in DocIO layer.

* Set 'handle->readbuffer' to NULL if allocation failed.

* Propagate error to caller layer, to handle it gracefully.

Change-Id: I5aa3b5dc3d7f777f2b48fc69a12e15e30c15a529
Reviewed-on: http://review.couchbase.org/76925
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
Tested-by: Jung-Sang Ahn <jungsang.ahn@gmail.com>

show more ...

74dd921417-Apr-2017 Jung-Sang Ahn <jungsang.ahn@gmail.com>

MB-23583: Gracefully handle errors during block reclaiming

* realloc() call or decompression may cause error in block reclaiming
task, but those errors do not propagate to the upper laye

MB-23583: Gracefully handle errors during block reclaiming

* realloc() call or decompression may cause error in block reclaiming
task, but those errors do not propagate to the upper layer so it causes
crash later.

* Immediately stop block reclaiming task if any kinds of errors happens.

Change-Id: I35ccace0afc03102fa55b889c148f4d4d65d2fbe
Reviewed-on: http://review.couchbase.org/76920
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
Tested-by: Jung-Sang Ahn <jungsang.ahn@gmail.com>

show more ...

cf87722c09-Feb-2017 olivermd <oliver.downard@couchbase.com>

Guard against LIBRT=NOTFOUND on APPLE

As a result of upgrading the version of google benchmark used we need to
guard against LIBRT being set in the CMakeCache on MacOS which does not

Guard against LIBRT=NOTFOUND on APPLE

As a result of upgrading the version of google benchmark used we need to
guard against LIBRT being set in the CMakeCache on MacOS which does not
have LIBRT. This is because google benchmark tries to find it without
checking it's not making for a non MacOS system. This means that when
it's not found it sets the variable as NOTFOUND rather than it not being
set at all. This interfered with the forestdb CMakeLists.txt.

Change-Id: I01b8ae6b041062f27eaa562722e016db1005910f
Reviewed-on: http://review.couchbase.org/73451
Reviewed-by: Jim Walker <jim@couchbase.com>
Tested-by: Jung-Sang Ahn <jungsang.ahn@gmail.com>

show more ...

e461559907-Feb-2017 Jung-Sang Ahn <jungsang.ahn@gmail.com>

[BP] MB-22538 Use file name instead of pointers for old/new file link

* Based on the same logic in
http://review.couchbase.org/#/c/66318

* Removed prev_file, new_file pointers

[BP] MB-22538 Use file name instead of pointers for old/new file link

* Based on the same logic in
http://review.couchbase.org/#/c/66318

* Removed prev_file, new_file pointers, and added new member variable
for new file name.

* Both old file name and new file name are used for tracking the
compaction order.

Change-Id: I211e3f6ac237305badae882dcd46cb81af30b01a
Reviewed-on: http://review.couchbase.org/73263
Reviewed-by: abhinav dangeti <abhinav@couchbase.com>
Tested-by: Jung-Sang Ahn <jungsang.ahn@gmail.com>

show more ...

1d16951015-Dec-2016 Sundar Sridharan <sundar.sridharan@gmail.com>

[BP] MB-21953: Re-open compacted file in rollback to 0 retry

The following racy sequence causes a hang in rollback..
1. Default KV Store opened, new handle created for default kvs.
2

[BP] MB-21953: Re-open compacted file in rollback to 0 retry

The following racy sequence causes a hang in rollback..
1. Default KV Store opened, new handle created for default kvs.
2. Compaction runs and changes file to REMOVED_PENDING
3. rollback attempted with Default KV store handle still pointing to
old file.
4. rollback to zero attempts to remove kvs but finds current file
of root handle in FILE_REMOVE_PENDING state and retries operation.
During retry, new file is not re-opened but rather same file is
examined repeatedly causing a hang.
Simple fix is to check for file reopen on retry even for
rollback to zero
+Test case to reproduce the hang.

Change-Id: Id7e89873c7e283711ce1a1d411ee76e804e0dd06
Reviewed-on: http://review.couchbase.org/70977
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Well-Formed: buildbot <build@couchbase.com>
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>

show more ...

12345678910>>...53