History log of /6.6.0/couchstore/src/ (Results 1 - 25 of 592)
Revision (<<< Hide revision tags) (Show revision tags >>>)Date Author Comments
(<<< Hide modified files)
(Show modified files >>>)
Revision tags: v6.6.0
f798e0b722-May-2020 Jim Walker <jim@couchbase.com>

[BP] MB-39561: Remove more unnecessary dynamic allocations

Changes for backport:

* include <vector> in reductions.h
* remove c++ initialisation in c code (bitmap.h)

Bui

[BP] MB-39561: Remove more unnecessary dynamic allocations

Changes for backport:

* include <vector> in reductions.h
* remove c++ initialisation in c code (bitmap.h)

Building on http://review.couchbase.org/c/couchstore/+/128724 this patch
improves more functions in the views code with the aim of reducing the
number of calls to cb_malloc/cb_free.

The patch aims to make minimal changes to the code avoiding too much
re-factoring, therefore some of the changes may not be completely
compatible with coding standards and cpp core guidelines.

Similar to http://review.couchbase.org/c/couchstore/+/128724 some
dynamic allocations are completely removed, for example the
'view_bitmask' function can now be called and makes no calls to
cb_malloc.

Change-Id: I851dc66fed4c44ee8667ae6da0731ab7866b8cbc
Reviewed-on: http://review.couchbase.org/c/couchstore/+/129366
Well-Formed: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>
Reviewed-by: <ankit.prabhu@couchbase.com>
Reviewed-by: Srinath Duvuru <srinath.duvuru@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

a741406e21-May-2020 Jim Walker <jim@couchbase.com>

[BP] MB-39561: Remove unnecessary dynamic allocations

Two changes made for back-port:
1) include of vector in values.h
2) remove C++17 structured-binding in purgers.cc (use std::ti

[BP] MB-39561: Remove unnecessary dynamic allocations

Two changes made for back-port:
1) include of vector in values.h
2) remove C++17 structured-binding in purgers.cc (use std::tie)

This patch improves a number of functions in the views code with
the aim of reducing the number of calls to cb_malloc/cb_free. The
modified functions were identified by profiling a rebalance whilst
a basic view was defined.

The patch aims to make minimal changes to the code avoiding too much
re-factoring, therefore some of the changes may not be completely
compatible with coding standards and cpp core guidelines.

Some of the primary changes are.

1. Remove dynamic allocation where the type is statically sized.

Example: decode_view_btree_value now takes a reference and no longer
allocates a fixed size structure to be filled in an returned.

2. Remove all dynamic allocation for reading the 'partition'

Example: reducers.cc was iterating and calling
decode_view_id_btree_value when all it needed was the first 2 bytes of
the input stream returning (partition).

3. Where dynamic memory is needed, remove multiple allocations and use
one larger buffer.

Example: view_btree_value_t now has a std::vector which holds all of the
memory replacing the need for many smaller cb_malloc/cb_free - this
change also gives automatic clean-up when the vector destructs.
decode_view_btree_value sees calls to cb_malloc removed, replacing many
calls with 1 vector resize.

Change-Id: Ibbe1525a896ac20c56bb4503cf7dc4880ae6cca5
Reviewed-on: http://review.couchbase.org/c/couchstore/+/129075
Well-Formed: Build Bot <build@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>
Reviewed-by: Sarath Lakshman <sarath@couchbase.com>
Reviewed-by: <ankit.prabhu@couchbase.com>

show more ...

5cd4091f22-Jan-2020 sduvuru <sduvuru@gmail.com>

MB-25982: Reduce memory usage during compaction

Compaction creates the ID index using merge sort of the keys.
To create the sorted runs a certain number of keys are loaded
into memor

MB-25982: Reduce memory usage during compaction

Compaction creates the ID index using merge sort of the keys.
To create the sorted runs a certain number of keys are loaded
into memory and sorted. The value passed in as the number of
keys to load is large. The fix is to base the number of keys
on the memory to use and the average key size for the ID index.
The assumption is the average key size is 200bytes (key,value
and overhead) and 100MB of memory available for sort.

Change-Id: If9a750a08659207c5e47235878b30314e96b793d
Reviewed-on: http://review.couchbase.org/120601
Well-Formed: Build Bot <build@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

9684039712-Dec-2019 Ben Huddleston <ben.huddleston@couchbase.com>

MB-36926: Use a PackedPtr for the type of a couchfile_modify_action

The couchfile_modify_action type field can be one of 4 values but is
currently stored as an int. If I change this to a

MB-36926: Use a PackedPtr for the type of a couchfile_modify_action

The couchfile_modify_action type field can be one of 4 values but is
currently stored as an int. If I change this to a uint8_t the size of
the item is not actually reduced due to padding. Store the type of the
item in the ptr top bits by using a folly PackedSyncPtr to save space
on x86-64 machines.

Before:

Run on (24 X 2400 MHz CPU s)
2020-01-07 14:11:38
***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead.
--------------------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations
UserCounters...--------------------------------------------------------------------------------------------------------
MemTrackingVBucketBench/FlushVBucket/0/1/0 10526240 ns 473020 ns 1000 PeakBytesPerItem=169.453k PeakFlushBytes=169.453k 2.06453k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/1/1 10050385 ns 545306 ns 1310 PeakBytesPerItem=174.109k PeakFlushBytes=174.109k 1.79085k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/100/0 32432631 ns 686847 ns 100 PeakBytesPerItem=2.12109k PeakFlushBytes=212.188k 142.18k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/100/1 10654749 ns 804115 ns 865 PeakBytesPerItem=2.32031k PeakFlushBytes=232.047k 121.446k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/10000/0 64317372 ns 39451200 ns 18 PeakBytesPerItem=541 PeakFlushBytes=5.16405M 247.537k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/10000/1 64357234 ns 53511370 ns 13 PeakBytesPerItem=624 PeakFlushBytes=5.95404M 182.496k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/1000000/0 6046917273 ns 5800217195 ns 1 PeakBytesPerItem=554 PeakFlushBytes=528.923M 168.367k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/1000000/1 8107168836 ns 7804534786 ns 1 PeakBytesPerItem=622 PeakFlushBytes=593.423M 125.128k items/s store:couchdb mode:replace

After:

Run on (24 X 2400 MHz CPU s)
2020-01-07 14:22:26
***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead.
--------------------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations
UserCounters...--------------------------------------------------------------------------------------------------------
MemTrackingVBucketBench/FlushVBucket/0/1/0 19700477 ns 315396 ns 1000 PeakBytesPerItem=169.453k PeakFlushBytes=169.453k 3.09631k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/1/1 16453375 ns 435352 ns 1000 PeakBytesPerItem=174.109k PeakFlushBytes=174.109k 2.24315k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/100/0 10685598 ns 666880 ns 952 PeakBytesPerItem=2.0918k PeakFlushBytes=209.188k 146.437k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/100/1 10630037 ns 809191 ns 871 PeakBytesPerItem=2.29004k PeakFlushBytes=229.078k 120.684k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/10000/0 64772284 ns 39635213 ns 18 PeakBytesPerItem=528 PeakFlushBytes=5.03905M 246.388k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/10000/1 64839296 ns 52869913 ns 13 PeakBytesPerItem=611 PeakFlushBytes=5.82904M 184.71k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/1000000/0 6028077858 ns 5765538078 ns 1 PeakBytesPerItem=537 PeakFlushBytes=512.923M 169.379k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/1000000/1 8008824879 ns 7734230045 ns 1 PeakBytesPerItem=605 PeakFlushBytes=577.423M 126.265k items/s store:couchdb mode:replace

Change-Id: I60b0c764fe9e80985b0fc166cefdcfd0a05aedf0
Reviewed-on: http://review.couchbase.org/120034
Reviewed-by: Jim Walker <jim@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

aa2f4cf212-Dec-2019 Ben Huddleston <ben.huddleston@couchbase.com>

MB-36926: Remove C linkage from couch_btree.h

To reduce flusher memory overhead I will replace a pointer with a
folly PackedSyncPtr which is C++ code. This header is only used from
o

MB-36926: Remove C linkage from couch_btree.h

To reduce flusher memory overhead I will replace a pointer with a
folly PackedSyncPtr which is C++ code. This header is only used from
other C++ compilation units.

Change-Id: I34ea76dc750c290bcb4ee1cc1db191acedd9a66f
Reviewed-on: http://review.couchbase.org/120033
Reviewed-by: Trond Norbye <trond.norbye@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

2b48820820-Nov-2019 Ben Huddleston <ben.huddleston@couchbase.com>

MB-36926: Reuse idacts for seqacts

To save memory when we update indexes swap the order of idacts and
seqacts so that we re-use some of the memory originally allocated for
idacts for

MB-36926: Reuse idacts for seqacts

To save memory when we update indexes swap the order of idacts and
seqacts so that we re-use some of the memory originally allocated for
idacts for seqacts if we have to remove any seqnos as a result of
idact Fetch callbacks. This is only possible as the buffer of memory
allocated for actions is contiguous.

Before:

Run on (24 X 2400 MHz CPU s)
2020-01-07 14:09:29
***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead.
--------------------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations
UserCounters...--------------------------------------------------------------------------------------------------------
MemTrackingVBucketBench/FlushVBucket/0/1/0 10968423 ns 311189 ns 1000 PeakBytesPerItem=169.453k PeakFlushBytes=169.453k 3.13817k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/1/1 12612678 ns 496806 ns 1000 PeakBytesPerItem=174.109k PeakFlushBytes=174.109k 1.96568k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/100/0 10656068 ns 650293 ns 1071 PeakBytesPerItem=2.1416k PeakFlushBytes=214.188k 150.173k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/100/1 10676463 ns 814919 ns 872 PeakBytesPerItem=2.33984k PeakFlushBytes=234.078k 119.835k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/10000/0 49862669 ns 39303727 ns 18 PeakBytesPerItem=580 PeakFlushBytes=5.53905M 248.466k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/10000/1 64845112 ns 54019393 ns 13 PeakBytesPerItem=663 PeakFlushBytes=6.32901M 180.78k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/1000000/0 6030438834 ns 5698634652 ns 1 PeakBytesPerItem=588 PeakFlushBytes=560.923M 171.368k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/1000000/1 8131300347 ns 7860270771 ns 1 PeakBytesPerItem=655 PeakFlushBytes=625.423M 124.24k items/s store:couchdb mode:replace

After:

Run on (24 X 2400 MHz CPU s)
2020-01-07 14:11:38
***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead.
--------------------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations
UserCounters...--------------------------------------------------------------------------------------------------------
MemTrackingVBucketBench/FlushVBucket/0/1/0 10526240 ns 473020 ns 1000 PeakBytesPerItem=169.453k PeakFlushBytes=169.453k 2.06453k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/1/1 10050385 ns 545306 ns 1310 PeakBytesPerItem=174.109k PeakFlushBytes=174.109k 1.79085k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/100/0 32432631 ns 686847 ns 100 PeakBytesPerItem=2.12109k PeakFlushBytes=212.188k 142.18k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/100/1 10654749 ns 804115 ns 865 PeakBytesPerItem=2.32031k PeakFlushBytes=232.047k 121.446k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/10000/0 64317372 ns 39451200 ns 18 PeakBytesPerItem=541 PeakFlushBytes=5.16405M 247.537k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/10000/1 64357234 ns 53511370 ns 13 PeakBytesPerItem=624 PeakFlushBytes=5.95404M 182.496k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/1000000/0 6046917273 ns 5800217195 ns 1 PeakBytesPerItem=554 PeakFlushBytes=528.923M 168.367k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/1000000/1 8107168836 ns 7804534786 ns 1 PeakBytesPerItem=622 PeakFlushBytes=593.423M 125.128k items/s store:couchdb mode:replace

Change-Id: I071fd21eb75d2cab655fc762769463f85e9968b4
Reviewed-on: http://review.couchbase.org/120032
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Jim Walker <jim@couchbase.com>

show more ...

c75b453712-Dec-2019 Ben Huddleston <ben.huddleston@couchbase.com>

MB-36926: Add FETCH_INSERT modify action

To save memory whilst flushing add a FETCH_INSERT action that
performs the FETCH and then the INSERT action on the given document
when updati

MB-36926: Add FETCH_INSERT modify action

To save memory whilst flushing add a FETCH_INSERT action that
performs the FETCH and then the INSERT action on the given document
when updating indexes. This saves us a O(n) amount of memory as we
halve the actions (1 per item) that we allocate memory for when
updating the by id index.

Before:

Run on (24 X 2400 MHz CPU s)
2020-01-07 14:04:53
***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead.
--------------------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations
UserCounters...--------------------------------------------------------------------------------------------------------
MemTrackingVBucketBench/FlushVBucket/0/1/0 10971667 ns 313077 ns 1000 PeakBytesPerItem=169.453k PeakFlushBytes=169.453k 3.11924k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/1/1 17295714 ns 526468 ns 1000 PeakBytesPerItem=174.078k PeakFlushBytes=174.078k 1.85493k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/100/0 11031869 ns 662786 ns 1045 PeakBytesPerItem=2.18164k PeakFlushBytes=218.188k 147.342k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/100/1 10627213 ns 804693 ns 874 PeakBytesPerItem=2.37988k PeakFlushBytes=238.078k 121.358k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/10000/0 57675779 ns 39132094 ns 18 PeakBytesPerItem=607 PeakFlushBytes=5.78905M 249.555k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/10000/1 64448036 ns 53727856 ns 13 PeakBytesPerItem=689 PeakFlushBytes=6.57901M 181.761k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/1000000/0 6059956216 ns 5724502548 ns 1 PeakBytesPerItem=621 PeakFlushBytes=592.923M 170.593k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/1000000/1 8639426887 ns 7868837390 ns 1 PeakBytesPerItem=689 PeakFlushBytes=657.423M 124.105k items/s store:couchdb mode:replace

After:

Run on (24 X 2400 MHz CPU s)
2020-01-07 14:09:29
***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead.
--------------------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations
UserCounters...--------------------------------------------------------------------------------------------------------
MemTrackingVBucketBench/FlushVBucket/0/1/0 10968423 ns 311189 ns 1000 PeakBytesPerItem=169.453k PeakFlushBytes=169.453k 3.13817k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/1/1 12612678 ns 496806 ns 1000 PeakBytesPerItem=174.109k PeakFlushBytes=174.109k 1.96568k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/100/0 10656068 ns 650293 ns 1071 PeakBytesPerItem=2.1416k PeakFlushBytes=214.188k 150.173k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/100/1 10676463 ns 814919 ns 872 PeakBytesPerItem=2.33984k PeakFlushBytes=234.078k 119.835k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/10000/0 49862669 ns 39303727 ns 18 PeakBytesPerItem=580 PeakFlushBytes=5.53905M 248.466k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/10000/1 64845112 ns 54019393 ns 13 PeakBytesPerItem=663 PeakFlushBytes=6.32901M 180.78k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/1000000/0 6030438834 ns 5698634652 ns 1 PeakBytesPerItem=588 PeakFlushBytes=560.923M 171.368k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/1000000/1 8131300347 ns 7860270771 ns 1 PeakBytesPerItem=655 PeakFlushBytes=625.423M 124.24k items/s store:couchdb mode:replace

Change-Id: Ib831af61b4401986f5bfec2bcd71cdf36f0c05c5
Reviewed-on: http://review.couchbase.org/120031
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Jim Walker <jim@couchbase.com>

show more ...

0dda1c6011-Dec-2019 Ben Huddleston <ben.huddleston@couchbase.com>

MB-39629: Move fetch_callback_ctx to couchfile_modify_request

Currently we use the arg field of couchfile_modify_action for a
context object to pass into the fetch_callback. As this is o

MB-39629: Move fetch_callback_ctx to couchfile_modify_request

Currently we use the arg field of couchfile_modify_action for a
context object to pass into the fetch_callback. As this is only used
in one place and every context object is identical, refactor the
couchfile_modify_request to instead take a pointer to the
callback_ctx and remove the now unnecessary union from
couchfile_modify_action. This will allow us to combine Fetch and Insert
actions in a future patch which will save us memory overhead when
updating indexes as part of flushing.

Change-Id: If1c561ed3f3c517fb97f0cba2fbc02cfc28ac4be
Reviewed-on: http://review.couchbase.org/120030
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Jim Walker <jim@couchbase.com>

show more ...

f8ebfb1120-Nov-2019 Ben Huddleston <ben.huddleston@couchbase.com>

MB-36926: Don't oversize actions buffer in update_indexes

This buffer is currently oversized by 20 bytes per item on linux.

Before:

Run on (24 X 2400 MHz CPU s)
2020-01

MB-36926: Don't oversize actions buffer in update_indexes

This buffer is currently oversized by 20 bytes per item on linux.

Before:

Run on (24 X 2400 MHz CPU s)
2020-01-07 14:27:10
***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead.
--------------------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations
UserCounters...--------------------------------------------------------------------------------------------------------
MemTrackingVBucketBench/FlushVBucket/0/1/0 10576568 ns 313547 ns 1000 PeakBytesPerItem=169.453k PeakFlushBytes=169.453k 3.11456k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/1/1 10279942 ns 582090 ns 1000 PeakBytesPerItem=174.109k PeakFlushBytes=174.109k 1.67768k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/100/0 11057268 ns 655322 ns 1056 PeakBytesPerItem=2.22168k PeakFlushBytes=222.188k 149.02k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/100/1 10723686 ns 789029 ns 900 PeakBytesPerItem=2.41992k PeakFlushBytes=242.047k 123.768k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/10000/0 63575784 ns 38858511 ns 18 PeakBytesPerItem=633 PeakFlushBytes=6.03905M 251.312k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/10000/1 64116211 ns 53274824 ns 13 PeakBytesPerItem=716 PeakFlushBytes=6.82904M 183.307k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/1000000/0 6085358211 ns 5724388638 ns 1 PeakBytesPerItem=655 PeakFlushBytes=624.923M 170.597k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/1000000/1 8237308870 ns 7816582048 ns 1 PeakBytesPerItem=722 PeakFlushBytes=689.423M 124.935k items/s store:couchdb mode:replace

After:

Run on (24 X 2400 MHz CPU s)
2020-01-07 14:04:53
***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead.
--------------------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations
UserCounters...--------------------------------------------------------------------------------------------------------
MemTrackingVBucketBench/FlushVBucket/0/1/0 10971667 ns 313077 ns 1000 PeakBytesPerItem=169.453k PeakFlushBytes=169.453k 3.11924k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/1/1 17295714 ns 526468 ns 1000 PeakBytesPerItem=174.078k PeakFlushBytes=174.078k 1.85493k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/100/0 11031869 ns 662786 ns 1045 PeakBytesPerItem=2.18164k PeakFlushBytes=218.188k 147.342k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/100/1 10627213 ns 804693 ns 874 PeakBytesPerItem=2.37988k PeakFlushBytes=238.078k 121.358k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/10000/0 57675779 ns 39132094 ns 18 PeakBytesPerItem=607 PeakFlushBytes=5.78905M 249.555k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/10000/1 64448036 ns 53727856 ns 13 PeakBytesPerItem=689 PeakFlushBytes=6.57901M 181.761k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/1000000/0 6059956216 ns 5724502548 ns 1 PeakBytesPerItem=621 PeakFlushBytes=592.923M 170.593k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/1000000/1 8639426887 ns 7868837390 ns 1 PeakBytesPerItem=689 PeakFlushBytes=657.423M 124.105k items/s store:couchdb mode:replace

Change-Id: I84a3fd3cc432e8ecbfd809f23db8e62b1d26ffd9
Reviewed-on: http://review.couchbase.org/120029
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Jim Walker <jim@couchbase.com>

show more ...

Revision tags: v6.5.0
cf120ada31-Oct-2019 Jim Walker <jim@couchbase.com>

MB-23086: Create libcouchstore and libcouchstore_views

Generate two libraries so that libcouchstore can lose the
dependency on libv8 and libicu. This will in turn mean that
ep-engine

MB-23086: Create libcouchstore and libcouchstore_views

Generate two libraries so that libcouchstore can lose the
dependency on libv8 and libicu. This will in turn mean that
ep-engine no longer has the dependency on those libraries.

Change-Id: I3e2f05ee1ee1b3acd75ad9a80c6faa92c8496135
Reviewed-on: http://review.couchbase.org/117265
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Trond Norbye <trond.norbye@couchbase.com>

show more ...

48c2917422-Oct-2019 Trond Norbye <trond.norbye@gmail.com>

Cleanup: remove unused method tmp_file_path

The implementation of the method caused a compiler warning as
part of trying to define deprecated preprocessor symbols.

Change-Id: Ic

Cleanup: remove unused method tmp_file_path

The implementation of the method caused a compiler warning as
part of trying to define deprecated preprocessor symbols.

Change-Id: Icc2acd1fd1fd396a203f7d5607b151e60729815e
Reviewed-on: http://review.couchbase.org/116764
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

61193ca317-Oct-2019 sduvuru <sduvuru@gmail.com>

MB-31930: couch_dbdump show expired keys as corrupted in its output

An expired key and deleted key was incorrectly determined as corrupt
by the dump utility. The code wrongly assumed doc

MB-31930: couch_dbdump show expired keys as corrupted in its output

An expired key and deleted key was incorrectly determined as corrupt
by the dump utility. The code wrongly assumed docinfo->bp set to 0
indicated a corruption. The value of docinfo->bp can be zero if the
document is deleted. If the document is not deleted and docinfo->bp is
0, then there is a corruption.

Change-Id: I7e864a7751b0099b14a6cf5de2b3b4ef92278f67
Reviewed-on: http://review.couchbase.org/116592
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

Revision tags: v6.0.4
c879f0d710-Oct-2019 Dave Rigby <daver@couchbase.com>

MB-36424: Ensure flush_mr_partial() obeys node size quota

When couchstore writes out modified node elements during document
saves, it it supposed to limit the number of bytes written in

MB-36424: Ensure flush_mr_partial() obeys node size quota

When couchstore writes out modified node elements during document
saves, it it supposed to limit the number of bytes written in a single
node to a _chunk_threshold - by default 1279 bytes. If the node is
larger than the limit it should be split into multiple sibling nodes.

However, this limit is _not_ respected, resulting in overly-large
nodes being written out. In the case of the by-seqno B-Tree (which
always writes values to the rightmost leaf as seqnos are increasing),
it results in all leaf elements residing in a single leaf
node. Moreover, this means that adding another element to the B-Tree
effectively re-writes the entire tree, resulting in massive Write
Amplification. Local testing saw Write Amplification up to 743x,
compared to around 5x immediately after compaction (when B-Tree
structure is repaired).

Problem is in the handling of the node quota in flush_mr_partial() -
the while loop terminating conditions is if mr_quota goes below zero
(i.e. quota has been consumed). mr_quota is provided as an input
parameter, and then decremented inside the while loop.

However, the type of mr_quota is an unsigned size_t - as such when it
is decremented in the while loop after writing the key/value pair it
goes to a number close to SIZE_T_MAX, not a negative value. As a
result the while loop is not exited.

This bug has been present since July 2012 - was introduced via
http://review.couchbase.org/#/c/18008/

Change-Id: I60200fcb100525ceb0fd91971c702645f0c4f3f5
Reviewed-on: http://review.couchbase.org/116219
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: James Harrison <james.harrison@couchbase.com>

show more ...

b8bc47e923-Sep-2019 AnkitPrabhu <ankit.prabhu@couchbase.com>

MB-31538: Log error number when fopen fails

Change-Id: Ic0b5ea87770dfb440524f40afefd46d881a1a776
Reviewed-on: http://review.couchbase.org/115195
Tested-by: Build Bot <build@couchbase

MB-31538: Log error number when fopen fails

Change-Id: Ic0b5ea87770dfb440524f40afefd46d881a1a776
Reviewed-on: http://review.couchbase.org/115195
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Suraj Naik <suraj.naik@couchbase.com>

show more ...

cfc9f88e14-Sep-2019 sduvuru <sduvuru@gmail.com>

Merge remote-tracking branch 'couchbase/vulcan'

* MB-35058: Couchstore-Trace operations on a file

Change-Id: Ia22ae175b980597f6686bf84f093696cbab7ed40


b87ff9fe04-Sep-2019 sduvuru <sduvuru@gmail.com>

MB-34395: couch_dbdump incorrectly prints doc body if compressed
and contains XATTRs

The dump utility instead of using the inflated document uses the
compressed document to

MB-34395: couch_dbdump incorrectly prints doc body if compressed
and contains XATTRs

The dump utility instead of using the inflated document uses the
compressed document to dump a document that contains XATTRs.
Fix is to use the inflated document

Change-Id: I78b83b672c22ebdda7df720bd5a06f871922948a
Reviewed-on: http://review.couchbase.org/114274
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

Revision tags: v6.0.3
c2c458ff19-Mar-2019 sduvuru <sduvuru@gmail.com>

MB-35058: Couchstore-Trace operations on a file

-Tracing support to couchstore and dump trace on detection of corruption
-mprotect of iobuffer
-verify write to buffer cache by readin

MB-35058: Couchstore-Trace operations on a file

-Tracing support to couchstore and dump trace on detection of corruption
-mprotect of iobuffer
-verify write to buffer cache by reading back

The validation checks are passed in as flags when opening the database file.
The options are tracing, write validation and mprotect
-- tracing is turned on
-- write verification by reading back the written block
-- mprotect of internal iobuffer

Note: mprotect is not supported for windows platform.

Change-Id: I7b6f29f493ecbad377df966d3004d2fbbb32e8e1
Reviewed-on: http://review.couchbase.org/113540
Well-Formed: Build Bot <build@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

eef37beb25-Jul-2019 sduvuru <sduvuru@gmail.com>

MB-35473: couchstore- return of 0 by pwrite() is not handled

Backport of MB-34862

A return value of 0 from pwrite() would actually mean that
no bytes were written and the caller

MB-35473: couchstore- return of 0 by pwrite() is not handled

Backport of MB-34862

A return value of 0 from pwrite() would actually mean that
no bytes were written and the caller should retry pwrite.
Couchstore code does not retry and it leads to assuming an
entire block has been written out when it hasn't.

The fix is to retry in the case of 0 bytes getting written.

Unit tests for buffered and unbuffered cases are added.

Change-Id: I71a1e8a247edfeb773db93906a70a6624de252b9
Reviewed-on: http://review.couchbase.org/113353
Tested-by: Build Bot <build@couchbase.com>
Well-Formed: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

5186902f25-Jul-2019 sduvuru <sduvuru@gmail.com>

MB-34862: couchstore- return of 0 by pwrite() is not handled

A return value of 0 from pwrite() would actually mean that
no bytes were written and the caller should retry pwrite.
Couc

MB-34862: couchstore- return of 0 by pwrite() is not handled

A return value of 0 from pwrite() would actually mean that
no bytes were written and the caller should retry pwrite.
Couchstore code does not retry and it leads to assuming an
entire block has been written out when it hasn't.

The fix is to retry in the case of 0 bytes getting written.

Unit tests for buffered and unbuffered cases are added.

Change-Id: I5f592d8a4084193e6bc9df88a1220a84854d113e
Reviewed-on: http://review.couchbase.org/112505
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

336ad22515-Jul-2019 AnkitPrabhu <ankit.prabhu@couchbase.com>

MB-34989: Pickup icu data file during initilization file merger

Change-Id: Ice19721dd26dd4d6005acce46f08f21d2a2db464
Reviewed-on: http://review.couchbase.org/111911
Reviewed-by: Sura

MB-34989: Pickup icu data file during initilization file merger

Change-Id: Ice19721dd26dd4d6005acce46f08f21d2a2db464
Reviewed-on: http://review.couchbase.org/111911
Reviewed-by: Suraj Naik <suraj.naik@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

5f50fb5b06-Jun-2019 AnkitPrabhu <ankit.prabhu@couchbase.com>

MB-32313: Upgrade to v8 v7.1

- Use the new v8 API
- CBD-2634 use ICU from v8
- Copy icudtl.dat to bin and initialize ICU with it

Change-Id: I1f5618ca26162630233bdbc718

MB-32313: Upgrade to v8 v7.1

- Use the new v8 API
- CBD-2634 use ICU from v8
- Copy icudtl.dat to bin and initialize ICU with it

Change-Id: I1f5618ca26162630233bdbc71837a1b4f12c3301
Reviewed-on: http://review.couchbase.org/110297
Reviewed-by: Sriram Melkote <siri@couchbase.com>
Tested-by: Suraj Naik <suraj.naik@couchbase.com>

show more ...

875284e615-Feb-2019 Dave Rigby <daver@couchbase.com>

dbdump: Add Synchronous Replication metadata

Update couch_dbdump to print the V3 metadata associated with
Synchronous Replication.

Change-Id: I61a3e28927d3b7e9a90128a9ed81664dea

dbdump: Add Synchronous Replication metadata

Update couch_dbdump to print the V3 metadata associated with
Synchronous Replication.

Change-Id: I61a3e28927d3b7e9a90128a9ed81664deaa06531
Reviewed-on: http://review.couchbase.org/104994
Reviewed-by: Daniel Owen <owend@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Paolo Cocchi <paolo.cocchi@couchbase.com>

show more ...

731bbc0c21-Mar-2019 Dave Rigby <daver@couchbase.com>

dbdump: Print keys in tree view correctly

With the advent of collections we need to use printDocId() to print
document keys and not just printsb().

Change-Id: I31224a94ae63a8871

dbdump: Print keys in tree view correctly

With the advent of collections we need to use printDocId() to print
document keys and not just printsb().

Change-Id: I31224a94ae63a88710fe0bce54af3773c77c280e
Reviewed-on: http://review.couchbase.org/106572
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Srinath Duvuru <srinath.duvuru@couchbase.com>
Reviewed-by: Trond Norbye <trond.norbye@couchbase.com>

show more ...

1670751321-Mar-2019 Trond Norbye <trond.norbye@gmail.com>

Rename config.h to couchstore_config.h

We had a number of projects with their own "config.h" and
depending on the compiler search order you could end up
getting another one than the

Rename config.h to couchstore_config.h

We had a number of projects with their own "config.h" and
depending on the compiler search order you could end up
getting another one than the one you expected.

Change-Id: I92430a321bdc51f29f5993a856e1adc59b7de019
Reviewed-on: http://review.couchbase.org/106560
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

b0f5a94820-Mar-2019 Trond Norbye <trond.norbye@gmail.com>

Don't include config.h from headers

Depending on which module the file is included from we might
get a different version of config.h (kv used to pick up its
own)

Change-Id:

Don't include config.h from headers

Depending on which module the file is included from we might
get a different version of config.h (kv used to pick up its
own)

Change-Id: Id2123e7de363cedf7b4d9ef9be96ffe99175362e
Reviewed-on: http://review.couchbase.org/106511
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

12345678910>>...24