History log of /6.6.0/couchstore/tests/ (Results 1 - 25 of 242)
Revision (<<< Hide revision tags) (Show revision tags >>>)Date Author Comments
(<<< Hide modified files)
(Show modified files >>>)
Revision tags: v6.6.0
f798e0b722-May-2020 Jim Walker <jim@couchbase.com>

[BP] MB-39561: Remove more unnecessary dynamic allocations

Changes for backport:

* include <vector> in reductions.h
* remove c++ initialisation in c code (bitmap.h)

Bui

[BP] MB-39561: Remove more unnecessary dynamic allocations

Changes for backport:

* include <vector> in reductions.h
* remove c++ initialisation in c code (bitmap.h)

Building on http://review.couchbase.org/c/couchstore/+/128724 this patch
improves more functions in the views code with the aim of reducing the
number of calls to cb_malloc/cb_free.

The patch aims to make minimal changes to the code avoiding too much
re-factoring, therefore some of the changes may not be completely
compatible with coding standards and cpp core guidelines.

Similar to http://review.couchbase.org/c/couchstore/+/128724 some
dynamic allocations are completely removed, for example the
'view_bitmask' function can now be called and makes no calls to
cb_malloc.

Change-Id: I851dc66fed4c44ee8667ae6da0731ab7866b8cbc
Reviewed-on: http://review.couchbase.org/c/couchstore/+/129366
Well-Formed: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>
Reviewed-by: <ankit.prabhu@couchbase.com>
Reviewed-by: Srinath Duvuru <srinath.duvuru@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

a741406e21-May-2020 Jim Walker <jim@couchbase.com>

[BP] MB-39561: Remove unnecessary dynamic allocations

Two changes made for back-port:
1) include of vector in values.h
2) remove C++17 structured-binding in purgers.cc (use std::ti

[BP] MB-39561: Remove unnecessary dynamic allocations

Two changes made for back-port:
1) include of vector in values.h
2) remove C++17 structured-binding in purgers.cc (use std::tie)

This patch improves a number of functions in the views code with
the aim of reducing the number of calls to cb_malloc/cb_free. The
modified functions were identified by profiling a rebalance whilst
a basic view was defined.

The patch aims to make minimal changes to the code avoiding too much
re-factoring, therefore some of the changes may not be completely
compatible with coding standards and cpp core guidelines.

Some of the primary changes are.

1. Remove dynamic allocation where the type is statically sized.

Example: decode_view_btree_value now takes a reference and no longer
allocates a fixed size structure to be filled in an returned.

2. Remove all dynamic allocation for reading the 'partition'

Example: reducers.cc was iterating and calling
decode_view_id_btree_value when all it needed was the first 2 bytes of
the input stream returning (partition).

3. Where dynamic memory is needed, remove multiple allocations and use
one larger buffer.

Example: view_btree_value_t now has a std::vector which holds all of the
memory replacing the need for many smaller cb_malloc/cb_free - this
change also gives automatic clean-up when the vector destructs.
decode_view_btree_value sees calls to cb_malloc removed, replacing many
calls with 1 vector resize.

Change-Id: Ibbe1525a896ac20c56bb4503cf7dc4880ae6cca5
Reviewed-on: http://review.couchbase.org/c/couchstore/+/129075
Well-Formed: Build Bot <build@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>
Reviewed-by: Sarath Lakshman <sarath@couchbase.com>
Reviewed-by: <ankit.prabhu@couchbase.com>

show more ...

9684039712-Dec-2019 Ben Huddleston <ben.huddleston@couchbase.com>

MB-36926: Use a PackedPtr for the type of a couchfile_modify_action

The couchfile_modify_action type field can be one of 4 values but is
currently stored as an int. If I change this to a

MB-36926: Use a PackedPtr for the type of a couchfile_modify_action

The couchfile_modify_action type field can be one of 4 values but is
currently stored as an int. If I change this to a uint8_t the size of
the item is not actually reduced due to padding. Store the type of the
item in the ptr top bits by using a folly PackedSyncPtr to save space
on x86-64 machines.

Before:

Run on (24 X 2400 MHz CPU s)
2020-01-07 14:11:38
***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead.
--------------------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations
UserCounters...--------------------------------------------------------------------------------------------------------
MemTrackingVBucketBench/FlushVBucket/0/1/0 10526240 ns 473020 ns 1000 PeakBytesPerItem=169.453k PeakFlushBytes=169.453k 2.06453k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/1/1 10050385 ns 545306 ns 1310 PeakBytesPerItem=174.109k PeakFlushBytes=174.109k 1.79085k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/100/0 32432631 ns 686847 ns 100 PeakBytesPerItem=2.12109k PeakFlushBytes=212.188k 142.18k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/100/1 10654749 ns 804115 ns 865 PeakBytesPerItem=2.32031k PeakFlushBytes=232.047k 121.446k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/10000/0 64317372 ns 39451200 ns 18 PeakBytesPerItem=541 PeakFlushBytes=5.16405M 247.537k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/10000/1 64357234 ns 53511370 ns 13 PeakBytesPerItem=624 PeakFlushBytes=5.95404M 182.496k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/1000000/0 6046917273 ns 5800217195 ns 1 PeakBytesPerItem=554 PeakFlushBytes=528.923M 168.367k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/1000000/1 8107168836 ns 7804534786 ns 1 PeakBytesPerItem=622 PeakFlushBytes=593.423M 125.128k items/s store:couchdb mode:replace

After:

Run on (24 X 2400 MHz CPU s)
2020-01-07 14:22:26
***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead.
--------------------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations
UserCounters...--------------------------------------------------------------------------------------------------------
MemTrackingVBucketBench/FlushVBucket/0/1/0 19700477 ns 315396 ns 1000 PeakBytesPerItem=169.453k PeakFlushBytes=169.453k 3.09631k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/1/1 16453375 ns 435352 ns 1000 PeakBytesPerItem=174.109k PeakFlushBytes=174.109k 2.24315k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/100/0 10685598 ns 666880 ns 952 PeakBytesPerItem=2.0918k PeakFlushBytes=209.188k 146.437k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/100/1 10630037 ns 809191 ns 871 PeakBytesPerItem=2.29004k PeakFlushBytes=229.078k 120.684k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/10000/0 64772284 ns 39635213 ns 18 PeakBytesPerItem=528 PeakFlushBytes=5.03905M 246.388k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/10000/1 64839296 ns 52869913 ns 13 PeakBytesPerItem=611 PeakFlushBytes=5.82904M 184.71k items/s store:couchdb mode:replace
MemTrackingVBucketBench/FlushVBucket/0/1000000/0 6028077858 ns 5765538078 ns 1 PeakBytesPerItem=537 PeakFlushBytes=512.923M 169.379k items/s store:couchdb mode:insert
MemTrackingVBucketBench/FlushVBucket/0/1000000/1 8008824879 ns 7734230045 ns 1 PeakBytesPerItem=605 PeakFlushBytes=577.423M 126.265k items/s store:couchdb mode:replace

Change-Id: I60b0c764fe9e80985b0fc166cefdcfd0a05aedf0
Reviewed-on: http://review.couchbase.org/120034
Reviewed-by: Jim Walker <jim@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

0dda1c6011-Dec-2019 Ben Huddleston <ben.huddleston@couchbase.com>

MB-39629: Move fetch_callback_ctx to couchfile_modify_request

Currently we use the arg field of couchfile_modify_action for a
context object to pass into the fetch_callback. As this is o

MB-39629: Move fetch_callback_ctx to couchfile_modify_request

Currently we use the arg field of couchfile_modify_action for a
context object to pass into the fetch_callback. As this is only used
in one place and every context object is identical, refactor the
couchfile_modify_request to instead take a pointer to the
callback_ctx and remove the now unnecessary union from
couchfile_modify_action. This will allow us to combine Fetch and Insert
actions in a future patch which will save us memory overhead when
updating indexes as part of flushing.

Change-Id: If1c561ed3f3c517fb97f0cba2fbc02cfc28ac4be
Reviewed-on: http://review.couchbase.org/120030
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Jim Walker <jim@couchbase.com>

show more ...

Revision tags: v6.5.0, v6.0.4
cfc9f88e14-Sep-2019 sduvuru <sduvuru@gmail.com>

Merge remote-tracking branch 'couchbase/vulcan'

* MB-35058: Couchstore-Trace operations on a file

Change-Id: Ia22ae175b980597f6686bf84f093696cbab7ed40


Revision tags: v6.0.3
c2c458ff19-Mar-2019 sduvuru <sduvuru@gmail.com>

MB-35058: Couchstore-Trace operations on a file

-Tracing support to couchstore and dump trace on detection of corruption
-mprotect of iobuffer
-verify write to buffer cache by readin

MB-35058: Couchstore-Trace operations on a file

-Tracing support to couchstore and dump trace on detection of corruption
-mprotect of iobuffer
-verify write to buffer cache by reading back

The validation checks are passed in as flags when opening the database file.
The options are tracing, write validation and mprotect
-- tracing is turned on
-- write verification by reading back the written block
-- mprotect of internal iobuffer

Note: mprotect is not supported for windows platform.

Change-Id: I7b6f29f493ecbad377df966d3004d2fbbb32e8e1
Reviewed-on: http://review.couchbase.org/113540
Well-Formed: Build Bot <build@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

eef37beb25-Jul-2019 sduvuru <sduvuru@gmail.com>

MB-35473: couchstore- return of 0 by pwrite() is not handled

Backport of MB-34862

A return value of 0 from pwrite() would actually mean that
no bytes were written and the caller

MB-35473: couchstore- return of 0 by pwrite() is not handled

Backport of MB-34862

A return value of 0 from pwrite() would actually mean that
no bytes were written and the caller should retry pwrite.
Couchstore code does not retry and it leads to assuming an
entire block has been written out when it hasn't.

The fix is to retry in the case of 0 bytes getting written.

Unit tests for buffered and unbuffered cases are added.

Change-Id: I71a1e8a247edfeb773db93906a70a6624de252b9
Reviewed-on: http://review.couchbase.org/113353
Tested-by: Build Bot <build@couchbase.com>
Well-Formed: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

5186902f25-Jul-2019 sduvuru <sduvuru@gmail.com>

MB-34862: couchstore- return of 0 by pwrite() is not handled

A return value of 0 from pwrite() would actually mean that
no bytes were written and the caller should retry pwrite.
Couc

MB-34862: couchstore- return of 0 by pwrite() is not handled

A return value of 0 from pwrite() would actually mean that
no bytes were written and the caller should retry pwrite.
Couchstore code does not retry and it leads to assuming an
entire block has been written out when it hasn't.

The fix is to retry in the case of 0 bytes getting written.

Unit tests for buffered and unbuffered cases are added.

Change-Id: I5f592d8a4084193e6bc9df88a1220a84854d113e
Reviewed-on: http://review.couchbase.org/112505
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

5f50fb5b06-Jun-2019 AnkitPrabhu <ankit.prabhu@couchbase.com>

MB-32313: Upgrade to v8 v7.1

- Use the new v8 API
- CBD-2634 use ICU from v8
- Copy icudtl.dat to bin and initialize ICU with it

Change-Id: I1f5618ca26162630233bdbc718

MB-32313: Upgrade to v8 v7.1

- Use the new v8 API
- CBD-2634 use ICU from v8
- Copy icudtl.dat to bin and initialize ICU with it

Change-Id: I1f5618ca26162630233bdbc71837a1b4f12c3301
Reviewed-on: http://review.couchbase.org/110297
Reviewed-by: Sriram Melkote <siri@couchbase.com>
Tested-by: Suraj Naik <suraj.naik@couchbase.com>

show more ...

1670751321-Mar-2019 Trond Norbye <trond.norbye@gmail.com>

Rename config.h to couchstore_config.h

We had a number of projects with their own "config.h" and
depending on the compiler search order you could end up
getting another one than the

Rename config.h to couchstore_config.h

We had a number of projects with their own "config.h" and
depending on the compiler search order you could end up
getting another one than the one you expected.

Change-Id: I92430a321bdc51f29f5993a856e1adc59b7de019
Reviewed-on: http://review.couchbase.org/106560
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

b0f5a94820-Mar-2019 Trond Norbye <trond.norbye@gmail.com>

Don't include config.h from headers

Depending on which module the file is included from we might
get a different version of config.h (kv used to pick up its
own)

Change-Id:

Don't include config.h from headers

Depending on which module the file is included from we might
get a different version of config.h (kv used to pick up its
own)

Change-Id: Id2123e7de363cedf7b4d9ef9be96ffe99175362e
Reviewed-on: http://review.couchbase.org/106511
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

4c48d57221-Feb-2019 Ben Huddleston <ben.huddleston@couchbase.com>

MB-30040: Update couchstore to use Folly

Change-Id: Id2eeb0205ff3c1ddf4269876366613b4860c6c2a
Reviewed-on: http://review.couchbase.org/105223
Reviewed-by: Trond Norbye <trond.norbye@

MB-30040: Update couchstore to use Folly

Change-Id: Id2eeb0205ff3c1ddf4269876366613b4860c6c2a
Reviewed-on: http://review.couchbase.org/105223
Reviewed-by: Trond Norbye <trond.norbye@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

f511f07715-Mar-2019 Dave Rigby <daver@couchbase.com>

MB-33373: couchstore_docinfos_by_id(RANGE): Don't process past end key

If couchstore_docinfos_by_id is used with the RANGES flag to perform a
range-scan, it can trigger an invalid memory

MB-33373: couchstore_docinfos_by_id(RANGE): Don't process past end key

If couchstore_docinfos_by_id is used with the RANGES flag to perform a
range-scan, it can trigger an invalid memory access due to incorrectly
accessing off the end of an array:

=================================================================
==38325==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000000ec0 at pc 0x00010a522b8c bp 0x7ffee5ac14d0 sp 0x7ffee5ac14c8
READ of size 8 at 0x602000000ec0 thread T0
#0 0x10a522b8b in btree_lookup_inner(couchfile_lookup_request*, unsigned long long, int, int) btree_read.cc:125
#1 0x10a52b88b in iterate_docinfos(...) couch_db.cc:1160
#2 0x10a52b229 in couchstore_docinfos_by_id couch_db.cc:1177
#3 0x10a18a7e2 in CouchstoreTest_MB33373_RangeScan_Test::TestBody() gtest_tests.cc:1378
#4 0x10a2269b6 in void testing::internal::HandleExceptionsInMethodIfSupported<...>(...) gtest.cc:2438
#5 0x10a226824 in testing::Test::Run() gtest.cc:2474
#6 0x10a2281b0 in testing::TestInfo::Run() gtest.cc:2656
#7 0x10a229441 in testing::TestCase::Run() gtest.cc:2774
#8 0x10a237011 in testing::internal::UnitTestImpl::RunAllTests() gtest.cc:4649
#9 0x10a2369b6 in bool testing::internal::HandleExceptionsInMethodIfSupported<...>(...) gtest.cc:2438
#10 0x10a236779 in testing::UnitTest::Run() gtest.cc:4257
#11 0x10a198839 in main gtest_tests.cc:1695
#12 0x7fff5eaf1014 in start (libdyld.dylib:x86_64+0x1014)

0x602000000ec0 is located 0 bytes to the right of 16-byte region [0x602000000eb0,0x602000000ec0)
allocated by thread T0 here:
#0 0x10de3cf53 in wrap_malloc (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x56f53)
#1 0x10dd8055e in cb_malloc cb_malloc.cc:60
#2 0x10a52b5aa in iterate_docinfos(...) couch_db.cc:1133
#3 0x10a52b229 in couchstore_docinfos_by_id couch_db.cc:1177
#4 0x10a18a7e2 in CouchstoreTest_MB33373_RangeScan_Test::TestBody() gtest_tests.cc:1378
#5 0x10a2269b6 in void testing::internal::HandleExceptionsInMethodIfSupported<...>(...) gtest.cc:2438
#6 0x10a226824 in testing::Test::Run() gtest.cc:2474
#7 0x10a2281b0 in testing::TestInfo::Run() gtest.cc:2656
#8 0x10a229441 in testing::TestCase::Run() gtest.cc:2774
#9 0x10a237011 in testing::internal::UnitTestImpl::RunAllTests() gtest.cc:4649
#10 0x10a2369b6 in bool testing::internal::HandleExceptionsInMethodIfSupported<...>(...) gtest.cc:2438
#11 0x10a236779 in testing::UnitTest::Run() gtest.cc:4257
#12 0x10a198839 in main gtest_tests.cc:1695
#13 0x7fff5eaf1014 in start (libdyld.dylib:x86_64+0x1014)

SUMMARY: AddressSanitizer: heap-buffer-overflow btree_read.cc:125 in btree_lookup_inner(couchfile_lookup_request*, unsigned long long, int, int)

Note: While this appears to affect versions of couchstore prior to
Mad-Hatter, couchstore_docinfos_by_id with the RANGES flag isn't used
by any currently shipping versions of KV-Engine or View engine, so it
shouldn't actually be exposed.

Fix by breaking the loop once we've gone past the last key.

Change-Id: I224c0d3cfc7038e46afba6a4ac2b9b82de4a0d7b
Reviewed-on: http://review.couchbase.org/106275
Reviewed-by: Trond Norbye <trond.norbye@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

5484b87e08-Mar-2019 Trond Norbye <trond.norbye@gmail.com>

Include what you use

Remove the catch-all include-file platform/platform.h

Change-Id: I40f36bc1bd3e10511ca6d7e2ee9aa620403ba0cf
Reviewed-on: http://review.couchbase.org/105908

Include what you use

Remove the catch-all include-file platform/platform.h

Change-Id: I40f36bc1bd3e10511ca6d7e2ee9aa620403ba0cf
Reviewed-on: http://review.couchbase.org/105908
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Daniel Owen <owend@couchbase.com>

show more ...

c042a1fe07-Mar-2019 Trond Norbye <trond.norbye@gmail.com>

Replace cb_mktemp with cb::io::mktemp

Thanks to clang-format the patch looks bigger than it is.

Change-Id: I4443740b4b2731582d40edb89ae3173615beb143
Reviewed-on: http://review.c

Replace cb_mktemp with cb::io::mktemp

Thanks to clang-format the patch looks bigger than it is.

Change-Id: I4443740b4b2731582d40edb89ae3173615beb143
Reviewed-on: http://review.couchbase.org/105832
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Paolo Cocchi <paolo.cocchi@couchbase.com>

show more ...

Revision tags: v5.1.3
2b2ba19329-Oct-2018 sduvuru <sduvuru@gmail.com>

MB-31835: Couchstore- Log messages when a corruption is detected

MB-31767: Couchstore- Log messages when a corruption is detected

When a corruption is detected, log the file offset,

MB-31835: Couchstore- Log messages when a corruption is detected

MB-31767: Couchstore- Log messages when a corruption is detected

When a corruption is detected, log the file offset, the checksum
and length fields. When an index node corruption was seen it
showed checksum and length fied corruption. Some corruptions
indicated the checksum was ok but snappy uncompression failed
to uncompress. If the length and checksum are 0, it is possible
for checksum validation to pass, but the snappy uncompress will
report an error. The new log will help verify if the length is 0.

When writing, snappy compression failures and unexpected increase
in size after compression are logged. The size increase is possible
but rare.

A new interface "couchstore_last_internal_error" which reports the last
internal error in case of a failure is returned.

There is a test case to inject an error to corrupt the length and
checksum to 0. This will not result in a checksum error but a
corruption error.

Change-Id: I2c3788510291b6eee7f43b1b22f0ed6eabe2359c
Reviewed-on: http://review.couchbase.org/101163
Reviewed-on: http://review.couchbase.org/101393
Well-Formed: Build Bot <build@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Tai <tai.qtt@gmail.com>

show more ...

db295fc329-Oct-2018 sduvuru <sduvuru@gmail.com>

MB-31767: Couchstore- Log messages when a corruption is detected

When a corruption is detected, log the file offset, the checksum
and length fields. When an index node corruption was see

MB-31767: Couchstore- Log messages when a corruption is detected

When a corruption is detected, log the file offset, the checksum
and length fields. When an index node corruption was seen it
showed checksum and length fied corruption. Some corruptions
indicated the checksum was ok but snappy uncompression failed
to uncompress. If the length and checksum are 0, it is possible
for checksum validation to pass, but the snappy uncompress will
report an error. The new log will help verify if the length is 0.

When writing, snappy compression failures and unexpected increase
in size after compression are logged. The size increase is possible
but rare.

A new interface "couchstore_last_internal_error" which reports the last
internal error in case of a failure is returned.

There is a test case to inject an error to corrupt the length and
checksum to 0. This will not result in a checksum error but a
corruption error.

Change-Id: I2c3788510291b6eee7f43b1b22f0ed6eabe2359c
Reviewed-on: http://review.couchbase.org/101163
Reviewed-by: Jim Walker <jim@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

Revision tags: v5.5.3
8dc7b5d008-Aug-2018 Jim Walker <jim@couchbase.com>

MB-30125: Provide old and new DocInfo for savedocs_and_callback

Update the API so both old and new DocInfo objects can be
accessed.

1) Removes the updated_how enum and requires

MB-30125: Provide old and new DocInfo for savedocs_and_callback

Update the API so both old and new DocInfo objects can be
accessed.

1) Removes the updated_how enum and requires users assume replace
when old and new are valid pointers

2) ADD - only newInfo is valid pointer

3) REPLACE - both old and new are valid pointers

Change-Id: I3ec64d1e7c6c4da977a8ac8fa05ebfd38fd7eb35
Reviewed-on: http://review.couchbase.org/97992
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Paolo Cocchi <paolo.cocchi@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

Revision tags: v5.1.2, v5.1.0
7106ad1b08-Aug-2017 Jim Walker <jim@couchbase.com>

MB-30125: Add a callback to discover if a key is added/replaced

Add a new couchstore_save_docs method which allows the caller to
specify a callback, the callback is invoked by the btree

MB-30125: Add a callback to discover if a key is added/replaced

Add a new couchstore_save_docs method which allows the caller to
specify a callback, the callback is invoked by the btree "builder"
with how the key was changed, i.e. is this an add or a replace.

Change-Id: I2cd9dbf3c0e8a7a1154f11e95d89461de3c42d60
Reviewed-on: http://review.couchbase.org/96051
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

3b4c35d713-Jun-2018 Jim Walker <jim@couchbase.com>

MB-29816: btree_lookup is skipping a key

In btree_lookup_inner when processing a KV node a while loop
exits too soon.

The loop condition looks at the distance into the node and

MB-29816: btree_lookup is skipping a key

In btree_lookup_inner when processing a KV node a while loop
exits too soon.

The loop condition looks at the distance into the node and
the index into the input keys.

while (bufpos < nodebuflen && current < end)

In the test we have many input keys, but 1 key in the
KV node.

The test has keys which after sorting means the first evaluated
key is not in the btree, so we will iterate once and revaluate
the loop condition. However bufpos = 50 and nodebuflen = 50
as we have read a key, however the second iteration doesn't
want to read a new key, it just wants to evaluate the second
input key. However bufpos < nodebuflen cause the loop to end
and we return with no callback for our key.

The commit updates the loop condition so that we keep going
whilst we have input keys and we explictly break from the
loop if an attempt to read a new key occurs and we have
no more buf space.

Change-Id: Ib93e11b5d5f03211f4a13301b64f20c7664b4ecd
Reviewed-on: http://review.couchbase.org/95669
Well-Formed: Build Bot <build@couchbase.com>
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Reviewed-by: Tim Bradgate <tim.bradgate@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

8de65b7e04-May-2018 Dave Rigby <daver@couchbase.com>

Avoid undefined behaviour for zero-length Document

As reported by UBSan, it is undefined behaviour to call memcpy() with
a null argument - in this case when Document has zero-length data

Avoid undefined behaviour for zero-length Document

As reported by UBSan, it is undefined behaviour to call memcpy() with
a null argument - in this case when Document has zero-length data.

Change-Id: Iabba97b79185eb4d5d4c1d8e73a421fbc1e19e47
Reviewed-on: http://review.couchbase.org/93749
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

6ef489f302-Feb-2018 harsha <harsha.hs.81@gmail.com>

MB-27758 Use aligned access for array of double values

Newer gcc optimizes double arrays by storing them into
wider(128 to 256 bits) registers like xmm/sse registers.
Memory accesses

MB-27758 Use aligned access for array of double values

Newer gcc optimizes double arrays by storing them into
wider(128 to 256 bits) registers like xmm/sse registers.
Memory accesses from/to xmm registers should be aligned
to double boundary.
This patch introduces a temporary double pointer for
aligned access and copies result back to unaligned pointer.

Change-Id: I4f38f8c19389f81a56cfe7f72f08095f17ee2f7e
Reviewed-on: http://review.couchbase.org/88776
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Jeelan Basha Poola <jeelan.poola@couchbase.com>

show more ...

21c7409724-Oct-2017 Eugen-Alexandru Virtan <eugen.virtan@couchbase.com>

MB-26047:[7-c] Switch over to using MicrosecondHistogram

Part of a cross repo change.

Change the code to use MicrosecondHistogram and related types.

Change-Id: Ic061e11a79b

MB-26047:[7-c] Switch over to using MicrosecondHistogram

Part of a cross repo change.

Change the code to use MicrosecondHistogram and related types.

Change-Id: Ic061e11a79b09663b980e173dc4191d46d5aa35d
Reviewed-on: http://review.couchbase.org/84745
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

Revision tags: v5.0.0
ea89a54301-Aug-2017 Dave Rigby <daver@couchbase.com>

MB-25509: Add periodic sync()ing after N bytes written

Add the ability for couchstore to automatically issue sync()
operations after every N bytes written.

When writing large am

MB-25509: Add periodic sync()ing after N bytes written

Add the ability for couchstore to automatically issue sync()
operations after every N bytes written.

When writing large amounts of data (e.g during compaction), read
latency can be adversely affected if a single sync() is made at the
end of writing all the data; as the IO subsystem has a large amount
of outstanding writes to flush to disk. By issuing periodic syncs
the affect on read latency can be signifcantly reduced.

This functionality is disabld by default, but may be enabled by
specifying a new flag to couchstore_open_db():
COUCHSTORE_OPEN_WITH_PERIODIC_SYNC.

Encoded as a power-of-2 KB value, ranging from 1KB .. 1TB (5 bits). A
value of N=0 specifies that automatic fsync is disabled.

Change-Id: If63f52680f2d0cc650e49773e970920611b97cc2
Reviewed-on: http://review.couchbase.org/81694
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

7a26ee4508-Jun-2017 James Harrison <00jamesh@gmail.com>

Fix clang static analysis bugs

Ubuntu 16.04 (clang-3.9, scan-build-3.9) identifies new bugs which
Ubuntu 12.04 (clang-3.6, scan-build-3.6) did not.

These are listed at
http:

Fix clang static analysis bugs

Ubuntu 16.04 (clang-3.9, scan-build-3.9) identifies new bugs which
Ubuntu 12.04 (clang-3.6, scan-build-3.6) did not.

These are listed at
http://cv.jenkins.couchbase.com/view/Commit%20Validation/view/couchstore/job/16.04-couchstore-clang_analyzer-master/6/clangScanBuildBugs/

Change-Id: I3ea1d2788665e66a6dc82c6c4bc5bb869310f4c0
Reviewed-on: http://review.couchbase.org/79258
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>
Reviewed-by: Will Gardner <willg@rdner.io>

show more ...

12345678910