History log of /5.5.2/kv_engine/engines/ep/src/callbacks.h (Results 1 - 25 of 48)
Revision (<<< Hide revision tags) (Show revision tags >>>) Date Author Comments
Revision tags: v7.0.2, v6.6.3, v7.0.1, v7.0.0, v6.6.2, v6.5.2, v6.6.1, v6.0.5, v6.6.0, v6.5.1, v6.0.4, v6.5.0, v6.0.3, v5.5.4, v5.5.5, v5.5.6, v6.0.1, v5.5.3, v6.0.0, v5.1.3, v5.5.2, v5.5.1, v5.1.2, v5.1.1, v5.0.1
# dea18910 25-Oct-2017 Jim Walker <jim@couchbase.com>

MB-26455: Allow correct processing of 'old' keys and separator changes

The collection's separator can only be changed when 0 collections
exist. However nothing stops the separator from c

MB-26455: Allow correct processing of 'old' keys and separator changes

The collection's separator can only be changed when 0 collections
exist. However nothing stops the separator from changing if there are
deleted collections (and their keys) in play.

Prior to this commit each separator change resulted in a single system
event being mutated, that event had a static key. Thus a VB could have
the following sequence of keys/events.

1 fruit::1
2 fruit::2
<fruit collection is deleted>
<separator is changed from :: to - >
<fruit collection is recreated>
6 fruit-1
7 fruit-2
<fruit collection is deleted>
<separator is changed from - to # >
9 $collections_separator (the Item value contains the new separator)
10 $collections::fruit (fruit recreated)
11 fruit#1
12 fruit#2

In this sequence, some of the changes are lost historically because a
static key is used for the separator change. Between seqno 2 and 6 the
separator changed from :: to -, but the separator change system event
is currently at seqno 9 with # as its value.

With this setup a problem exists if we were to process the historical
data e.g. whilst backfilling for DCP or performing a compaction
collection erase. The problem is that when we go back to seqno 1 and
2, we have no knowledge of the separator for those items, all we have
is the current # separator. We cannot determine that fruit::1 is a
fruit collection key.

This commit addresses this problem by making each separator change
generate a unique key. The key itself will encode the new separator,
and because the key is unique it will reside at the correct point in
history for each separator change.

The unique key format will be:

$collections_separator:<seqno>:<new_separator>

With this change the above sequence now looks as:

1 fruit::1
2 fruit::2
<fruit collection is deleted>
4 $collections_separator:3:- (change separator to -)
<fruit collection is recreated>
6 fruit-1
7 fruit-2
<fruit collection is deleted>
9 $collections_separator:8:# (change separator to #)
10 $collections::fruit (fruit recreated)
11 fruit#1
12 fruit#2

Now the code which looks at the historical data (e.g. backfill) will
encounter these separator change keys before it encounters collection
keys using that separator. Backfill and collections-erase can just
maintain a current separator and can now correctly split keys to
discover the collection they belong to. The collections eraser and
KVStore scanning code now include a collections context which has data
and associated code for doing this tracking.

A final piece of the commit is the garbage collection of these unique
keys. i.e. if each separator change puts a unique key into the seqno
index, how can we clean these away when they're no longer needed (i.e.
all fruit:: keys purged)?

Whilst the eraser runs it tracks all 'separator change' keys, because
a separator change can only happen when 0 collections exist, it can
assume that all but the latest separator change key are redundant once
the erase has completed. This list of keys are simply deleted in the
normal way by pushing a deleted Item into the checkpoint once
compaction is complete.

Change-Id: I4b38b04ed72d6b39ceded4a860c15260fd394118
Reviewed-on: http://review.couchbase.org/84801
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...


# 61d9ac92 04-Dec-2017 Dave Rigby <daver@couchbase.com>

MB-26021: Add StatusCallback<> subclass

Refactor the Callback<> template so the statusCode is moved into a
seperate subclass - StatusCallback.

This shrinks the size of Callback<

MB-26021: Add StatusCallback<> subclass

Refactor the Callback<> template so the statusCode is moved into a
seperate subclass - StatusCallback.

This shrinks the size of Callback<> (and any subclasses) which don't
need the status code by at least 4 bytes (can be more due to
alignment).

Specifically, this reduces PersistenceCallback from 48 to 32 bytes.

Change-Id: I1866d0cc618af55a3d7c5809fcd768504f7204c7
Reviewed-on: http://review.couchbase.org/86323
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...


Revision tags: v5.1.0, v5.0.0
# 5e08ba29 24-Aug-2017 Dave Rigby <daver@couchbase.com>

Move callbacks impl to callbacks.cc

Move implementation details of classes in Callbacks to a new file
callbacks.cc. Similar to previous changes, this reduces the number of
files whic

Move callbacks impl to callbacks.cc

Move implementation details of classes in Callbacks to a new file
callbacks.cc. Similar to previous changes, this reduces the number of
files which need to include the implementation details of callbacks.h;
specifically item.h which indirectly pulls in a number of header
files.

Fix up missing / incorrect #includes as a result of this.

Change-Id: Icc565894b746e688d0c59a532e7ff98e22683959
Reviewed-on: http://review.couchbase.org/82686
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...


# cdfa9ea6 09-Aug-2017 Jim Walker <jim@couchbase.com>

Refactor: CacheLookup doesn't need to copy the DocKey

Replace StoredDocKey with DocKey so that it doesn't take a copy of the
key.

Change-Id: Ibf310e849549153f94254e45c6907497803

Refactor: CacheLookup doesn't need to copy the DocKey

Replace StoredDocKey with DocKey so that it doesn't take a copy of the
key.

Change-Id: Ibf310e849549153f94254e45c690749780309800
Reviewed-on: http://review.couchbase.org/82115
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Oliver Downard <oliver.downard@couchbase.com>
Reviewed-by: Will Gardner <willg@rdner.io>

show more ...


# b7acc401 18-May-2017 Dave Rigby <daver@couchbase.com>

Move VBucketBGFetchItem to own header file

Move VBucketBGFetchItem and assocated types from kvstore.h to a new
header file vbucket_bgfetch_item.h. This removes the need for files
whi

Move VBucketBGFetchItem to own header file

Move VBucketBGFetchItem and assocated types from kvstore.h to a new
header file vbucket_bgfetch_item.h. This removes the need for files
which don't need to use bgfetch types from including kvstore.h

Change-Id: I8af795f12def4ac07c11ca92c36a2bf23131414f
Reviewed-on: http://review.couchbase.org/78654
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...


# 6d4fe478 22-May-2017 olivermd <oliver.downard@couchbase.com>

MB-23377: Alter Item* to unique_ptr in GetValue

This patch adds a mechanism to transfer ownership of the Item out of the
GetValue which a number of call sites need to make use of.

MB-23377: Alter Item* to unique_ptr in GetValue

This patch adds a mechanism to transfer ownership of the Item out of the
GetValue which a number of call sites need to make use of.

Change-Id: Id06ea6af6b51bdb9018bab11f56f52c0f8adc999
Reviewed-on: http://review.couchbase.org/78601
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...


# b11f569e 24-May-2017 olivermd <oliver.downard@couchbase.com>

MB-23377: Raise GetValue out of callback

This commit changes the kvstore get and getheader interfaces to pass the
value directly via a GetValue rather than a callback which saves a
G

MB-23377: Raise GetValue out of callback

This commit changes the kvstore get and getheader interfaces to pass the
value directly via a GetValue rather than a callback which saves a
GetValue. This can be done because all the work is done synchronously.
It also simplifies the interface to allow for additional changes to
GetValue in later patches.

Change-Id: I1c8ccc4e8af39d533ae2a92a1a777d6899b717ea
Reviewed-on: http://review.couchbase.org/78600
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...


# ef22f9b0 25-May-2017 Dave Rigby <daver@couchbase.com>

Move ep-engine to engines/ep


# 9c7b2ace 11-May-2017 Manu Dhundi <manu@couchbase.com>

Use 'const' for the member 'Item* value' in class 'GetValue'

Class GetValue contains a pointer to an item which is only used to
transfer out the item contained from one layer to another.

Use 'const' for the member 'Item* value' in class 'GetValue'

Class GetValue contains a pointer to an item which is only used to
transfer out the item contained from one layer to another. It does not/
should not change the item it contains.

Hence it would be best to explicitly use const so that the item
contained is not modified by mistake in the future.

Further, it could be a future task to use std::unique_ptr<Item> in
GetValue to explicitly transfer the ownership of the item that
gets handed over from one layer to another.

Change-Id: Ic0a880f3070e299e8ff1628f3ac329aaa1a1b0cb
Reviewed-on: http://review.couchbase.org/78040
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...


Revision tags: v4.6.2_ep, v4.6.2_mc, v4.6.1_ep
# 606a8c7c 13-Jan-2017 Manu Dhundi <manu@couchbase.com>

[Refactor]: Move completeBGFetchForSingleItem() go from KVBucket to VBucket

Change-Id: Ib474a3773fa1f0cf80b14eacfa6881a55839f999
Reviewed-on: http://review.couchbase.org/71572
Tested

[Refactor]: Move completeBGFetchForSingleItem() go from KVBucket to VBucket

Change-Id: Ib474a3773fa1f0cf80b14eacfa6881a55839f999
Reviewed-on: http://review.couchbase.org/71572
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Manu Dhundi <manu@couchbase.com>

show more ...


Revision tags: v4.6.0_ep
# f734f13f 05-Dec-2016 Jim Walker <jim@couchbase.com>

MB-21916: Make use of StoredDocKey

Where std::string stored a key, now we use StoredDocKey
When we passed std::string& through interfaces we now
pass DocKey as much as possible, thus

MB-21916: Make use of StoredDocKey

Where std::string stored a key, now we use StoredDocKey
When we passed std::string& through interfaces we now
pass DocKey as much as possible, thus delaying the
heap alloc and memcpy to the places we really need
to store a key.

This patch does not store the namespace in StoredValue
This patch does not store the namespace in couchstore/fdb

Thus any keys created in say Collections namespace will
not work. This is correct as this patch does not make
any assertions about the support of non-DefaultCollection

Change-Id: Ibc7f59183e59f43d55fad5e582232e2891231179
Reviewed-on: http://review.couchbase.org/70698
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: buildbot <build@couchbase.com>

show more ...


Revision tags: v4.5.1-MP1_mc, v4.6.0-DP_mc, v4.6.0-DP_ep, v4.5.1-MP1_ep, v4.1.2-MP2_mc, v4.5.1_mc, v4.6.0_mc, v4.1.2-MP1_ep, v3.1.6_ep, v4.5.0_mc, v4.5.0_ep, v4.1.1_ep, v3.1.5_ep, v4.1.1_mc, v3.1.4_ep, v3.1.4_mc, v3.1.5_mc, v3.1.3_ep
# b9334207 30-Nov-2015 Dave Rigby <daver@couchbase.com>

Mutex modernization [2/2]: Use std::condition_variable for SyncObject

Change Mutex to a typedef to std::mutex, and use
std::condition_variable for the implementation of SyncObject.

Mutex modernization [2/2]: Use std::condition_variable for SyncObject

Change Mutex to a typedef to std::mutex, and use
std::condition_variable for the implementation of SyncObject.

Note these two changes are mutually dependent hence being done in the
same patch.

Change-Id: Ife74822e9c4e04efefff446e964a952b672a1f91
Reviewed-on: http://review.couchbase.org/57328
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...


# df3730be 30-Nov-2015 Dave Rigby <daver@couchbase.com>

Mutex modernization [1/2]: Replace with std::mutex

Update the API of Mutex to match that of std::mutex, and SyncObject to
match that of std::condition_variable in preparation for replaci

Mutex modernization [1/2]: Replace with std::mutex

Update the API of Mutex to match that of std::mutex, and SyncObject to
match that of std::condition_variable in preparation for replacing
Mutex and SyncObject with the C++11 standard library equivilents.

Change-Id: I5625d980b11144f681f7e717df87c8b5f323dc7c
Reviewed-on: http://review.couchbase.org/57327
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: buildbot <build@couchbase.com>

show more ...


Revision tags: v4.1.0_ep
# e7754280 05-Nov-2015 Dave Rigby <daver@couchbase.com>

Simplify common.h: Remove unnecessary headers [3/3]

Remove all #includes which arn't actually needed by the file including
them (aka "include what you use").

Reduces build time

Simplify common.h: Remove unnecessary headers [3/3]

Remove all #includes which arn't actually needed by the file including
them (aka "include what you use").

Reduces build time down from before this patch series from:

31.82 real 222.13 user 16.09 sys

down to:

26.95 real 188.54 user 13.29 sys

giving a 15% speedup in (user+sys).

(Built using `make clean && CCACHE_DISABLE=1 time make -j8`)

Change-Id: Ia47f08487d20c94167d93de8dd6590f7703dd898
Reviewed-on: http://review.couchbase.org/56705
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Manu Dhundi <manu@couchbase.com>
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...


# 44137813 05-Nov-2015 Dave Rigby <daver@couchbase.com>

Simplify common.h: add utility.h [2/3]

Create a new utility.h header file and move some of the functions /
macros which virtually everyone uses to there. utility.h has
significantly

Simplify common.h: add utility.h [2/3]

Create a new utility.h header file and move some of the functions /
macros which virtually everyone uses to there. utility.h has
significantly fewer dependancies, and such is much cheaper to include
than common.h

Update users of these functions / macros (DISALLOW_COPY_ON_ASSIGN, LOG
etc) to use utility.h instead of common.h

Change-Id: I3f93435c82950bd2a20b416af2b2e5011a931e04
Reviewed-on: http://review.couchbase.org/56704
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: abhinav dangeti <abhinav@couchbase.com>
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...


Revision tags: v3.1.2_ep, v4.1.0_mc
# 9b87636e 25-Sep-2015 Dave Rigby <daver@couchbase.com>

MB-16056: Use exceptions instead of cb_assert() [10]

Tenth set of changes to migrate away from cb_assert() and use
exceptions where applicable for invalid input / logic errors.

MB-16056: Use exceptions instead of cb_assert() [10]

Tenth set of changes to migrate away from cb_assert() and use
exceptions where applicable for invalid input / logic errors.

This is a "tidy-up" patch - fixes the remaining 13 or so cb_asserts
which I skipped over in the first pass as they had some form of subtly
to them. Highlights of interesting asserts:

- BgFetcher::start/stop, ConnNotifier::start/stop : These were
asserting on taskId being >0. However taskId is set in the ExTask
constructor using a atomic increment so cannot see any need for this
assert.

- FailoverTable : These were postcondition checks that we had at least
1 failover entry after various operations. However we can change
them into precondiion checks on the input, and throw an exception if
the proposed new value is not valid (i.e. turn it into a
precondition check).

Change-Id: Iac6fff0b42e55e62fa6d0eeca446f8bfb094f3aa
Reviewed-on: http://review.couchbase.org/55608
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...


Revision tags: v3.1.2_mc, v3.1.1_mc, v3.1.1_ep, v4.0.0_ep, v4.0.0_mc, v3.1.0_ep, v3.1.0_mc, v3.1.6_mc
# 1d770846 16-Mar-2015 Mike Wiederhold <mikewied06@gmail.com>

Make callback template varadic

Change-Id: I7240db6c9040a03c95c7ffd4f04fba86377dce55
Reviewed-on: http://review.couchbase.org/48338
Reviewed-by: Manu Dhundi <manu@couchbase.com>
R

Make callback template varadic

Change-Id: I7240db6c9040a03c95c7ffd4f04fba86377dce55
Reviewed-on: http://review.couchbase.org/48338
Reviewed-by: Manu Dhundi <manu@couchbase.com>
Reviewed-by: abhinav dangeti <abhinav@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>

show more ...


Revision tags: v3.0.2-MP2_mc, v3.0.2_ep, v3.0.2_mc
# 4dc2c64c 15-Nov-2014 abhinavdangeti <abhinav@couchbase.com>

Removed unused class in callback.h

Change-Id: I2bd7423d12fd69d8ee1a36fafd8f77ec99370992
Reviewed-on: http://review.couchbase.org/43273
Reviewed-by: Michael Wiederhold <mike@couchbase

Removed unused class in callback.h

Change-Id: I2bd7423d12fd69d8ee1a36fafd8f77ec99370992
Reviewed-on: http://review.couchbase.org/43273
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: abhinav dangeti <abhinav@couchbase.com>

show more ...


Revision tags: v3.0.1_mc, v3.0.0_ep, v3.0.0-beta3_mc, v3.0.0_mc, v3.0.0-beta2_mc, v3.0.0-beta1_mc
# 4664d274 19-May-2014 Mike Wiederhold <mike@couchbase.com>

MB-10941: When backfill is neccessary always do a full backfill

When sending backfill we need to send the entire thing as a single
snapshot in order to avoid potentially missing de-dupli

MB-10941: When backfill is neccessary always do a full backfill

When sending backfill we need to send the entire thing as a single
snapshot in order to avoid potentially missing de-duplicated items
when then checkpoint ends.

Change-Id: If898be3414bb51b4248dad03176255e5934ac6a2
Reviewed-on: http://review.couchbase.org/37295
Tested-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Volker Mische <volker.mische@gmail.com>
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...


# c649b2d9 17-Mar-2014 Trond Norbye <trond.norbye@gmail.com>

MB-10476: Use cb_assert instead of assert

If compiled with -DNDEBUG assert is an empty macro

Change-Id: I266ddf8a8fe3f1a388a9d8ce3d21f28614c35315
Reviewed-on: http://review.couc

MB-10476: Use cb_assert instead of assert

If compiled with -DNDEBUG assert is an empty macro

Change-Id: I266ddf8a8fe3f1a388a9d8ce3d21f28614c35315
Reviewed-on: http://review.couchbase.org/34579
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...


Revision tags: v2.5.1_ep, v2.5.1-MP1-A_mc, v2.5.0_mc, v2.5.0-dp1_mc
# 9a8679e2 17-Oct-2013 Chiyoung Seo <chiyoung.seo@gmail.com>

CBD-1034 Support the full eviction as an optional eviction policy

When an item is evicted from the cache, its value is only
evicted from memory by default. To further reduce the memory

CBD-1034 Support the full eviction as an optional eviction policy

When an item is evicted from the cache, its value is only
evicted from memory by default. To further reduce the memory
overhead, this change supports the full eviction as an additional
option, which removes an item's key, metadata, and value from the
cache. The eviction policy can be chosen when a bucket is
instantiated.

If the full eviction option is chosen, this results in affecting
lots of engine APIs and also causes more disk IO overhead.
For example, CAS for a non-resident item will require a backgroud
fetch because it should know the item's CAS value.

Change-Id: I2edbe2b12527ef1c1fb31992cb04afbb7c078dac
Reviewed-on: http://review.couchbase.org/29155
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...


# 3ea83dc4 16-Sep-2013 Mike Wiederhold <mike@couchbase.com>

MB-9107: During Backfill don't read resident values off disk

We remove in-memroy backfilling in a previous commit and this
can cause backfilling to be slow if most of the items are in

MB-9107: During Backfill don't read resident values off disk

We remove in-memroy backfilling in a previous commit and this
can cause backfilling to be slow if most of the items are in
memory. This change allows us to skip reading values off of disk
if the values already exist in memory.

Change-Id: I51aad158e9bab15cc8fb78f3c140c1d87c224101
Reviewed-on: http://review.couchbase.org/29115
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>

show more ...


Revision tags: v2.2.0_mc, 2.1.1r_ep, 2.1.0r_ep, 2.0.2r_ep, 2.0.2r_mc, 2.0.1-macosx_ep, 2.0.1-linux_ep
# 66eb94d0 17-Jan-2013 Mike Wiederhold <mike@couchbase.com>

MB-7728: Add copyright headers to all files

This is something that our cpplint tool turns on and it makes
our product look a little bit more profesional. I have also
enabled the tool

MB-7728: Add copyright headers to all files

This is something that our cpplint tool turns on and it makes
our product look a little bit more profesional. I have also
enabled the tool to begin checking for these headers so it will
start complaining if we add a new file without a copyright header.

Change-Id: Ie956fdf7027bf56d9665ee455acd3c109fcacc76
Reviewed-on: http://review.couchbase.org/24560
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Reviewed-by: Jin Lim <jin@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>

show more ...


Revision tags: 2.0.1-linux_mc, 2.0.0-couchbase_ep
# 244c0146 26-Nov-2012 Mike Wiederhold <mike@couchbase.com>

MB-7461: Alphabetize includes to conform with c++ standards

Change-Id: I71333dedada6530713b90184857bab9ac49d13df
Reviewed-on: http://review.couchbase.org/23515
Reviewed-by: Jin Lim <

MB-7461: Alphabetize includes to conform with c++ standards

Change-Id: I71333dedada6530713b90184857bab9ac49d13df
Reviewed-on: http://review.couchbase.org/23515
Reviewed-by: Jin Lim <jin@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>

show more ...


# 8f48f64b 23-Nov-2012 Mike Wiederhold <mike@couchbase.com>

MB-7426: Fix all header defines to conform with cpplint

Change-Id: I8253b640bc49ee8a0fabbb39ece73c2efdb906d5
Reviewed-on: http://review.couchbase.org/23510
Reviewed-by: Michael Wiede

MB-7426: Fix all header defines to conform with cpplint

Change-Id: I8253b640bc49ee8a0fabbb39ece73c2efdb906d5
Reviewed-on: http://review.couchbase.org/23510
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>

show more ...


12