MB-29675: Optimize collections lookups when disabled

While collections code is present in Vulcan, it is not enabled in
production configurations - only the default collection is present.

MB-29675: Optimize collections lookups when disabled

While collections code is present in Vulcan, it is not enabled in
production configurations - only the default collection is present.

As such, optimize a number of functions which show up in 'linux perf'
profiles:

- Use std::string[] instead of string::at() do avoid the runtime size
check (and setup for throwing an exception) in StoredDocKey.

- Split Collections::VB::Filter::checkAndUpdate() into a hot and slow
path, and inline the hot path (This is called for every outgoing DCP
mutation).

- Add an explicit case for DefaultCollection in
Collections::DocKey::make() - this avoids an unnecessary call to
findCollection which otherwise needs to scan the document key.

Change-Id: I4c4eeb6ab26dc616728f4dc89ac1fd4243c21e98
Reviewed-on: http://review.couchbase.org/94214
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

MB-26455: Allow correct processing of 'old' keys and separator changes

The collection's separator can only be changed when 0 collections
exist. However nothing stops the separator from c

MB-26455: Allow correct processing of 'old' keys and separator changes

The collection's separator can only be changed when 0 collections
exist. However nothing stops the separator from changing if there are
deleted collections (and their keys) in play.

Prior to this commit each separator change resulted in a single system
event being mutated, that event had a static key. Thus a VB could have
the following sequence of keys/events.

1 fruit::1
2 fruit::2
<fruit collection is deleted>
<separator is changed from :: to - >
<fruit collection is recreated>
6 fruit-1
7 fruit-2
<fruit collection is deleted>
<separator is changed from - to # >
9 $collections_separator (the Item value contains the new separator)
10 $collections::fruit (fruit recreated)
11 fruit#1
12 fruit#2

In this sequence, some of the changes are lost historically because a
static key is used for the separator change. Between seqno 2 and 6 the
separator changed from :: to -, but the separator change system event
is currently at seqno 9 with # as its value.

With this setup a problem exists if we were to process the historical
data e.g. whilst backfilling for DCP or performing a compaction
collection erase. The problem is that when we go back to seqno 1 and
2, we have no knowledge of the separator for those items, all we have
is the current # separator. We cannot determine that fruit::1 is a
fruit collection key.

This commit addresses this problem by making each separator change
generate a unique key. The key itself will encode the new separator,
and because the key is unique it will reside at the correct point in
history for each separator change.

The unique key format will be:

$collections_separator:<seqno>:<new_separator>

With this change the above sequence now looks as:

1 fruit::1
2 fruit::2
<fruit collection is deleted>
4 $collections_separator:3:- (change separator to -)
<fruit collection is recreated>
6 fruit-1
7 fruit-2
<fruit collection is deleted>
9 $collections_separator:8:# (change separator to #)
10 $collections::fruit (fruit recreated)
11 fruit#1
12 fruit#2

Now the code which looks at the historical data (e.g. backfill) will
encounter these separator change keys before it encounters collection
keys using that separator. Backfill and collections-erase can just
maintain a current separator and can now correctly split keys to
discover the collection they belong to. The collections eraser and
KVStore scanning code now include a collections context which has data
and associated code for doing this tracking.

A final piece of the commit is the garbage collection of these unique
keys. i.e. if each separator change puts a unique key into the seqno
index, how can we clean these away when they're no longer needed (i.e.
all fruit:: keys purged)?

Whilst the eraser runs it tracks all 'separator change' keys, because
a separator change can only happen when 0 collections exist, it can
assume that all but the latest separator change key are redundant once
the erase has completed. This list of keys are simply deleted in the
normal way by pushing a deleted Item into the checkpoint once
compaction is complete.

Change-Id: I4b38b04ed72d6b39ceded4a860c15260fd394118
Reviewed-on: http://review.couchbase.org/84801
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

MB-27661 [10/n]: Fix MSVC warnings - C4267

This patch addresses the following generated warnings:

C4267 - var : conversion from 'size_t' to 'type', possible loss of data

MB-27661 [10/n]: Fix MSVC warnings - C4267

This patch addresses the following generated warnings:

C4267 - var : conversion from 'size_t' to 'type', possible loss of data
The compiler detected a conversion from size_t to a smaller type.

Since there are a lot of these warnings, use multiple commits to sort
them out to make reviewing easier.

Change-Id: I223e3880196558d4df926f01c60e05f5062e56e1
Reviewed-on: http://review.couchbase.org/90539
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

MB-16181: Fix Collections::DocKey::make when separator is a dollar

The system keys currently have a $ prefix which is breaking the
findCollection methods when the separator is configured

MB-16181: Fix Collections::DocKey::make when separator is a dollar

The system keys currently have a $ prefix which is breaking the
findCollection methods when the separator is configured as $. Only
namespace::System keys are affected as if a user wrote a key with
a $ prefix and $ separator, they're saying the collection is empty
(and not actually legal when collections is on).

Change-Id: I949f435cfd79e6b71a3e280984698951ab9de540
Reviewed-on: http://review.couchbase.org/84594
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

MB-16181: Rename and update doesKeyContainDeletingCollection

Rename this function isLogicallyDeleted to test if a key belongs to
a deleted collection.

Update the function to con

MB-16181: Rename and update doesKeyContainDeletingCollection

Rename this function isLogicallyDeleted to test if a key belongs to
a deleted collection.

Update the function to consider System keys because a Collection
event key may also be considered deleted by future callers.

Change-Id: I76e07d26e9236d4a5e48be2b7e3c0f41ba0d7221
Reviewed-on: http://review.couchbase.org/83631
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

Move ep-engine to engines/ep

MB-16181: Add Collections::DocKey

An extension to the DocKey class that adds the number of bytes
which maybe a collection, i.e. how many bytes are before the prefix.

E.g. beer:b

MB-16181: Add Collections::DocKey

An extension to the DocKey class that adds the number of bytes
which maybe a collection, i.e. how many bytes are before the prefix.

E.g. beer:budweiser
- len = 15
- collection len = 4 (if separator is : then beer is the collection)

Change-Id: Ia877273bef18b92b2c53c161bb4b9c465eac834b
Reviewed-on: http://review.couchbase.org/72458
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...