MB-36948: Track and store maxVisibleEndSeqno in Checkpoints

Add a maxVisibleEndSeqno to Checkpoint and then maintain that
value for active/replica.

Replica vbuckets set this val

MB-36948: Track and store maxVisibleEndSeqno in Checkpoints

Add a maxVisibleEndSeqno to Checkpoint and then maintain that
value for active/replica.

Replica vbuckets set this value on receipt of a snapshot-marker.
They will either call createSnapshot or updateCurrentSnapshot both
methods now require the visible snap-end value. The value then
remains in the Checkpoint object and is now visible to subsequent
DCP producers (producing the replica) or readable from
CheckpointManager (which will be wired into GetAllVBSeqnos).

Active VBuckets track/set the maxVisibleEndSeqno in a way which
mirrors the visible seqno (which is CheckpointManager::lastBySeqno).
As items are queued (queueDirty) lastBySeqno is increased and
the current Checkpoint will have its end seqno adjusted.
The same logic applies to a new member, maxVisibleSeqno
which only increments for visible items.

Note on tests using "createSnapshot": to avoid a default parameter
the createSnapshot method has been updated in many unit tests and
in some places what seems like the correct maxVisibleSeqno is
added and sometime later tested. Many tests though are just using
zero, as for many tests the maxVisibleSeqno isn't of interest.

Change-Id: I0e2a6aafeeef3a334766e8bdc868471462510d85
Reviewed-on: http://review.couchbase.org/118637
Well-Formed: Build Bot <build@couchbase.com>
Reviewed-by: Paolo Cocchi <paolo.cocchi@couchbase.com>
Tested-by: Jim Walker <jim@couchbase.com>

show more ...

MB-36971: Never skip op::checkpoint_start in CM::getItemsForCursor

In MB-35245 we have introduced a bug where we may skip checkpoint_start
items in CheckpointManager::getItemsForCursor()

MB-36971: Never skip op::checkpoint_start in CM::getItemsForCursor

In MB-35245 we have introduced a bug where we may skip checkpoint_start
items in CheckpointManager::getItemsForCursor().
The side effect is that the MARKER_FLAG_CHK may not be set when sending
a SnapshotMarker to Replica. That may cause a Replica "merging"
snapshots of different types (Disk/Memory) into the same open
checkpoint.

Change-Id: Ie1ed7edef1914f365786ea4a70cccac1038f8350
Reviewed-on: http://review.couchbase.org/118409
Well-Formed: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>
Reviewed-by: Ben Huddleston <ben.huddleston@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

Rename connCursors to cursors

CheckpointManager::connCursors also includes the persistence cursor
(if it exists) so rename the field to cursors.

Change-Id: Ia86eaa3ab7eade9e019e

Rename connCursors to cursors

CheckpointManager::connCursors also includes the persistence cursor
(if it exists) so rename the field to cursors.

Change-Id: Ia86eaa3ab7eade9e019ea06a43d1aa2cda71ed04
Reviewed-on: http://review.couchbase.org/116376
Reviewed-by: James Harrison <james.harrison@couchbase.com>
Tested-by: Dave Rigby <daver@couchbase.com>

show more ...

MB-36119: Add debug info/comments to expel code

1) Add comments to expel code to explain how ordering cursors by
seqno is actually valid.

2) Replace the pre-condition with an ex

MB-36119: Add debug info/comments to expel code

1) Add comments to expel code to explain how ordering cursors by
seqno is actually valid.

2) Replace the pre-condition with an exception to get useful debug
information should it ever fire again.

3) Order by checkpoint ID before seqno as we then don't have to worry
about the dummy item (which does not bump the seqno) when we create
a new checkpoint.

4) Make incrCursor protected as unintended use could cause cursors to
end up in invalid positions.

Change-Id: I74f5e749a0d90afe9402942d1eb5445173146288
Reviewed-on: http://review.couchbase.org/115426
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

MB-36014: Fix tracking number of cursors in checkpoint for replica

Prior to this change, if any cursors other than the persistence cursor
were created /before/ receiving a snapshot from

MB-36014: Fix tracking number of cursors in checkpoint for replica

Prior to this change, if any cursors other than the persistence cursor
were created /before/ receiving a snapshot from the active, the cursors
would be reset to the first checkpoint, and would increment the number
of cursors in that checkpoint again.

The major result of this was that affected checkpoints, and all
following checkpoints, would never be removed as they would forever be
seen as "referenced" because of the erroneous second increment.

Fixed by only incrementing if resetting the cursor to a different
checkpoint.

Change-Id: Iaad476cf2ebeca53597856022848ad10a8811276
Reviewed-on: http://review.couchbase.org/114960
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

MB-35654: Make expelling only operate on the oldest checkpoint

Currently, expelling will operate on the oldest checkpoint _containing
cursors_, which may not be the _oldest checkpoint_.

MB-35654: Make expelling only operate on the oldest checkpoint

Currently, expelling will operate on the oldest checkpoint _containing
cursors_, which may not be the _oldest checkpoint_.

Expelling from a checkpoint which is _not_ the oldest is unsafe as it
leaves holes in the items a cursor will read. This patch prevents this.

If the oldest checkpoint is unref'd (i.e., all cursors are in other
later checkpoints) it can be deleted as a whole; expelling from such a
checkpoint would be inefficient. Given this, trying to expel in this
state should be a noop - the oldest checkpoint will be dropped if it
remains unreferenced, and no other checkpoints are valid candidates.

Change-Id: Id72a613930909103fabd4265d63a5f500567d627
Reviewed-on: http://review.couchbase.org/113706
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Daniel Owen <owend@couchbase.com>

show more ...

MB-32591: Trigger expelling to occur before cursor dropping

Currently checkpoint expelling is applied every time a vbucket is
visited via CheckpointVisitor::visitBucket. This aggressive

MB-32591: Trigger expelling to occur before cursor dropping

Currently checkpoint expelling is applied every time a vbucket is
visited via CheckpointVisitor::visitBucket. This aggressive approach
means that items can be expelled from a checkpoint even if memory usage
is below the low watermark. If a new DCP producer DCP stream creates a
cursor to send items to a consumer there is an increased chance that
items will have to be retrieved via backfilling, where previously they
would have been sent directly from the checkpoint. Consequently
over expelling could unnecessarily degrade performance.

Therefore this patch changes checkpoint expelling to be triggered only
when ClosedUnrefCheckpointRemoverTask is run and the existing criteria
for cursor droping is satisified.

We first apply checkpoint expelling, then if the memory target is still
not met we perform cursor dropping. The reason for prioritising
expelling over cursor dropping is that it is considered less disruptive.

Change-Id: I92cbf31390e1cd88f5f3c64cca05fa348441ec5b
Reviewed-on: http://review.couchbase.org/108904
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

MB-32591: Estimate memory recovered from expelling

We want to be able to trigger expelling using the same mechanism that
is currently used for cursor dropping.

For cursor droppi

MB-32591: Estimate memory recovered from expelling

We want to be able to trigger expelling using the same mechanism that
is currently used for cursor dropping.

For cursor dropping we calculate how much memory we want to recover.
We then iterate through each vbucket performing cursor dropping
and removeClosedUnrefCheckpoints (which actually frees the memory).
Once we have freed the required amount of memory we break out of the
vbucket loop.

Given we want to apply a similiar mechanism we need to calculate the
amount of memory recovered by expelling. It is important to note that
the value returned is an optimistic estimate (similar to the value
returned by removeClosedUnrefCheckpoints), as it assumes that each
queued_item is not referenced by anyone else (e.g. a DCP stream) and
therefore its reference count will drop to zero on exiting the function
allowing the memory to be freed.

Change-Id: I997576374dbc59923f0d08f7ca93e172a1af074a
Reviewed-on: http://review.couchbase.org/108453
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

Refactor: Move test code out of CheckpointManager

The following methods in CheckpointManager are only used for testing
purposes.

- getNumOfCursors
- getNumCheckpoint
- n

Refactor: Move test code out of CheckpointManager

The following methods in CheckpointManager are only used for testing
purposes.

- getNumOfCursors
- getNumCheckpoint
- nextItem

This patch moves them out into a newly created MockCheckpointManager
class that is only used for testing purposes.

Change-Id: Ib3c0c112ed4936d05a0ea3484747a9d8fd634908
Reviewed-on: http://review.couchbase.org/106129
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...