Rename all classes/variables with upr to dcp

Change-Id: Iaf644f5034fdb30a14c400ec55def22d8bb0ba87
Reviewed-on: http://review.couchbase.org/40942
Reviewed-by: Chiyoung Seo <chiyoung@c

Rename all classes/variables with upr to dcp

Change-Id: Iaf644f5034fdb30a14c400ec55def22d8bb0ba87
Reviewed-on: http://review.couchbase.org/40942
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>

show more ...

MB-11237: Add backoff stats to dcp and dcpagg stats

This stat will be incremented each time the consumer is unable
to process items due to memory usage being too high.

Change-Id

MB-11237: Add backoff stats to dcp and dcpagg stats

This stat will be incremented each time the consumer is unable
to process items due to memory usage being too high.

Change-Id: Ie79c4c57f8b0c364f6075f9ba1be776f5bcc490a
Reviewed-on: http://review.couchbase.org/40061
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>

show more ...

MB-11721: Send acks for snapshot markers during the takeover phase

By sending acks for snapshot markers we can make sure that all items
hve actually been processed on the new master befo

MB-11721: Send acks for snapshot markers during the takeover phase

By sending acks for snapshot markers we can make sure that all items
hve actually been processed on the new master before sending any
set vbucket state messages. This change will reduce the time that
a vbucket might be unavailable due to state changes.

Change-Id: I70b64035c4684754bfcdb632f29cb83fecf161a1
Reviewed-on: http://review.couchbase.org/39375
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>

show more ...

MB-11365:Datatype to be in response where required

- Return datatype in responses for GAT, GATQ,
GetLocked, GetRandomKey

Change-Id: Ia7e40c7d4a79efae3f9cfff41b51d099e89b0621

MB-11365:Datatype to be in response where required

- Return datatype in responses for GAT, GATQ,
GetLocked, GetRandomKey

Change-Id: Ia7e40c7d4a79efae3f9cfff41b51d099e89b0621
Reviewed-on: http://review.couchbase.org/38078
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
Tested-by: abhinav dangeti <abhinav@couchbase.com>

show more ...

Move command opcodes to memcached

And rename them all to PROTOCOL_BINARY_

Change-Id: I674cfb8dcf5817d76a973cfda8ef86df32815f51
Reviewed-on: http://review.couchbase.org/37115

Move command opcodes to memcached

And rename them all to PROTOCOL_BINARY_

Change-Id: I674cfb8dcf5817d76a973cfda8ef86df32815f51
Reviewed-on: http://review.couchbase.org/37115
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

Fix sporadic unit test failure in upr end stream test

Because we now se flow control the actual end stream reasponse is
processed by another thread. As a result it is possible that the

Fix sporadic unit test failure in upr end stream test

Because we now se flow control the actual end stream reasponse is
processed by another thread. As a result it is possible that the
other thread hasn't processed this item when we check the stats. As
a result we need to wait until the stat changes.

Change-Id: Icdb71d61e5842da462b9668321f444a6868bee05
Reviewed-on: http://review.couchbase.org/37015
Reviewed-by: abhinav dangeti <abhinav@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>

show more ...

Check if datatype is supported for Meta commands

For the following commands:
- SET_WITH_META
- ADD_WITH_META
- SET_RET_META
- ADD_RET_META
We need to inspect the document

Check if datatype is supported for Meta commands

For the following commands:
- SET_WITH_META
- ADD_WITH_META
- SET_RET_META
- ADD_RET_META
We need to inspect the document for JSONness
only if datatype isn't supported for the connection.

Change-Id:I51331552b21d97243ec5a94549efb917b65e179e
Reviewed-on: http://review.couchbase.org/36738
Reviewed-by: abhinav dangeti <abhinav@couchbase.com>
Tested-by: abhinav dangeti <abhinav@couchbase.com>

show more ...

MB-9898: Add flow control to the upr producer

Change-Id: Ia94099d249c0624d153be948ac085163608de9db
Reviewed-on: http://review.couchbase.org/35431
Reviewed-by: Chiyoung Seo <chiyoung@

MB-9898: Add flow control to the upr producer

Change-Id: Ia94099d249c0624d153be948ac085163608de9db
Reviewed-on: http://review.couchbase.org/35431
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>

show more ...

MB-10702: Include the proper NRU bits in an UPR mutation

When sending an UPR mutation, we need to include the proper
NRU bits of the stored value.

Change-Id: If7fd01e34079161212

MB-10702: Include the proper NRU bits in an UPR mutation

When sending an UPR mutation, we need to include the proper
NRU bits of the stored value.

Change-Id: If7fd01e34079161212cf213fb62cafefa9248f7c
Reviewed-on: http://review.couchbase.org/35897
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

MB-10711: Datatype adjustment for legacy clients

New engine API, for memcached to update datatype,
in case of legacy clients

For Unknown commands:
- SET_RET_META
- ADD_R

MB-10711: Datatype adjustment for legacy clients

New engine API, for memcached to update datatype,
in case of legacy clients

For Unknown commands:
- SET_RET_META
- ADD_RET_META
- SET_WITH_META
- ADD_WITH_META
check if document is JSON, if datatype is set to
RAW_BYTES, once before going ahead with the operation.
If the document were JSON and the datatype indicates
otherwise, adjust the datatype.

Change-Id:I5ffdd66062ceba79306495fa1c94e7f311a77150
Reviewed-on: http://review.couchbase.org/35170
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

Extend test-datatype to test set_with_meta

Change-Id: I42efab4d11d5db97abda33f9ec106ee469604608
Reviewed-on: http://review.couchbase.org/35020
Reviewed-by: Michael Wiederhold <mike@c

Extend test-datatype to test set_with_meta

Change-Id: I42efab4d11d5db97abda33f9ec106ee469604608
Reviewed-on: http://review.couchbase.org/35020
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: abhinav dangeti <abhinav@couchbase.com>

show more ...

MB-10488: Add a notifier upr connection type

The upr notifier connection is used to get updates through an upr
stream about when new items are available to stream.

Change-Id: I8

MB-10488: Add a notifier upr connection type

The upr notifier connection is used to get updates through an upr
stream about when new items are available to stream.

Change-Id: I8f568f7696b09f64703b9bab50a9c5420c9d0c80
Reviewed-on: http://review.couchbase.org/34868
Reviewed-by: David Liao <david.liao@couchbase.com>
Tested-by: buildbot <build@couchbase.com>

show more ...

Revert "MB-10488: Add a notifier upr connection type"

This will revert commit ba01f68086c3d9b62bbeed9629674efece3fc61d.

Change-Id: Ife7c4bb2c4bbcad4932e4cac53a004bbe90f25e0
Revi

Revert "MB-10488: Add a notifier upr connection type"

This will revert commit ba01f68086c3d9b62bbeed9629674efece3fc61d.

Change-Id: Ife7c4bb2c4bbcad4932e4cac53a004bbe90f25e0
Reviewed-on: http://review.couchbase.org/34859
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>

show more ...

MB-10488: Add a notifier upr connection type

The upr notifier connection is used to get updates through an upr
stream about when new items are available to stream.

Change-Id: I2

MB-10488: Add a notifier upr connection type

The upr notifier connection is used to get updates through an upr
stream about when new items are available to stream.

Change-Id: I276e558bf36c43f427d221351495587ee5063154
Reviewed-on: http://review.couchbase.org/34572
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>

show more ...

MB-10210: Handling Appends/Prepends/Incrs/Decrs

- Appends/Prepends:
Handle the scenario where data were compressed
and data being added was not or vice versa.
Return value:

MB-10210: Handling Appends/Prepends/Incrs/Decrs

- Appends/Prepends:
Handle the scenario where data were compressed
and data being added was not or vice versa.
Return value:
ENGINE_SUCCESS: if succeeds
ENGINE_E2BIG: if new value exceeds limit
ENGINE_FAILED: if snappy operations fail, check
memory to ensure out-of-memory situation.

- Incrs/Decrs:
Return ENGINE_EINVAL in case of arithmetic operations
on compressed data.

Change-Id: I6f7d8b09e86572e199a8ece9a40fa2afe7669db4
Reviewed-on: http://review.couchbase.org/33587
Reviewed-by: abhinav dangeti <abhinav@couchbase.com>
Tested-by: abhinav dangeti <abhinav@couchbase.com>

show more ...

Return ENGINE_WANT_MORE when upr step should be called again

When the engine wants the memcached layer to call the step function
again then we should return ENGINE_WANT_MORE. For example

Return ENGINE_WANT_MORE when upr step should be called again

When the engine wants the memcached layer to call the step function
again then we should return ENGINE_WANT_MORE. For example, this
error code should be returned when the engine has more messages that
are ready to be written to the wire. If the engine does not have any
more messages to send then we should return ENGINE_SUCCESS. This will
cause the memcached layer to stop calling the step function. When
more messages do become ready then we need to notify the memcached
layer that more messages are ready.

There is a corresponding change in the memcached layer that allows us
to use these two error codes. This change corrects the engine code to
return the proper error codes when we merge the memcached change. A
later change will allow us to get the full behavior for pausing and
resuming the calling of the step function.

Change-Id: I5e92386327ece55bab344b8093bda4a3f1009ca4
Reviewed-on: http://review.couchbase.org/33477
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>

show more ...

MB-9900 Rollback implementation on a replica vbucket

This change implements the rollback support on a replica
vbucket in case where a replica vbucket is ahead of its
active vbucket.

MB-9900 Rollback implementation on a replica vbucket

This change implements the rollback support on a replica
vbucket in case where a replica vbucket is ahead of its
active vbucket.

If the number of items to be rollbacked is more than
50% of the number of items on the replica vbucket, we then
reset the replica vbucket and ask the active vbucket to send
the entire snapshot to the replica side.

Change-Id: I0fbe8d91994807073be61d65b735ca79fe6a42ba
Reviewed-on: http://review.couchbase.org/32782
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

Don't use the vbucket uuid in persist seqno messages

The vbucket uuid field was added in order to make sure that things
like failover didn't happen when you issued this command. This is

Don't use the vbucket uuid in persist seqno messages

The vbucket uuid field was added in order to make sure that things
like failover didn't happen when you issued this command. This is
unnecessary because if you want to watch seqno persistence and
there are failures then your going to have to get the latest seqno
and try again anyways.

Change-Id: If1033d03597474942216c3ca3dc4d793994d4c76
Reviewed-on: http://review.couchbase.org/33227
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>

show more ...

MB-9821: Add takeover stream logic

The change adds two more states to the producer stream state
machine to allow a producer to properly hand off a vbucket to
another node.

C

MB-9821: Add takeover stream logic

The change adds two more states to the producer stream state
machine to allow a producer to properly hand off a vbucket to
another node.

Change-Id: Ia7925442b9f8cb984639b331ae642d90c9ef94a0
Reviewed-on: http://review.couchbase.org/31515
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

Add a test api to get 64 bit stat numbers

Not being able to get these numbers can cause crashes in
the testsuite since the values can be truncated.

Change-Id: Ic289144c637a840ea

Add a test api to get 64 bit stat numbers

Not being able to get these numbers can cause crashes in
the testsuite since the values can be truncated.

Change-Id: Ic289144c637a840ea2816e0a6ab9e668804510d8
Reviewed-on: http://review.couchbase.org/31650
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>

show more ...

Fix compilation on WIN32

Change-Id: I3e320969e882530c7a9aa42b5ba7287f75cbc622
Reviewed-on: http://review.couchbase.org/31571
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Te

Fix compilation on WIN32

Change-Id: I3e320969e882530c7a9aa42b5ba7287f75cbc622
Reviewed-on: http://review.couchbase.org/31571
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

MB-9638 UPR persistence sequence number command

This command is similar to the Tap checkpoint persistence command.
But we use sequence number instead of checkpoint id for UPR.
Refact

MB-9638 UPR persistence sequence number command

This command is similar to the Tap checkpoint persistence command.
But we use sequence number instead of checkpoint id for UPR.
Refactored code to support both checkpoint id and seqno persistence.
The new memcached command opcode is CMD_SEQNO_PERSISTENCE 0xb7.
Added a stat "persistence_seqno".
Added a test case "upr_persistence_seqno".

Change-Id: Iec5db260e05cfa8889f253c33bb08209eb87ca2c
Reviewed-on: http://review.couchbase.org/30899
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>

show more ...

Remove compiler warnings

Change-Id: I48743038fba8bb9df4a8d392b48bd79d8029a16d
Reviewed-on: http://review.couchbase.org/30703
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Re

Remove compiler warnings

Change-Id: I48743038fba8bb9df4a8d392b48bd79d8029a16d
Reviewed-on: http://review.couchbase.org/30703
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

MB-9311: Notify all the pending connections on vbucket delete

If a given vbucket is deleted, we should notify it to all of its
pending connections, so that the connections are released

MB-9311: Notify all the pending connections on vbucket delete

If a given vbucket is deleted, we should notify it to all of its
pending connections, so that the connections are released
immediately and the error messages should be sent to the clients.

Change-Id: Ic230a803f0b26944f98e84ed4d21f8d834e3a561
Reviewed-on: http://review.couchbase.org/30353
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>

show more ...

CBD-485: Move DB file compaction into EP-Engine

As detailed in CBD-485, the compactor module is moved into EP-Engine
New Memcached Command:
+ CMD_COMPACT_DB with protocol_binary_requ

CBD-485: Move DB file compaction into EP-Engine

As detailed in CBD-485, the compactor module is moved into EP-Engine
New Memcached Command:
+ CMD_COMPACT_DB with protocol_binary_request_compact_db
New Stats:
+ ep_pending_compactions
+ compaction timing histogram
New unit test case to exercise the code path
New command line utility to perform compaction:
+ cbcompact 127.0.0.1:12000 compact <vbid> --wait
New MCCouch Notification cmd
+ VB_COMPACTION_DONE VB_COMPACT_OPENDB_ERROR VB_COMPACT_RENAME_ERROR
Change-Id: I0606a6e06edc41c5265f3661d97d178165c7f19a
Reviewed-on: http://review.couchbase.org/29258
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...