History log of /5.5.2/couchdb/src/couch_dcp/ (Results 1 - 25 of 54)
Revision (<<< Hide revision tags) (Show revision tags >>>)Date Author Comments
(<<< Hide modified files)
(Show modified files >>>)
8be914dd15-Apr-2018 harsha <harsha.hs.81@gmail.com>

MB-27546 Separate out ipv4 and ipv6 tests

It is observed that dcp fake server failed to start when the
bind address is changed from ipv4 to ipv6 mode.
This patch runs whole gamut of

MB-27546 Separate out ipv4 and ipv6 tests

It is observed that dcp fake server failed to start when the
bind address is changed from ipv4 to ipv6 mode.
This patch runs whole gamut of unit tests in ipv4 followed by ipv6
mode. Mode is decided by environment variable ipv6, which is
set to either true or false.

Change-Id: I5f9c6a6176f47294f78bbc56e88f807a80fd96a4
Reviewed-on: http://review.couchbase.org/92759
Reviewed-by: Aman Kishore Achpal <aman.achpal@couchbase.com>
Tested-by: Jeelan Basha Poola <jeelan.poola@couchbase.com>

show more ...

94134d1105-Mar-2018 harsha <harsha.hs.81@gmail.com>

MB-27546 Make dcp client and fake server robust to socket failures

noop_interval was set in control thread which handled
messages received from socket. This patch moves setting
noop

MB-27546 Make dcp client and fake server robust to socket failures

noop_interval was set in control thread which handled
messages received from socket. This patch moves setting
noop interval to main thread if noop is enabled by KV.

Fake server closes socket so that dcp client can restart
receive worker whenever there is are transient errors
during socket send/recv operations.
Fake server handle_calls had default timeout of 5s which is now
made infinite.

Change-Id: I5336c55d0ce7ea85ef1b4e31830ba11007630619
Reviewed-on: http://review.couchbase.org/90495
Reviewed-by: Aman Kishore Achpal <aman.achpal@couchbase.com>
Reviewed-by: Jeelan Basha Poola <jeelan.poola@couchbase.com>
Tested-by: Jeelan Basha Poola <jeelan.poola@couchbase.com>

show more ...

73648b9013-Jan-2018 harsha <harsha.hs.81@gmail.com>

MB-26674 Log redaction support in views.

This patch identifies user data and redacts customer user data
that are logged in the views under <ud> </ud> tags.

Change-Id: Ic61c92179

MB-26674 Log redaction support in views.

This patch identifies user data and redacts customer user data
that are logged in the views under <ud> </ud> tags.

Change-Id: Ic61c9217908c455ef4e4dce987a2295add9c5515
Reviewed-on: http://review.couchbase.org/87818
Reviewed-by: Aman Kishore Achpal <aman.achpal@couchbase.com>
Reviewed-by: Jeelan Basha Poola <jeelan.poola@couchbase.com>
Tested-by: Jeelan Basha Poola <jeelan.poola@couchbase.com>

show more ...

f7298f3214-Nov-2017 harsha <harsha.hs.81@gmail.com>

MB-26479 Add ipv6 protocol support in views

Change-Id: Ife18b0318d9505772d71dfeb7de71d8f6a1cb043
Reviewed-on: http://review.couchbase.org/86613
Reviewed-by: Aman Kishore Achpal <aman

MB-26479 Add ipv6 protocol support in views

Change-Id: Ife18b0318d9505772d71dfeb7de71d8f6a1cb043
Reviewed-on: http://review.couchbase.org/86613
Reviewed-by: Aman Kishore Achpal <aman.achpal@couchbase.com>
Reviewed-by: Jeelan Basha Poola <jeelan.poola@couchbase.com>
Tested-by: Jeelan Basha Poola <jeelan.poola@couchbase.com>

show more ...

8b994cdb08-Nov-2017 harsha <harsha.hs.81@gmail.com>

MB-100 Retire unused couchdb functionality

Change-Id: Icacf54e764d5cd30b1a25967ebbf45bc00b559b8
Reviewed-on: http://review.couchbase.org/85235
Reviewed-by: Aman Kishore Achpal <aman.

MB-100 Retire unused couchdb functionality

Change-Id: Icacf54e764d5cd30b1a25967ebbf45bc00b559b8
Reviewed-on: http://review.couchbase.org/85235
Reviewed-by: Aman Kishore Achpal <aman.achpal@couchbase.com>
Tested-by: Aman Kishore Achpal <aman.achpal@couchbase.com>

show more ...

3cf258de21-Jul-2017 Aman Achpal <aman.achpal@couchbase.com>

MB-25063 Print latest snapshot boundaries

Upon receiving a snapshot mutation, we immediately update
the Start-sequence number. However, end sequence number is
not updated, causing th

MB-25063 Print latest snapshot boundaries

Upon receiving a snapshot mutation, we immediately update
the Start-sequence number. However, end sequence number is
not updated, causing the "Obtaining mutations timed out"
message to print seemingly stale and misleading information.
This fix prints the latest snapshot boundaries that are
persisted when a snapshot marker is received, as well as
the Start-sequence number as reflected by the index header.

Change-Id: I03f6e0696c91746ba62fbc80c10fea26cedd36b9
Reviewed-on: http://review.couchbase.org/80698
Tested-by: Aman Kishore Achpal <aman.achpal@couchbase.com>
Reviewed-by: Abhishek Singh <abhishek@couchbase.com>
Reviewed-by: Harsha H S <hhs.couchbase@gmail.com>

show more ...

dd64cc6a11-Jul-2017 harsha <hhs.couchbase@gmail.com>

MB-17025 Terminate updater if kv_engine is silent for 2*NOOP_INTERVAL

If KV_Engine fails to stream mutation or high sequence numbers for a
period of 2 * NOOP_INTERVAL (default is 4 min)

MB-17025 Terminate updater if kv_engine is silent for 2*NOOP_INTERVAL

If KV_Engine fails to stream mutation or high sequence numbers for a
period of 2 * NOOP_INTERVAL (default is 4 min) we terminate updater
process with timeout reason, instead of retrying perpetually.

Also correct the set_noop_interval value string to send in seconds
instead of miliseconds

Change-Id: I11b7ddee604e654f36b610c67268a09cbe3a266b
Reviewed-on: http://review.couchbase.org/80527
Reviewed-by: Aman Kishore Achpal <aman.achpal@couchbase.com>
Reviewed-by: Abhishek Singh <abhishek@couchbase.com>
Tested-by: Harsha H S <hhs.couchbase@gmail.com>

show more ...

30ee623713-Jul-2017 Aman Achpal <aman.achpal@couchbase.com>

MB-25235 Enabling Noop for view-engine

As per MB-23649, KV Engine now enforces Noop for components
using v5 features. Since view-engine uses XAttrs, failing
to enable dead connection

MB-25235 Enabling Noop for view-engine

As per MB-23649, KV Engine now enforces Noop for components
using v5 features. Since view-engine uses XAttrs, failing
to enable dead connection detection causes "not supported"
errors. This fix addresses the issue by modifying the
view-engine DCP client/consumer to enable Noop.
Additionally, dummy implementation is added to
couch_dcp_fake_server to prevent failing tests.

Change-Id: Ie2917f34730363f8c1b408bf498eece5dd6da84f
Reviewed-on: http://review.couchbase.org/80637
Reviewed-by: Harsha H S <hhs.couchbase@gmail.com>
Tested-by: Harsha H S <hhs.couchbase@gmail.com>

show more ...

Revision tags: v4.6.0
a004585521-Oct-2016 harsha <hhs.couchbase@gmail.com>

MB-21303 Handle race between stream addition and updater exit

DocLoader sub-process adds dcp stream for a vbucket.
Streams are added into active streams list after a response is
rece

MB-21303 Handle race between stream addition and updater exit

DocLoader sub-process adds dcp stream for a vbucket.
Streams are added into active streams list after a response is
received from dcp producer. If updater exits before the stream
is added into active list, there is no way of closing this stream
until updater is stopped in subsequent iteration, which could
potentially result in not acknowledgeing any stream events for
this vbucket.

This patch avoids race by adding, listing and removing
streams from the parent updater process.

Change-Id: Ide67f6a3ec061cfe7c279f98d199da32dc0cb47c
Reviewed-on: http://review.couchbase.org/76980
Tested-by: Harsha H S <hhs.couchbase@gmail.com>
Reviewed-by: Abhishek Singh <abhishek@couchbase.com>

show more ...

5f5f70d811-Oct-2016 harsha <hhs.couchbase@gmail.com>

MB-21303 Address leak in stream_info queue

In the error condition of add_stream, stream_info
queue entry was not flushed out which resulted in leak.

Change-Id: If54ef45ffd25fe1c

MB-21303 Address leak in stream_info queue

In the error condition of add_stream, stream_info
queue entry was not flushed out which resulted in leak.

Change-Id: If54ef45ffd25fe1c9f4615a29bf8d615609dfc20
Reviewed-on: http://review.couchbase.org/76964
Reviewed-by: Abhishek Singh <abhishek@couchbase.com>
Tested-by: Harsha H S <hhs.couchbase@gmail.com>

show more ...

773ad42f10-Oct-2016 harsha <hhs.couchbase@gmail.com>

MB-21303 Handle ack from dcp producer for control_request correctly

control_request for flow control buffer size is added into the
pending request dictionary with nil as SendTo parameter

MB-21303 Handle ack from dcp producer for control_request correctly

control_request for flow control buffer size is added into the
pending request dictionary with nil as SendTo parameter as
gen_server need not reply when it gets response from producer.
This was not handled correctly when stream_response is handled
in the client which is fixed in this patch.

Change-Id: Iaaa2c8b62be838ed6d03fbe56c5041b2a8445607
Reviewed-on: http://review.couchbase.org/76962
Tested-by: Harsha H S <hhs.couchbase@gmail.com>
Reviewed-by: Abhishek Singh <abhishek@couchbase.com>

show more ...

8463251518-Apr-2017 Sriram Melkote <siri@couchbase.com>

CBD-1983: Remove DCP flow control changes from master

This reverts commit e4767cfeae434e2e63480e7bc6b50395f0738a5d.

Remove the experimenal flow control changes from master. These

CBD-1983: Remove DCP flow control changes from master

This reverts commit e4767cfeae434e2e63480e7bc6b50395f0738a5d.

Remove the experimenal flow control changes from master. These
were reverted on watson as they introduced more problems, while
the root cause of the regression was a bug in ep-engine, and hence
were not strictly necessary.

Change-Id: I5da23439012d5e5097841477305fbb9ee4c383f9
Reviewed-on: http://review.couchbase.org/76951
Reviewed-by: Harsha H S <hhs.couchbase@gmail.com>
Tested-by: Harsha H S <hhs.couchbase@gmail.com>

show more ...

584de8c001-Jun-2017 harsha <hhs.couchbase@gmail.com>

MB-24616 Control indexing of system _xattrs on document deletion

Indexing xattributes on document deletion has a potential problem,
that the index structure can grow out of proportions,

MB-24616 Control indexing of system _xattrs on document deletion

Indexing xattributes on document deletion has a potential problem,
that the index structure can grow out of proportions, if care is
not taken to delete them from the index structure.

Views depend on the dcp mutations to insert or delete a document
into the index structure. Dcp set triggers value insertion whereas
dcp deletion triggers value deletion from view index structure.

Since, we index xattrs on document deletion, there is no way to
delete this value from index structure, unless same document is
created and deleted without setting xattrs. In order to avoid
unscrupulous use of indexing xattrs, a new design doc level
attribute is added to enable indexing of xattrs on document deletion.
This attribute can be set using RESTful API on CAPI REST port,
and is not exposed in UI.

For example on development server where CAPI REST port is 9500,
design doc named ddoc which can index xattrs on document deletion
and smaple view called view1 is created using
curl -X PUT -H 'Content-Type: application/json'
http://Administrator:asdasd@localhost:9500/default/_design/ddoc -d
@views_def.json
where contents of views_def.json is
{
"views" : {
"view1": {
"map" : "function(doc, meta){emit(meta.id,
meta.xattrs);}"
}
},
"index_xattr_on_deleted_docs" : true
}

Change-Id: I3d977812409d6bba205431e7bc408aab4864b4e2
Reviewed-on: http://review.couchbase.org/78907
Reviewed-by: Abhishek Singh <abhishek@couchbase.com>
Tested-by: Harsha H S <hhs.couchbase@gmail.com>
Reviewed-by: Aman Kishore Achpal <aman.achpal@couchbase.com>

show more ...

045f5a6425-May-2017 harsha <hhs.couchbase@gmail.com>

MB-23888 Unit test to stream xattrs during soft deletion of document

As part of mobile convergence plan, system xattributes are streamed
during document soft deletion.
This unit test

MB-23888 Unit test to stream xattrs during soft deletion of document

As part of mobile convergence plan, system xattributes are streamed
during document soft deletion.
This unit test simulates this feature by having a deleted attribute
in the document which is parsed, and delete mutation is generated,
along with streaming of xattribute body

Change-Id: I360ed51074e9f44f6baaab43538c00d04ed90940
Reviewed-on: http://review.couchbase.org/78582
Tested-by: Harsha H S <hhs.couchbase@gmail.com>
Reviewed-by: Abhishek Singh <abhishek@couchbase.com>

show more ...

de563bee23-May-2017 harsha <hhs.couchbase@gmail.com>

MB-23888 Fix spec for couch_dcp_consumer:parse_snapshot_deletion/2

Snapshot deletion now returns system xattributes set on the document.
Correcting the spec for this function to overcome

MB-23888 Fix spec for couch_dcp_consumer:parse_snapshot_deletion/2

Snapshot deletion now returns system xattributes set on the document.
Correcting the spec for this function to overcome dialyzer failures.

Change-Id: I47f1d0fa62271542e086f0a07452ac7f1cad293f
Reviewed-on: http://review.couchbase.org/78469
Tested-by: Harsha H S <hhs.couchbase@gmail.com>
Reviewed-by: Abhishek Singh <abhishek@couchbase.com>

show more ...

123f480311-May-2017 harsha <hhs.couchbase@gmail.com>

MB-23888 Treat deleted documents with system _xattrs as set operation

As part of mobile convergence, any document with system xattributes,
upon deletion streams those system xattributes

MB-23888 Treat deleted documents with system _xattrs as set operation

As part of mobile convergence, any document with system xattributes,
upon deletion streams those system xattributes over DCP.
Views will treat these document as soft deletion and index them, by
treating them as KV set operation. Views will treat the documents
without any system xattributes as hard deletion and remove them
from index structure.

Change-Id: I4ca6ac5e12509a36fc6354d189daa39f51ff0426
Reviewed-on: http://review.couchbase.org/78004
Reviewed-by: Harsha H S <hhs.couchbase@gmail.com>
Tested-by: Harsha H S <hhs.couchbase@gmail.com>

show more ...

8da028d828-Apr-2017 harsha <hhs.couchbase@gmail.com>

MB-21350 Ignore system xattrs on snapshot deletion

Until we support indexing of system xattrs on snapshot deletion, we
are ignoring them.

Change-Id: I943a8acf67d442dba4de52f92d1

MB-21350 Ignore system xattrs on snapshot deletion

Until we support indexing of system xattrs on snapshot deletion, we
are ignoring them.

Change-Id: I943a8acf67d442dba4de52f92d1e940e37fabe84
Reviewed-on: http://review.couchbase.org/77489
Tested-by: Harsha H S <hhs.couchbase@gmail.com>
Reviewed-by: Abhishek Singh <abhishek@couchbase.com>

show more ...

3db6751517-Feb-2017 harsha <hhs.couchbase@gmail.com>

MB-21350 Expose Extended attributes a.k.a XATTR to views

Extended Attributes that are streamed by KV are now included
in the metadata information of the document and are accessible
b

MB-21350 Expose Extended attributes a.k.a XATTR to views

Extended Attributes that are streamed by KV are now included
in the metadata information of the document and are accessible
by views as `meta.xattrs` property

Sample metadata of document with xattrs now looks like
{
"id": "xattr_demo_doc",
"rev": "4-14a501b8539400000000000002000000",
"seq": "4",
"vb": "321",
"expiration": 0,
"flags": 33554432,
"type": "json",
"xattrs": {
"doc": {
"readcount": [
10,
11
],
"author": "Jack"
},
"_sync": {
"channels": [
"UK",
"USA",
"EU"
]
}
}
}

whereas sample metadata of document not having xattrs looks like
{
"id": "pymc0",
"rev": "1-14a501f28e5b00000000000000000000",
"seq": "1",
"vb": "449",
"expiration": 0,
"flags": 0,
"type": "json",
"xattrs": {}
}

Change-Id: Iefb8d507a23737acdbe98ee632a0e7f09bdcde38
Reviewed-on: http://review.couchbase.org/73817
Tested-by: Harsha H S <hhs.couchbase@gmail.com>
Reviewed-by: Abhishek Singh <abhishek@couchbase.com>

show more ...

d759605531-Jan-2017 harsha <hhs.couchbase@gmail.com>

MB-22525 DCP_OPEN_NO_VALUE is made global per connection flag

For views that do not use document fields, only metadata was
streamed from dcp. Earlier this was done by setting NO_VALUE

MB-22525 DCP_OPEN_NO_VALUE is made global per connection flag

For views that do not use document fields, only metadata was
streamed from dcp. Earlier this was done by setting NO_VALUE
flag for every open stream request.
Now this flag is a global flag which is set while opening a
new DCP connection, as there was no use-case for selectively
streaming metadata from specific subset of vbuckets.

Change-Id: Ibb50e70aa2817d9c8a29876adf178b5b1afacce6
Reviewed-on: http://review.couchbase.org/72946
Reviewed-by: Harsha H S <hhs.couchbase@gmail.com>
Tested-by: Harsha H S <hhs.couchbase@gmail.com>

show more ...

e4767cfe01-Nov-2016 Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>

Revert "Merge remote-tracking branch 'couchbase/watson'"

I didn't mean to revert master to the state before the dcp client
changes. So this reverts this faulty merge.

This rever

Revert "Merge remote-tracking branch 'couchbase/watson'"

I didn't mean to revert master to the state before the dcp client
changes. So this reverts this faulty merge.

This reverts commit 39c7fbc82ace9bf7955673b8a19c844f7f4ea40c, reversing
changes made to a6402a915bd2ea574ef5e32b0c247b727142209a.

Change-Id: I42b097ff809aa9f073bb4c6c228d400def5d122f
Reviewed-on: http://review.couchbase.org/69386
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Tested-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>

show more ...

39c7fbc801-Nov-2016 Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>

Merge remote-tracking branch 'couchbase/watson'

* couchbase/watson:
Revert dcp client changes.

Change-Id: I78ff3321010ddefe9f9b7acd76f7c5755ddd8c26


a6402a9101-Nov-2016 Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>

Merge remote-tracking branch 'couchbase/3.x' into work

* couchbase/3.x:
MB-21509 Always ack whatever is in the queue when removing it.
MB-21509 Crash if close_stream returns unex

Merge remote-tracking branch 'couchbase/3.x' into work

* couchbase/3.x:
MB-21509 Always ack whatever is in the queue when removing it.
MB-21509 Crash if close_stream returns unexpected error.

Change-Id: I5f3371f01575cac198134ceb8524e16d11fbb471

show more ...


37036aa931-Oct-2016 Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>

MB-21509 Always ack whatever is in the queue when removing it.

The problem with the original patch was that when the client got DOWN
message from a process, it would just delete the queu

MB-21509 Always ack whatever is in the queue when removing it.

The problem with the original patch was that when the client got DOWN
message from a process, it would just delete the queue without
acknowledging anything.

Now I just call check_and_send_buffer_ack from inside
remove_request_queue. Incidentally, this means that it gets called
even after the client just receives stream_end. And this means, that
even if ep-engine continues sending mutations after stream_end, we'll
still acknowledge them.

Change-Id: I18f55f54deff30c0a914803869c3fe62d64f99f7
Reviewed-on: http://review.couchbase.org/69349
Well-Formed: buildbot <build@couchbase.com>
Reviewed-by: Sarath Lakshman <sarath@couchbase.com>
Tested-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>

show more ...

57c534b431-Oct-2016 Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>

MB-21509 Crash if close_stream returns unexpected error.

It's much better than leaving the client in an undefined state.

Change-Id: I76afddcb33ae7db25609b88517c64d6e1f2099f1
Rev

MB-21509 Crash if close_stream returns unexpected error.

It's much better than leaving the client in an undefined state.

Change-Id: I76afddcb33ae7db25609b88517c64d6e1f2099f1
Reviewed-on: http://review.couchbase.org/69348
Well-Formed: buildbot <build@couchbase.com>
Reviewed-by: Sarath Lakshman <sarath@couchbase.com>
Tested-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>

show more ...

63c71f0b23-Oct-2016 Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>

MB-21509 [Backport] MB-21303 Handle race between stream addition and updater exit

DocLoader sub-process adds dcp stream for a vbucket.
Streams are added into active streams list after a

MB-21509 [Backport] MB-21303 Handle race between stream addition and updater exit

DocLoader sub-process adds dcp stream for a vbucket.
Streams are added into active streams list after a response is
received from dcp producer. If updater exits before the stream
is added into active list, there is no way of closing this stream
until updater is stopped in subsequent iteration, which could
potentially result in not acknowledgeing any stream events for
this vbucket.

This patch avoids race by monitoring the DocLoader process and
removes the streams whenevr the process terminates or the stream
is explicitly closed.

Change-Id: Ib36bb8d6fe0d07c220b16c231e2762d4dfeb33d9
Reviewed-on: http://review.couchbase.org/68808
Tested-by: Harsha H S <hhs.couchbase@gmail.com>
Reviewed-by: Harsha H S <hhs.couchbase@gmail.com>
Reviewed-on: http://review.couchbase.org/69315
Reviewed-by: Sarath Lakshman <sarath@couchbase.com>
Tested-by: Sarath Lakshman <sarath@couchbase.com>

show more ...

123