MB-31828: Stop analytics overwriting diag.log during cbcollect_info

Backport of d5c1206

Also rename pprof_url to cbas_pprof_url and eventing_pprof_url in the
same patch to stop

MB-31828: Stop analytics overwriting diag.log during cbcollect_info

Backport of d5c1206

Also rename pprof_url to cbas_pprof_url and eventing_pprof_url in the
same patch to stop this kind of behaviour happening elsewhere without
realisation

Change-Id: I3c6e0b7baf1d5b8a062722d65757c1623ce848f5
Reviewed-on: http://review.couchbase.org/101651
Well-Formed: Build Bot <build@couchbase.com>
Tested-by: Tim Bradgate <tim.bradgate@couchbase.com>
Reviewed-by: Aliaksey Artamonau <aliaksey.artamonau@couchbase.com>

show more ...

MB-31353: query analytics metadata (cbcollect_info)

Change-Id: I921002b20ac251eaf1f00e06205a506cec49dcdf
Reviewed-on: http://review.couchbase.org/99810
Reviewed-by: Murtadha Hubail <

MB-31353: query analytics metadata (cbcollect_info)

Change-Id: I921002b20ac251eaf1f00e06205a506cec49dcdf
Reviewed-on: http://review.couchbase.org/99810
Reviewed-by: Murtadha Hubail <Murtadha.hubail@couchbase.com>
Reviewed-by: Michael Blow <michael.blow@couchbase.com>
Tested-by: Michael Blow <michael.blow@couchbase.com>
Reviewed-by: Ajit Yagaty <ajit.yagaty@couchbase.com>

show more ...

MB-30906: += analytics_access.log to cbcollect_info

Change-Id: I925688dc1e7a4750af5312aa4df6fe06b81f9c72
Reviewed-on: http://review.couchbase.org/98242
Tested-by: Michael Blow <micha

MB-30906: += analytics_access.log to cbcollect_info

Change-Id: I925688dc1e7a4750af5312aa4df6fe06b81f9c72
Reviewed-on: http://review.couchbase.org/98242
Tested-by: Michael Blow <michael.blow@couchbase.com>
Reviewed-by: Murtadha Hubail <Murtadha.hubail@couchbase.com>
Reviewed-by: Aliaksey Artamonau <aliaksey.artamonau@couchbase.com>

show more ...

MB-30665: Remove cbstats hash task from cbcollect_info

Remove the call to the cbstats hash command from cbcollect_info
as it causes significant intra-cluster replication delay
(See M

MB-30665: Remove cbstats hash task from cbcollect_info

Remove the call to the cbstats hash command from cbcollect_info
as it causes significant intra-cluster replication delay
(See MB-30553)

Change-Id: I420b56bd159d8362577fe377b5e775a32557e9d5
Reviewed-on: http://review.couchbase.org/97502
Tested-by: Ben Huddleston <ben.huddleston@couchbase.com>
Well-Formed: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

MB-30465: update analytics log files

Change-Id: Iafe60fbec34d64cbcd3c381de042b21d55bb94fb
Reviewed-on: http://review.couchbase.org/97173
Reviewed-by: Michael Blow <michael.blow@couch

MB-30465: update analytics log files

Change-Id: Iafe60fbec34d64cbcd3c381de042b21d55bb94fb
Reviewed-on: http://review.couchbase.org/97173
Reviewed-by: Michael Blow <michael.blow@couchbase.com>
Reviewed-by: Aliaksey Artamonau <aliaksey.artamonau@couchbase.com>
Tested-by: Michael Blow <michael.blow@couchbase.com>

show more ...

MB-30339: [FTS] cbcollect to collect stack traces from @fts/dumps

Change-Id: Id2e96e37d9c2a732c11c784cc32d7c499bba8ed0
Reviewed-on: http://review.couchbase.org/96647
Reviewed-by: Ste

MB-30339: [FTS] cbcollect to collect stack traces from @fts/dumps

Change-Id: Id2e96e37d9c2a732c11c784cc32d7c499bba8ed0
Reviewed-on: http://review.couchbase.org/96647
Reviewed-by: Steve Yen <steve.yen@gmail.com>
Tested-by: Abhinav Dangeti <abhinav@couchbase.com>

show more ...

[cbcollect_info] Collect more memory related files from procfs.

Change-Id: I51d5a638cf68813cd55358cca4b8b606799a47ba
Reviewed-on: http://review.couchbase.org/94993
Tested-by: Aliakse

[cbcollect_info] Collect more memory related files from procfs.

Change-Id: I51d5a638cf68813cd55358cca4b8b606799a47ba
Reviewed-on: http://review.couchbase.org/94993
Tested-by: Aliaksey Artamonau <aliaksey.artamonau@couchbase.com>
Well-Formed: Build Bot <build@couchbase.com>
Reviewed-by: Poonam Dhavale <poonam@couchbase.com>

show more ...

Capture full stats from Eventing

Full stats will be necessary to debug cases related to missed or
duplicate event execution.

Change-Id: Ic874e897f2210d7f7f4b2a7230ef14bb8aa6db94

Capture full stats from Eventing

Full stats will be necessary to debug cases related to missed or
duplicate event execution.

Change-Id: Ic874e897f2210d7f7f4b2a7230ef14bb8aa6db94
Reviewed-on: http://review.couchbase.org/94580
Well-Formed: Build Bot <build@couchbase.com>
Reviewed-by: Aliaksey Artamonau <aliaksey.artamonau@couchbase.com>
Tested-by: Abhishek Singh <abhishek@couchbase.com>

show more ...

MB-29431 no need to cleanup {redact_salt, _} tuple from the logs anymore

Change-Id: Ib30ed64ff96f7ac21c59895e873c3599dabff714
Reviewed-on: http://review.couchbase.org/94174
Tested-by

MB-29431 no need to cleanup {redact_salt, _} tuple from the logs anymore

Change-Id: Ib30ed64ff96f7ac21c59895e873c3599dabff714
Reviewed-on: http://review.couchbase.org/94174
Tested-by: Artem Stemkovski <artem@couchbase.com>
Well-Formed: Build Bot <build@couchbase.com>
Reviewed-by: Amit Krishnan <krish.amit@gmail.com>

show more ...

Revert "MB-29431: Hash log redaction salt in redacted logs"

This reverts commit 43909d7f5f79be38f152d0c39f6202619352fe3d.

Because the fix is incorrect. Using anonymous functions acr

Revert "MB-29431: Hash log redaction salt in redacted logs"

This reverts commit 43909d7f5f79be38f152d0c39f6202619352fe3d.

Because the fix is incorrect. Using anonymous functions across the
nodes requires that the nodes run exactly the same compiled beam
file. Which we cannot afford.

Change-Id: I2ea649c772bfdaee3698d572f753eb694cfda511
Reviewed-on: http://review.couchbase.org/94157
Reviewed-by: Aliaksey Artamonau <aliaksey.artamonau@couchbase.com>
Tested-by: Aliaksey Artamonau <aliaksey.artamonau@couchbase.com>

show more ...

MB-29431: Hash log redaction salt in redacted logs

Change-Id: I0b6d3c1e1862f613990b0260c1ab7301b929119e
Reviewed-on: http://review.couchbase.org/93960
Reviewed-by: Artem Stemkovski <

MB-29431: Hash log redaction salt in redacted logs

Change-Id: I0b6d3c1e1862f613990b0260c1ab7301b929119e
Reviewed-on: http://review.couchbase.org/93960
Reviewed-by: Artem Stemkovski <artem@couchbase.com>
Tested-by: Artem Stemkovski <artem@couchbase.com>

show more ...

cbft-bleve with zap option to be invoked from cbcollectInfo

- rename fts_mossScope_stats.log to fts_store_stats.log.
- footer stats for all the fts's scorch zap files will now
be p

cbft-bleve with zap option to be invoked from cbcollectInfo

- rename fts_mossScope_stats.log to fts_store_stats.log.
- footer stats for all the fts's scorch zap files will now
be populated into fts_store_stats.log as well.
- invoke mossScope over a fts folder only if it contains
.moss files.

Change-Id: I49aa03dc543e46bce70c9d8afd489343dee92004
Reviewed-on: http://review.couchbase.org/93967
Well-Formed: Build Bot <build@couchbase.com>
Reviewed-by: Steve Yen <steve.yen@gmail.com>
Tested-by: Abhinav Dangeti <abhinav@couchbase.com>

show more ...

Add eviction stats to cbcollect_info

The addition of the eviction stats will aid in the debugging of
potential issues with the new hifi_mfu hash table eviction policy used
by kv_engi

Add eviction stats to cbcollect_info

The addition of the eviction stats will aid in the debugging of
potential issues with the new hifi_mfu hash table eviction policy used
by kv_engine.

An example of the eviction stats outputted in the stats.log are given
below:

========================================================================
memcached stats eviction
['cbstats', '-a', '127.0.0.1:11999', 'eviction', '-u', '@ns_server']
========================================================================
************************************************************************
daniel

ep_active_or_pending_eviction_values_evicted (151699 total)
0 - 0 : ( 5.44%) 8245 ##
1 - 1 : ( 10.33%) 7419 ##
2 - 2 : ( 15.02%) 7118 ##
3 - 3 : ( 21.07%) 9179 ##
4 - 4 : ( 87.23%) 100359 ################################
5 - 5 : ( 99.64%) 18839 ######
6 - 6 : ( 99.97%) 497
7 - 7 : (100.00%) 38
8 - 8 : (100.00%) 4
9 - 9 : (100.00%) 1
Avg : ( 3)
ep_active_or_pending_eviction_values_snapshot (36 total)
0 - 0 : ( 11.11%) 4 #####
1 - 1 : ( 16.67%) 2 ##
2 - 2 : ( 22.22%) 2 ##
3 - 3 : ( 25.00%) 1 #
4 - 4 : ( 30.56%) 2 ##
5 - 5 : ( 41.67%) 4 #####
255 - 255 : (100.00%) 21 ############################
Avg : ( 149)

Change-Id: I80d7637d335bf9ec38be00e8ccbfefe52b8ef194
Reviewed-on: http://review.couchbase.org/93532
Tested-by: Daniel Owen <owend@couchbase.com>
Well-Formed: Build Bot <build@couchbase.com>
Reviewed-by: Aliaksey Artamonau <aliaksey.artamonau@couchbase.com>

show more ...

MB-29428: Redact user document ID's in access log

The pools/default/buckets/<ID>/docs/<DocID> REST endpoint exposes
the user document ID in the http_access_log.
This patch adds code

MB-29428: Redact user document ID's in access log

The pools/default/buckets/<ID>/docs/<DocID> REST endpoint exposes
the user document ID in the http_access_log.
This patch adds code to cbcollect_info to redact this as part of
the log redaction work.

Change-Id: Ic7d9ac475121d3247f9e24f10203fa8203e0b738
Reviewed-on: http://review.couchbase.org/93524
Well-Formed: Build Bot <build@couchbase.com>
Tested-by: Amit Krishnan <krish.amit@gmail.com>
Reviewed-by: Artem Stemkovski <artem@couchbase.com>

show more ...

MB-28942: Redact occurances of log redaction salt

We see the salt value used in log redaction occuring in plain text in
the logs. This is undesirable therefore we redact them in the post

MB-28942: Redact occurances of log redaction salt

We see the salt value used in log redaction occuring in plain text in
the logs. This is undesirable therefore we redact them in the post
processing step.

Change-Id: I4cef9cd8cb09460097d80ff55df2c64696e5fec8
Reviewed-on: http://review.couchbase.org/92812
Tested-by: Amit Krishnan <krish.amit@gmail.com>
Well-Formed: Build Bot <build@couchbase.com>
Reviewed-by: Artem Stemkovski <artem@couchbase.com>

show more ...

Capture stats and pprof dump from Eventing

Change-Id: I94dba517b5a151d03eff4f94ed764451981a096c
Reviewed-on: http://review.couchbase.org/92629
Well-Formed: Build Bot <build@couchbase

Capture stats and pprof dump from Eventing

Change-Id: I94dba517b5a151d03eff4f94ed764451981a096c
Reviewed-on: http://review.couchbase.org/92629
Well-Formed: Build Bot <build@couchbase.com>
Reviewed-by: Aliaksey Artamonau <aliaksey.artamonau@couchbase.com>
Tested-by: Abhishek Singh <abhishek@couchbase.com>

show more ...

MB-28904: retain analytics_debug.log in cbcollect_info

Change-Id: Idb42391cda2a7976ffa50666727881e044d8f022
Reviewed-on: http://review.couchbase.org/91998
Well-Formed: Build Bot <bui

MB-28904: retain analytics_debug.log in cbcollect_info

Change-Id: Idb42391cda2a7976ffa50666727881e044d8f022
Reviewed-on: http://review.couchbase.org/91998
Well-Formed: Build Bot <build@couchbase.com>
Tested-by: Michael Blow <michael.blow@couchbase.com>
Reviewed-by: Aliaksey Artamonau <aliaksey.artamonau@couchbase.com>

show more ...

MB-28805: collect goroutine dump for analytics

Change-Id: Id3ec49b2f7874ea0642ead73f2ac3918552e38d8
Reviewed-on: http://review.couchbase.org/91559
Reviewed-by: Aliaksey Artamonau <al

MB-28805: collect goroutine dump for analytics

Change-Id: Id3ec49b2f7874ea0642ead73f2ac3918552e38d8
Reviewed-on: http://review.couchbase.org/91559
Reviewed-by: Aliaksey Artamonau <aliaksey.artamonau@couchbase.com>
Tested-by: Aliaksey Artamonau <aliaksey.artamonau@couchbase.com>

show more ...

MB-28900: -= analytics_cbas.log

Change-Id: I622c24873ea084f14a95e8bef931e3b8f87963c9
Reviewed-on: http://review.couchbase.org/91636
Well-Formed: Build Bot <build@couchbase.com>
R

MB-28900: -= analytics_cbas.log

Change-Id: I622c24873ea084f14a95e8bef931e3b8f87963c9
Reviewed-on: http://review.couchbase.org/91636
Well-Formed: Build Bot <build@couchbase.com>
Reviewed-by: Till Westmann <till@couchbase.com>
Reviewed-by: Aliaksey Artamonau <aliaksey.artamonau@couchbase.com>
Tested-by: Aliaksey Artamonau <aliaksey.artamonau@couchbase.com>

show more ...

MB-28519 Fix typo in cbcollect_info help output

Diagnostics was missplet.

Change-Id: Ie356834354035d5b838abcaedcda6e2486a402c3
Reviewed-on: http://review.couchbase.org/90857

MB-28519 Fix typo in cbcollect_info help output

Diagnostics was missplet.

Change-Id: Ie356834354035d5b838abcaedcda6e2486a402c3
Reviewed-on: http://review.couchbase.org/90857
Well-Formed: Build Bot <build@couchbase.com>
Reviewed-by: Aliaksey Artamonau <aliaksey.artamonau@couchbase.com>
Tested-by: Aliaksey Artamonau <aliaksey.artamonau@couchbase.com>

show more ...

MB-28572: ignore broken pipe on stdin close

If a task which receives input dies quickly, an instance of:
"IOError: [Errno 32] Broken pipe" may be raised when writing to
or closing st

MB-28572: ignore broken pipe on stdin close

If a task which receives input dies quickly, an instance of:
"IOError: [Errno 32] Broken pipe" may be raised when writing to
or closing stdin; print a warning and continue if this occurs

Change-Id: Iec815614d15a4ffbe0341e8202739779c71c6855
Reviewed-on: http://review.couchbase.org/90673
Well-Formed: Build Bot <build@couchbase.com>
Tested-by: Michael Blow <michael.blow@couchbase.com>
Reviewed-by: Aliaksey Artamonau <aliaksey.artamonau@couchbase.com>

show more ...

do not try to collect old xdcr ddocs

Change-Id: I95260ba9630568dc0bb5f4c4a0b9e4bb451fccb9
Reviewed-on: http://review.couchbase.org/90400
Tested-by: Artem Stemkovski <artem@couchbase.

do not try to collect old xdcr ddocs

Change-Id: I95260ba9630568dc0bb5f4c4a0b9e4bb451fccb9
Reviewed-on: http://review.couchbase.org/90400
Tested-by: Artem Stemkovski <artem@couchbase.com>
Well-Formed: Build Bot <build@couchbase.com>
Reviewed-by: Timofey Barmin <timofey.barmin@couchbase.com>

show more ...

MB-27682: collect indexer minidumps during cbcollect.

By default, minidump dirs for memcached and indexer are the same.
If minidump dir for memcached is overridden, collect indexer minid

MB-27682: collect indexer minidumps during cbcollect.

By default, minidump dirs for memcached and indexer are the same.
If minidump dir for memcached is overridden, collect indexer minidumps
from indexer minidump dir.

Change-Id: I8ee79fea617462616c7611bf6495e5797595e954
Reviewed-on: http://review.couchbase.org/90052
Reviewed-by: Ajit Yagaty <ajit.yagaty@couchbase.com>
Tested-by: <amit.kulkarni@couchbase.com>

show more ...

reshuffling of xdcr logging

xdcr.log, xdcr_error.log - removed

ns_couchdb node will continue to log everything xdcr related to
ns_couchdb.log.

new log file xdcr_target.

reshuffling of xdcr logging

xdcr.log, xdcr_error.log - removed

ns_couchdb node will continue to log everything xdcr related to
ns_couchdb.log.

new log file xdcr_target.log will be used on ns_server node to
log traces from xdcr target endpoints handlers. I would personally
prefer not to have this file and simply return traces to goxdcr
to log into goxdcr.log, but according to Yu Sui, goxdcr.log is
full of junk and rotates too quickly, and these traces are too important
to be in that file

Change-Id: I05b0d5991887966fd13f9c8de08e3ce45455115e
Reviewed-on: http://review.couchbase.org/90010
Tested-by: Artem Stemkovski <artem@couchbase.com>
Well-Formed: Build Bot <build@couchbase.com>
Reviewed-by: Aliaksey Artamonau <aliaksey.artamonau@couchbase.com>

show more ...

MB-28204: Make cbcollect_info IPv6 aware.

Modify dump-guts to return if the cluster has been configured in
IPv6 mode or not. This information can then be used in cbcollect
to either

MB-28204: Make cbcollect_info IPv6 aware.

Modify dump-guts to return if the cluster has been configured in
IPv6 mode or not. This information can then be used in cbcollect
to either use IPv4 or IPv6 loopback addresses.

Change-Id: I129914424db8d31cbefa85fcf101476abbeed793
Reviewed-on: http://review.couchbase.org/89607
Reviewed-by: Artem Stemkovski <artem@couchbase.com>
Tested-by: Artem Stemkovski <artem@couchbase.com>

show more ...