Explicitly enable ASan for KV-Engine targets needed for cluster_run

To be used with the new CB_ADDRESSSANITIZER=2 mode; enable ASan for
the targets needed to run KV-Engine as part of a l

Explicitly enable ASan for KV-Engine targets needed for cluster_run

To be used with the new CB_ADDRESSSANITIZER=2 mode; enable ASan for
the targets needed to run KV-Engine as part of a larger cluster-run;
but without having to enable for all targets.

For example; enabling ASan for libcouchstore.so causes problems for
the view engine as the main executable which loads libcouchstore
(Erlang VM) doesn't have ASan enabled and hence it crashes on startup.

Change-Id: Ieb77f0622d581e5d7767b2a4fba66853416dd342
Reviewed-on: http://review.couchbase.org/90866
Reviewed-by: Jim Walker <jim@couchbase.com>
Reviewed-by: Tim Bradgate <tim.bradgate@couchbase.com>
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

MB-27986,MB-28225,MB-27882: Remove log dedupe and flush at 200ms

This patch removes the deduplication logic from the logger as
we want to make sure that we dump (and flush) our log "imme

MB-27986,MB-28225,MB-27882: Remove log dedupe and flush at 200ms

This patch removes the deduplication logic from the logger as
we want to make sure that we dump (and flush) our log "immediately"
to the files. All of the log is written in a dedicated thread
which use a buffered FILE stream. We don't want to flush this
stream for _every_ log message (as that is typically 100 bytes
or so) which would cause a significant increse of file IO.

Instead we want the logger to keep writing data to the FILE
pointer, but if no log message is provided within 200ms we'll
cause FILE stream to be flushed to disk. This means that we'll
try to fill up the FILE streams buffer issue fewer actual
filesystem calls, but at the same time we won't keep the data
in those buffers hanging around for a long time.

Change-Id: I1f320dc540d39d9890c8ba84e89b0ce1f203aaa1
Reviewed-on: http://review.couchbase.org/90146
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

Add logger benchmark

Add a small micro benchmark for the logger testing 3 different scenarios:

1) GetLogger - Test the cost of fetching the logger object (means copying

Add logger benchmark

Add a small micro benchmark for the logger testing 3 different scenarios:

1) GetLogger - Test the cost of fetching the logger object (means copying
the shared pointer)

2) LogToLoggerWithDisabeldLogLevel - Test the cost of the log statements
we have in our code when the logger will drop the log
statement becuase the log level isn't enabled (note:
this does _NOT_ include the cost it could be to generate
the parameters passed in!)

3) LogToLoggerWithEnabledLogLevel - Test the cost of adding the log
statement into the queue of statements to pass off to the
async logger. Note that the async logger will drop all
of them. The test will probably return "incorrect" results
if we manage to fill up the queue between the two threads.

Run on (8 X 2500 MHz CPU s)
2018-02-09 11:41:04
---------------------------------------------------------------------------------
Benchmark Time CPU Iterations
---------------------------------------------------------------------------------
GetLogger/threads:1 18 ns 18 ns 38555385
GetLogger/threads:2 46 ns 92 ns 7759112
GetLogger/threads:4 50 ns 198 ns 3510752
GetLogger/threads:8 53 ns 414 ns 1910896
LogToLoggerWithDisabeldLogLevel/threads:1 26 ns 26 ns 26377668
LogToLoggerWithDisabeldLogLevel/threads:2 57 ns 114 ns 6165690
LogToLoggerWithDisabeldLogLevel/threads:4 76 ns 304 ns 2282880
LogToLoggerWithDisabeldLogLevel/threads:8 59 ns 462 ns 1506520
LogToLoggerWithEnabledLogLevel/threads:1 100 ns 100 ns 7002031
LogToLoggerWithEnabledLogLevel/threads:2 125 ns 249 ns 2677172
LogToLoggerWithEnabledLogLevel/threads:4 115 ns 458 ns 1610576
LogToLoggerWithEnabledLogLevel/threads:8 87 ns 681 ns 1083992

Change-Id: I040327e1be9dca10a219df726d530d5b942d01dd
Reviewed-on: http://review.couchbase.org/89170
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Daniel Owen <owend@couchbase.com>

show more ...

MB-25624[2]: Implement custom sink which deduplicates log messages

Plug dedupe_sink into spdlogger and remove redundant code.

Change-Id: I81b0487754a90e57a7135684adc262be54fdb90f

MB-25624[2]: Implement custom sink which deduplicates log messages

Plug dedupe_sink into spdlogger and remove redundant code.

Change-Id: I81b0487754a90e57a7135684adc262be54fdb90f
Reviewed-on: http://review.couchbase.org/86828
Reviewed-by: Dave Rigby <daver@couchbase.com>
Reviewed-by: Tim Bradgate <tim.bradgate@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

MB-27724: Install libmemcached_logger in <root>/lib

The RPATH on macosx is incorrectly specified and includes
the "install path" as a fallback. It should be replaced
with @executable

MB-27724: Install libmemcached_logger in <root>/lib

The RPATH on macosx is incorrectly specified and includes
the "install path" as a fallback. It should be replaced
with @executable_path instead.

Change-Id: I788df3e50a06c856914c3656e53eb956ba581c0c
Reviewed-on: http://review.couchbase.org/88393
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

MB-27673: Add option to bypass building unit tests

Add kv_engine into COUCHBASE_DISABLED_UNIT_TESTS to skip building
all of the unit tests.

Change-Id: I2eca3773ba0cfe71c2df4d48e

MB-27673: Add option to bypass building unit tests

Add kv_engine into COUCHBASE_DISABLED_UNIT_TESTS to skip building
all of the unit tests.

Change-Id: I2eca3773ba0cfe71c2df4d48e46f551686a71076
Reviewed-on: http://review.couchbase.org/88354
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Jim Walker <jim@couchbase.com>

show more ...

MB-27665: Fix problems with EMFILE for spdlogger

Continue to log to the same file if we encounter problems opening
the next log file.

Change-Id: Ia35a5aa4f6f320704725ae1c987bd58

MB-27665: Fix problems with EMFILE for spdlogger

Continue to log to the same file if we encounter problems opening
the next log file.

Change-Id: Ia35a5aa4f6f320704725ae1c987bd581d605a637
Reviewed-on: http://review.couchbase.org/88134
Reviewed-by: Tim Bradgate <tim.bradgate@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

Refactor: move the spdlogger to memcached_logger

Instead of doing dynamic loading of the logger it is linked into
the binary.

Change-Id: I1577397cc29ecd4f73a50836391a985bac94fe7

Refactor: move the spdlogger to memcached_logger

Instead of doing dynamic loading of the logger it is linked into
the binary.

Change-Id: I1577397cc29ecd4f73a50836391a985bac94fe7f
Reviewed-on: http://review.couchbase.org/88092
Reviewed-by: Dave Rigby <daver@couchbase.com>
Reviewed-by: Tim Bradgate <tim.bradgate@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...