MB-30041: Remove linking of cJSON wherever possible

Remove the linking of cJSON from any objects that no longer
need it.

Change-Id: I2fe8354f469283faf3a47b91ee811686270dd278

MB-30041: Remove linking of cJSON wherever possible

Remove the linking of cJSON from any objects that no longer
need it.

Change-Id: I2fe8354f469283faf3a47b91ee811686270dd278
Reviewed-on: http://review.couchbase.org/104392
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Daniel Owen <owend@couchbase.com>
Reviewed-by: Trond Norbye <trond.norbye@couchbase.com>

show more ...

MB-30041: Migrate logger to nlohmann::json

Migrate settings.cc to nlohmann::json by migrating individual
handlers, which requires interim parsing of cJSON objects
to nlohmann::json o

MB-30041: Migrate logger to nlohmann::json

Migrate settings.cc to nlohmann::json by migrating individual
handlers, which requires interim parsing of cJSON objects
to nlohmann::json objects.

Migrate logger and logger_config to nlohmann::json.

Change-Id: I2fc8c649e4aa5d50fed93a19ca01fa36e74adf0c
Reviewed-on: http://review.couchbase.org/100182
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

ASan: Add missing add_sanitizers() to test executables

Various test executables links against libraries which enable
AddressSanitizer - as such we need to enable ASan on the target itsel

ASan: Add missing add_sanitizers() to test executables

Various test executables links against libraries which enable
AddressSanitizer - as such we need to enable ASan on the target itself
to avoid linker errors when building with CB_ADDRESSSANITIZER=2
(i.e. explicit target linking). For example:

[ 65%] Linking CXX executable ../../tracing_bench
/usr/bin/ld: ../../libmcd_tracing.a(tracer.cc.o): undefined reference to symbol '__asan_report_store4'
//usr/local/lib64/libasan.so.4: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status

Change-Id: Ia4257bb664ae2472e3c8f5de8af0b48c5463289e
Reviewed-on: http://review.couchbase.org/101541
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Daniel Owen <owend@couchbase.com>

show more ...

Optimise logger.h header

- Move logger::Config into it's own file (logger_config.h), changing
all users which just need the logger Config to #include
logger_config.h instead of l

Optimise logger.h header

- Move logger::Config into it's own file (logger_config.h), changing
all users which just need the logger Config to #include
logger_config.h instead of logger.h

- Minimise includes of logger.h; removing where no longer needed in
header files and moving instead to necessary .cc files.

Clean build times:

$ ninja clean
$ CCACHE_DISABLE=true time ninja kv_engine/all -j6

Before (6a3ec65a7):
254.75 real 1426.10 user 78.62 sys
user+sys: 1504

After (d8e692c45):
243.33 real 1366.29 user 76.51 sys
user+sys: 1442

Results in 4% speedup in user+sys build (62s reduction).

Change-Id: I81625a830c3bd7fb7bd0a62a98138334fc3887f5
Reviewed-on: http://review.couchbase.org/99954
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

MB-31325: Add Google Benchmark include directory

This change is in preparation of the upgrade of the Google Benchmark
library.
Multiple library commits between our current revision a

MB-31325: Add Google Benchmark include directory

This change is in preparation of the upgrade of the Google Benchmark
library.
Multiple library commits between our current revision and the new
revision change the way GBench is built.
That requires that we specify the 'target_include_directories' in the
compilation units where we use the library.

Change-Id: Ia7766d9c3868e3a604d3637a1225c37ade3f8f77
Reviewed-on: http://review.couchbase.org/99669
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

MB-30302: Spdlog style formatting for EP Engine

Modification of SERVER_LOG_API and EP Engine to allow for spdlog style
formatting in EP Engine. Requires passing a pointer to the spdlog l

MB-30302: Spdlog style formatting for EP Engine

Modification of SERVER_LOG_API and EP Engine to allow for spdlog style
formatting in EP Engine. Requires passing a pointer to the spdlog logger
object to create a new logger to format the log messages and prepend with
the engine name before using the original spdlog logger to print them.

Change-Id: I3711229fba45002d07bf40e2c6cb7f796e91a7bf
Reviewed-on: http://review.couchbase.org/96605
Reviewed-by: Dave Rigby <daver@couchbase.com>
Reviewed-by: Tim Bradgate <tim.bradgate@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

Explicitly enable ASan for KV-Engine targets needed for cluster_run

To be used with the new CB_ADDRESSSANITIZER=2 mode; enable ASan for
the targets needed to run KV-Engine as part of a l

Explicitly enable ASan for KV-Engine targets needed for cluster_run

To be used with the new CB_ADDRESSSANITIZER=2 mode; enable ASan for
the targets needed to run KV-Engine as part of a larger cluster-run;
but without having to enable for all targets.

For example; enabling ASan for libcouchstore.so causes problems for
the view engine as the main executable which loads libcouchstore
(Erlang VM) doesn't have ASan enabled and hence it crashes on startup.

Change-Id: Ieb77f0622d581e5d7767b2a4fba66853416dd342
Reviewed-on: http://review.couchbase.org/90866
Reviewed-by: Jim Walker <jim@couchbase.com>
Reviewed-by: Tim Bradgate <tim.bradgate@couchbase.com>
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

MB-27986,MB-28225,MB-27882: Remove log dedupe and flush at 200ms

This patch removes the deduplication logic from the logger as
we want to make sure that we dump (and flush) our log "imme

MB-27986,MB-28225,MB-27882: Remove log dedupe and flush at 200ms

This patch removes the deduplication logic from the logger as
we want to make sure that we dump (and flush) our log "immediately"
to the files. All of the log is written in a dedicated thread
which use a buffered FILE stream. We don't want to flush this
stream for _every_ log message (as that is typically 100 bytes
or so) which would cause a significant increse of file IO.

Instead we want the logger to keep writing data to the FILE
pointer, but if no log message is provided within 200ms we'll
cause FILE stream to be flushed to disk. This means that we'll
try to fill up the FILE streams buffer issue fewer actual
filesystem calls, but at the same time we won't keep the data
in those buffers hanging around for a long time.

Change-Id: I1f320dc540d39d9890c8ba84e89b0ce1f203aaa1
Reviewed-on: http://review.couchbase.org/90146
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

Add logger benchmark

Add a small micro benchmark for the logger testing 3 different scenarios:

1) GetLogger - Test the cost of fetching the logger object (means copying

Add logger benchmark

Add a small micro benchmark for the logger testing 3 different scenarios:

1) GetLogger - Test the cost of fetching the logger object (means copying
the shared pointer)

2) LogToLoggerWithDisabeldLogLevel - Test the cost of the log statements
we have in our code when the logger will drop the log
statement becuase the log level isn't enabled (note:
this does _NOT_ include the cost it could be to generate
the parameters passed in!)

3) LogToLoggerWithEnabledLogLevel - Test the cost of adding the log
statement into the queue of statements to pass off to the
async logger. Note that the async logger will drop all
of them. The test will probably return "incorrect" results
if we manage to fill up the queue between the two threads.

Run on (8 X 2500 MHz CPU s)
2018-02-09 11:41:04
---------------------------------------------------------------------------------
Benchmark Time CPU Iterations
---------------------------------------------------------------------------------
GetLogger/threads:1 18 ns 18 ns 38555385
GetLogger/threads:2 46 ns 92 ns 7759112
GetLogger/threads:4 50 ns 198 ns 3510752
GetLogger/threads:8 53 ns 414 ns 1910896
LogToLoggerWithDisabeldLogLevel/threads:1 26 ns 26 ns 26377668
LogToLoggerWithDisabeldLogLevel/threads:2 57 ns 114 ns 6165690
LogToLoggerWithDisabeldLogLevel/threads:4 76 ns 304 ns 2282880
LogToLoggerWithDisabeldLogLevel/threads:8 59 ns 462 ns 1506520
LogToLoggerWithEnabledLogLevel/threads:1 100 ns 100 ns 7002031
LogToLoggerWithEnabledLogLevel/threads:2 125 ns 249 ns 2677172
LogToLoggerWithEnabledLogLevel/threads:4 115 ns 458 ns 1610576
LogToLoggerWithEnabledLogLevel/threads:8 87 ns 681 ns 1083992

Change-Id: I040327e1be9dca10a219df726d530d5b942d01dd
Reviewed-on: http://review.couchbase.org/89170
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Daniel Owen <owend@couchbase.com>

show more ...

MB-25624[2]: Implement custom sink which deduplicates log messages

Plug dedupe_sink into spdlogger and remove redundant code.

Change-Id: I81b0487754a90e57a7135684adc262be54fdb90f

MB-25624[2]: Implement custom sink which deduplicates log messages

Plug dedupe_sink into spdlogger and remove redundant code.

Change-Id: I81b0487754a90e57a7135684adc262be54fdb90f
Reviewed-on: http://review.couchbase.org/86828
Reviewed-by: Dave Rigby <daver@couchbase.com>
Reviewed-by: Tim Bradgate <tim.bradgate@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

MB-27724: Install libmemcached_logger in <root>/lib

The RPATH on macosx is incorrectly specified and includes
the "install path" as a fallback. It should be replaced
with @executable

MB-27724: Install libmemcached_logger in <root>/lib

The RPATH on macosx is incorrectly specified and includes
the "install path" as a fallback. It should be replaced
with @executable_path instead.

Change-Id: I788df3e50a06c856914c3656e53eb956ba581c0c
Reviewed-on: http://review.couchbase.org/88393
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

MB-27673: Add option to bypass building unit tests

Add kv_engine into COUCHBASE_DISABLED_UNIT_TESTS to skip building
all of the unit tests.

Change-Id: I2eca3773ba0cfe71c2df4d48e

MB-27673: Add option to bypass building unit tests

Add kv_engine into COUCHBASE_DISABLED_UNIT_TESTS to skip building
all of the unit tests.

Change-Id: I2eca3773ba0cfe71c2df4d48e46f551686a71076
Reviewed-on: http://review.couchbase.org/88354
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Jim Walker <jim@couchbase.com>

show more ...

MB-27665: Fix problems with EMFILE for spdlogger

Continue to log to the same file if we encounter problems opening
the next log file.

Change-Id: Ia35a5aa4f6f320704725ae1c987bd58

MB-27665: Fix problems with EMFILE for spdlogger

Continue to log to the same file if we encounter problems opening
the next log file.

Change-Id: Ia35a5aa4f6f320704725ae1c987bd581d605a637
Reviewed-on: http://review.couchbase.org/88134
Reviewed-by: Tim Bradgate <tim.bradgate@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

Refactor: move the spdlogger to memcached_logger

Instead of doing dynamic loading of the logger it is linked into
the binary.

Change-Id: I1577397cc29ecd4f73a50836391a985bac94fe7

Refactor: move the spdlogger to memcached_logger

Instead of doing dynamic loading of the logger it is linked into
the binary.

Change-Id: I1577397cc29ecd4f73a50836391a985bac94fe7f
Reviewed-on: http://review.couchbase.org/88092
Reviewed-by: Dave Rigby <daver@couchbase.com>
Reviewed-by: Tim Bradgate <tim.bradgate@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...