MB-29377: Change return signature of cb::logger::get()

Change the return signature of getting the logger object
to return a direct pointer to the object. This has a notable
affect on

MB-29377: Change return signature of cb::logger::get()

Change the return signature of getting the logger object
to return a direct pointer to the object. This has a notable
affect on the logger benchmark results, in the case where we are
A) getting the logger or B) logging at a level which is disabled.

Before change:

Run on (8 X 2300 MHz CPU s)
2018-04-25 15:37:36
----------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations
----------------------------------------------------------------------------------------------
LoggerBench_Blackhole/GetLogger/threads:1 13 ns 13 ns 50902427
LoggerBench_Blackhole/GetLogger/threads:16 38 ns 320 ns 2124144
LoggerBench/LogToLoggerWithDisabledLogLevel/threads:1 13 ns 13 ns 50811533
LoggerBench/LogToLoggerWithDisabledLogLevel/threads:16 54 ns 483 ns 1284736
LoggerBench/LogToLoggerWithEnabledLogLevel/threads:1 475 ns 473 ns 1485288
LoggerBench/LogToLoggerWithEnabledLogLevel/threads:16 606 ns 1065 ns 550688

After change:

Run on (8 X 2300 MHz CPU s)
2018-04-25 15:37:36
----------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations
----------------------------------------------------------------------------------------------
LoggerBench_Blackhole/GetLogger/threads:1 2 ns 2 ns 335836112
LoggerBench_Blackhole/GetLogger/threads:16 0 ns 3 ns 220889872
LoggerBench/LogToLoggerWithDisabledLogLevel/threads:1 2 ns 2 ns 334633624
LoggerBench/LogToLoggerWithDisabledLogLevel/threads:16 0 ns 4 ns 156165040
LoggerBench/LogToLoggerWithEnabledLogLevel/threads:1 472 ns 471 ns 1490142
LoggerBench/LogToLoggerWithEnabledLogLevel/threads:16 423 ns 1075 ns 590576

Change-Id: I82ebd5c1776ff0efad6b9316e1ff35a20887892f
Reviewed-on: http://review.couchbase.org/93286
Reviewed-by: Dave Rigby <daver@couchbase.com>
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

MB-29377: Improve spdlog usage within kv_engine

[1]: Add isInitialized method to cb::logger and use this in
auditd.cc instead of cb::logger::get
[2]: Replace all usages of cb::l

MB-29377: Improve spdlog usage within kv_engine

[1]: Add isInitialized method to cb::logger and use this in
auditd.cc instead of cb::logger::get
[2]: Replace all usages of cb::logger::get in daemon with
the LOG_* macros
[3]: Add default log level to the logger config object and
use this to set the log level during initialization

Change-Id: I3728ec786eef7729609f65a36a35267a3b7e2b78
Reviewed-on: http://review.couchbase.org/93201
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

MB-27986,MB-28225,MB-27882: Remove log dedupe and flush at 200ms

This patch removes the deduplication logic from the logger as
we want to make sure that we dump (and flush) our log "imme

MB-27986,MB-28225,MB-27882: Remove log dedupe and flush at 200ms

This patch removes the deduplication logic from the logger as
we want to make sure that we dump (and flush) our log "immediately"
to the files. All of the log is written in a dedicated thread
which use a buffered FILE stream. We don't want to flush this
stream for _every_ log message (as that is typically 100 bytes
or so) which would cause a significant increse of file IO.

Instead we want the logger to keep writing data to the FILE
pointer, but if no log message is provided within 200ms we'll
cause FILE stream to be flushed to disk. This means that we'll
try to fill up the FILE streams buffer issue fewer actual
filesystem calls, but at the same time we won't keep the data
in those buffers hanging around for a long time.

Change-Id: I1f320dc540d39d9890c8ba84e89b0ce1f203aaa1
Reviewed-on: http://review.couchbase.org/90146
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

Don't require the server API when creating logger

It's only used for setting the log level, and we may do that
later on if we really want to change it after we've created
the logger.

Don't require the server API when creating logger

It's only used for setting the log level, and we may do that
later on if we really want to change it after we've created
the logger. (If verbosity is being used it is automatically
updated anyway).

Change-Id: Ided21e53e016980e04e3acb9d601de8dd282b708
Reviewed-on: http://review.couchbase.org/89169
Reviewed-by: Tim Bradgate <tim.bradgate@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

Don't use an async logger for blackhole logger

There is an issue [1] during shutdown on windows for async loggers
causing a deadlock.

There isn't much of a benefit putting the e

Don't use an async logger for blackhole logger

There is an issue [1] during shutdown on windows for async loggers
causing a deadlock.

There isn't much of a benefit putting the entry into a queue to
be dropped by another thread anyway ;-)

The shutdown method should drop all loggers (and clients using it
should know that after calling the method there is no loggers)

[1] https://github.com/gabime/spdlog/wiki/6.-Asynchronous-logging

Change-Id: I7fd90a037145626fe27a92d4769702b0318806fc
Reviewed-on: http://review.couchbase.org/89162
Reviewed-by: Tim Bradgate <tim.bradgate@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

Make flush and shutdown available through the "new" api

Change-Id: Ie5fab68bbbee30f2db74d78e0ead135be118656b
Reviewed-on: http://review.couchbase.org/89065
Reviewed-by: Paolo Cocchi

Make flush and shutdown available through the "new" api

Change-Id: Ie5fab68bbbee30f2db74d78e0ead135be118656b
Reviewed-on: http://review.couchbase.org/89065
Reviewed-by: Paolo Cocchi <paolo.cocchi@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

Update the LOG_ macros to not call the logger function when not enabled

The problem with calling:

LOG_DEBUG("Foo {} {} {}", getId(), getBlah(), convert());

Is that all o

Update the LOG_ macros to not call the logger function when not enabled

The problem with calling:

LOG_DEBUG("Foo {} {} {}", getId(), getBlah(), convert());

Is that all of the methods provided on the LOG_DEBUG line would have
to be called even if the logger would drop the log message.

Tweak the log macros so that we check if the level is enabled
_before_ trying to call the method.

Change-Id: I7a67e54ff3189bfad4d0ea8ee25a7b0dd06bf64c
Reviewed-on: http://review.couchbase.org/89171
Reviewed-by: Tim Bradgate <tim.bradgate@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

Use logger directly in the core

Changed the LOG_ macros used in the core and call
them directly instead of going through the old
logger interface using the printf-style of formatting

Use logger directly in the core

Changed the LOG_ macros used in the core and call
them directly instead of going through the old
logger interface using the printf-style of formatting

Change-Id: I7927bfdd98a843a605fd418ab343f166c25c7b74
Reviewed-on: http://review.couchbase.org/88814
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

Use spdlogger for everything

Remove the old stderr and stdout loggers and use the spdlogger
everywhere (and remove the ability to install custom loggers).

Change-Id: Ibd5a5d6775

Use spdlogger for everything

Remove the old stderr and stdout loggers and use the spdlogger
everywhere (and remove the ability to install custom loggers).

Change-Id: Ibd5a5d6775970e33664341f2476dd2dee709abff
Reviewed-on: http://review.couchbase.org/88781
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

Replace blackhole logger with spdlogs null sink

Add a new property "console" to the logger config
which may be set to false (it is enabled by default)
If console is set to false we'l

Replace blackhole logger with spdlogs null sink

Add a new property "console" to the logger config
which may be set to false (it is enabled by default)
If console is set to false we'll use null sink instead
of stderr.

As an extra bonus we'll use the colored sinks when
the unit_test option is set to true.

Change-Id: I690b7022e2cef04b98fd8ebc4ddd39d9913a0d42
Reviewed-on: http://review.couchbase.org/88493
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

Refactor: Don't use the on_log_level from spdlogger

Now that we can fetch the logger object we may set it
directly on the underlying object rather than using
the callback method (The

Refactor: Don't use the on_log_level from spdlogger

Now that we can fetch the logger object we may set it
directly on the underlying object rather than using
the callback method (The callback is still fired
as ep-engine tries to cache the current verbosity
setting)

Change-Id: I8d20800246d432dd95c29a4342dea5357e514188
Reviewed-on: http://review.couchbase.org/88491
Reviewed-by: Tim Bradgate <tim.bradgate@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

Refactor: Add a method to fetch the logger

A number of cmake targets had to be updated to reference
the memcached_logger library to have the correct include
path being propagated.

Refactor: Add a method to fetch the logger

A number of cmake targets had to be updated to reference
the memcached_logger library to have the correct include
path being propagated.

Change-Id: I36e905b57f86e714ec0b05c36497162b3b5804aa
Reviewed-on: http://review.couchbase.org/88365
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

Refactor: move the spdlogger to memcached_logger

Instead of doing dynamic loading of the logger it is linked into
the binary.

Change-Id: I1577397cc29ecd4f73a50836391a985bac94fe7

Refactor: move the spdlogger to memcached_logger

Instead of doing dynamic loading of the logger it is linked into
the binary.

Change-Id: I1577397cc29ecd4f73a50836391a985bac94fe7f
Reviewed-on: http://review.couchbase.org/88092
Reviewed-by: Dave Rigby <daver@couchbase.com>
Reviewed-by: Tim Bradgate <tim.bradgate@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...