MB-28735: Skip HandleOpenFileErrors under UBSan

This test fails under UBSan, when spdlog fails to open a new file (in
custom_rotating_file_sink::_sink_it):

common.h:139:9: r

MB-28735: Skip HandleOpenFileErrors under UBSan

This test fails under UBSan, when spdlog fails to open a new file (in
custom_rotating_file_sink::_sink_it):

common.h:139:9: runtime error: member access within address <ADDR> which does not point to an object of type 'spdlog::spdlog_ex'
<ADDR>: note: object has invalid vptr

Examining <ADDR> in a debugger indicates a valid object, and I cannot
identify any issue here. Additinally, this code is only called in an
error path if we have run out of file descriptors (so likely Bad
Things will already be happing).

As such, suppress this warning by skipping the test under UBSan.

Change-Id: Ieebe7992bbd1a0c038c2db99536032e38e74423f
Reviewed-on: http://review.couchbase.org/91097
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

MB-27986,MB-28225,MB-27882: Remove log dedupe and flush at 200ms

This patch removes the deduplication logic from the logger as
we want to make sure that we dump (and flush) our log "imme

MB-27986,MB-28225,MB-27882: Remove log dedupe and flush at 200ms

This patch removes the deduplication logic from the logger as
we want to make sure that we dump (and flush) our log "immediately"
to the files. All of the log is written in a dedicated thread
which use a buffered FILE stream. We don't want to flush this
stream for _every_ log message (as that is typically 100 bytes
or so) which would cause a significant increse of file IO.

Instead we want the logger to keep writing data to the FILE
pointer, but if no log message is provided within 200ms we'll
cause FILE stream to be flushed to disk. This means that we'll
try to fill up the FILE streams buffer issue fewer actual
filesystem calls, but at the same time we won't keep the data
in those buffers hanging around for a long time.

Change-Id: I1f320dc540d39d9890c8ba84e89b0ce1f203aaa1
Reviewed-on: http://review.couchbase.org/90146
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

Simplify logger tests

Increase the file rotation size to 20MB for the "basic" tests
to avoid having to deal with multiple files. Create a separate
test batch for all of the tests whi

Simplify logger tests

Increase the file rotation size to 20MB for the "basic" tests
to avoid having to deal with multiple files. Create a separate
test batch for all of the tests which expects file rotation
with a 2k file rotation threshold.

Disable time-based flushing by setting sleeptime to 0

Dump the log contents if the deduplication tests fail (to ease
debugging from the CV machines)

Change-Id: Iec7f9ae2e0aabacf60008f140e71ee967cb3904c
Reviewed-on: http://review.couchbase.org/89320
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

MB-27882: Tighten up logger tests

The deduplication tests was a bit flaky as they would potentially
fail under load as the logger could flush the data to the logfiles
due to the 1 se

MB-27882: Tighten up logger tests

The deduplication tests was a bit flaky as they would potentially
fail under load as the logger could flush the data to the logfiles
due to the 1 sec threshold for the automatic flushing

Change-Id: If71307e68acd2f4ec2b43c4a3f75b4ff3a45bb8e
Reviewed-on: http://review.couchbase.org/89246
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

Create unit test to verify that the logger formats correctly

We want to move the unit tests to use the spdlogging api directly.
Create a new unit tests which use the "old" API, and one w

Create unit test to verify that the logger formats correctly

We want to move the unit tests to use the spdlogging api directly.
Create a new unit tests which use the "old" API, and one which
use the spdlogger API.

Change-Id: Ib49adbca8aa05e639fb27cb5f1cf67c860e88082
Reviewed-on: http://review.couchbase.org/89244
Reviewed-by: Dave Rigby <daver@couchbase.com>
Reviewed-by: Tim Bradgate <tim.bradgate@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

Simplify helper function to search a file

Just memorymap the entire file and use std::search instead
of parsing the file line by line.

Change-Id: I94137cb84bb9e4368c11a58ffef836

Simplify helper function to search a file

Just memorymap the entire file and use std::search instead
of parsing the file line by line.

Change-Id: I94137cb84bb9e4368c11a58ffef83666a1b4e8dd
Reviewed-on: http://review.couchbase.org/89243
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>
Reviewed-by: Tim Bradgate <tim.bradgate@couchbase.com>

show more ...

Don't require the server API when creating logger

It's only used for setting the log level, and we may do that
later on if we really want to change it after we've created
the logger.

Don't require the server API when creating logger

It's only used for setting the log level, and we may do that
later on if we really want to change it after we've created
the logger. (If verbosity is being used it is automatically
updated anyway).

Change-Id: Ided21e53e016980e04e3acb9d601de8dd282b708
Reviewed-on: http://review.couchbase.org/89169
Reviewed-by: Tim Bradgate <tim.bradgate@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

Remove unused part of the old logger API

Change-Id: I111500132370add975c23b6d69a1358816006fb0
Reviewed-on: http://review.couchbase.org/89067
Reviewed-by: Paolo Cocchi <paolo.cocchi@c

Remove unused part of the old logger API

Change-Id: I111500132370add975c23b6d69a1358816006fb0
Reviewed-on: http://review.couchbase.org/89067
Reviewed-by: Paolo Cocchi <paolo.cocchi@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

Refactor: Remove unused argument to shutdown

Change-Id: I42034b142e8b1e574db61d18e6368012afb6f090
Reviewed-on: http://review.couchbase.org/89064
Reviewed-by: Paolo Cocchi <paolo.cocc

Refactor: Remove unused argument to shutdown

Change-Id: I42034b142e8b1e574db61d18e6368012afb6f090
Reviewed-on: http://review.couchbase.org/89064
Reviewed-by: Paolo Cocchi <paolo.cocchi@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

Refactor: Remove support for extensions in configuration

It was only used by the stdin_listener. Drop that part of the
API, and make the stdin_listener a part of the core.

Chang

Refactor: Remove support for extensions in configuration

It was only used by the stdin_listener. Drop that part of the
API, and make the stdin_listener a part of the core.

Change-Id: I32fa173814b18e0018dfa2979ded853ce4112076
Reviewed-on: http://review.couchbase.org/89020
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

Use spdlogger for everything

Remove the old stderr and stdout loggers and use the spdlogger
everywhere (and remove the ability to install custom loggers).

Change-Id: Ibd5a5d6775

Use spdlogger for everything

Remove the old stderr and stdout loggers and use the spdlogger
everywhere (and remove the ability to install custom loggers).

Change-Id: Ibd5a5d6775970e33664341f2476dd2dee709abff
Reviewed-on: http://review.couchbase.org/88781
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

MB-25624[2]: Implement custom sink which deduplicates log messages

Plug dedupe_sink into spdlogger and remove redundant code.

Change-Id: I81b0487754a90e57a7135684adc262be54fdb90f

MB-25624[2]: Implement custom sink which deduplicates log messages

Plug dedupe_sink into spdlogger and remove redundant code.

Change-Id: I81b0487754a90e57a7135684adc262be54fdb90f
Reviewed-on: http://review.couchbase.org/86828
Reviewed-by: Dave Rigby <daver@couchbase.com>
Reviewed-by: Tim Bradgate <tim.bradgate@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

Refactor: Add a method to fetch the logger

A number of cmake targets had to be updated to reference
the memcached_logger library to have the correct include
path being propagated.

Refactor: Add a method to fetch the logger

A number of cmake targets had to be updated to reference
the memcached_logger library to have the correct include
path being propagated.

Change-Id: I36e905b57f86e714ec0b05c36497162b3b5804aa
Reviewed-on: http://review.couchbase.org/88365
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

Remove (almost) unused DETAIL log level

The detail level was only used to print out when we moved to
closing state (or all state transitions in the state machinery).

It is alway

Remove (almost) unused DETAIL log level

The detail level was only used to print out when we moved to
closing state (or all state transitions in the state machinery).

It is always disabled in production, and if enabled it'll generate
so much log data that it is pretty much useless ;)

Change-Id: Ie487674274bb270998afb99e105607a164717175
Reviewed-on: http://review.couchbase.org/88233
Reviewed-by: Tim Bradgate <tim.bradgate@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...

MB-27665: Fix problems with EMFILE for spdlogger

Continue to log to the same file if we encounter problems opening
the next log file.

Change-Id: Ia35a5aa4f6f320704725ae1c987bd58

MB-27665: Fix problems with EMFILE for spdlogger

Continue to log to the same file if we encounter problems opening
the next log file.

Change-Id: Ia35a5aa4f6f320704725ae1c987bd581d605a637
Reviewed-on: http://review.couchbase.org/88134
Reviewed-by: Tim Bradgate <tim.bradgate@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...