MB-12762: Fix build break introduced with backport

Change-Id: Ica414b586f9f1e154fa01e478e3e2b939945b638
Reviewed-on: http://review.couchbase.org/43624
Reviewed-by: Trond Norbye <tron

MB-12762: Fix build break introduced with backport

Change-Id: Ica414b586f9f1e154fa01e478e3e2b939945b638
Reviewed-on: http://review.couchbase.org/43624
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

MB-12762: Close SSL connection on error during handshake

If a receive error occured during SSL handshake (e.g. received
incorrect handshake data) then instead of closing the connection (

MB-12762: Close SSL connection on error during handshake

If a receive error occured during SSL handshake (e.g. received
incorrect handshake data) then instead of closing the connection (as
expected) memcached would belive it needed to retry and return the
connection to the conn_read state. This resulted in the state machine
for the connection essentially looping forever attempting to read
data. This has two observabl side-effects:

1) A connection slot is essentially consumed (forever) by the failed
connection attempt.

2) The frontend thread that the failed connection was assigned to will
spin (at 100% CPU), if no other connections owned by the worker have
any events waiting.

(Note for (2), the CPU isn't "consumed" - other connections if present
*will* be serviced as normal, the issue is simply that the failed
connection immediately posts an event to retry the read, which
libevent will immediately schedule if there are no other events
(i.e. other connections) ready).

Fix is to ensure that the ssl.error status is checked after draining
the SSL recv pipe, and if set then reseting the connection.

Change-Id: I1775b7ad16c6e12fc619da2ba15c93f6e07cd096
Reviewed-on: http://review.couchbase.org/43608
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Dave Rigby <daver@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...

testapp failure observed when linked against a newer OpenSSL (seen on OSX).

The fix though is not actually a library issue but just an issue with our send
code returning the wrong send l

testapp failure observed when linked against a newer OpenSSL (seen on OSX).

The fix though is not actually a library issue but just an issue with our send
code returning the wrong send length, we must return the SSL_write number,
not the send number (the send number will be bigger as SSL inflates the data
by some amount).

Change-Id: I93f33c9ec791f3f174f624afd8d8adaf6c5f5d48
Reviewed-on: http://review.couchbase.org/40532
Reviewed-by: Dave Rigby <daver@couchbase.com>
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

Fix compile warning from clang

Change-Id: If5bfed8fd995f4feeceafe4f6cd618e5a259dd81
Reviewed-on: http://review.couchbase.org/40488
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>

Fix compile warning from clang

Change-Id: If5bfed8fd995f4feeceafe4f6cd618e5a259dd81
Reviewed-on: http://review.couchbase.org/40488
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

[MB-11203] SSL-enabled memcached will hang...

A large pipeline input lead to memcached effectively
performing a under-read of the input buffer. This occured
when memcached moved to

[MB-11203] SSL-enabled memcached will hang...

A large pipeline input lead to memcached effectively
performing a under-read of the input buffer. This occured
when memcached moved to conn_waiting as part of yield, we
only check rbuf for data when a SSL_peek is needed for SSL
connections.

Also on the drain_recv code we could get into a situation
where we'd read everything off the network but didn't quite
handle the SSL_read -1 error correctly. Again we could end up
yielding and never wake up again because there's no more data
on the network, yet there's data in our intermediate ssl read
buffer.

testapp changes are required because this defect occured when
a large single sendto occurs. The old SSL code in testapp
used openssl methods to send/recv, which actually chunked up
large pipelines using smaller writes, hiding the bug because
we kept stimulating memcached.

New code allows testapp to take full control of sends using
whatever chunking scheme we wish.

Also removed some time tests which weren't needed and were
causing some occasional failures because they were a little
intolerant.

Change-Id: Ie2a3374fc32ad57a122040ad359e3148d6bdeffd
Reviewed-on: http://review.couchbase.org/40417
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

[MB-11203] Enable SSL for all tests that use a socket.

Add some more focused stream/pipeline tests.

Overall increase of about 2% code coverage in memcached.c

Change-Id: Ia0

[MB-11203] Enable SSL for all tests that use a socket.

Add some more focused stream/pipeline tests.

Overall increase of about 2% code coverage in memcached.c

Change-Id: Ia0b8ffbb84e53e0bd38eaf3d3641ae92161e2d92
Reviewed-on: http://review.couchbase.org/40185
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

[MB-11548] Memcached does not handle going back in time.

1) Update testapp so that we can expose the bug.

The idea here is to move us to using a platform method that abstracts getti

[MB-11548] Memcached does not handle going back in time.

1) Update testapp so that we can expose the bug.

The idea here is to move us to using a platform method that abstracts gettimeofday.
With that in place we can apply time shifts in a programmatic fashion using the
memcached protocol extension framework.

Testapp uses testapp_extension.so to send time offset values into memcached and
cause a jump in time.

2) Add mc_time module and move memcached.c methods + add bug fix.
- This module provides the legacy API but
a) current_time is now monotonic and immune to system clock changes
b) allow correct absolute system time calculations without the I/O path needing
a system call. This works alongside real system clock changes.
c) validate that existing expiry tests and new expiry tests are now operational.

3) Remove any gettimeofday reference from memcached so that we use the new platform method.

4) As part of final testing, some system tests were performed where I manually
changed the real system clock using "sudo date" commands.

Change-Id: I2048c536f344050191feeed16635196f44b70b37
Reviewed-on: http://review.couchbase.org/39813
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

Refactor: rename UPR to DCP

Change-Id: Idc9f5c4561fbf5bf011ca1dbed5ad00a64c67fa0
Reviewed-on: http://review.couchbase.org/39747
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>

Refactor: rename UPR to DCP

Change-Id: Idc9f5c4561fbf5bf011ca1dbed5ad00a64c67fa0
Reviewed-on: http://review.couchbase.org/39747
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

MB-11772: Add `release_free_memory` ioctl and `mcctl` program to access them

Add two new binary protocol commands:

* PROTOCOL_BINARY_CMD_IOCTL_GET - Retrieve a property from the s

MB-11772: Add `release_free_memory` ioctl and `mcctl` program to access them

Add two new binary protocol commands:

* PROTOCOL_BINARY_CMD_IOCTL_GET - Retrieve a property from the server
* PROTOCOL_BINARY_CMD_IOCTL_SET - Modify a property on the server.

Using these, implement a `set release_free_memory` command which will
request the memory allocator to return to the OS any free memory
inside the process.

To access this new functionality, add a `mcctl` command to set/get CTL
properties. Example usage (and currently only supported
functionality):

./mcctl -h localhost:12000 set release_free_memory

Change-Id: I35c40ff9c41c0f00802ff9db9202865ec146f46c
Reviewed-on: http://review.couchbase.org/39608
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

Fix bug introduced by MB-11259 [Buffer acknowledgment should have extlen of 4]

Update testapp so `make test` passes. Bug introduced by
b4a135a2e6c25c01dec9b09e51ca2ac6e838bcb8

C

Fix bug introduced by MB-11259 [Buffer acknowledgment should have extlen of 4]

Update testapp so `make test` passes. Bug introduced by
b4a135a2e6c25c01dec9b09e51ca2ac6e838bcb8

Change-Id: I58e164a3670562d4ec0822ab6d566ae4f90d4e27
Reviewed-on: http://review.couchbase.org/38966
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

Initialize control cas to 0xdeadbeef

If its default value is 0 there is no way to change it without
using 0 as the cas value (which means override) introducing
the possibility for ra

Initialize control cas to 0xdeadbeef

If its default value is 0 there is no way to change it without
using 0 as the cas value (which means override) introducing
the possibility for race conditions. With a non-null value it
may be changed by someone who requested its value and protects
against race conditions.

Allow for using cas value 0 to override cas for setting the cas
(to make it behave consistently with the rest of the cas usage)

Change-Id: I637104df4f65527d8ad10636c01a0e7f53930c53
Reviewed-on: http://review.couchbase.org/38784
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

MB-11394: Disable use of the datatype feature

Introduced a new configuration attribute in the configuration
to enable it: "datatype_supported" : true

Change-Id: I0de2f5315de94ee

MB-11394: Disable use of the datatype feature

Introduced a new configuration attribute in the configuration
to enable it: "datatype_supported" : true

Change-Id: I0de2f5315de94eec14670218505dc36406303f3d
Reviewed-on: http://review.couchbase.org/38228
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

MB-11284: Support "admin" user in memcached core

Change-Id: I8e990e147ea870857483638bd1e9fa8e6fea551b
Reviewed-on: http://review.couchbase.org/37830
Reviewed-by: Trond Norbye <trond.

MB-11284: Support "admin" user in memcached core

Change-Id: I8e990e147ea870857483638bd1e9fa8e6fea551b
Reviewed-on: http://review.couchbase.org/37830
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

MB-11305 Fix memory leak in testapp

As identified by Clang static analyzer; fix memory leak in the
memcached testapp.

Change-Id: I1738288aa648b6b680dd205446dea6badc80b863
Re

MB-11305 Fix memory leak in testapp

As identified by Clang static analyzer; fix memory leak in the
memcached testapp.

Change-Id: I1738288aa648b6b680dd205446dea6badc80b863
Reviewed-on: http://review.couchbase.org/37962
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

MB-11036: Support for session cas validation

- Provide API for ns_server to set the session_cas.
- Provide API for ns_server to get current session_cas.
- Provide an API for ep_engin

MB-11036: Support for session cas validation

- Provide API for ns_server to set the session_cas.
- Provide API for ns_server to get current session_cas.
- Provide an API for ep_engine to validate the session_cas
for every request from ns_server
- Validate the session_cas before creating and deleting a
bucket instance.

Change-Id: Iea8ff75c4d19b312c538b9c66eb3fedca7cbfd38
Reviewed-on: http://review.couchbase.org/36735
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

MB-10711: Datatype detection for legacy clients

- In case of legacy clients that do not support
datatype, check whether document is of type JSON
and set the datatype as it should be,

MB-10711: Datatype detection for legacy clients

- In case of legacy clients that do not support
datatype, check whether document is of type JSON
and set the datatype as it should be, before the
saving the document.

Change-Id: I90e579e21bd35ce1b8d7ac4418135c9d92d04148
Reviewed-on: http://review.couchbase.org/35169
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

Add test that we don't allow invalid datatypes

Clients trying to send an illegal datatype will be disconnected

Change-Id: Icb6b3193b053619823b8dd99424b5cf1b1bb5985
Reviewed-on:

Add test that we don't allow invalid datatypes

Clients trying to send an illegal datatype will be disconnected

Change-Id: Icb6b3193b053619823b8dd99424b5cf1b1bb5985
Reviewed-on: http://review.couchbase.org/35451
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

MB-10476: Use cb_assert which is never a noop

Change-Id: Ibf478e0dc55697ea7b9fe1aff1ac18a4ec1ea0c7
Reviewed-on: http://review.couchbase.org/34580
Reviewed-by: Michael Wiederhold <mik

MB-10476: Use cb_assert which is never a noop

Change-Id: Ibf478e0dc55697ea7b9fe1aff1ac18a4ec1ea0c7
Reviewed-on: http://review.couchbase.org/34580
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: buildbot <build@couchbase.com>

show more ...

MB-10416: Add UPR control

Change-Id: Id0c28d2ebea002db93ebfb5c36622df5f79a286e
Reviewed-on: http://review.couchbase.org/34417
Reviewed-by: Michael Wiederhold <mike@couchbase.com>

MB-10416: Add UPR control

Change-Id: Id0c28d2ebea002db93ebfb5c36622df5f79a286e
Reviewed-on: http://review.couchbase.org/34417
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>

show more ...

MB-10416: Add upr buffer acknowledgement

Change-Id: Id9cf495b09228823a667704e9368e40f0b48df00
Reviewed-on: http://review.couchbase.org/34416
Reviewed-by: Michael Wiederhold <mike@cou

MB-10416: Add upr buffer acknowledgement

Change-Id: Id9cf495b09228823a667704e9368e40f0b48df00
Reviewed-on: http://review.couchbase.org/34416
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>

show more ...

MB-10416: Add upr no-op message

Change-Id: Idca071d2abb241862a7507498c4b87a9d8543c88
Reviewed-on: http://review.couchbase.org/34415
Reviewed-by: Michael Wiederhold <mike@couchbase.co

MB-10416: Add upr no-op message

Change-Id: Idca071d2abb241862a7507498c4b87a9d8543c88
Reviewed-on: http://review.couchbase.org/34415
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>

show more ...

Refactor: move test programs to tests

Change-Id: I009d56a4cf3e876d19dec2987d3aada76d58e66a
Reviewed-on: http://review.couchbase.org/34339
Reviewed-by: Trond Norbye <trond.norbye@gmai

Refactor: move test programs to tests

Change-Id: I009d56a4cf3e876d19dec2987d3aada76d58e66a
Reviewed-on: http://review.couchbase.org/34339
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

Add support for configuration file

This patch cleans up the handling of all of the command line
options and replace them with a single configuration file.

Change-Id: I6a7994427b

Add support for configuration file

This patch cleans up the handling of all of the command line
options and replace them with a single configuration file.

Change-Id: I6a7994427b33d6a54e4fa901d0f5f3836a6b3472
Reviewed-on: http://review.couchbase.org/33119
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

Add testcase for MB-10114

Change-Id: I35b7558f9cef549a5658f692d4ca467a6f4d59eb
Reviewed-on: http://review.couchbase.org/33321
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
T

Add testcase for MB-10114

Change-Id: I35b7558f9cef549a5658f692d4ca467a6f4d59eb
Reviewed-on: http://review.couchbase.org/33321
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

Add support for parent monitor for unix platforms

Let memcached look for the availability for the parent process
and kill itself if its gone. This piece of code is only used by
the u

Add support for parent monitor for unix platforms

Let memcached look for the availability for the parent process
and kill itself if its gone. This piece of code is only used by
the unit tests to ensure that we don't have zombie memcached
processes running if the unit tests fail.

Change-Id: I462af679d7691a1897b46448e9dc6a2613a56372
Reviewed-on: http://review.couchbase.org/33170
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

show more ...