MB-11870 Avoid polling starvation in mixed priorities

1) Wake up threads from the correct Q in mixed priorities.
2) If a thread has just woken up off a high priority Q
then make sure

MB-11870 Avoid polling starvation in mixed priorities

1) Wake up threads from the correct Q in mixed priorities.
2) If a thread has just woken up off a high priority Q
then make sure it polls the low priority Q next.
Not doing this causes starvation in the low prioirty Q
resuling in rebalance hangs.
3) Avoid sleeping threads for longer than 2 seconds to avoid
missing posts from other threads
Change-Id: I6cef6153285098e2f77639c50b194c9a9541dd24
Reviewed-on: http://review.couchbase.org/40242
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

correctly decrement numToWake to avoid waking more

Change-Id: I2a0e5f2bb38a0d8831a05028b89bee98fa48c22a
Reviewed-on: http://review.couchbase.org/40154
Tested-by: Sundararaman Sridhar

correctly decrement numToWake to avoid waking more

Change-Id: I2a0e5f2bb38a0d8831a05028b89bee98fa48c22a
Reviewed-on: http://review.couchbase.org/40154
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>
Reviewed-by: abhinav dangeti <abhinav@couchbase.com>

show more ...

MB-11405 Dedicated Threads that poll & sleep on own Q only

Threads only poll their own Q type toggling between high and low
priority queues and sleep on their own queue type only.
Ev

MB-11405 Dedicated Threads that poll & sleep on own Q only

Threads only poll their own Q type toggling between high and low
priority queues and sleep on their own queue type only.
Even if there are more tasks ready, do not wake threads from
other queues.
This is to reduce kernel CPU context switching overhead

Change-Id: I6d77930d1c8f4f6b363fbcf5cdabcf4db0cf6b06
Reviewed-on: http://review.couchbase.org/40080
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

prevent task state change from overriding task cancel

task states of snooze and running must not override task cancel
otherwise it can cause a hang while shutting down bucket
use ato

prevent task state change from overriding task cancel

task states of snooze and running must not override task cancel
otherwise it can cause a hang while shutting down bucket
use atomic compare and swap for safe lockless operation.

Change-Id: Ibfc3c7745d0c40893c577d3d94775f523ecc2861
Reviewed-on: http://review.couchbase.org/40060
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

MB-11799 Throttle vbucket compaction with the disk write queue size.

If a disk write queue size is above the threshold, it is better to
throttle vbucket compaction tasks, so that the min

MB-11799 Throttle vbucket compaction with the disk write queue size.

If a disk write queue size is above the threshold, it is better to
throttle vbucket compaction tasks, so that the minimum number of
compaction tasks can be running at the same time.

As another commit, we will make both the write queue size threshold and
the minimum number of compaction tasks configurable at runtime.

Change-Id: I70d6c3f02507895ed767060870e9e16265cd60ac
Reviewed-on: http://review.couchbase.org/40043
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

Add stats for measuring ExecutorPool scheduling delay & runtimes

Usage: cbstats ip:port -b <bucketname> scheduler

It returns the timing histograms of the scheduling overhead incurre

Add stats for measuring ExecutorPool scheduling delay & runtimes

Usage: cbstats ip:port -b <bucketname> scheduler

It returns the timing histograms of the scheduling overhead incurred by
all tasks in ep-engine. Scheduling overhead is computed as the difference
between the scheduled runtime of a task and the actual runtime.
This will give us definitive answers to how much delay is introduced by
the scheduling overhead of the global thread pool and how it affects
each task in ep-engine.

Usage: cbstats ip:port -b <bucketname> runtimes

It returns the timing histograms of various task runtimes measured from
the ExecutorPool.

Change-Id: Icb34d4a87c8659fb22b7d2c1ba6bef0836045c3e
Reviewed-on: http://review.couchbase.org/39705
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

MB-11731 remove shard level locking from ExecutorPool

we no longer need to block threads at shard level since
locks are acquired at the vbucket level

Change-Id: Iebc368d243499b7

MB-11731 remove shard level locking from ExecutorPool

we no longer need to block threads at shard level since
locks are acquired at the vbucket level

Change-Id: Iebc368d243499b76b5cf1f4f0d50c487866960dc
Reviewed-on: http://review.couchbase.org/39502
Reviewed-by: abhinav dangeti <abhinav@couchbase.com>
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>

show more ...

MB-11434 distributed sleep on taskQ

Make shared thread pool workers sleep inside their respective
TaskQueue mutexes as opposed to a single global mutex to reduce
the CPU overhead of

MB-11434 distributed sleep on taskQ

Make shared thread pool workers sleep inside their respective
TaskQueue mutexes as opposed to a single global mutex to reduce
the CPU overhead of putting threads to sleep and waking them up.
We still try to wake only as many threads as ready tasks.
First try waking local threads, if still more tasks are ready,
then visit other TaskQueues to wake their sleeping threads

Results in 15% improvement in view operations local testing
Lowered scheduling overhead with larger number of threads

Change-Id: I8ca746a696b4b2166f327385125b8510c0567407
Reviewed-on: http://review.couchbase.org/39210
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>

show more ...

MB-11488 Set the thread local to NULL before calling thread pool API

Memory allocations and de-allocations inside a global thread pool
API should not be accounted in a bucket memory quot

MB-11488 Set the thread local to NULL before calling thread pool API

Memory allocations and de-allocations inside a global thread pool
API should not be accounted in a bucket memory quota because they
might not be executed in the same bucket context. As a todo task,
we need to keep track of the memory overhead of the global thread
pool separately.

Change-Id: I328dc39c98a23c2e397ccae08b0dc76936d6d2f6
Reviewed-on: http://review.couchbase.org/38942
Reviewed-by: abhinav dangeti <abhinav@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

MB-11405 Wake only as many threads as ready tasks

+ If a thread has uncovered only one ready task then do not
wake more threads, since current thread will suffice.
+ Only signal th

MB-11405 Wake only as many threads as ready tasks

+ If a thread has uncovered only one ready task then do not
wake more threads, since current thread will suffice.
+ Only signal threads if they are sleeping.
+ cond_broadcast and wake all threads if we have more ready tasks
than sleeping threads
+ Aggregate and signal threads to avoid repeated mutex contentions
Change-Id: Ic130b87c19a2c5cd9eb971078c9a40b3612b5da0
Reviewed-on: http://review.couchbase.org/38756
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

MB-11524 Check pendingQueue on poll to avoid starvation

We limit number of threads that can work on a task queue by
blocking excess tasks in a pendingQueue.
However in mixed priority

MB-11524 Check pendingQueue on poll to avoid starvation

We limit number of threads that can work on a task queue by
blocking excess tasks in a pendingQueue.
However in mixed priority scenarios, we can end up with
starvation if tasks are blocked in low priority pendingQueue
while capacity is released to a high priority task queue.

Fix is to release capacity when task finishes execution, and
have the task queues poll the pendingQueue.

This avoids starvation and also reduces mutex contentions.

Change-Id: Iaeb08a53d31d21b463502085198d07923c0731c7
Reviewed-on: http://review.couchbase.org/38825
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>

show more ...

MB-11287 Lock TaskQ selectively, avoid busy loops

Global Thread Pool offers 2 serialization features:
1. lockless shard serialization (Flusher, VBDelete, Snapshot etc)
2. task-queue

MB-11287 Lock TaskQ selectively, avoid busy loops

Global Thread Pool offers 2 serialization features:
1. lockless shard serialization (Flusher, VBDelete, Snapshot etc)
2. task-queue based thread control (NonIO and AuxIO)
If thread is working on tasks that need neither of the above two,
then avoid any additional locking overhead.

Also isLock variable ends up causing more overhead, removing same

Change-Id: I38b060573975786973aa608277d40851adb64e45
Reviewed-on: http://review.couchbase.org/38102
Reviewed-by: abhinav dangeti <abhinav@couchbase.com>
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>

show more ...

MB-11287 Lower default atomic operation ordering

Lowering global default memory ordering stringency to
memory_order_acq_rel to avoid costly CPU fence operations

Change-Id: Ic8f1

MB-11287 Lower default atomic operation ordering

Lowering global default memory ordering stringency to
memory_order_acq_rel to avoid costly CPU fence operations

Change-Id: Ic8f14abc7843f931242c61ba98c5cc667f52023d
Reviewed-on: http://review.couchbase.org/37981
Reviewed-by: abhinav dangeti <abhinav@couchbase.com>
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>

show more ...

MB-11287 use less stringent memory_order for TaskQueue lock

The default memory_order for the atomic variable isLock in
TaskQueue is sequential ordering which imposes severe CPU
fence

MB-11287 use less stringent memory_order for TaskQueue lock

The default memory_order for the atomic variable isLock in
TaskQueue is sequential ordering which imposes severe CPU
fence and results in high CPU usage on machines with many cores.

The acquire-release ordering should offer a balanced mix of
ordering and performance for this use-case.
http://en.cppreference.com/w/cpp/atomic/memory_order

Change-Id: Ida12cfaf428b725f5a960f4f34e527f548e01c91
Reviewed-on: http://review.couchbase.org/37976
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

MB-11287 move taskLogs to threads, lessen locking

This fix is meant to both reduce CPU and memory usage when
there are a large number of buckets by moving non-essential
task logging

MB-11287 move taskLogs to threads, lessen locking

This fix is meant to both reduce CPU and memory usage when
there are a large number of buckets by moving non-essential
task logging to ExecutorThread to avoid lock contention in
TaskQueue mutex

Change-Id: I8c82db1fb333febcead7e981e1dcffef3ce0b0a2
Reviewed-on: http://review.couchbase.org/37881
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

MB-11232 Wake up tasks in pendingQueue as well

TaskQueue::wake() routine was missing to look at the
tasks blocked in the pendingQueue causing the
ExecutorPool::unregister() routine t

MB-11232 Wake up tasks in pendingQueue as well

TaskQueue::wake() routine was missing to look at the
tasks blocked in the pendingQueue causing the
ExecutorPool::unregister() routine to hang occasionally.

Change-Id: I3592a820ae7ef0d1c587a459f0f34ab6731bbdde
Reviewed-on: http://review.couchbase.org/37688
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

MB-11194 correct order of serializing checks in taskqueue

If the tryNewTask check was done first and the checkOutShard() check
done later, then a accounting mismatch happens in the numbe

MB-11194 correct order of serializing checks in taskqueue

If the tryNewTask check was done first and the checkOutShard() check
done later, then a accounting mismatch happens in the number of
curWorkers for a particular taskType since the tryNewTask has incremented
the number of curWorkers already. Writer tasks are affected by this and
this resulted in unnecessarily blocking flusher tasks causing slow disk
persistence.

Reverse the order of checks to correct the situation above.

Change-Id: If566f6025aec5ccc25b1f06db6338f12eba4a108
Reviewed-on: http://review.couchbase.org/37528
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

MB-10383 AuxIO, NonIO taskQ serialization must not affect CPU

If a thread cannot pick up an AuxIO task or a NonIO task, then stash
it away in a pendingQueue to avoid busy looping and hig

MB-10383 AuxIO, NonIO taskQ serialization must not affect CPU

If a thread cannot pick up an AuxIO task or a NonIO task, then stash
it away in a pendingQueue to avoid busy looping and high cpu usage,
performance problems.
added stats for the new pendingQueue
Change-Id: I00c40bab8bd8d4dd8e691d3e532aa1c427066f6a
Reviewed-on: http://review.couchbase.org/37367
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Reviewed-by: abhinav dangeti <abhinav@couchbase.com>
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>

show more ...

MB-11083: block shard serial runnable tasks to save CPU

Ready tasks in the task queue that cannot be picked up by
a thread because of shard serialization are now put in
a shard-local

MB-11083: block shard serial runnable tasks to save CPU

Ready tasks in the task queue that cannot be picked up by
a thread because of shard serialization are now put in
a shard-local pending list to prevent threads from busy looping
Change-Id: I84b999df2da6e769f0da49bfda0409cae564792a
Reviewed-on: http://review.couchbase.org/37056
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

Serialize Shard Level Operations from ExecutorPool

Add a flag in every shard to indicate that a thread is
currently working on it.
In TaskQueue, threads check this flag first to ensu

Serialize Shard Level Operations from ExecutorPool

Add a flag in every shard to indicate that a thread is
currently working on it.
In TaskQueue, threads check this flag first to ensure that
only one thread picks up task from this shard
Mark certain tasks like VBDeleteTask and VBSnapshotTask
as needing serialization

Change-Id: Ib4f0ae14d90a3d95ea0a59531197da4f87f388d3
Reviewed-on: http://review.couchbase.org/36286
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: abhinav dangeti <abhinav@couchbase.com>

show more ...

MB-10539: use less than eq comparison in wake()

ExecutorPool::wake() was wrongly using less_tv()
in its wake routine, resulting in un-woken tasks
This was causing large delays in sch

MB-10539: use less than eq comparison in wake()

ExecutorPool::wake() was wrongly using less_tv()
in its wake routine, resulting in un-woken tasks
This was causing large delays in scheduling
Fix by using less_eq_tv

Change-Id: I52e97c6308c70cfc415dd731a50dc8e00a7d834a
Reviewed-on: http://review.couchbase.org/35586
Reviewed-by: abhinav dangeti <abhinav@couchbase.com>
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>

show more ...

MB-9986 ensure futureQueue's heap sort

Windows ep-engine unit tests enabling fix..
Microsoft Visual Studio has a debug heap checker which
requires that STL container priority_queue a

MB-9986 ensure futureQueue's heap sort

Windows ep-engine unit tests enabling fix..
Microsoft Visual Studio has a debug heap checker which
requires that STL container priority_queue always satisfies
the HEAPIFY property. In GIO's implementation it is possible
that futureQueue can disobey the heap sort property temporarily
when we wake and cancel tasks.
To fix, always re-sort queue when task is woken or cancelled
Change-Id: I5631a4ec7e609953f037cd5a7c689ed876b40c12
Reviewed-on: http://review.couchbase.org/34264
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

MB-9939: worker stats fix, debug for MB-9675

Move tasklog and slow log to the taskQueue so that worker
stats are consistent and allows debugging of low disk utilization
seen in MB-96

MB-9939: worker stats fix, debug for MB-9675

Move tasklog and slow log to the taskQueue so that worker
stats are consistent and allows debugging of low disk utilization
seen in MB-9675
Change-Id: I9184a5b87ea91eebc3db0e7c14b7e406245ea468
Reviewed-on: http://review.couchbase.org/32887
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>

show more ...

Refactor ExecutorPool out of scheduler.cc/h

Change-Id: Ie67a093ebcd1ed3a645847810f2050439e9283ed
Reviewed-on: http://review.couchbase.org/31721
Reviewed-by: Sundararaman Sridharan <s

Refactor ExecutorPool out of scheduler.cc/h

Change-Id: Ie67a093ebcd1ed3a645847810f2050439e9283ed
Reviewed-on: http://review.couchbase.org/31721
Reviewed-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>
Tested-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>

show more ...

Extract TaskQueue out of scheduler.h

Change-Id: If46478214e81defa42d17f3e7c63f639ac266756
Reviewed-on: http://review.couchbase.org/31720
Reviewed-by: Sundararaman Sridharan <sundar.s

Extract TaskQueue out of scheduler.h

Change-Id: If46478214e81defa42d17f3e7c63f639ac266756
Reviewed-on: http://review.couchbase.org/31720
Reviewed-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>
Tested-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>

show more ...