MB-11731 remove shard level locking from ExecutorPool

we no longer need to block threads at shard level since
locks are acquired at the vbucket level

Change-Id: Iebc368d243499b7

MB-11731 remove shard level locking from ExecutorPool

we no longer need to block threads at shard level since
locks are acquired at the vbucket level

Change-Id: Iebc368d243499b76b5cf1f4f0d50c487866960dc
Reviewed-on: http://review.couchbase.org/39502
Reviewed-by: abhinav dangeti <abhinav@couchbase.com>
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>

show more ...

MB-11434 distributed sleep on taskQ

Make shared thread pool workers sleep inside their respective
TaskQueue mutexes as opposed to a single global mutex to reduce
the CPU overhead of

MB-11434 distributed sleep on taskQ

Make shared thread pool workers sleep inside their respective
TaskQueue mutexes as opposed to a single global mutex to reduce
the CPU overhead of putting threads to sleep and waking them up.
We still try to wake only as many threads as ready tasks.
First try waking local threads, if still more tasks are ready,
then visit other TaskQueues to wake their sleeping threads

Results in 15% improvement in view operations local testing
Lowered scheduling overhead with larger number of threads

Change-Id: I8ca746a696b4b2166f327385125b8510c0567407
Reviewed-on: http://review.couchbase.org/39210
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>

show more ...

MB-11488 Set the thread local to NULL before calling thread pool API

Memory allocations and de-allocations inside a global thread pool
API should not be accounted in a bucket memory quot

MB-11488 Set the thread local to NULL before calling thread pool API

Memory allocations and de-allocations inside a global thread pool
API should not be accounted in a bucket memory quota because they
might not be executed in the same bucket context. As a todo task,
we need to keep track of the memory overhead of the global thread
pool separately.

Change-Id: I328dc39c98a23c2e397ccae08b0dc76936d6d2f6
Reviewed-on: http://review.couchbase.org/38942
Reviewed-by: abhinav dangeti <abhinav@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

MB-11405 Wake only as many threads as ready tasks

+ If a thread has uncovered only one ready task then do not
wake more threads, since current thread will suffice.
+ Only signal th

MB-11405 Wake only as many threads as ready tasks

+ If a thread has uncovered only one ready task then do not
wake more threads, since current thread will suffice.
+ Only signal threads if they are sleeping.
+ cond_broadcast and wake all threads if we have more ready tasks
than sleeping threads
+ Aggregate and signal threads to avoid repeated mutex contentions
Change-Id: Ic130b87c19a2c5cd9eb971078c9a40b3612b5da0
Reviewed-on: http://review.couchbase.org/38756
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

MB-11524 Check pendingQueue on poll to avoid starvation

We limit number of threads that can work on a task queue by
blocking excess tasks in a pendingQueue.
However in mixed priority

MB-11524 Check pendingQueue on poll to avoid starvation

We limit number of threads that can work on a task queue by
blocking excess tasks in a pendingQueue.
However in mixed priority scenarios, we can end up with
starvation if tasks are blocked in low priority pendingQueue
while capacity is released to a high priority task queue.

Fix is to release capacity when task finishes execution, and
have the task queues poll the pendingQueue.

This avoids starvation and also reduces mutex contentions.

Change-Id: Iaeb08a53d31d21b463502085198d07923c0731c7
Reviewed-on: http://review.couchbase.org/38825
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>

show more ...

MB-11287 Lock TaskQ selectively, avoid busy loops

Global Thread Pool offers 2 serialization features:
1. lockless shard serialization (Flusher, VBDelete, Snapshot etc)
2. task-queue

MB-11287 Lock TaskQ selectively, avoid busy loops

Global Thread Pool offers 2 serialization features:
1. lockless shard serialization (Flusher, VBDelete, Snapshot etc)
2. task-queue based thread control (NonIO and AuxIO)
If thread is working on tasks that need neither of the above two,
then avoid any additional locking overhead.

Also isLock variable ends up causing more overhead, removing same

Change-Id: I38b060573975786973aa608277d40851adb64e45
Reviewed-on: http://review.couchbase.org/38102
Reviewed-by: abhinav dangeti <abhinav@couchbase.com>
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>

show more ...

MB-11287 move taskLogs to threads, lessen locking

This fix is meant to both reduce CPU and memory usage when
there are a large number of buckets by moving non-essential
task logging

MB-11287 move taskLogs to threads, lessen locking

This fix is meant to both reduce CPU and memory usage when
there are a large number of buckets by moving non-essential
task logging to ExecutorThread to avoid lock contention in
TaskQueue mutex

Change-Id: I8c82db1fb333febcead7e981e1dcffef3ce0b0a2
Reviewed-on: http://review.couchbase.org/37881
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

MB-10383 AuxIO, NonIO taskQ serialization must not affect CPU

If a thread cannot pick up an AuxIO task or a NonIO task, then stash
it away in a pendingQueue to avoid busy looping and hig

MB-10383 AuxIO, NonIO taskQ serialization must not affect CPU

If a thread cannot pick up an AuxIO task or a NonIO task, then stash
it away in a pendingQueue to avoid busy looping and high cpu usage,
performance problems.
added stats for the new pendingQueue
Change-Id: I00c40bab8bd8d4dd8e691d3e532aa1c427066f6a
Reviewed-on: http://review.couchbase.org/37367
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Reviewed-by: abhinav dangeti <abhinav@couchbase.com>
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>

show more ...

MB-11083: block shard serial runnable tasks to save CPU

Ready tasks in the task queue that cannot be picked up by
a thread because of shard serialization are now put in
a shard-local

MB-11083: block shard serial runnable tasks to save CPU

Ready tasks in the task queue that cannot be picked up by
a thread because of shard serialization are now put in
a shard-local pending list to prevent threads from busy looping
Change-Id: I84b999df2da6e769f0da49bfda0409cae564792a
Reviewed-on: http://review.couchbase.org/37056
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

Serialize Shard Level Operations from ExecutorPool

Add a flag in every shard to indicate that a thread is
currently working on it.
In TaskQueue, threads check this flag first to ensu

Serialize Shard Level Operations from ExecutorPool

Add a flag in every shard to indicate that a thread is
currently working on it.
In TaskQueue, threads check this flag first to ensure that
only one thread picks up task from this shard
Mark certain tasks like VBDeleteTask and VBSnapshotTask
as needing serialization

Change-Id: Ib4f0ae14d90a3d95ea0a59531197da4f87f388d3
Reviewed-on: http://review.couchbase.org/36286
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: abhinav dangeti <abhinav@couchbase.com>

show more ...

MB-9986 ensure futureQueue's heap sort

Windows ep-engine unit tests enabling fix..
Microsoft Visual Studio has a debug heap checker which
requires that STL container priority_queue a

MB-9986 ensure futureQueue's heap sort

Windows ep-engine unit tests enabling fix..
Microsoft Visual Studio has a debug heap checker which
requires that STL container priority_queue always satisfies
the HEAPIFY property. In GIO's implementation it is possible
that futureQueue can disobey the heap sort property temporarily
when we wake and cancel tasks.
To fix, always re-sort queue when task is woken or cancelled
Change-Id: I5631a4ec7e609953f037cd5a7c689ed876b40c12
Reviewed-on: http://review.couchbase.org/34264
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

show more ...

MB-9939: worker stats fix, debug for MB-9675

Move tasklog and slow log to the taskQueue so that worker
stats are consistent and allows debugging of low disk utilization
seen in MB-96

MB-9939: worker stats fix, debug for MB-9675

Move tasklog and slow log to the taskQueue so that worker
stats are consistent and allows debugging of low disk utilization
seen in MB-9675
Change-Id: I9184a5b87ea91eebc3db0e7c14b7e406245ea468
Reviewed-on: http://review.couchbase.org/32887
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>

show more ...

Extract TaskQueue out of scheduler.h

Change-Id: If46478214e81defa42d17f3e7c63f639ac266756
Reviewed-on: http://review.couchbase.org/31720
Reviewed-by: Sundararaman Sridharan <sundar.s

Extract TaskQueue out of scheduler.h

Change-Id: If46478214e81defa42d17f3e7c63f639ac266756
Reviewed-on: http://review.couchbase.org/31720
Reviewed-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>
Tested-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>

show more ...