History log of /6.0.3/kv_engine/engines/ep/src/globaltask.h (Results 1 - 16 of 16)
Revision (<<< Hide revision tags) (Show revision tags >>>) Date Author Comments
Revision tags: v7.0.2, v6.6.3, v7.0.1, v7.0.0, v6.6.2, v6.5.2, v6.6.1, v6.0.5, v6.6.0, v6.5.1, v6.0.4, v6.5.0, v6.0.3, v5.5.4, v5.5.5, v5.5.6, v6.0.1, v5.5.3, v6.0.0, v5.1.3, v5.5.2, v5.5.1, v5.1.2, v5.1.1
# 44a3fd5d 15-May-2018 Dave Rigby <daver@couchbase.com>

MB-29675: Change Task::getDescription() back to std::string

Task::getDescription() currently returns a non-owning string
(const_char_buffer), instead of an owning std::string. This was c

MB-29675: Change Task::getDescription() back to std::string

Task::getDescription() currently returns a non-owning string
(const_char_buffer), instead of an owning std::string. This was chocen
to minimise string copying as getDescription was called multiple times
per task execution (for example for taskLog / slowTaskLog - see
d26ea7d1f12aa021e74663a304d81d1f9e512097 which originally changed to
const_char_buffer.

However now taskLog / slowTaskLog have been removed, we will no longer
call getDescription anywhere near as frequently - short-running tasks
will likely never have their description requested. As such
pre-calculating the description (and having to ensure it's lifetime is
as long as any caller requesting it) is now less efficient than simply
generating it on-demand.

Change getDescription back to std::string, only creating a string
object on demand when it is requested.

Change-Id: I5f003b76d0ac73b2abc3ac5cd3d985a6ceb867c3
Reviewed-on: http://review.couchbase.org/94283
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>

show more ...


# 482fe784 26-Feb-2018 Paolo Cocchi <paolo.cocchi@couchbase.com>

MB-25822: Log datetime of long schedules for NON_IO tasks

It would be useful to have the exact datetime of long unexpected
schedule for debugging, the same as we have for long runtimes.

MB-25822: Log datetime of long schedules for NON_IO tasks

It would be useful to have the exact datetime of long unexpected
schedule for debugging, the same as we have for long runtimes.
Compared to runtime, deciding for a sensible expected schedule
time is more difficult, mainly for 2 reasons:
- the schedule time depends on things "external" to the task itself
(e.g., the number of tasks in the same priority-group)
- the schedule time depends on the runtime of the previous run of the
same task; so, for IO tasks (Read/Write/AuxIO) is even more difficult
to predict
Thus, with this chagnge we log unexpected long schedule time only for
NON_IO tasks. We pick 1 seconds as schedule overhead limit and log only
schedule overheads that exceed that limit.

Change-Id: I68b921b783c7bcd9644565e7e9a14f2497763ce0
Reviewed-on: http://review.couchbase.org/90047
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...


Revision tags: v5.0.1, v5.1.0, v5.0.0
# 0ca3887c 05-Sep-2017 Dave Rigby <daver@couchbase.com>

MB-25822: Recalibrate maxExpectedDuration for all Tasks

Each GlobalTask object defines a maximum expected duration for
executions of that task type; executions which exceed this threshol

MB-25822: Recalibrate maxExpectedDuration for all Tasks

Each GlobalTask object defines a maximum expected duration for
executions of that task type; executions which exceed this threshold
are logged (currently into an in-memory circular buffer).

However, all subclasses currently just inherit the value from the base
class of 3.6ms. This is not very useful - some tasks (e.g. warmup, or
disk-related) often exceed that value; for other tasks that value is
far too generious. As a result the current "slow tasks" log is of
little value.

To address this, make GlobalTask::maxExpectedDuration() pure virtual;
and define a suitable value for *all* subclasses. This has been done
empirically by looking at runtimes from statistics from various
cbcollect_info logs.

This means that entries in the 'slow tasks' log are actually
meaningful. A following patch will also record these slow executions
in memcached log file (similar to Slow Ops), to make their occurance
more explicit.

Change-Id: Ic7a4141ef65162f01ce9c3b1b4cd8a0f4f1c463b
Reviewed-on: http://review.couchbase.org/83096
Well-Formed: Build Bot <build@couchbase.com>
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...


# fe6b9e03 12-Jul-2017 Dave Rigby <daver@couchbase.com>

MB-23797: Revert "Introduce testing exception for rescheduled dead tasks"

In http://review.couchbase.org/#/c/76394/ the ability to reschedule a
cancelled (and now in state TASK_DEAD) Glo

MB-23797: Revert "Introduce testing exception for rescheduled dead tasks"

In http://review.couchbase.org/#/c/76394/ the ability to reschedule a
cancelled (and now in state TASK_DEAD) GlobalTask was fixed.

It did not appear that any tasks other than the ItemPager for
ephemeral buckets are rescheduled once dead, but to avoid changing
existing behaviour a "paranoia" check was introduced (6e6429b) to
throw an exception if any task other than the ItemPager is
rescheduled, to bring them to our attention.

This patch has had 3 months of soak time, and hasn't exposed any
unexpected wakups, but to ensure we don't fire such exceptions in
production the check is being reverted.

This reverts commit 6e6429b0b1e20aa44211452198a7b5c80b9ae835.

Change-Id: I82f8d4068e03c451493ef7ac17bd3d21c37f4166
Reviewed-on: http://review.couchbase.org/80594
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Daniel Owen <owend@couchbase.com>

show more ...


# da1ddf1e 19-Jun-2017 James Harrison <00jamesh@gmail.com>

Add last runtime to cbstats tasks

When tasks are sleeping, the runtime field was somewhat wasted. We now
reuse it to show the last runtime, marking currently running tasks with
an "*

Add last runtime to cbstats tasks

When tasks are sleeping, the runtime field was somewhat wasted. We now
reuse it to show the last runtime, marking currently running tasks with
an "*".

Change-Id: I639b8da9e03f83584eda48e8496c5d459c44ba87
Reviewed-on: http://review.couchbase.org/79785
Reviewed-by: Oliver Downard <oliver.downard@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...


# e158d66e 12-Jun-2017 Dave Rigby <daver@couchbase.com>

Add GlobalTask::wakeUp, remove unnecessary snooze() calls

Add a method to wake up a task immediatatly -
GlobalTask::wakeUp().

This is logically identical to calling GlobalTask::

Add GlobalTask::wakeUp, remove unnecessary snooze() calls

Add a method to wake up a task immediatatly -
GlobalTask::wakeUp().

This is logically identical to calling GlobalTask::snooze() with the
argument 0, but (IMHO) is more self-documenting.

Additionally, remove unnecessary calls to snooze(0) - a task is
rescheduled to run immediately by default if it returns true from
run(), it is not necessary to call snooze(0) explicitly.

Change-Id: If70f77cecfe7c085b901c871fa9eb37b5751fa52
Reviewed-on: http://review.couchbase.org/79356
Reviewed-by: James Harrison <james.harrison@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...


# f48fae53 16-May-2017 Jim Walker <jim@couchbase.com>

MB-24394: Peform an unlocked reset of ExecutorThread::currentTask

When performing currentTask.reset with currentTaskMutex held a
lock inversion situation occurs if:

* the task h

MB-24394: Peform an unlocked reset of ExecutorThread::currentTask

When performing currentTask.reset with currentTaskMutex held a
lock inversion situation occurs if:

* the task hits a ref-count of 0
* the task owns a VBucketPtr (shared_ptr)
* the VBucket is marked for deferred deletion
* the VBucketPtr hits a ref-count of 0

This occurs because the VBucketPtr deleter will schedule a new task
to delete the VBucket and thus we re-enter the task code with a
different lock ordering (TaskQueue::tMutex->currentTaskMutex vs
currentTaskMutex->TaskQueue::tMutex).

To fix this issue we simply need to not hold the currentTaskMutex
whilst calling currentTask.reset().

We switch ExTask over to being a std::shared_ptr and use its well
define move constructor (via std::move) to move the ref-counted
pointer into unlocked scope, and then reset that.

Change-Id: I883cb4d0e8a2c4dbc91cdf93f0a9a967863dcb31
Reviewed-on: http://review.couchbase.org/78643
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...


# ef22f9b0 25-May-2017 Dave Rigby <daver@couchbase.com>

Move ep-engine to engines/ep


# 74551d34 28-Apr-2017 Dave Rigby <daver@couchbase.com>

Remove unnecessary #include "tasks.h"

tasks.h is a relatively expensive header to include, as it contains
the definitions of various disjoint Task types, so in turn it includes
a lar

Remove unnecessary #include "tasks.h"

tasks.h is a relatively expensive header to include, as it contains
the definitions of various disjoint Task types, so in turn it includes
a large proportion of ep-engine headers.

However, it is unneessarily included in a large number of places. This
is probably at last in part due to the fact that globaltask.h didn't
used to be it's own header, and hence to get the GlobalTask definition
one needed to include tasks.h

Change-Id: I0971eab4808ce51d470fe061c13796d18b444234
Reviewed-on: http://review.couchbase.org/77498
Reviewed-by: Manu Dhundi <manu@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...


# 9f76e3c1 27-Apr-2017 Dave Rigby <daver@couchbase.com>

MB-24141: Use correct unit for slowTask recording

Change-Id: I388eb43fea01f8b79f5a122afa2c68757736fb81
Reviewed-on: http://review.couchbase.org/77448
Tested-by: Build Bot <build@couc

MB-24141: Use correct unit for slowTask recording

Change-Id: I388eb43fea01f8b79f5a122afa2c68757736fb81
Reviewed-on: http://review.couchbase.org/77448
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Manu Dhundi <manu@couchbase.com>

show more ...


# 6e6429b0 07-Apr-2017 James Harrison <00jamesh@gmail.com>

Introduce testing exception for rescheduled dead tasks

In http://review.couchbase.org/#/c/76394/ the ability to reschedule a
cancelled (and now in state TASK_DEAD) GlobalTask was fixed.

Introduce testing exception for rescheduled dead tasks

In http://review.couchbase.org/#/c/76394/ the ability to reschedule a
cancelled (and now in state TASK_DEAD) GlobalTask was fixed.

it does not appear that any tasks other than the ItemPager for ephemeral
buckets are rescheduled once dead, but to avoid changing existing
behaviour this introduces an exception if any task other than the
ItemPager is rescheduled, to bring them to our attention.

NB: This patch will be reverted to remove this exception for Spock.
(MB-23797)

Change-Id: If44b7cf8a71c3dc4d85fba98d65c4f608d449460
Reviewed-on: http://review.couchbase.org/76551
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...


Revision tags: v4.6.2_ep, v4.6.2_mc, v4.6.1_ep
# 7dfe4a57 24-Feb-2017 James Harrison <00jamesh@gmail.com>

MB-22018: Add tasks stats to cbstats

Initial work on adding a "top" like tool showing tasks running and
sleeping in a cluster.

This can be viewed by running

cbstats

MB-22018: Add tasks stats to cbstats

Initial work on adding a "top" like tool showing tasks running and
sleeping in a cluster.

This can be viewed by running

cbstats host:port tasks

This shows the information about the tasks such as the state, next
runtime, description, type etc.

Future additions to the tool are intended to add features such as load
averages for task types (of the style shown by `uptime`) and percentage
utilisation over a period of time per thread.

Change-Id: I005a2589a35b4c4364ba46cf2b44b71061ba06b3
Reviewed-on: http://review.couchbase.org/74510
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...


# 4bf14710 24-Mar-2017 James Harrison <00jamesh@gmail.com>

MB-22018: Add method mapping TaskId to task_type_t

Allows identification of the task type without needing to know what
queue it is scheduled in. At the moment this is not /guaranteed/ to

MB-22018: Add method mapping TaskId to task_type_t

Allows identification of the task type without needing to know what
queue it is scheduled in. At the moment this is not /guaranteed/ to be
entirely true, but another patch will alter scheduling to use this,
ensuring consistency.

Change-Id: Icf03d9f459f0db3b480df61734c390580954bd6d
Reviewed-on: http://review.couchbase.org/75775
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...


# d26ea7d1 20-Mar-2017 Dave Rigby <daver@couchbase.com>

Change GlobalTask::getDescription to return cb::const_char_buffer

The pure virtual method GlobalTask::getDescription() returns a
std::string object. This method is actually called very f

Change GlobalTask::getDescription to return cb::const_char_buffer

The pure virtual method GlobalTask::getDescription() returns a
std::string object. This method is actually called very frequently by
the ExecutorThreads (to log task execution / slow tasks), and hence
passing a concrete object is potentially costly, requiring an
allocation (and then deletion) for each call.

Given that the description is mostly constant for the lifetime of a
Task, change the method to return cb::const_char_buffer - i.e. a
read-only non-owning view. This reduces to passing essentially two
words, removing all memory allocation.

Change-Id: I3cd68798d97401a3d5b50c9720c87dade2a1b32e
Reviewed-on: http://review.couchbase.org/75435
Reviewed-by: Jim Walker <jim@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

show more ...


Revision tags: v4.6.0_ep
# d55c867e 30-Nov-2016 Sergey Avseyev <sergey.avseyev@gmail.com>

Explicitly specify type for tp_ns to help template resolving

Fixes the following build error

Building CXX object ep-engine/CMakeFiles/ep_testsuite.dir/src/objectregistry.cc.o
In

Explicitly specify type for tp_ns to help template resolving

Fixes the following build error

Building CXX object ep-engine/CMakeFiles/ep_testsuite.dir/src/objectregistry.cc.o
In file included from ep-engine/src/objectregistry.cc:23:
In file included from ep-engine/src/ep_engine.h:23:
In file included from ep-engine/src/ep.h:21:
In file included from ep-engine/src/executorpool.h:71:
In file included from ep-engine/src/tasks.h:29:
ep-engine/src/globaltask.h:154:9: error: no matching function for call to 'atomic_setIfBigger'
atomic_setIfBigger(waketime, tp_ns);
^~~~~~~~~~~~~~~~~~
ep-engine/src/atomic.h:30:6: note: candidate template ignored: deduced conflicting types for parameter 'T' ('long' vs. 'long long')
void atomic_setIfBigger(std::atomic<T> &obj, const T &newValue) {
^
1 error generated.

Change-Id: I792db0ef847cc4bada2bf6a93918c603a31886e3
Reviewed-on: http://review.couchbase.org/70470
Tested-by: buildbot <build@couchbase.com>
Tested-by: Sergey Avseyev <sergey.avseyev@gmail.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>

show more ...


Revision tags: v4.5.1-MP1_mc, v4.6.0-DP_mc, v4.6.0-DP_ep, v4.5.1-MP1_ep, v4.1.2-MP2_mc, v4.5.1_mc, v4.6.0_mc, v4.1.2-MP1_ep, v3.1.6_ep
# 24c408e6 30-Jun-2016 Jim Walker <jim@couchbase.com>

MB-20061: Improved futureQueue heap property fix

MB-9986 found that we break the heap property of the underlying
container of the std::priority_queue used in scheduling. This is
beca

MB-20061: Improved futureQueue heap property fix

MB-9986 found that we break the heap property of the underlying
container of the std::priority_queue used in scheduling. This is
because we use the wakeTime to sort the queue, but then later
change the value of a task which is still in the queue, thus
order is broken. MB-9986 emptied the futureQueue and rebuilt it.

In this fix we utilise the C++ standard which shows that we can
access the container of the priority_queue. Thus we can sub-class
it and create a method that allows us to update a tasks time and
maintain the heap order.

Change-Id: I238d36ea684d59ef06326183fa1f16c04f8d29ad
Reviewed-on: http://review.couchbase.org/65394
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Manu Dhundi <manu@couchbase.com>
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>

show more ...