History log of /5.5.2/testrunner/pysystests/rabbit_helper.py (Results 1 - 12 of 12)
Revision (<<< Hide revision tags) (Show revision tags >>>) Date Author Comments
Revision tags: 4.6.0, testrunner-ng-old, watson-moxi, 4.5.0, 4.1.0, 4.0.0, 3.0.2, 3.0.1, 3.0.0, 2.5.1
# c205f8e6 20-Nov-2013 Tommie McAfee <tommie@couchbase.com>

CBQE-1712: support gevented python sdk

changes the underlying sdk implementation to the source in consumer.py,
which uses the python gevented sdk.

Also this change isolates the

CBQE-1712: support gevented python sdk

changes the underlying sdk implementation to the source in consumer.py,
which uses the python gevented sdk.

Also this change isolates the loader from the worker so that cluster can be loaded
as standalone without celery,rabbitmq,memcached with the --standalone option:

python cbsystest.py run workload --create 100 --ops 80000 --cluster default --standalone

Another feature is that only 1 worker needs to be started in the entire client-ecosystem.
any helper vms will simply need to run ./consumer.py --cluster <name>

Change-Id: I6d0241d13c5e377d08e906ba0b5e40c08402548d
Reviewed-on: http://review.couchbase.org/30429
Reviewed-by: Tommie McAfee <tommie@couchbase.com>
Tested-by: Tommie McAfee <tommie@couchbase.com>

show more ...


Revision tags: 2.1.1r, 2.1.0r, 2.0.2r
# 7a76cbb6 19-Apr-2013 Tommie McAfee <tommie@couchbase.com>

CBQE-1233: adding raw task publisher method

This method assembles celery tasks and publishes them
to workers listening to any broker

Change-Id: I15e2d0d80e59f6b2abadda508e0914c7

CBQE-1233: adding raw task publisher method

This method assembles celery tasks and publishes them
to workers listening to any broker

Change-Id: I15e2d0d80e59f6b2abadda508e0914c716075007
Reviewed-on: http://review.couchbase.org/25802
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Chisheng Hong <chisheng@couchbase.com>

show more ...


# 11ff3f61 12-Apr-2013 Tommie McAfee <tommie@couchbase.com>

CBQE-0: rabbit_helper allow vhost arg

we also need to support passing the --cluster
value used in cbsystest to automate other workers

Change-Id: I2acdbdb82730c90f532bd875f004ca2

CBQE-0: rabbit_helper allow vhost arg

we also need to support passing the --cluster
value used in cbsystest to automate other workers

Change-Id: I2acdbdb82730c90f532bd875f004ca2496203540
Reviewed-on: http://review.couchbase.org/25644
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Chisheng Hong <chisheng@couchbase.com>

show more ...


Revision tags: 2.0.1-macosx, 2.0.1-windows, 2.0.1-linux
# 215405d8 11-Feb-2013 Tommie McAfeee <tommie@couchbase.com>

CBQE-819: rename purge/delete queue methods

Current purge queue method in rabbit_helper actually deletes the queue.
This change replaces it with the proper purging functionality and

CBQE-819: rename purge/delete queue methods

Current purge queue method in rabbit_helper actually deletes the queue.
This change replaces it with the proper purging functionality and
makes a separate method for deletes.

Change-Id: I83cad7b47c12940bb5f2de72eaf5e1f8da355f9f
Reviewed-on: http://review.couchbase.org/24516
Tested-by: Tommie McAfee <tommie@couchbase.com>
Reviewed-by: Tommie McAfee <tommie@couchbase.com>

show more ...


# 34d6496e 29-Oct-2012 Tommie McAfee <tommie@couchbase.com>

CBQE-572: stop using default vhost

use CB_CLUSTER_TAG in as broker virtual host path where queues can be found
to further ensure no conflict in sending tasks from multiple workers.

CBQE-572: stop using default vhost

use CB_CLUSTER_TAG in as broker virtual host path where queues can be found
to further ensure no conflict in sending tasks from multiple workers.

Change-Id: If5c43920f5570512136e37b0f83ea7bb38288b06
Reviewed-on: http://review.couchbase.org/22040
Reviewed-by: Chisheng Hong <chisheng@couchbase.com>
Tested-by: Tommie McAfee <tommie@couchbase.com>

show more ...


# b78a0728 14-Oct-2012 Tommie McAfee <tommie@couchbase.com>

CBQE-572: Enhancements to systest framework

Allow for shared broker across clusters
* testcfg.py specify cluster name as prefix to queues

Better message routing
* all

CBQE-572: Enhancements to systest framework

Allow for shared broker across clusters
* testcfg.py specify cluster name as prefix to queues

Better message routing
* all tasks for each worker type has a specific route
so that traffic will be isolated and not picked up
by any worker listening to 'default' queue

Auto-delete unused queues
* set auto-delete = True for all queues to have them
removed when worker no longer needs them.

Support for using celeryd-multi to start several
worker instances on a single vm:
* this is useful to protecting against losing
jobs across the entire worker when an unrelated
task crashes the worker. I.E key-value loader crashes
and stop collection is killed.

examaple of starting 4 isolated workers on one vm with 2 process dedicated to each:

celeryd-multi start kv query admin stats -A app --purge -l ERROR -B -I:kv app.init \
-n:kv kv -n:query query -n:admin admin -n:stats stats -c 2

Where syntax (-n:kv kv) means for worker named kv, start the kv scheduler , create it's queues and routes
Note also init is only started once along with the kv worker, although it can be started as standalone.

To start all types in a single worker do:

celeryd-multi start all -A app --purge -l ERROR -B -n:all all -c 8

One worker with two schedulers:

celeryd-multi start kv query -A app --purge -l ERROR -B -I:kv app.init \
-n:kv kv -n:query query -c 4

Change-Id: I0054da50e31fdbe9b279daa033e461bfa00f31b2
Reviewed-on: http://review.couchbase.org/21632
Reviewed-by: Karan Kumar <karan@couchbase.com>
Tested-by: Tommie McAfee <tommie@couchbase.com>

show more ...


# 41be4345 03-Oct-2012 Tommie McAfee <tommie@couchbase.com>

CBQE-572: add broker arg to cbsystest cli

Allow sending commands to different brokers.
Currently only the rabbit server specified
in the config file is used

Change-Id: I5248

CBQE-572: add broker arg to cbsystest cli

Allow sending commands to different brokers.
Currently only the rabbit server specified
in the config file is used

Change-Id: I52486d7fd07ffddff4147188dbaf26ab1d67c48c
Reviewed-on: http://review.couchbase.org/21289
Reviewed-by: Tommie McAfee <tommie@couchbase.com>
Tested-by: Tommie McAfee <tommie@couchbase.com>

show more ...


# b14a4fec 26-Sep-2012 Karan Kumar <karan@couchbase.com>

CBQE-624: Support admin operations

Support following admin ops:
* rebalance in
* rebalance out
* failover+rebalance out and only failover
* soft/hard restart

Change

CBQE-624: Support admin operations

Support following admin ops:
* rebalance in
* rebalance out
* failover+rebalance out and only failover
* soft/hard restart

Change-Id: If4f62457aa1882bf3470e768b58ea22c90221283
Reviewed-on: http://review.couchbase.org/21103
Reviewed-by: Tommie McAfee <tommie@couchbase.com>
Tested-by: Tommie McAfee <tommie@couchbase.com>

show more ...


# 80ea2fc8 18-Sep-2012 Tommie McAfeee <tommie@couchbase.com>

MB-6686: fix for memory leak in systest platform

Do not persist channels within rabbitmq connection,
for each request open/close new channel.
Also using json for loading/dumping larg

MB-6686: fix for memory leak in systest platform

Do not persist channels within rabbitmq connection,
for each request open/close new channel.
Also using json for loading/dumping large objects,
as there appears to be leak in yajl package somewhere.

Change-Id: I2fc081ef21fec951296604bd474c968952f80745
Reviewed-on: http://review.couchbase.org/20937
Tested-by: Tommie McAfee <tommie@couchbase.com>
Reviewed-by: Tommie McAfee <tommie@couchbase.com>

show more ...


# 0649c220 08-Sep-2012 Tommie McAfee <tommie@couchbase.com>

CBQE-572: support for multi-delete in systest fw

Retrieve batch keys and via deleteq
using ruby client which operates in quiet mode
by default.

Change-Id: I9626ff0425de277c4

CBQE-572: support for multi-delete in systest fw

Retrieve batch keys and via deleteq
using ruby client which operates in quiet mode
by default.

Change-Id: I9626ff0425de277c46489e774da8e4970b80c21e
Reviewed-on: http://review.couchbase.org/20676
Reviewed-by: Tommie McAfee <tommie@couchbase.com>
Tested-by: Tommie McAfee <tommie@couchbase.com>

show more ...


# 436262b3 03-Sep-2012 Tommie McAfee <tommie@couchbase.com>

CBQE-572: getJsonMsg method from message broker

wrapper around regular getMsg function for
converting result from mq broker to json

Change-Id: If2e43184af9d5905870080372a2693e48

CBQE-572: getJsonMsg method from message broker

wrapper around regular getMsg function for
converting result from mq broker to json

Change-Id: If2e43184af9d5905870080372a2693e48f7a6d5c
Reviewed-on: http://review.couchbase.org/20445
Reviewed-by: Tommie McAfee <tommie@couchbase.com>
Tested-by: Tommie McAfee <tommie@couchbase.com>

show more ...


# 330a0995 30-Jul-2012 Tommie McAfee <tommie@couchbase.com>

Systest Platform

Initial check in of platform for system testing.
Adding pysystest directory which contains
a python-celery application that operates as a
worker for performing t

Systest Platform

Initial check in of platform for system testing.
Adding pysystest directory which contains
a python-celery application that operates as a
worker for performing tasks. Package also
consists of a cbsystest.py command line interface
for defining and running new workloads.
Currently python and ruby are supported sdks
for handling underlying requests to the couch cluster.

Environmental Vars:
testcfg.py

Starting celery worker with scheduler(-B):
celery worker -A app -B

Running a workload:
1) Define simple document template:
python cbsystest.py import template --name "default" --kvpair ' "hello":"world"
2) Run workload using predifined template at 1k ops/sec
python cbsystest.py run workload --create 50 --get 50 --ops 1000 --cc_queues docs_to_consume --template "default

Change-Id: I5979b5f9ac3ddc4581f7f64899abdd4c8166a510
Reviewed-on: http://review.couchbase.org/19019
Reviewed-by: Tommie McAfee <tommie@couchbase.com>
Tested-by: Tommie McAfee <tommie@couchbase.com>

show more ...