Name Date Size

..11-Feb-20204 KiB

.gitignoreH A D11-Feb-20201,007

.gitmodulesH A D11-Feb-202097

.shipitH A D11-Feb-2020152

agent.hH A D11-Feb-20202.7 KiB

agent_config.cH A D11-Feb-202051.8 KiB

agent_ping.cH A D11-Feb-202010 KiB

agent_stats.cH A D11-Feb-202070.2 KiB

assoc.cH A D11-Feb-20207.2 KiB

assoc.hH A D11-Feb-2020309

AUTHORSH A D11-Feb-202069

cache.cH A D11-Feb-20203.7 KiB

cache.hH A D11-Feb-20204.2 KiB

ChangeLogH A D11-Feb-202018.9 KiB

check_moxi.cH A D11-Feb-202016.3 KiB

check_moxi_agent.cH A D11-Feb-202018.8 KiB

check_util.cH A D11-Feb-20207.7 KiB

check_work.cH A D11-Feb-20202.4 KiB

cJSON.cH A D11-Feb-202019.7 KiB

cJSON.hH A D11-Feb-20205.4 KiB

config/H11-Feb-20204 KiB

config_static.hH A D11-Feb-20202 KiB

configure.acH A D11-Feb-20205.9 KiB

COPYINGH A D11-Feb-20201.5 KiB

cproxy.cH A D11-Feb-2020114.1 KiB

cproxy.hH A D11-Feb-202030.5 KiB

cproxy_config.cH A D11-Feb-202029.6 KiB

cproxy_front.cH A D11-Feb-202012.3 KiB

cproxy_multiget.cH A D11-Feb-202015.8 KiB

cproxy_protocol_a.cH A D11-Feb-202018.3 KiB

cproxy_protocol_a2a.cH A D11-Feb-202019.4 KiB

cproxy_protocol_a2b.cH A D11-Feb-202063.5 KiB

cproxy_protocol_b.cH A D11-Feb-202013.6 KiB

cproxy_protocol_b2b.cH A D11-Feb-202016.5 KiB

cproxy_stats.cH A D11-Feb-202017.1 KiB

daemon.cH A D11-Feb-20203 KiB

debian/H11-Feb-20204 KiB

devtools/H11-Feb-20204 KiB

doc/H11-Feb-20204 KiB

DoxyfileH A D11-Feb-202061.3 KiB

genhash.cH A D11-Feb-20206.7 KiB

genhash.hH A D11-Feb-20205.2 KiB

genhash_int.hH A D11-Feb-2020324

globals.cH A D11-Feb-2020808

HACKINGH A D11-Feb-20201.2 KiB

hash.cH A D11-Feb-202014.4 KiB

hash.hH A D11-Feb-2020205

htgram.cH A D11-Feb-20208.6 KiB

htgram.hH A D11-Feb-20204 KiB

htgram_test.cH A D11-Feb-20203.7 KiB

items.cH A D11-Feb-202016.9 KiB

items.hH A D11-Feb-20201,001

LICENSEH A D11-Feb-20201.5 KiB

log.cH A D11-Feb-20206.3 KiB

log.hH A D11-Feb-20201.3 KiB

m4/H11-Feb-20204 KiB

Makefile.amH A D11-Feb-20205.4 KiB

matcher.cH A D11-Feb-20205.3 KiB

matcher.hH A D11-Feb-20201 KiB

mcs.cH A D11-Feb-202019.5 KiB

mcs.hH A D11-Feb-20202.9 KiB

memcached.cH A D11-Feb-2020161.6 KiB

memcached.hH A D11-Feb-202022.2 KiB

memcached.spec.inH A D11-Feb-20202.9 KiB

murmur_hash.cH A D11-Feb-20201.6 KiB

NEWSH A D11-Feb-202040

protocol_binary.hH A D11-Feb-202013 KiB

READMEH A D11-Feb-20206.4 KiB

redirects.cH A D11-Feb-20202 KiB

redirects.hH A D11-Feb-20201.6 KiB

scripts/H11-Feb-20204 KiB

sizes.cH A D11-Feb-2020912

slabs.cH A D11-Feb-202016 KiB

slabs.hH A D11-Feb-20201.3 KiB

solaris_priv.cH A D11-Feb-20201.2 KiB

stats.cH A D11-Feb-202012.2 KiB

stats.hH A D11-Feb-2020342

stdin_check.cH A D11-Feb-2020740

stdin_check.hH A D11-Feb-202076

strsep.cH A D11-Feb-2020454

t/H11-Feb-20204 KiB

testapp.cH A D11-Feb-202014.5 KiB

thread.cH A D11-Feb-202020.5 KiB

timedrun.cH A D11-Feb-20202.3 KiB

TODOH A D11-Feb-2020985

TODO.moxiH A D11-Feb-202011.3 KiB

trace.hH A D11-Feb-20203 KiB

util.cH A D11-Feb-20204.8 KiB

util.hH A D11-Feb-20201.8 KiB

win32/H11-Feb-20204 KiB

work.cH A D11-Feb-20207.1 KiB

work.hH A D11-Feb-20201.4 KiB

README

1
2                                moxi
3           a memcached/membase proxy with energy and pep
4
5What is it?
6-----------
7
8moxi is a proxy capable of handling many connections for client
9applications, providing those clients simplified management and
10increased performance.  It can be used with memcached servers or
11a Membase Cluster hosting both membase and memcached type buckets.
12
13For more information see http://membase.org
14
15Running
16-------
17
18To have moxi load a "vBucket" json configuration from a REST/HTTP
19server, try...
20
21  moxi http://host:port/url/of/vBucketServerMapJSON
22
23Against a NorthScale server, for example, this would look like...
24
25  moxi http://host:8091/pools/default/bucketsStreamingConfig/default
26
27The above, also, is just shorthand (assuming you have no other flags
28specified), for the following explicit command-line...
29
30  moxi -z url=http://host:8091/url/of/vBucketServerMap.json
31
32To provide reliability when running with multiple servers, moxi will 
33accept multiple URLs.  For example...
34
35  moxi \ 
36  http://host:8091/pools/default/bucketsStreamingConfig/default,http://host2:8091/pools/default/bucketsStreamingConfig/default
37
38When used with particular buckets, authentication may be required.
39This can be done with the additional arguments to the -z flag...
40
41  moxi -z auth=USER%PASSWD,url=http://host:8091...
42
43To get more command line usage info:
44
45  moxi -h
46
47  Keeping moxi Running
48  --------------------
49
50On UNIX and UNIX-like systems, moxi may be started with an external
51service management system.  This will vary from system to system
52but many have BSD or SysV style init systems.  Other, more advanced
53service systems may be used too. 
54
55  File-based configuration:
56  -------------------------
57
58You may also provide a configuration file to moxi that holds a static
59vBucket server map.  This is most often used during development and
60testing.  The file (such as vbucket1.cfg) would look like...
61
62  11211 = {
63    "hashAlgorithm": "CRC",
64    "numReplicas": 0,
65    "serverList": ["memcached_svr1:11311"],
66    "vBucketMap":
67      [
68        [0],
69        [0]
70      ]
71  }
72
73The above configuration would tell moxi to listen on port 11211, and
74proxy to memcached_svr1:11311.  To use a static configuration file,
75you would start moxi like...
76
77  moxi -z ./vbucket1.cfg
78
79The "./" path prefix is required so that moxi knows you're passing in
80a config file.
81
82Developing
83----------
84
85Dependencies:
86
87   -- libevent, http://www.monkey.org/~provos/libevent/ (libevent-dev)
88
89   -- libconflate, http://github.com/northscale/libconflate
90
91   -- libvbucket, http://github.com/northscale/libvbucket
92
93As a backwards-compatible (ketama/consistent-hashing) alternative to
94libvbucket, you may instead use libmemcached instead of libvbucket...
95
96   -- libmemcached, http://tangent.org/552/libmemcached.html
97
98To use moxi against membase, however, you'll want libvbucket.
99
100                     -   -   -
101
102To compile moxi (assuming you want libvbucket), after you got the
103dependencies built and installed:
104
105  ./config/autorun.sh
106  ./configure
107  make
108
109For example, if libevent is installed in /opt/local, you'd do...
110
111  ./config/autorun.sh
112  ./configure --with-libevent=/opt/local
113  make
114
115  Testing:
116  --------
117
118  # To test that moxi still behaves like memcached
119  # and passes all the tests that memcached passes...
120  #
121  make test
122
123  # To test moxi in a simple proxy topology of...
124  #  client <-> moxi <-> memcached
125  #
126  ./t/moxi.pl
127
128  # To test moxi in a chained proxy topology of...
129  #   client <-> moxi <-> moxi <-> moxi <-> memcached
130  #
131  ./t/moxi.pl chain
132
133  # To test moxi in a fanout topology of...
134  #   client <---> moxi <---> memcached
135  #                       |-> memcached
136  #                       |-> memcached
137  #                       |-> memcached
138  #
139  ./t/moxi.pl fanout
140
141  # To test moxi in a fanout and back in again topology of...
142  #   client <---> moxi <---> moxi <-> memcached
143  #            |-> moxi <-|
144  #            |-> moxi <-|
145  #            |-> moxi <-|
146  #
147  ./t/moxi.pl fanoutin
148
149  # To test moxi proxy cases...
150  #
151  ./moxi -z 11333=localhost:11311 -t 1
152  python t/moxi_mock.py
153
154                     -   -   -
155
156For vbucket development, start the following...
157
158  ruby ./t/rest_mock.rb
159
160Then start a "pretend" memcached server...
161
162  ./moxi -vvv -p 11311
163
164Then...
165
166  ./moxi -vvv -z url=http://localhost:4567/pools/default/bucketsStreaming/default -Z port_listen=11211
167
168Then...
169
170  telnet localhost 11211
171
172                     -   -   -
173More notes:
174
175If using Linux, you need a kernel with epoll (it's better than select()).
176
177epoll isn't in Linux 2.4 yet, but there's a backport at:
178
179    http://www.xmailserver.org/linux-patches/nio-improve.html
180
181You want the epoll-lt patch (level-triggered).
182
183If you're using MacOS, you'll want libevent 1.1 or higher to deal with
184a kqueue bug.
185
186Also, be warned that the -k (mlockall) option to memcached might be
187dangerous when using a large cache.  Just make sure the memcached machines
188don't swap.  memcached does non-blocking network I/O, but not disk.  (it
189should never go to disk, or you've lost the whole point of it)
190
191The memcached website is at:
192
193    http://memcached.org
194
195                     -   -   -
196
197Building moxi for use with libmemcached
198---------------------------------------
199
200If you want to use moxi explicitly using ketama / consistent-hashing,
201instead of libvbucket with membase, you'll need to compile moxi with
202libmemcached.
203
204First, you would configure moxi differently at build-time...
205
206  ./configure --enable-moxi-vbucket=no \
207     CFLAGS=<libmemcached/include> \
208     LDFLAGS=<libmemcahed/lib>
209
210After building, moxi understands the following kind of command-line...
211
212  moxi -z <port=<more-config>>
213
214  moxi -z <port=<memcached_host:memcached_port(,*)>>
215
216moxi will listen on the given port and accept connections from
217upstream memcached clients.  It will forward requests to downstream
218memcached servers running on memcached_host:memcached_port.  For
219example...
220
221  moxi -z 11211=my_memcached_server:11222
222
223Above, moxi will listen on port 11211 and forward requests
224to the memcached running on my_memcached_server that's listening
225on port 11222.
226
227You can list more than one memcached_host:memcached_port,
228separated by commas.  For example...
229
230  moxi -z 11211=memcached_1:11211,memcached_2:11211
231
232The default downstream port is 11211, so you can also
233just use...
234
235  moxi -z 11211=memcached_1,memcached_2
236
237If you have some memcached servers running on the same server,
238but on different ports, you can put moxi in front of them
239with something like...
240
241  moxi -z 11211=server:11222,server:11233
242
243