Name Date Size

..11-Feb-20204 KiB

.gitignoreH A D11-Feb-20201.7 KiB

.gitmodulesH A D11-Feb-202097

.shipitH A D11-Feb-2020152

AUTHORSH A D11-Feb-202069

ChangeLogH A D11-Feb-202018.9 KiB

CMakeLists.txtH A D11-Feb-20207.2 KiB

conflate/H11-Feb-20204 KiB

COPYINGH A D11-Feb-20201.5 KiB

debian/H11-Feb-20204 KiB

devtools/H11-Feb-20204 KiB

doc/H11-Feb-20204 KiB

DoxyfileH A D11-Feb-202061.3 KiB

HACKINGH A D11-Feb-20201.2 KiB

include/H11-Feb-20204 KiB

libmemcached/H11-Feb-20204 KiB

LICENSEH A D11-Feb-20201.5 KiB

memcached.spec.inH A D11-Feb-20202.9 KiB

NEWSH A D11-Feb-202040

READMEH A D11-Feb-20206.4 KiB

scripts/H11-Feb-20204 KiB

src/H11-Feb-20204 KiB

t/H11-Feb-20204 KiB

tests/H11-Feb-20204 KiB

TODOH A D11-Feb-2020985

TODO.moxiH A D11-Feb-202011.3 KiB

vbucket/H11-Feb-20204 KiB

README

1
2                                moxi
3           a memcached/membase proxy with energy and pep
4
5What is it?
6-----------
7
8moxi is a proxy capable of handling many connections for client
9applications, providing those clients simplified management and
10increased performance.  It can be used with memcached servers or
11a Membase Cluster hosting both membase and memcached type buckets.
12
13For more information see http://membase.org
14
15Running
16-------
17
18To have moxi load a "vBucket" json configuration from a REST/HTTP
19server, try...
20
21  moxi http://host:port/url/of/vBucketServerMapJSON
22
23Against a NorthScale server, for example, this would look like...
24
25  moxi http://host:8091/pools/default/bucketsStreamingConfig/default
26
27The above, also, is just shorthand (assuming you have no other flags
28specified), for the following explicit command-line...
29
30  moxi -z url=http://host:8091/url/of/vBucketServerMap.json
31
32To provide reliability when running with multiple servers, moxi will 
33accept multiple URLs.  For example...
34
35  moxi \ 
36  http://host:8091/pools/default/bucketsStreamingConfig/default,http://host2:8091/pools/default/bucketsStreamingConfig/default
37
38When used with particular buckets, authentication may be required.
39This can be done with the additional arguments to the -z flag...
40
41  moxi -z auth=USER%PASSWD,url=http://host:8091...
42
43To get more command line usage info:
44
45  moxi -h
46
47  Keeping moxi Running
48  --------------------
49
50On UNIX and UNIX-like systems, moxi may be started with an external
51service management system.  This will vary from system to system
52but many have BSD or SysV style init systems.  Other, more advanced
53service systems may be used too. 
54
55  File-based configuration:
56  -------------------------
57
58You may also provide a configuration file to moxi that holds a static
59vBucket server map.  This is most often used during development and
60testing.  The file (such as vbucket1.cfg) would look like...
61
62  11211 = {
63    "hashAlgorithm": "CRC",
64    "numReplicas": 0,
65    "serverList": ["memcached_svr1:11311"],
66    "vBucketMap":
67      [
68        [0],
69        [0]
70      ]
71  }
72
73The above configuration would tell moxi to listen on port 11211, and
74proxy to memcached_svr1:11311.  To use a static configuration file,
75you would start moxi like...
76
77  moxi -z ./vbucket1.cfg
78
79The "./" path prefix is required so that moxi knows you're passing in
80a config file.
81
82Developing
83----------
84
85Dependencies:
86
87   -- libevent, http://www.monkey.org/~provos/libevent/ (libevent-dev)
88
89   -- libconflate, http://github.com/northscale/libconflate
90
91   -- libvbucket, http://github.com/northscale/libvbucket
92
93As a backwards-compatible (ketama/consistent-hashing) alternative to
94libvbucket, you may instead use libmemcached instead of libvbucket...
95
96   -- libmemcached, http://tangent.org/552/libmemcached.html
97
98To use moxi against membase, however, you'll want libvbucket.
99
100                     -   -   -
101
102To compile moxi (assuming you want libvbucket), after you got the
103dependencies built and installed:
104
105  ./config/autorun.sh
106  ./configure
107  make
108
109For example, if libevent is installed in /opt/local, you'd do...
110
111  ./config/autorun.sh
112  ./configure --with-libevent=/opt/local
113  make
114
115  Testing:
116  --------
117
118  # To test that moxi still behaves like memcached
119  # and passes all the tests that memcached passes...
120  #
121  make test
122
123  # To test moxi in a simple proxy topology of...
124  #  client <-> moxi <-> memcached
125  #
126  ./t/moxi.pl
127
128  # To test moxi in a chained proxy topology of...
129  #   client <-> moxi <-> moxi <-> moxi <-> memcached
130  #
131  ./t/moxi.pl chain
132
133  # To test moxi in a fanout topology of...
134  #   client <---> moxi <---> memcached
135  #                       |-> memcached
136  #                       |-> memcached
137  #                       |-> memcached
138  #
139  ./t/moxi.pl fanout
140
141  # To test moxi in a fanout and back in again topology of...
142  #   client <---> moxi <---> moxi <-> memcached
143  #            |-> moxi <-|
144  #            |-> moxi <-|
145  #            |-> moxi <-|
146  #
147  ./t/moxi.pl fanoutin
148
149  # To test moxi proxy cases...
150  #
151  ./moxi -z 11333=localhost:11311 -t 1
152  python t/moxi_mock.py
153
154                     -   -   -
155
156For vbucket development, start the following...
157
158  ruby ./t/rest_mock.rb
159
160Then start a "pretend" memcached server...
161
162  ./moxi -vvv -p 11311
163
164Then...
165
166  ./moxi -vvv -z url=http://localhost:4567/pools/default/bucketsStreaming/default -Z port_listen=11211
167
168Then...
169
170  telnet localhost 11211
171
172                     -   -   -
173More notes:
174
175If using Linux, you need a kernel with epoll (it's better than select()).
176
177epoll isn't in Linux 2.4 yet, but there's a backport at:
178
179    http://www.xmailserver.org/linux-patches/nio-improve.html
180
181You want the epoll-lt patch (level-triggered).
182
183If you're using MacOS, you'll want libevent 1.1 or higher to deal with
184a kqueue bug.
185
186Also, be warned that the -k (mlockall) option to memcached might be
187dangerous when using a large cache.  Just make sure the memcached machines
188don't swap.  memcached does non-blocking network I/O, but not disk.  (it
189should never go to disk, or you've lost the whole point of it)
190
191The memcached website is at:
192
193    http://memcached.org
194
195                     -   -   -
196
197Building moxi for use with libmemcached
198---------------------------------------
199
200If you want to use moxi explicitly using ketama / consistent-hashing,
201instead of libvbucket with membase, you'll need to compile moxi with
202libmemcached.
203
204First, you would configure moxi differently at build-time...
205
206  ./configure --enable-moxi-vbucket=no \
207     CFLAGS=<libmemcached/include> \
208     LDFLAGS=<libmemcahed/lib>
209
210After building, moxi understands the following kind of command-line...
211
212  moxi -z <port=<more-config>>
213
214  moxi -z <port=<memcached_host:memcached_port(,*)>>
215
216moxi will listen on the given port and accept connections from
217upstream memcached clients.  It will forward requests to downstream
218memcached servers running on memcached_host:memcached_port.  For
219example...
220
221  moxi -z 11211=my_memcached_server:11222
222
223Above, moxi will listen on port 11211 and forward requests
224to the memcached running on my_memcached_server that's listening
225on port 11222.
226
227You can list more than one memcached_host:memcached_port,
228separated by commas.  For example...
229
230  moxi -z 11211=memcached_1:11211,memcached_2:11211
231
232The default downstream port is 11211, so you can also
233just use...
234
235  moxi -z 11211=memcached_1,memcached_2
236
237If you have some memcached servers running on the same server,
238but on different ports, you can put moxi in front of them
239with something like...
240
241  moxi -z 11211=server:11222,server:11233
242
243