Name Date Size

..11-Feb-20204 KiB

.gitignoreH A D11-Feb-202012

cmake/Modules/H11-Feb-20204 KiB

CMakeLists-moxi.txtH A D11-Feb-20201.2 KiB

CMakeLists.txtH A D11-Feb-20207.1 KiB

deps/H11-Feb-20204 KiB

GNUmakefileH A D11-Feb-20201.1 KiB

MakefileH A D11-Feb-20202.7 KiB

memcached-wrapper.inH A D11-Feb-2020822

moxi-wrapper.inH A D11-Feb-2020125

README.markdownH A D11-Feb-202019.4 KiB

tag_release.pyH A D11-Feb-20201.3 KiB

tsan.suppressionsH A D11-Feb-20202.4 KiB

valgrind.suppH A D11-Feb-20203.9 KiB

win32/H11-Feb-20204 KiB

README.markdown

1# tlm - Top-level Makefile
2
3What's this, then? The tlm project contains the make information for
4building Couchbase on multiple platforms.
5
6**Table of Contents**
7
8- [Content](#user-content-content)
9- [How to build](#user-content-how-to-build)
10	- [Simple](#user-content-simple-build)
11	- [Customize your builds](#user-content-customize-your-builds)
12- [Microsoft Windows 2008R2](#user-content-microsoft-windows-2008r2)
13	- [Configuration](#user-content-configuration)
14		- [git](#user-content-git)
15	- [How to build](#user-content-how-to-build-1)
16- [MacOSX](#user-content-macosx)
17- [Ubuntu 14.04](#user-content-ubuntu-1404)
18- [Fedora 21](#user-content-fedora-21)
19- [OpenSUSE](#opensuse)
20- [SmartOS containers](#user-content-smartos)
21	- [SmartOS](#user-content-smartos-container)
22	- [CentOS 7](#user-content-centos-7)
23	- [Ubuntu](#user-content-ubuntu)
24	- [Debian 7](#user-content-debian7)
25- [Static Analysis](#user-content-static-analysis)
26
27## Content
28
29The file named `CMakeLists.txt` contains the full build description
30for Couchbase. It should be copied to the top of your source directory
31(this happens automatically when you are using repo).
32
33The `cmake` directory contains macros used by cmake to configure a
34build of Couchbase.
35
36`Makefile` is a convenience file that repo will put in the root of
37your source directory. It invokes `cmake` with a specific set of
38options. "All" flavors of make should be able to parse this makefile,
39and its defaults are set to match building on Microsoft Windows.
40
41`GNUmakefile` is another convenience file that repo will put in the
42root of your source directory. GNU make will favor this file over
43`Makefile` and this file just overrides the defaults specified in
44`Makefile`.
45
46## How to build
47
48Couchbase utilizes [CMake][cmake_link] in order to provide build
49support for a wide range of platforms. CMake isn't a build system
50like GNU Autotools, but a tool that generates build information for
51external systems like: Visual Studio projects, XCode projects and
52Makefiles to name a few. Their good support for Microsoft Windows and
53Makefiles is the primary reason why we decided to move away from GNU
54Autotools. CMake isn't a magic pill that solves all our problems; it
55comes with its own list of challenges.
56
57It is recommended to perform "out of source builds", which means that
58the build artifacts is stored _outside_ the source directory.
59
60### Simple build
61
62If you just want to build Couchbase and without any special
63configuration, you may use the Makefile we've supplied for your
64convenience:
65
66    trond@ok > mkdir source
67    trond@ok > cd source
68    trond@ok source> repo init -u git://github.com/couchbase/manifest -m branch-master.xml
69    trond@ok source> repo sync
70    trond@ok source> make
71
72This would install the build software in a subdirectory named
73`install`. To change this you may run:
74
75    trond@ok source> make PREFIX=/opt/couchbase
76
77### Customize your builds
78
79CMake offers a wide range of customizations, and this chapter won't
80try to cover all of them. There is plenty of documentation available
81on the [webpage](http://www.cmake.org/cmake/help/documentation.html).
82
83There is no point of trying to keep a list of all tunables in this
84document. To find the tunables you have two options: look in
85`cmake/Modules/*.cmake` or you may look in the cache file generated
86during a normal build (see `build/CMakeCache.txt`)
87
88There are two ways to customize your own builds. You can do it all by
89yourself by invoking cmake yourself:
90
91    trond@ok > mkdir source
92    trond@ok > mkdir build
93    trond@ok > cd source
94    trond@ok source> repo init -u git://github.com/couchbase/manifest -m branch-master.xml
95    trond@ok source> repo sync
96    trond@ok source> cd ../build
97    trond@ok build> cmake -D CMAKE_INSTALL_PREFIX=/opt/couchbase -D CMAKE_BUILD_TYPE=Debug -D DTRACE_FOUND:BOOL=True -D DTRACE:FILEPATH=/usr/sbin/dtrace CMAKE_PREFIX_PATH="/opt/r14b04;/opt/couchbase"
98    trond@ok build> gmake all install
99
100Or pass extra options to the convenience Makefile provided:
101
102    trond@ok > mkdir source
103    trond@ok > mkdir build
104    trond@ok > cd source
105    trond@ok source> repo init -u git://github.com/couchbase/manifest -m branch-master.xml
106    trond@ok source> repo sync
107    trond@ok source> make PREFIX=/opt/couchbase CMAKE_PREFIX_PATH="/opt/r14b04;/opt/couchbase" EXTRA_CMAKE_OPTIONS='-D DTRACE_FOUND:BOOL=True -D DTRACE:FILEPATH=/usr/sbin/dtrace'
108
109Use `CMAKE_PREFIX_PATH` to specify a "list" of directories to search
110for tools/libraries if they are stored in "non-standard"
111locations. Ex:
112
113    CMAKE_PREFIX_PATH="/opt/r14b04;/opt/couchbase;/opt/local"
114
115## Microsoft Windows 2008R2
116
117The following steps are needed to build Couchbase on Microsoft Windows 2008R2:
118
119* Install OS, activate and run Windows Update and install all updates
120* Install Google Chrome (optional, but makes your life easier)
121* Install [Visual Studio 2013 Professional][win_visual_studio_link]
122* Install all updates from microsoft update
123* Install [GIT][win_git_link] and select the option to add GIT to path
124* Install [Python 2.7][win_python_link] and add c:\python27 to path (manually)
125* Install [7-ZIP][win_7zip_link] and add the installation to path (manually)
126* Install [CMake][win_cmake_link] and add to path
127* Install [MinGW][mingw_link] and add to path
128* Download and install [2008 runtime extensions][win_2008_runtime_ext_link]
129
130Note: Significant portions of Couchbase Server are written in Go. Go itself
131is automatically downloaded as part of the build, but Go requires gcc (and
132not Visual Studio) in order to interface with C libraries. Our builds are
133tested with MinGW 4.8.3.
134
135### Configuration
136
137#### git
138
139Repo will complain if git isn't properly configured. Setting name and
140email should be sufficient, but you also may at least want to set the
141two additional settings suggested:
142
143    C:\> git config --global user.email trond.norbye@gmail.com
144    C:\> git config --global user.name "Trond Norbye"
145    C:\> git config --global color.ui false
146    C:\> git config --global core.autocrlf true
147
148### How to build
149
150Before you may start to build on Microsoft Windows you have to set up
151the environment. The script `environment.bat` is located in the `win32`
152directory.
153
154Open cmd.com and type in the following (assuming c:\compile\couchbase
155is the directory holding your source):
156
157    C:\> set source_root=c:\compile\couchbase
158    C:\> set target_arch=amd64
159    C:\> environment
160
161You may now follow the build description outlined in [How to
162build](#user-content-how-to-build). Please note that the make utility
163on windows is named `nmake`.
164
165## MacOSX
166
167Multiple versions of Mac OSX may work, but this list is verified with
168El Capitan
169
170* Install XCode 7.2
171* Install [Homebrew][homebrew_link]
172
173Install the following packages from homebrew:
174
175    trond@ok> brew install cmake git ccache openssl
176
177You should be all set to start compile the server as described above.
178
179## Ubuntu 14.04
180
181The steps below may work on other versions of Ubuntu as well, but this
182procedure is verified with a clean installation of Ubuntu 14.04.1
183
184    sudo su -
185    wget https://storage.googleapis.com/git-repo-downloads/repo
186    chmod a+x repo
187    mv repo /usr/local/bin
188    apt-get install -y git gcc g++ ccache cmake libssl-dev libicu-dev \
189                       erlang
190
191## Fedora 21
192
193The steps below may work on other versions of Fedora as well, but this
194procedure is verified with a clean installation of Fedora 21
195
196    sudo su -
197    wget https://storage.googleapis.com/git-repo-downloads/repo
198    chmod a+x repo
199    mv repo /usr/local/bin
200    yum install -y gcc gcc-c++ git cmake ccache redhat-lsb-core \
201                   erlang openssl-devel libicu-devel
202
203## OpenSUSE
204
205I tested this on a clean install of OpenSUSE 13.2 by choosing the
206defaults during the installer except choosing gnome desktop and enable
207ssh access.
208
209    sudo zypper install gcc gcc-c++ autoconf automake ncurses-devel \
210                        git ccache libopenssl-devel cmake
211
212Open a new terminal to ensure you get an updated environment (the
213package install modifies some of the environement variables)
214
215    curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
216    chmod a+x ~/bin/repo
217    sudo mkdir /opt/couchbase
218    sudo chown `whoami` /opt/couchbase
219    mkdir -p compile/couchbase
220    cd compile/couchbase
221    repo init -u git://github.com/couchbase/manifest -m branch-master.xml -g default,build
222    repo sync
223    repo start opensuse --all
224    mkdir cbdeps && cd cbdeps
225    ../cbbuild/cbdeps/build-all-sherlock.sh
226    export CB_DOWNLOAD_DEPS_CACHE=`pwd`/output
227    export CB_DOWNLOAD_DEPS_MANIFEST=`pwd`/output/manifest.cmake
228    unset GOBIN
229    cd ..
230    gmake PREFIX=/opt/couchbase
231
232Note: Unfortunately the build-all-sherlock.sh script mentioned above is
233likely out of date for Couchbase Server builds later than 4.1.
234
235You should be able to start the server by running
236
237    /opt/couchbase/bin/couchbase-server start
238
239## SmartOS
240
241The following chapters describes how to configure and build under
242various containers hosted by SmartOS. [Joyent][joyent_link] provides a
243variety of datasets for various operating systems (CentOS, Fedora,
244FreeBSD, Debian, SmartOS, ...). This section is not intended to cover
245all of these, but covers a set of configurations known to work.
246
247### SmartOS container
248
249The following descrtiption use the standard64 (14.2.1) image imported by:
250
251    root@smartos~> imgadm import 3f57ffe8-47da-11e4-aa8b-dfb50a06586a
252
253The KVM may be created with the following attributes (store in `smartos.json`):
254
255    {
256      "alias" : "compilesrv",
257      "autoboot": true,
258      "brand": "joyent",
259      "dns_domain" : "norbye.org",
260      "resolvers" : [ "8.8.4.4" ],
261      "image_uuid" : "3f57ffe8-47da-11e4-aa8b-dfb50a06586a",
262      "hostname" : "compilesrv",
263      "filesystems" : [
264       {
265          "type" : "lofs",
266          "source" : "/zones/home",
267          "target" : "/home",
268          "options" : "nodevices"
269        }
270      ],
271      "max_physical_memory": 4096,
272      "nics": [
273         {
274          "nic_tag": "admin",
275          "ip": "10.0.0.207",
276          "netmask": "255.255.255.0",
277          "gateway": "10.0.0.1"
278        }
279      ]
280    }
281
282Create the KVM with:
283
284    root@smartos~> vmadm create -f smartos.json
285
286Log into the newly created zone and install the following packages:
287
288    root@compilesrv~> pkgin update
289    root@compilesrv~> pkgin -y in py27-expat-2.7.7 icu-53.1 erlang-16.1.2nb3 go-1.3
290
291[Install Google repo][google_repo_link] and you should be all set to
292start building the code as described above.
293
294### CentOS 7
295
296The following descrtiption use the Centos-7 image imported by:
297
298    root@smartos~> imgadm import 553da8ba-499e-11e4-8bee-5f8dadc234ce
299
300The KVM may be created with the following attributes (store in `centos.json`):
301
302    {
303      "alias": "centos-7",
304      "brand": "kvm",
305      "resolvers": [
306        "10.0.0.1",
307        "8.8.4.4"
308      ],
309      "default-gateway": "10.0.0.1",
310      "hostname":"centos",
311      "ram": "6144",
312      "vcpus": "2",
313      "nics": [
314        {
315          "nic_tag": "admin",
316          "ip": "10.0.0.201",
317          "netmask": "255.255.255.0",
318          "gateway": "10.0.0.1",
319          "model": "virtio",
320          "primary": true
321        }
322      ],
323      "disks": [
324        {
325          "image_uuid": "553da8ba-499e-11e4-8bee-5f8dadc234ce",
326          "boot": true,
327          "model": "virtio",
328          "image_size": 10240
329        },
330       {
331          "model": "virtio",
332          "size": 10240
333        }
334      ],
335    "customer_metadata": {
336        "root_authorized_keys": "<my ssh key>"
337      }
338    }
339
340Create the KVM with:
341
342    root@smartos~> vmadm create -f centos.json
343
344You should now be able to ssh into the machine and run `yum update` and
345install all of the updates ;-)
346
347Install as much as possible of the precompiled dependencies with:
348
349    yum install -y gcc gcc-c++ make redhat-lsb git openssl-devel
350
351A newer version of cmake and repo is needed so we have to compile
352them from source with:
353
354    wget http://www.cmake.org/files/v3.2/cmake-3.2.1.tar.gz
355    tar xfz cmake-3.2.1.tar.gz
356    cd cmake-3.2.1
357    ./bootstrap && make && make install
358    cd /usr/local/bin
359    curl https://storage.googleapis.com/git-repo-downloads/repo > repo
360    chmod a+x repo
361
362And you should be all set to start building the code as described above.
363
364### Ubuntu
365
366The following descrtiption use the Ubuntu 14.04 image imported by:
367
368    root@smartos~> imgadm import c864f104-624c-43d2-835e-b49a39709b6b
369
370The KVM may be created with the following attributes (store in `ubuntu.json`):
371
372    {
373      "alias": "ubuntu-1404",
374      "brand": "kvm",
375      "resolvers": [
376        "10.0.0.1",
377        "8.8.4.4"
378      ],
379      "default-gateway": "10.0.0.1",
380      "hostname":"ubuntu",
381      "ram": "6144",
382      "vcpus": "2",
383      "nics": [
384        {
385          "nic_tag": "admin",
386          "ip": "10.0.0.203",
387          "netmask": "255.255.255.0",
388          "gateway": "10.0.0.1",
389          "model": "virtio",
390          "primary": true
391        }
392      ],
393      "disks": [
394        {
395          "image_uuid": "c864f104-624c-43d2-835e-b49a39709b6b",
396          "boot": true,
397          "model": "virtio",
398          "image_size": 10240
399        },
400       {
401          "model": "virtio",
402          "size": 20480
403        }
404      ],
405    "customer_metadata": {
406        "root_authorized_keys": "<my ssh key>"
407      }
408    }
409
410Create the KVM with:
411
412    root@smartos~> vmadm create -f ubuntu.json
413
414You should now be able to ssh into the machine and run `aptitude` and
415install all of the updates ;-)
416
417Install as much as possible of the precompiled dependencies with:
418
419    apt-get update --fix-missing
420    apt-get install -y git gcc g++ make ccache lsb-release libssl-dev cmake
421
422A newer version of repo is needed:
423
424    cd /usr/local/bin
425    curl https://storage.googleapis.com/git-repo-downloads/repo > repo
426    chmod a+x repo
427
428And you should be all set to start building the code as described above.
429
430### Debian7
431
432The following descrtiption use the Debian image imported by:
433
434    root@smartos~> imgadm import 5f41692e-a70d-11e4-8c2d-afc6735144dc
435
436The KVM may be created with the following attributes (store in `debian7.json`):
437
438    {
439      "alias": "debian-7",
440      "brand": "kvm",
441      "resolvers": [
442        "10.0.0.1",
443        "8.8.4.4"
444      ],
445      "default-gateway": "10.0.0.1",
446      "hostname":"debian",
447      "ram": "6144",
448      "vcpus": "2",
449      "nics": [
450        {
451          "nic_tag": "admin",
452          "ip": "10.0.0.200",
453          "netmask": "255.255.255.0",
454          "gateway": "10.0.0.1",
455          "model": "virtio",
456          "primary": true
457        }
458      ],
459      "disks": [
460        {
461          "image_uuid": "5f41692e-a70d-11e4-8c2d-afc6735144dc",
462          "boot": true,
463          "model": "virtio",
464          "image_size": 10240
465        }
466      ],
467      "customer_metadata": {
468        "root_authorized_keys": "ssh-rsa <my-personal-public-key>"
469      }
470    }
471
472Create the KVM with:
473
474    root@smartos~> vmadm create -f debian7.json
475
476You should now be able to ssh into the machine and run `aptitude` and
477install all of the updates ;-)
478
479Install as much as possible of the precompiled dependencies with:
480
481    apt-get update --fix-missing
482    apt-get install -y git gcc g++ make ccache lsb-release libssl-dev
483
484A newer version of cmake and repo is needed so we have to compile
485it from source with:
486
487    wget http://www.cmake.org/files/v3.2/cmake-3.2.1.tar.gz
488    tar xfz cmake-3.2.1.tar.gz
489    cd cmake-3.2.1
490    ./bootstrap && make && make install
491    cd /usr/local/bin
492    curl https://storage.googleapis.com/git-repo-downloads/repo > repo
493    chmod a+x repo
494
495And you should be all set to start building the code as described above.
496
497## Static Analysis
498
499There are pre-canned build rules to allow you to run the
500[Clang Static Analyzer][clang_static_analyzer_link] against the Couchbase
501codebase.
502
503So far this has only been tested on OS X, using Clang shipping as part
504of OS X Developer Tools. It *should* be possible to also run on other
505platforms which Clang/LLVM is available, however this isn't tested.
506
507### Prerequisites
508
509* Install `clang` (from OS X Developer Tools). If you can build from source you should already have this :)
510* Download and extract clang Static Analyzer tools
511  (from [clang-analyzer.llvm.org][clang_static_analyzer_link]).
512  Note that while the actual analyzer functionality is built into
513  clang, this is needed for `scan-build` and `scan-view` tools to
514  invoke and display the analyser results.
515
516### Running
517
518*  Add `scan-build` and `scan-view` to your path:
519
520        export PATH=$PATH:/path/to/scan-build
521
522*  Run `make analyze` at the top-level to configure clang-analyser as the 'compiler':
523
524        make analyze
525
526*  At the end you will see a message similar to the following - Invoke the specified command to browse the found bugs:
527
528
529        scan-build: 31 bugs found.
530        scan-build: Run 'scan-view /source/build-analyzer/analyser-results/2014-06-05-173247-52416-1' to examine bug reports.
531
532## Thread Sanitizer and Address Sanitizer
533
534There are pre-canned build rules to allow you to build with
535[ThreadSanitizer][thread_sanitizer_link], to detect threading issues;
536or [AddressSanitizer][address_sanitizer_link], to detect memory
537errors.
538
539### Prerequities
540
541* A compiler which supports ThreadSanitizer / AddressSanitizer. Recent
542  version of Clang (3.2+) or GCC (4.8+) are claimed to work, however
543  only Clang (3.5+) has been tested.
544
545### Running
546
547* Ensure that the compiler supporting Thread/Address Sanitizer is chosen by
548  CMake. If it's the system default compiler there is nothing to do;
549  otherwise you will need to set both `CC` and `CXX` environment
550  variables to point to the C / C++ compiler before calling the build
551  system.
552
553* Pass the variable `CB_THREADSANITIZER=1` / `CB_ADDRESSSANITIZER=1`to CMake.
554
555ThreadSanitizer one liner for a Ubuntu-based system where Clang isn't
556the default system compiler:
557
558        CC=clang CXX=clang++ make EXTRA_CMAKE_OPTIONS="-D CB_THREADSANITIZER=1"
559
560and for AddressSanitizer:
561
562        CC=clang CXX=clang++ make EXTRA_CMAKE_OPTIONS="-D CB_ADDRESSSANITIZER=1"
563
564* Run one or more tests. Any issues will be reported (to stderr by default).
565
566### Customizing ThreadSanitizer / AddressSanitizer
567
568See `cmake/Modules/CouchbaseThreadSanitizer.cmake` CMake fragment for
569how ThreadSanizer is configured.
570
571See the `TSAN_OPTIONS` environment variable (documented on the
572ThreadSanitizer [Flags][thread_sanitizer_flags] wiki page) for more
573information on configuring.
574
575Similarly for AddressSanitizer see
576`cmake/Modules/CouchbaseAddressSanitizer.cmake`, and the
577`ASAN_OPTIONS` environment variable (documented on the
578AddressSanitizer [Flags][address_sanitizer_flags] wiki page) for
579details..
580
581[win_visual_studio_link]: http://hub.internal.couchbase.com/confluence/download/attachments/7242678/en_visual_studio_professional_2013_x86_web_installer_3175305.exe?version=1&modificationDate=1389383332000
582[win_git_link]: http://git-scm.com/download/win
583[win_python_link]: http://www.python.org/download/releases/2.7/
584[win_7zip_link]: http://downloads.sourceforge.net/sevenzip/7z920-x64.msi
585[win_cmake_link]: http://www.cmake.org/cmake/resources/software.html
586[win_mingw_link]: http://www.mingw.org/
587[win_2008_runtime_ext_link]: http://www.microsoft.com/en-us/download/confirmation.aspx?id=15336
588[google_repo_link]: http://source.android.com/source/downloading.html#installing-repo
589[homebrew_link]: http://brew.sh/
590[cmake_link]: http://www.cmake.org/cmake/
591[clang_static_analyzer_link]: http://clang-analyzer.llvm.org
592[thread_sanitizer_link]: https://code.google.com/p/thread-sanitizer/wiki/CppManual
593[thread_sanitizer_flags]: https://code.google.com/p/thread-sanitizer/wiki/Flags
594[address_sanitizer_link]: https://github.com/google/sanitizers/wiki/AddressSanitizer
595[address_sanitizer_flags]: https://github.com/google/sanitizers/wiki/AddressSanitizerFlags
596