Name Date Size

..19-Aug-20204 KiB

.gitattributesH A D19-Aug-2020104

.gitignoreH A D19-Aug-202012

clang-format-pre-commit-hookH A D19-Aug-20201.6 KiB

cmake/Modules/H19-Aug-20204 KiB

CMakeLists-cbq.txtH A D19-Aug-2020537

CMakeLists.txtH A D19-Aug-202011.5 KiB

deps/H19-Aug-20204 KiB

dot-clang-formatH A D19-Aug-20202.8 KiB

GNUmakefileH A D19-Aug-20201.1 KiB

install_hook.shH A D19-Aug-20202.1 KiB

MakefileH A D19-Aug-20202.7 KiB

memcached-wrapper.inH A D19-Aug-2020822

python/H19-Aug-20204 KiB

README.markdownH A D19-Aug-202010.9 KiB

scripts/H19-Aug-20204 KiB

tag_release.pyH A D19-Aug-20201.5 KiB

third-party-CMakeLists.txtH A D19-Aug-2020803

tsan.suppressionsH A D19-Aug-20202.4 KiB

valgrind.suppH A D19-Aug-20207.5 KiB

win32/H19-Aug-20204 KiB

README.markdown

1# tlm - "Top-level Makefile" for Couchbase Server
2
3This repository contains a number of tools and scripts for building
4Couchbase Server. The main interesting part is the top-level CMakeLists.txt,
5which is the entry point for a complete Server build. There are also a number
6of utility CMake libraries in cmake/Modules.
7
8## Software requirements
9
10* C/C++ compiler; one of:
11  * gcc 7.3 or newer
12  * Visual Studio 2017 or newer
13  * Xcode
14  * clang
15* CMake 3.12 or newer
16* Google repo (in order to fetch all of the source code)
17* A build tool such as Make or Ninja
18* ccache may speed up the development cycle when clang / gcc is used
19
20Our production builds currently use gcc 7.3.0 on most Linux platforms;
21Visual Studio 2017 on Windows; and Xcode 9.3.1 on MacOS.
22
23### Requirements on Windows
24
25In addition to Visual Studio, gcc must be installed and on the PATH for
26building some of the Go language tools. We use a recent version of MinGW for
27this.
28
29Couchbase uses Google repo to stitch together all of the individual git
30repositories. Repo is implemented in python, but it's unfortunately using
31features not available on python for windows. We use a modified version of
32repo from http://github.com/esrlabs/git-repo.
33
34It is important to set the git config option `core.longpaths` to `true`.
35
36In general it is quite challenging to get a Windows box configured perfectly
37for building Couchbase Server. If you are familiar with Ansible, it may be
38useful to look at the Ansible scripts we use to configure our build VMs.
39They are available here: https://github.com/couchbase/build-infra/tree/master/ansible/windows/couchbase-server/window2016
40
41### Additional software requirements on unsupported platforms
42
43Couchbase Server requires a great many libraries, computer languages, and
44build tools to successfully build. The list in the previous section should
45be all that is required to be installed prior to starting building, however.
46The remaining packages are pre-built by Couchbase and downloaded as part of
47the build on supported platforms.
48
49Supported platforms include, at this time of writing:
50
51* Windows (10, 2016, or newer)
52* MacOS (10.12 or newer)
53* Linux
54  * Ubuntu 16.04 or 18.04
55  * Debian 8 or 9
56  * SUSE 12 or 15
57  * Centos 7 or 8
58  * Amazon Linux 2
59
60If you are building on another platform, you will need to also provide all
61of the required tools and libraries. The canonical list of these packages
62can be found in the file `tlm\deps\manifest.cmake`. For the most part, if
63you install these tools and then ensure that `CMAKE_PREFIX_PATH` points to
64their installation directories, CMake will pick them up as part of the
65build. It is however beyond the scope of this document to cover exactly how
66all of those tools must be built and installed for use in a Couchbase Server
67build. We strongly recommend restricting building to supported platforms.
68
69If you are building on a platform which is similar to a supported platform
70but not exactly the same, you may be able to "lie" to the build about what
71platform you are on and have it download the supported pre-built binaries for
72a different platform. For instance, if you are building on Ubuntu 18.10,
73it may work to tell the build system that you're actually on Ubuntu 18.04 and
74have it download the required packages for you. To do this, set the CMake
75variable `CB_DOWNLOAD_DEPS_PLATFORM` to one of the platform strings from
76`manifest.cmake`, eg.
77
78    cmake -D CB_DOWNLOAD_DEPS_PLATFORM=ubuntu18.04 .....
79
80## How to build
81
82Couchbase utilizes [CMake][cmake_link] in order to provide build support for
83a wide range of platforms. CMake isn't a build system like GNU Autotools,
84but a tool that generates build information for external systems like:
85Visual Studio projects, XCode projects and Makefiles to name a few. Internal
86builds of Couchbase (and hence what we test) use Makefiles on Linux and
87MacOS and Ninja on Windows. Other systems _may_ however work, but you're
88pretty much on your own if you try to use them.
89
90### Simple build (Linux and MacOS)
91
92If you just want to build Couchbase and without any special
93configuration, you may use the Makefile we've supplied for your
94convenience:
95
96    trond@ok > mkdir source
97    trond@ok > cd source
98    trond@ok source> repo init -u git://github.com/couchbase/manifest -m branch-master.xml
99    trond@ok source> repo sync
100    trond@ok source> make
101
102This would install the build software in a subdirectory named
103`install`. To change this you may run:
104
105    trond@ok source> make EXTRA_CMAKE_OPTIONS='-DCMAKE_INSTALL_PREFIX=/opt/couchbase'
106
107If you want to build the Enterprise Edition (requires access to
108git repositories containing closed source) you need to tell repo
109to fetch additional source by adding `-g enterprise,default` to
110repo init:
111
112    trond@ok source> repo init -u git://github.com/couchbase/manifest -m branch-master.xml -g enterprise,default
113
114### Simple build (Windows)
115
116The build is not optimized for Windows, but the following steps should work. Start with the
117same "repo init" and "repo sync" steps as above, then run:
118
119    tlm\win32\environment.bat
120    mkdir build
121    cd build
122    cmake -G Ninja -D CMAKE_C_COMPILER=cl -D CMAKE_CXX_COMPILER=cl -D CMAKE_BUILD_TYPE=RelWithDebInfo ..
123    ninja install
124
125# End of the basic build information
126
127The remainder of this document covers certain special cases for building
128Couchbase Server. You likely will not be interested in anything beyond this
129point unless you work for Couchbase and have specific development issues to
130work on.
131
132### Customize your builds
133
134CMake offers a wide range of customizations, and this chapter won't
135try to cover all of them. There is plenty of documentation available
136on the [webpage](http://www.cmake.org/cmake/help/documentation.html).
137
138There is no point of trying to keep a list of all tunables in this
139document. To find the tunables you have two options: look in
140`cmake/Modules/*.cmake` or you may look in the cache file generated
141during a normal build (see `build/CMakeCache.txt`)
142
143There are two ways to customize your own builds. You can do it all by
144yourself by invoking cmake yourself:
145
146    trond@ok > mkdir source
147    trond@ok > mkdir build
148    trond@ok > cd source
149    trond@ok source> repo init -u git://github.com/couchbase/manifest -m branch-master.xml
150    trond@ok source> repo sync
151    trond@ok source> cd ../build
152    trond@ok build> cmake -D CMAKE_INSTALL_PREFIX=/opt/couchbase -D CMAKE_BUILD_TYPE=Debug -D DTRACE_FOUND:BOOL=True -D DTRACE:FILEPATH=/usr/sbin/dtrace CMAKE_PREFIX_PATH="/opt/r14b04;/opt/couchbase"
153    trond@ok build> gmake all install
154
155Or pass extra options to the convenience Makefile provided:
156
157    trond@ok > mkdir source
158    trond@ok > mkdir build
159    trond@ok > cd source
160    trond@ok source> repo init -u git://github.com/couchbase/manifest -m branch-master.xml
161    trond@ok source> repo sync
162    trond@ok source> make PREFIX=/opt/couchbase CMAKE_PREFIX_PATH="/opt/r14b04;/opt/couchbase" EXTRA_CMAKE_OPTIONS='-D DTRACE_FOUND:BOOL=True -D DTRACE:FILEPATH=/usr/sbin/dtrace'
163
164Use `CMAKE_PREFIX_PATH` to specify a "list" of directories to search
165for tools/libraries if they are stored in "non-standard"
166locations. Ex:
167
168    CMAKE_PREFIX_PATH="/opt/r14b04;/opt/couchbase;/opt/local"
169
170## Static Analysis
171
172There are pre-canned build rules to allow you to run the
173[Clang Static Analyzer][clang_static_analyzer_link] against the Couchbase
174codebase.
175
176So far this has only been tested on OS X, using Clang shipping as part
177of OS X Developer Tools. It *should* be possible to also run on other
178platforms which Clang/LLVM is available, however this isn't tested.
179
180### Prerequisites
181
182* Install `clang` (from OS X Developer Tools). If you can build from source
183  you should already have this :)
184* Download and extract clang Static Analyzer tools
185  (from [clang-analyzer.llvm.org][clang_static_analyzer_link]).
186  Note that while the actual analyzer functionality is built into
187  clang, this is needed for `scan-build` and `scan-view` tools to
188  invoke and display the analyser results.
189
190### Running
191
192*  Add `scan-build` and `scan-view` to your path:
193
194        export PATH=$PATH:/path/to/scan-build
195
196*  Run `make analyze` at the top-level to configure clang-analyser as the
197   'compiler':
198
199        make analyze
200
201*  At the end you will see a message similar to the following - Invoke the
202   specified command to browse the found bugs:
203
204        scan-build: 31 bugs found.
205        scan-build: Run 'scan-view /source/build-analyzer/analyser-results/2014-06-05-173247-52416-1' to examine bug reports.
206
207## Address / Thread / UndefinedBehavior Sanitizers
208
209There are pre-canned build rules to allow you to build with
210[ThreadSanitizer][thread_sanitizer_link] to detect threading issues,
211[AddressSanitizer][address_sanitizer_link] to detect memory errors, or
212[UndefinedBehaviorSanitizer][undefined_sanitizer_link] to detect
213undefined behavior.
214
215### Prerequities
216
217* A compiler which supports Address / Thread / UndefinedBehavior
218  Sanitizer. Recent version of Clang (3.2+) or GCC (4.8+) are claimed
219  to work. Currently automatied tests use GCC 7 / Clang 3.9.
220
221### Running
222
223* Ensure that the compiler supporting *Sanitizer is chosen by
224  CMake. If it's the system default compiler there is nothing to do;
225  otherwise you will need to set both `CC` and `CXX` environment
226  variables to point to the C / C++ compiler before calling the build
227  system.
228
229* Pass the variable `CB_THREADSANITIZER=1` / `CB_ADDRESSSANITIZER=1` /
230  `CB_UNDEFINEDSANITIZER=1` to CMake.
231
232ThreadSanitizer one liner for a Ubuntu-based system where Clang isn't
233the default system compiler:
234
235        CC=clang CXX=clang++ make EXTRA_CMAKE_OPTIONS="-D CB_THREADSANITIZER=1"
236
237and for AddressSanitizer:
238
239        CC=clang CXX=clang++ make EXTRA_CMAKE_OPTIONS="-D CB_ADDRESSSANITIZER=1"
240
241similary for UndefinedBehaviorSanitizer:
242
243        CC=clang CXX=clang++ make EXTRA_CMAKE_OPTIONS="-D CB_UNDEFINEDSANITIZER=1"
244
245* Run one or more tests. Any issues will be reported (to stderr by default).
246
247### Customizing Address / Thread / UndefinedBehavior Sanitizer
248
249See `cmake/Modules/CouchbaseThreadSanitizer.cmake` CMake fragment for
250how ThreadSanizer is configured.
251
252See the `TSAN_OPTIONS` environment variable (documented on the
253ThreadSanitizer [Flags][thread_sanitizer_flags] wiki page) for more
254information on configuring.
255
256Similarly for AddressSanitizer / UndefinedBehaviorSanitizer see
257`cmake/Modules/CouchbaseAddressSanitizer.cmake` or
258`cmake/Modules/CouchbassUndefinedBehaviorSanitizer.cmake`, and the
259`ASAN_OPTIONS` / `UBSAN_OPTIONS` environment variable (documented on
260the AddressSanitizer [Flags][address_sanitizer_flags] wiki page) for
261details..
262
263[cmake_link]: http://www.cmake.org/cmake/
264[clang_static_analyzer_link]: http://clang-analyzer.llvm.org
265[thread_sanitizer_link]: https://code.google.com/p/thread-sanitizer/wiki/CppManual
266[thread_sanitizer_flags]: https://code.google.com/p/thread-sanitizer/wiki/Flags
267[address_sanitizer_link]: https://github.com/google/sanitizers/wiki/AddressSanitizer
268[address_sanitizer_flags]: https://github.com/google/sanitizers/wiki/AddressSanitizerFlags
269[undefined_sanitizer_link]: https://github.com/google/sanitizers/wiki/AddressSanitizer
270