1621b031fSJens Alfke# The Binary (Termless) Format for Views. #
2621b031fSJens Alfke
3621b031fSJens Alfke**(Damien Katz -- July 19 2012)**
4621b031fSJens Alfke
5621b031fSJens AlfkeThis documents the format of the Key/Value pairs in leaf nodes, and the reduction format in inner nodes.
6621b031fSJens Alfke
7621b031fSJens AlfkeThe underlying b-tree format is the same as already used in CouchStore.
8621b031fSJens Alfke
9621b031fSJens Alfke* All integers are network byte order.
10621b031fSJens Alfke* All strings are encoded as UTF-8.
11621b031fSJens Alfke* Adjacent fields are tightly packed with no padding.
12621b031fSJens Alfke
13621b031fSJens Alfke
14621b031fSJens Alfke## Primary Index Key Values ##
15621b031fSJens Alfke
16621b031fSJens AlfkeIn leaf nodes, `KeyValues` have the following format:
17621b031fSJens Alfke
18621b031fSJens Alfke* `Key`:
198b33a414SJens Alfke  *  `EmittedJsonKeyLength` -- 16bit integer
208b33a414SJens Alfke  *  `EmittedJSONKey` -– JSON -– The key emitted by the map function
218b33a414SJens Alfke  *  `UnquotedDocId` –- String -– The raw doc ID (occupies the remaining bytes)
22621b031fSJens Alfke* Value:
238b33a414SJens Alfke  *  `PartitionId` -- 16bit integer -- This is the partitionId (vbucket) from which this document id maps to.
24621b031fSJens Alfke  *  1 to infinity `JSONStringValue`s -- These are all the values that were emitted for this `EmittedJSONKey`.  
25621b031fSJens Alfke     Each `JSONStringValue` is of the form:
26621b031fSJens Alfke		* `ValueLength` -- 24bit unsigned integer
27621b031fSJens Alfke		* `JSONValue` - string that is `ValueLength` bytes long
28621b031fSJens Alfke
29621b031fSJens Alfke(Parsing the `JSONStringValue`s is simply reading the first 24 bits, getting the length of the following string and extracting the string. If there is still buffer left, the process is repeated until the is no value buffer left.)
30621b031fSJens Alfke
31621b031fSJens AlfkeWhen an emit happens, and the Key is different from all other keys emitted for that document, then there is only one `JSONStringValue`.
32621b031fSJens AlfkeBut when multiple identical keys are emitted, the values are coalesced into a list of Values, and there will be multiple values.
33621b031fSJens Alfke
34621b031fSJens Alfke
35621b031fSJens Alfke### Primary Index Inner Node Reductions (KeyPointerNodes and Root) ###
36621b031fSJens Alfke
37621b031fSJens Alfke* `SubTreeCount` -- 40bit integer -- Count of all Values in subtree.  
38621b031fSJens Alfke	NOTE: this is possibly greater than the `KeyId` count in the subtree, because a document can emit multiple identical keys, and they are coalesced into single `KeyId`, with all the values emitted in a list as the value.
39621b031fSJens Alfke* `SubTreePartitionBitmap` -- 1024 bits -- a bitfield of all partition keys in the subtree. Currently this is hardcoded at 1024 bits in length, but in the future we may change this to a variable size. Until then, it works with any # of vbuckets ≤ 1024.
40621b031fSJens Alfke* `JSONReductions` -- remaining bytes -- Zero or more `JSONReductions`, each consisting of:
41621b031fSJens Alfke  *  `JSONLen` -- 16bit integer
42621b031fSJens Alfke  *  `JSON` -- the actual JSON string
43621b031fSJens Alfke
44621b031fSJens Alfke
45621b031fSJens Alfke## Back Index ##
46621b031fSJens Alfke
47621b031fSJens AlfkeIn leaf nodes, `KeyValues` have the following format:
48621b031fSJens Alfke
49621b031fSJens Alfke* `Key` -- blob -- The raw docId, not quoted or JSONified in any way.
50621b031fSJens Alfke* `Value`:
51621b031fSJens Alfke  * `PartitionId` -- 16bit integer -- This is the partitionId (vbucket) from which this document id maps to.
52621b031fSJens Alfke  * 1-*n* `ViewKeysMappings`, where *n* ≤ the # of mapfunctions defined in the design document.  
53621b031fSJens Alfke    A `ViewKeysMapping` is:
54621b031fSJens Alfke	    * `ViewId` -- 8bit integer -- the ordinal id of the map view in the group the following keys were emitted from
55621b031fSJens Alfke	    * `NumKeys` -- 16bit integer -- the number of `JSONKeys` that follow
56621b031fSJens Alfke		* `JSONKeys` -- a sequence of:
57621b031fSJens Alfke			  * `KeyLen` -- 16bit integer -- Length of the following `JSONKey`
58621b031fSJens Alfke		  	  * `Key` -- JSON string -- Emitted JSON key
59621b031fSJens Alfke
60621b031fSJens Alfke
61621b031fSJens Alfke### Back Index Inner Node Reductions (KeyPointerNodes and Root) ###
62621b031fSJens Alfke
63621b031fSJens Alfke* `SubTreeCount` -- 40bit integer -- count of all Keys in subtree.
64621b031fSJens Alfke* `SubTreePartitionBitmap` -- 1024 bits -- a bitfield of all partition keys in the subtree. Currently this is hardcoded at 1024 in length, but in the future we may change this to a variable size. Until then, it works with any # of vbuckets ≤ 1024.
65