1# N1QL—Query Language for N1NF (Non-1st Normal Form): Logic
2
3* Status: DRAFT/PROPOSAL
4* Latest: [n1ql-logic](https://github.com/couchbase/query/blob/master/docs/n1ql-logic.md)
5* Modified: 2013-07-19
6
7## Summary
8
9N1QL is a query language for Couchbase, and is a continuation of
10[UNQL](https://github.com/couchbaselabs/tuqqedin/blob/master/docs/unql-2013.md).
11This document builds on the [N1QL
12Select](https://github.com/couchbase/query/blob/master/docs/n1ql-select.md)
13and [N1QL
14DML](https://github.com/couchbase/query/blob/master/docs/n1ql-dml.md)
15specs.
16
17This document describes the syntax and semantics of the logic
18statements in the language.  These are sometimes called procedural or
19compound statements in other database systems.  DDL for stored
20programs (procedures and functions) is defined in a separate spec.
21
22N1QL logic statements can be submitted directly or embedded in stored
23programs.
24
25## Motivation
26
27There are several reasons for providing these logic capabilities in
28the query language:
29
30* Application developers sometimes want to push logic and processing
31  down into the database.  This could be to reduce network traffic, to
32  leverage database hardware, and to avoid complexities and risks of
33  managing connections, connectivity, etc.
34
35* We could (and might) also embed host languages like Javascript and
36  Lua in the database server to get similar logic capabilities.
37  However, application developers would still be left with the
38  impedance mismatches between the query language and the host
39  language (our host language bindings might mitigate this).  By
40  providing N1QL logic statements in the query language, it becomes
41  natural and convenient for application developers to combine queries
42  and logic.
43
44* These logic-in-QL capabilities are well validated.  All major
45  database systems provide logic capabilities that are actively used.
46
47* N1QL borrows features from [The Go Programming
48  Language](http://golang.org) and other newer programming languages.
49  These features distinguish N1QL from other procedural database
50  languages, and may offer unique benefits to Couchbase users.
51
52## Data types
53
54N1QL has a type system, as do RDBMS procedural languages and languages
55such as C, Java, Python, and Javascript.
56
57In N1QL, variables are strongly but dynamically typed.  This means
58that variables are not statically tied to specific data types, but
59runtime values do have specific data types.
60
61The N1QL data types are:
62
63* JSON value (primitive, object, or array)
64* Blob (uninterpreted and / or encoded byte sequence)
65* Channel (golang-style message-passing concurrency construct)
66* Prepared (prepared statement)
67* Cursor (bound, open, fetchable cursor)
68* Range (golang- and python-style iterable numeric range)
69* Function / closure???? (TBD)
70
71N1QL aims to be more streamlined than other database procedural
72languages by treating prepared statements, cursors, ranges (and
73possibly functions) as first-class data types that can be used in
74expressions, assignments, function calls, etc.  This reduces the need
75for special syntax.
76
77## Termination
78
79N1QL statements are terminated by a semicolon or a newline (similar to
80golang). A semicolon termination is explicit. If the semicolon is
81omitted and a newline appears where a statement could end, the
82semicolon is inferred and the statement is concluded.
83
84terminated-stmt:
85
86![](diagram/terminated-stmt.png)
87
88## Blocks
89
90Blocks introduce variable and namespace scope and masking.
91
92block:
93
94![](diagram/block.png)
95
96## Statement types
97
98N1QL logic supports several types of statements.
99
100stmt:
101
102![](diagram/stmt.png)
103
104#### Expressions
105
106Several statements below use *mexpr*, *mcond*, and *lexpr*:
107
108* *mexpr* and *mcond*: all in-memory expression types (all expressions
109  excluding cursors or other objects that retain network or other
110  non-memory dependencies or resources)
111* *lexpr*: all expression types in N1QL logic, including mexpr and
112  cursor.
113
114Mexpr, mcond, and lexpr are specified later in this document, under
115"Expressions".
116
117### Begin
118
119A BEGIN statement creates an explicit block.
120
121begin:
122
123![](diagram/begin.png)
124
125### Declare
126
127A DECLARE statement explicitly declares one or more variables in the
128current scope, and optionally initialize them.  Like golang, multiple
129variables are supported.
130
131decl:
132
133![](diagram/decl.png)
134
135var:
136
137![](diagram/var.png)
138
139### Initializer
140
141An initializer statement implicitly declares one or more variables in
142the current scope, and always initializes them.  An initializer cannot
143be used to shadow or mask a name that is already visible in the
144current scope.  Like golang, multiple variables are supported.
145
146init:
147
148![](diagram/init.png)
149
150### Assignment
151
152An assignment statement assigns values to one or more variables that
153have already been declared and are visible in the current scope.  Like
154golang, multiple variables are supported.
155
156assign:
157
158![](diagram/assign.png)
159
160### Send
161
162A send statement is a golang-style send to a channel.
163
164sendop:
165
166![](diagram/sendop.png)
167
168### Unset
169
170Unset removes a field name and value from an object or map.
171
172unset:
173
174![](diagram/unset.png)
175
176## Control flow statements
177
178ctrl:
179
180![](diagram/ctrl.png)
181
182### If
183
184This is the standard IF-THEN-ELSEIF-ELSE form.
185
186if:
187
188![](diagram/if.png)
189
190### Case
191
192There are 2 forms of the CASE statement.
193
194case:
195
196![](diagram/case.png)
197
198#### Full case
199
200Full-case is equivalent to IF-THEN-ELSEIF-ELSE.
201
202full-case:
203
204![](diagram/full-case.png)
205
206#### Searched case
207
208Searched-case is equivalent to a SWITCH statement in C / C++ / Java /
209Go.
210
211searched-case:
212
213![](diagram/searched-case.png)
214
215### Loops
216
217loop:
218
219![](diagram/loop.png)
220
221for:
222
223![](diagram/for.png)
224
225### While
226
227This is a standard WHILE loop.
228
229while:
230
231![](diagram/while.png)
232
233### For (over iterables)
234
235This form of FOR loop iterates over iterable expressions (e.g. arrays,
236ranges) and cursors (queries and EXECUTEs).  It is a runtime error if
237the argument does not evaluate to an iterable expression or cursor.
238
239for-iter:
240
241![](diagram/for-iter.png)
242
243### For (over maps and objects)
244
245This form of FOR loop iterates over maps and JSON objects.  It is a
246runtime error if the mexpr does not evaluate to a map or JSON object.
247
248for-map:
249
250![](diagram/for-map.png)
251
252key-var:
253
254![](diagram/key-var.png)
255
256val-var:
257
258![](diagram/val-var.png)
259
260### Break
261
262The BREAK statement exits a loop, which may be labeled (e.g. when
263exiting an outer loop).
264
265break:
266
267![](diagram/break.png)
268
269### Continue
270
271The CONTINUE statement jumps to the next iteration of a loop, which
272may be labeled (e.g. when continuing an outer loop).
273
274continue:
275
276![](diagram/continue.png)
277
278### Pass
279
280A PASS statement does nothing; it is a no-op. It is useful in places
281where at least one statement is required (e.g. in blocks).  PASS is
282borrowed from Python.  A NULL expression can also be used to
283equivalent effect.
284
285pass:
286
287![](diagram/pass.png)
288
289### Return
290
291A RETURN statement returns from the current function or procedure,
292optionally setting the values of the return variables.  It is an error
293if there is no current function or procedure (e.g. in a standalone
294BEGIN block).
295
296return:
297
298![](diagram/return.png)
299
300### Deliver
301
302DELIVER is a concurrency construct borrowed from golang.  It randomly
303selects a send or receive operation and performs it, or blocks until
304an operation is available.  DELIVER is called SELECT in golang.
305
306deliver:
307
308![](diagram/deliver.png)
309
310commop:
311
312![](diagram/commop.png)
313
314#### Send op
315
316Send to a channel.
317
318sendop:
319
320![](diagram/sendop.png)
321
322#### Receive op
323
324Receive from a channel. The optional second variable is for status.
325
326rcvop:
327
328![](diagram/rcvop.png)
329
330## Expressions
331
332N1QL Logic uses a superset of the expression types used by N1QL Select
333and N1QL DML.
334
335### Memory expressions
336
337Memory expressions include all expressions that do not retain
338non-memory dependencies or resources (e.g. cursors are excluded
339because they retain network resources).
340
341mexpr:
342
343![](diagram/mexpr.png)
344
345mcond:
346
347![](diagram/mcond.png)
348
349#### Receive expr
350
351Receive a value and optional status from a channel.
352
353rcvexpr:
354
355![](diagram/rcvexpr.png)
356
357#### First
358
359FIRST returns the first result of a cursor, or NULL if the cursor's
360result set is empty.
361
362FIRST is equivalent to the SQL constructs SELECT ... INTO and UPDATE /
363INSERT / DELETE ... RETURNING INTO.  N1QL aims to be more streamlined
364by providing FIRST expressions instead of alternate syntax for query
365statements.
366
367first:
368
369![](diagram/first.png)
370
371### Logic expressions
372
373Logic expressions encompass all expression types in N1QL Logic.
374
375lexpr:
376
377![](diagram/lexpr.png)
378
379#### Prepare
380
381PREPARE evaluates to a prepared statement.  Its argument may be a
382query or a string-valued mexpr.  In either case, a special syntax is
383used to identify embedded query parameters.
384
385The arguments to USING introduce names of query parameters.
386
387prepare:
388
389![](diagram/prepare.png)
390
391#### Cursor
392
393A cursor expression is either a direct query or an EXECUTE, both of
394which evaluate to a logical cursor.
395
396cursor:
397
398![](diagram/cursor.png)
399
400query:
401
402![](diagram/query.png)
403
404#### Execute
405
406The argument to EXECUTE must evaluate to a prepared statement
407(e.g. the result of a PREPARE) or a string.
408
409The arguments to USING are bound to the query parameters.
410
411execute:
412
413![](diagram/execute.png)
414
415## About this Document
416
417The
418[grammar](https://github.com/couchbase/query/blob/master/docs/n1ql-logic.ebnf)
419forming the basis of this document is written in a [W3C dialect of
420EBNF](http://www.w3.org/TR/REC-xml/#sec-notation).
421
422This grammar has not yet been converted to an actual implementation,
423ambiguities and conflicts may still be present.
424
425Diagrams were generated by [Railroad Diagram
426Generator](http://railroad.my28msec.com/) ![](diagram/.png)
427
428### Document History
429
430* 2013-07-13 - Initial checkin
431    * Logic inspired by RDBMS stored procedures, and by [The Go
432      Programming Language](http://golang.org).
433* 2013-07-14 - First meaningful (non-stub) checkin
434    * Logic inspired by RDBMS stored procedures, and by [The Go
435      Programming Language](http://golang.org).
436* 2013-07-14 - Cosmetics
437    * Typos, wording.
438* 2013-07-15 - Cosmetics
439    * Typos, wording.
440* 2013-07-16 - Cosmetics
441    * Spacing, wording.
442* 2013-07-19 - mcond
443    * Added mcond to EBNF diagrams
444
445### Open Issues
446
447This meta-section records open issues in this document, and will
448eventually disappear.
449
4501. Should we provide first-class functions and closures?  Likely yes.
451
4521. Should EXECUTE ... USING accept values, name=value pairs, or both?
453
4541. Do we need syntax for embedded query parameters in PREPARE
455   expressions, e.g. a leading colon (:)?
456