Open Q Language

The q surface

Keywords, parse-time rewrites, pattern matching, namespaces

Required One chapter of the Open Q Language specification. README.txt states what must be implemented, how conformance is defined, and how rule IDs work.

q is the readable surface over the shared core that core.txt specifies. This chapter covers ONLY what q adds or changes relative to that core; everything not mentioned here (evaluation model, literals, types, verbs, adverbs, lambdas, assignment, control flow, lists/dicts/tables) is identical to core.txt and is not repeated.

The query sublanguage (select/exec/update/delete, joins, functional forms) is large enough to warrant its own chapter: see qsql.txt. qSQL is a q-only surface; it does not exist in the k dialect.

Read order: core.txt -> q.txt (this chapter) -> qsql.txt.

Q-00011. Relationship to k

Dialect is chosen per line: a q) prefix (or a .q file) selects q; k) selects k. A script is split into maximal same-dialect runs. The default dialect is q.

q differs from k in exactly four surface areas: A. Named keywords ARE the primary spelling of the primitives; bare monadic

glyphs are not the everyday form (section 2, 4).

B. Some q keywords rewrite to a verb (or derived function) AT PARSE TIME, so

they are indistinguishable from the glyph downstream (section 3).

C. Word-spelled infix verbs and a reserved keyword set exist (section 5). D. q 4.1 pattern matching: pattern assignment, parameter patterns, and the

pattern conditional (section 6).

Plus the whole qSQL surface (qsql.txt) and richer namespace conventions (section 7).

Q-00022. The keyword layer

In q the primitives are usually written as NAMES, not glyphs:

count x        rather than   #x
first x                      *x
reverse x                    |x
string x                     $x
key d                        !d
flip t                       +t

These names lex as ordinary identifiers; the lexer does not know them. They resolve to functions at evaluation time, EXCEPT the parse-time rewrites in section 3. An unknown identifier is simply a name reference.

The full library is the same set listed in core.txt section 7 (aggregates, list ops, ordering/search, math, stats and moving windows, text, dict/table, joins, reflection/system, IO/IPC/JSON), plus the query keywords of qsql.txt.

Q-00033. Parse-time keyword rewrites (q-specific)

q's parser rewrites certain keywords to their underlying verb before evaluation, so a global variable can never shadow them. Two rewrite tables are needed. A k parser does NOT need these; a q parser does, to match q's `parse` output and to make the keyword un-shadowable.

Q-00043.1 T1 — monadic keyword to verb-monad value

These keywords resolve at parse time to the verb's MONADIC overload as a value (core.txt section 9.10). parse "count x" yields (#:;x).

count -> #:      neg   -> -:      flip  -> +:      first -> *:
reverse -> |:    where -> &:      group -> =:      distinct -> ?:
string -> $:     floor -> _:      null  -> ^:      type  -> @:
not   -> ~:      key   -> !:      value -> .:

(Only keywords whose builtin is IDENTICAL to the glyph's monad are rewritten. enlist, get, iasc/idesc, hdel/hclose etc. stay name-resolved because their behaviour differs from the raw glyph: parse "enlist x" is (enlist;`x) and parse "iasc x" expands to a k lambda, not a glyph.)

Q-00053.2 T2 — derived-function keyword to adverb+verb

These resolve to a derived function (adverb applied to a verb):

sums -> +\     prds -> *\     maxs -> |\     mins -> &\
fills -> ^\    raze -> ,/     deltas -> -':  ratios -> %':
prev -> :':

next is NOT in this table: parse "next x" expands to a k lambda, so the pair prev/next is not symmetric.

Q-00063.3 Namespace-member keywords

A large set of q keywords (aj asof lj ij uj ej wj cross cut asc desc iasc idesc rank meta cols keys xasc xcol xkey xgroup insert upsert parse eval value get set save load read0 read1 til sv vs except inter union ...) resolve as .q NAMESPACE MEMBERS at parse time. They behave as their builtin but are looked up through the namespace.

Q-00073.4 k-native words that stay bare

abs avg in div and the adverb-words (over scan each peach prior) are NOT namespace members; they resolve through the ordinary builtin fallback or live in the prelude. show and view are prelude globals.

Q-00084. Monadic glyphs in q

k's defining rule — a bare glyph in monadic position is the glyph's monad (#x, |x, $x) — is the k surface. In q you write the keyword instead (count x, reverse x, string x). The verb-monad VALUE form v: (#:, -:, $:) still exists in q and is exactly what the T1 keywords rewrite to (section 3.1). Applied bare monadic glyphs are therefore uncommon in idiomatic q source but the machinery is shared; the difference is which spelling the surface encourages, not the underlying operation.

Q-00095. Infix keyword verbs and reserved words

Word-spelled DYADIC verbs (no monad), active in q:

div    integer divide (floor)
mod    modulo (floor; sign of divisor)
xexp   power (float)
xbar   bucket down to a multiple of n
in     membership -> boolean
and    -> the & glyph verb (min / and)      \ lexes to the glyph in q
or     -> the | glyph verb (max / or)       \ lexes to the glyph in q
mmu    -> the $ glyph verb (matrix multiply) \ lexes to the glyph in q

(In k, and/or/mmu are plain identifiers, not verbs — see core.txt section 2.)

within is a builtin (x within (lo;hi) -> boolean), not a glyph verb.

The reserved query keywords select exec update delete by from where are recognized only in query position (qsql.txt).

Q-00106. q 4.1 pattern matching

q 4.1 adds destructuring/matching in three positions. A minimal q parser may implement only plain names and skip the exotic pattern forms; they are listed here for completeness.

Q-00116.1 The pattern forms

Name        a          bind the matched value to a
Blank       _  or ;    match anything, bind nothing (an elided slot)
Const       1          value must ~-match the constant, else 'match
NameIndex   a[1]       amend an existing variable at an index
List        (b;c)      match elements positionally; lengths must agree,
                       else 'length
Type        p:`f       type check: value's type must be `f (lowercase=atom,
                       uppercase=list), else 'type
Filter      a:3+       apply the function to the value; match/return that
Bang        (k!v)      operator-dict pattern: match keys against k, values
            (k!)       against v (v optional)
Dict        ([key]col) dict/table pattern: match a dict by key columns, or a
                       (keyed) table's value columns

Q-00126.2 Pattern assignment

(pattern): value       destructure and bind         (a;b):1 2
(pattern):: value      same, binding to the global scope

Produces a PatternAssign node. Ordinary name:value stays a plain Assign (core.txt section 9.3).

Q-00136.3 Parameter patterns

A lambda parameter may be any pattern, not just a name:

{[(a;b)] a+b}[(1;2)]        destructure the single argument
{[a:`f] a}                  type-checked parameter

The everyday forms remain the plain name and [] (niladic).

Q-00146.4 Pattern conditional

:[value; pat1; res1; pat2; res2; ...; default] Match value against each pattern in order; return the first match's result (its bindings persist); default when none match. Lazy — only the taken result is evaluated. Produces a PatternCond node. The pattern positions are parsed as ordinary expressions and reinterpreted (ast_to_pattern):

an identifier -> Name, an elision -> Blank, name:`t -> Type,
name:expr -> Filter, (a;b) -> List, anything else -> Const.

Do not confuse this with $[c;t;f] (the scalar conditional, core.txt 9.7): $[..] branches on truth of a condition; :[..] branches on a pattern MATCH.

Q-00157. Namespaces and scope

A name beginning with . is namespaced; a dotted path builds nested dicts automatically:

.ns.a: 1
.ns.b: 2
key .ns        ->  `a`b
.ns`a          ->  1

Scope lookup order inside a lambda: local frame -> globals -> builtins.

Assignment forms (shared with k, listed for the q reader):

name: v        local (or global at top level)
name:: v       force the global binding
`name set v    set a global by symbol name
name op: v     in-place amend  (r+:5  r-:3  r,:x)
name[i]: v     indexed amend (desugars to @[name;i;:;v]; core.txt section 9.4)

Built-in namespaces of note (a q parser treats these as ordinary dotted names; they matter to the evaluator, not the grammar):

.z   process/runtime: .z.d date, .z.p timestamp, .z.t time, .z.x argv,
     .z.ts timer callback, .z.pg/.z.ps sync/async handlers, .z.po/.z.pc
     port open/close, .z.pw auth check
.Q   utilities and constants (prelude)
.q   the standard library namespace the T3 keywords resolve through
.j   JSON (.j.j serialize, .j.k parse)

An implementation may add its own namespaces; see runtime.txt.

Q-00168. System commands

A line beginning with \ (not a block-comment toggle) is a system command Examples: \l file.q (load), \t expr (time), \ts (time+space), \p port (listen), \Z 0|1 (temporal cast epoch), \\ (exit). These are lines, not expressions; a parser hands the remainder of the line to the command handler.

Q-00179. Notes for the q parser implementer

  • Start from the k core parser (core.txt section 10). q adds: a keyword-rewrite pass (T1/T2/namespace members), the query templates (qsql.txt), and the pattern forms. None of these change the right-to-left, precedence-free skeleton.
  • Apply T1/T2 rewrites at parse time so a user global cannot shadow count, sums, etc. This also makes `parse` output match kdb.
  • The query keywords are contextual: select/exec/update/delete are query starters only in statement/argument position; elsewhere they are names. by/from/where terminate query phrases.
  • Pattern positions are parsed as normal expressions then reinterpreted; you do not need a separate pattern grammar for the conditional.
  • within / and / or / mmu spellings: in q, and/or/mmu fold into the glyph verbs during lexing; within stays a builtin name.

END

Source: spec/q.txt