IPC
The wire format: framing, handshake, and the serialized value encoding
Required One chapter of the Open Q Language specification. README.txt states what must be implemented, how conformance is defined, and how rule IDs work.
This chapter specifies the bytes on the wire. It is the interoperability contract: two implementations conform when either can read what the other writes. The same encoding is what -8! produces and -9! consumes, so an implementation can be checked against it without a network.
Every byte sequence below was observed from the reference implementation.
IPC-00011. Framing
Every message is an 8-byte header followed by one serialized value.
offset size meaning
0 1 endianness of this message: 1 little, 0 big
1 1 message type: 0 async, 1 sync, 2 response
2 1 compressed flag: 0 uncompressed, 1 compressed
3 1 unused (0)
4 4 TOTAL message length, header included, in the
endianness given by byte 0The length counts the header, so a boolean atom message is 10 bytes, not 2.
-8!0b -> 01 00 00 00 0a 00 00 00 ff 00
| | | | \--------/ \---/
| | | unused len=10 payload
| | not compressed
| async (what -8! stamps)
little-endianIPC-00021.1 Endianness
A reader MUST honour byte 0 rather than assume its own order: the length field and every multi-byte value in the payload are in the sender's order. JVM clients send big-endian. A reply need not match the request's order; the peer adapts from byte 0.
IPC-00031.2 Message type
0 async fire and forget. The receiver MUST NOT reply. 1 sync the receiver MUST reply with exactly one type-2 message. 2 response the reply to a sync request, or an error (section 5).
-8! stamps type 0. The transport sets the real type; the body encoding is identical in all three cases.
Verified against the reference over a socket: a type-1 "2+3" is answered with 01 02 00 00 11 00 00 00 f9 05 00 00 00 00 00 00 00 — a type-2 frame carrying the long atom 5. The same body sent as type 0 draws no reply.
IPC-00042. Handshake
Before any framed message, the client sends a raw, UNFRAMED byte string:
"user:password" , capability_byte , 0x00The capability byte is the highest protocol version the client supports. The server replies with ONE raw byte: the version it agrees to use, which is at most what the client asked for. No header is involved in either direction.
client -> 03 00 (empty credentials, capability 3)
server -> 03 (agreed)Credentials may be empty, as above. A server that rejects the connection closes it instead of replying. After the single reply byte, both directions speak framed messages only.
IPC-00053. Value encoding
A payload is one value. Every value begins with a one-byte TYPE CODE, which is the language's type code (core.txt 4) taken as a SIGNED byte: atoms are negative and appear as 0xff..0xee, vectors are positive.
IPC-00063.1 Atoms: type byte, then the value
No count, no attribute byte.
code hex type payload
-1 ff boolean 1 byte
-2 fe guid 16 bytes, big-endian as displayed
-4 fc byte 1 byte
-5 fb short 2 bytes
-6 fa int 4 bytes
-7 f9 long 8 bytes
-8 f8 real 4 bytes (IEEE)
-9 f7 float 8 bytes (IEEE)
-10 f6 char 1 byte
-11 f5 symbol NUL-TERMINATED bytes, variable length
-12 f4 timestamp 8 bytes ns since 2000.01.01
-13 f3 month 4 bytes months since 2000.01
-14 f2 date 4 bytes days since 2000.01.01
-15 f1 datetime 8 bytes IEEE days since 2000.01.01
-16 f0 timespan 8 bytes ns duration
-17 ef minute 4 bytes
-18 ee second 4 bytes
-19 ed time 4 bytes msSymbols are the only variable-length atom: `abc is f5 61 62 63 00, and the empty symbol ` is f5 00.
Null and infinity are the ordinary sentinels, not a special encoding: 0N is f9 followed by i64 MIN, 0Ng is fe followed by 16 zero bytes.
IPC-00073.2 Simple vectors: type, attribute, count, elements
A positive type code, then ONE attribute byte, then a 4-byte element COUNT, then the elements back to back with no per-element type byte.
1 2 3 -> 07 00 03000000 0100.. 0200.. 0300..
| | \------/
| | count = 3
| attribute (section 3.6)
type 7 = long vectorChar vectors carry no terminator: "abc" is 0a 00 03000000 61 62 63. Symbol vectors carry one NUL per element: `a`b is 0b 00 02000000 61 00 62 00.
IPC-00083.3 General list
Type 0, attribute byte, count, then each element as a COMPLETE value with its own type byte.
() -> 00 00 00000000
(1;`a) -> 00 00 02000000 f9 0100000000000000 f5 6100IPC-00093.4 Dictionary, table, keyed table
dict 99 (0x63) then TWO values: keys, then values. No attribute byte,
no count — the two values carry their own.
`a`b!1 2 -> 63 0b 00 02000000 610062 00 07 00 02000000 ...table 98 (0x62) then ONE attribute byte, then a DICTIONARY of column
name vector -> column list.
([]a:1 2) -> 62 00 63 0b 00 01000000 6100 00 00 01000000 ...keyed table
a DICTIONARY (99) whose key value and value value are both TABLES.
It is not a distinct wire type.
([k:1 2]v:3 4) -> 63 62 00 63 ... 62 00 63 ...IPC-00103.5 Functions and singletons
100 (0x64) lambda: a NUL-terminated context/namespace string, then the
source text as a char vector. {x+y} is
64 00 0a 05000000 7b782b797d — the lambda travels as SOURCE,
not as compiled form, which is why any runtime can read it.101 (0x65) unit / generic null (::) : one following byte, 0. 102 (0x66) a primitive verb, as a one-byte index into the primitive
table. (+) is 66 01. The index table is shared by convention
between implementations and is the least portable part of this
chapter.IPC-00113.6 The attribute byte
Simple vectors and tables carry their attribute on the wire; it is not lost in transit and not re-derived by the receiver.
0 none 1 `s (sorted) 2 `u (unique)
3 `p (parted) 4 `g (grouped)
`s#1 2 3 -> 07 01 03000000 ...
`u#`a`b -> 0b 02 02000000 ...
`p#`a`a -> 0b 03 02000000 ...
`g#`a`b -> 0b 04 02000000 ...IPC-00123.7 What does not survive
ENUMERATIONS ARE RESOLVED TO SYMBOLS. A value of type 20h is sent as a plain symbol vector (type 11); the domain does not travel and the receiver gets no enumeration back.
s:`a`b; -8!`s$`a`b -> 0b 00 02000000 610062 00This is the one place where a round-trip is not identity, and any runtime that keeps enumerations must accept that IPC flattens them. Handles are process-local integers and are never serialized at all.
IPC-00134. Compression
Header byte 2 flags a compressed payload. The reference implementation does NOT compress same-host connections: over loopback it left byte 2 at 0 for an 80,014-byte message of a single repeated long, which is as compressible as a message gets. -8! never compresses either — count -8!10000#1 is 80014.
OPEN: the compressed payload encoding is not pinned by this chapter. It
could not be observed without a non-loopback peer, and this specification
does not state bytes it has not seen. A conformant runtime must therefore
be prepared to RECEIVE a compressed frame from kdb+ over a real network,
and is free never to SEND one. This is a gap to close, not a claim that
compression is absent.IPC-00145. Errors and the reserved codes
An error in reply to a sync request is a type-2 frame whose payload is type 128 (0x80) followed by the NUL-terminated error text.
The ! verb's reserved negative left arguments (core.txt 5) expose this codec to the language:
-8!x serialize x to a byte vector (async header, type 0)
-9!b deserialize a byte vector back to a value
-18!x compress a byte vector
-19!x ... and the file variants
-9!-8!x is identity for every value except enumerations (section 3.7):
(-9!-8!([]a:1 2))~([]a:1 2) is 1b.Source: spec/ipc.txt