← History

Binaries and bit syntax: the BEAM's protocol superpower

Erlang's bit syntax turns byte-level protocol parsing into a few lines of pattern matching, and every BEAM language inherits some form of it except Lua, which reads bytes the hard way.

Ask an Erlang veteran why the BEAM ended up running so much of the world's telecom and messaging infrastructure and you will hear about processes and supervisors. But there is a quieter reason: the VM has a first-class type for raw bytes, and a declarative mini-language for slicing them apart. Where most runtimes make you reach for byte buffers, manual offsets, and shift-and-mask arithmetic, the BEAM lets you write the wire format down almost as it appears in the RFC and match on it directly. That feature is the binary type and its bit syntax, <<...>>.

Binaries and bitstrings in Erlang

A bitstring is a sequence of bits of any length. A binary is the common special case: a bitstring whose length is a whole number of bytes. So every binary is a bitstring, but <<1:1, 0:1, 1:1>> (three bits) is a bitstring that is not a binary.

Bin  = <<1, 2, 3>>,      %% 24 bits, byte-aligned: a binary
Bits = <<1:1, 0:1, 1:1>>,%% 3 bits: a bitstring, not a binary
true  = is_binary(Bin),
false = is_binary(Bits),
true  = is_bitstring(Bits).

Inside <<...>> you write segments, each shaped as Value:Size/TypeSpecifiers. The specifiers control how many bits the value occupies and how those bits are read. The defaults are the thing to memorise: a bare segment like <<X>> is an integer, 8 bits, unsigned, big-endian. That is why <<1, 2, 3>> is three bytes.

The real power shows up when you match. Here is the first 32-bit word of an IPv4 header, pulled apart in one line:

<<Version:4, IHL:4, DSCP:6, ECN:2, TotalLength:16, Rest/binary>> = Packet,

Version and IHL grab four bits each, DSCP six, ECN two, TotalLength a big-endian 16-bit integer, and Rest/binary scoops up every remaining byte. Sizes and units, endianness, and signedness are all just specifiers:

<<X:8/signed>>   = <<255>>,  %% X = -1  (read as a signed byte)
<<Y:8>>          = <<255>>,  %% Y = 255 (unsigned is the default)
<<Z:16/little>>  = <<0, 1>>. %% Z = 256 (little-endian)

This is the same matching engine described in pattern matching everywhere; binaries are just one more shape you can destructure.

Why this matters: zero-copy and protocols

Bit syntax is not only concise, it is fast. When you match Rest/binary, the VM usually does not copy those bytes. It hands you a sub-binary: a small reference (offset plus length) pointing into the original data. Parsing a megabyte packet stream into headers and payloads can proceed without duplicating the payload even once. The compiler further builds a match context so a chain of matches walks the buffer in place.

Combine that with cheap processes and you get the BEAM's networking sweet spot: one lightweight process per connection, each parsing frames with bit syntax and holding only sub-binary views of a shared, reference-counted buffer. Telecom stacks, MQTT brokers, and HTTP servers all lean on exactly this.

Elixir: same feature, different punctuation

Elixir compiles to the same VM, so it has the same binaries. The syntax uses <<>> too, but segment options are attached with :::

<<version::4, ihl::4, rest::binary>> = packet
<<256::16>>              # <<1, 0>>

The twist Elixir adds is cultural: its strings are binaries, UTF-8 encoded. There is no separate string type hiding a byte array; the string literal is the byte array.

"abc" == <<97, 98, 99>>   # true
<<"abc">> == "abc"        # true
byte_size("héllo")        # 6  (é is two UTF-8 bytes)
String.length("héllo")    # 5

That single fact explains a lot of Elixir: byte_size and String.length differ, and low-level string work is just binary matching.

Gleam: the typed BitArray

Gleam gives the feature a name and a static type. The <<...>> syntax builds a BitArray, and segment options are joined with a hyphen rather than a slash:

let header = <<version:size(4), ihl:size(4)>>
let word   = <<256:unsigned-little-size(16)>>

case packet {
  <<version:size(4), ihl:size(4), rest:bytes>> -> Ok(#(version, ihl, rest))
  _ -> Error(Nil)
}

Options include size, unit, int, float, signed/unsigned, big/little/native, bits, bytes, and utf8/utf16/utf32. The gleam/bit_array module rounds out the everyday helpers. One caveat worth knowing: bit-array pattern matching is fully supported on the Erlang target and only partially on the JavaScript target, so protocol code that matches on bits belongs on the BEAM.

LFE: the same power in parentheses

Lisp-Flavoured Erlang exposes the identical VM feature through s-expressions. Construction uses the binary form, and each segment is (value type (size n) ...):

;; Pack an RGB565 pixel into 16 bits
(binary (red (size 5)) (green (size 6)) (blue (size 5)))

;; Match the first two nibbles of a packet, keep the rest
(let (((binary (version (size 4)) (ihl (size 4)) (rest binary)) packet))
  (list version ihl rest))

Different surface, same defaults and same specifiers as Erlang, because it is the same bit syntax underneath.

Luerl: Lua has no bit syntax

Luerl runs Lua on the BEAM, but Lua the language never had structural pattern matching, let alone bit syntax. A Lua string is an immutable sequence of bytes, and you work with it imperatively: read a byte with string.byte, build one with string.char.

local first = string.byte("ABC", 1)   -- 65
local s     = string.char(65, 66, 67) -- "ABC"

Lua 5.3 added string.pack/string.unpack with printf-style format strings for binary layout, which is the closest Lua gets, though Luerl implements only a subset of the standard library. The honest contrast: on the native BEAM languages you declare the wire format and the VM parses it; in Lua you loop, index, and mask by hand. The runtime is shared, but this particular superpower is not.

One feature, four spellings

Binaries are the reason a language built for phone switches turned out to be a superb fit for modern network services. The four BEAM languages that inherit Erlang's lineage all get the bit syntax more or less for free, which is a good reminder that the one VM, many languages story is about capabilities, not just syntax. Luerl is the exception that proves the rule.