⟨⟩ Eubeamgua

← History

Metaprogramming: macros on the BEAM

How Elixir's quote/unquote turns code into AST tuples you can rewrite, how LFE gets the real Lisp deal with code-as-data macros, and why Gleam deliberately refuses to play the game at all.

ElixirLFE

Metaprogramming is the art of writing programs that write programs. On the BEAM, the appetite for it is unusually strong: the runtime is shared by everyone, so a language's personality lives almost entirely in its surface - and the most powerful way to shape a surface is to let the language extend itself. Three of the five BEAM languages take radically different positions on this. Elixir offers hygienic macros that operate on a tuple encoding of its syntax tree. LFE offers the genuine Lisp article: code that is data, transformed by unrestricted compile-time macros. And Gleam offers nothing at all - and means it.

This article walks through all three stances, because the contrast is the lesson. Macros are not a free lunch; each language's choice tells you what it values most.

What a macro actually is

A function takes values and returns a value. It runs at run time, and by the time it is called, all of its arguments have already been evaluated. A macro takes code and returns code. It runs at compile time, and its arguments arrive unevaluated - as data describing the program text - so the macro can inspect, rearrange, drop, or duplicate them before deciding what the compiler should actually see.

That single difference - operating on unevaluated code at compile time - is the whole game. It is why a macro can implement unless, a construct that must not evaluate its body when the condition is true; a function literally cannot, because its body argument would be evaluated before the function ever ran. It is why macros can build domain-specific languages - Phoenix's router, Ecto's queries, ExUnit's assert - that read like dedicated syntax yet compile down to ordinary BEAM code.

The prerequisite for sane macros is homoiconicity: the language must represent code as data the language itself can manipulate. Both Elixir and LFE clear that bar. Erlang technically can too (via -define macros, the preprocessor, and parse transforms), but those tools are coarse and awkward by comparison. Gleam clears the bar and then deliberately walks away from it.

Elixir: the AST is a three-element tuple

Elixir code, before it becomes bytecode, is represented as a tree of three-element tuples of the shape {operator, metadata, arguments}. You can see it for any expression with quote, which returns the AST instead of evaluating it:

iex> quote do: 1 + 2
{:+, [context: Elixir, imports: [{1, Kernel}, {2, Kernel}]], [1, 2]}

Read that tuple back as a sentence: "the operator :+, with some compiler metadata, applied to the arguments 1 and 2." A nested call nests the tuples, exactly mirroring the tree:

iex> quote do: sum(1, 2 * 3)
{:sum, [], [1, {:*, [...], [2, 3]}]}

A handful of literals are clever enough to represent themselves - atoms, integers, floats, strings, booleans, nil, two-element tuples, and lists of those - which is why 1 and 2 appear raw inside the tuple above rather than wrapped. Everything else is a {op, meta, args} node. Variables are a special case worth recognizing: a bare x quotes to {:x, [], SomeContext}, where the third element is an atom (the context) rather than an argument list. That context is the seed of Elixir's hygiene, which we will come back to.

quote and unquote: templates with holes

quote on its own builds a static template. To make it useful you need to inject runtime-known values into that template, and the tool for that is unquote. Think of quote as a quoted string and unquote as string interpolation - except the thing being interpolated is AST, not text:

iex> x = 41
iex> quote do: unquote(x) + 1
{:+, [context: Elixir, ...], [41, 1]}

Without unquote, you would get {:+, [...], [{:x, [], Elixir}, 1]} - the variable x, not its value 41. unquote is what bridges the two worlds: it evaluates an expression now, at macro-expansion time, and splices the resulting AST into the template. Its sibling unquote_splicing injects a list of AST nodes as separate arguments rather than as one list argument:

iex> args = [1, 2, 3]
iex> quote do: sum(unquote_splicing(args))
{:sum, [], [1, 2, 3]}        # three arguments, not one list argument

Put the pieces together and you have a macro. Here is the canonical unless, which must control evaluation of its branch and therefore cannot be a function:

defmodule MyMacros do
  # A macro, not a function: `block` arrives as AST and is only
  # evaluated if the (inverted) condition is true.
  defmacro unless(condition, do: block) do
    quote do
      if !unquote(condition), do: unquote(block)
    end
  end
end

At the call site, the macro expands before the code runs. You can watch it happen with Macro.expand_once/2 and turn AST back into readable source with Macro.to_string/1 - the indispensable pair for debugging macros:

iex> require MyMacros
iex> quoted = quote do: MyMacros.unless(x == 1, do: IO.puts("nope"))
iex> quoted |> Macro.expand_once(__ENV__) |> Macro.to_string() |> IO.puts()
if !(x == 1) do
  IO.puts("nope")
end

Two practical notes the example quietly relies on. First, you must require (or import) a module before using its macros, because the compiler has to know a name is a macro to expand it rather than call it. Second, unless is not a toy: if, unless, def, defmodule, |>, and much of the Elixir you write every day are themselves macros defined in Kernel, not built-in keywords. The language is, to a remarkable degree, written in itself.

Hygiene: macros that don't trample your variables

The subtle danger of macros is variable capture: a macro that introduces a temporary variable can accidentally clobber - or be clobbered by - a variable of the same name at the call site. Elixir solves this automatically. Macros are hygienic: variables a macro introduces live in the macro's own context and cannot collide with the caller's, even when the names are identical.

defmodule Hygiene do
  defmacro double_it(expr) do
    quote do
      result = unquote(expr) * 2   # this `result` is the macro's, not yours
      result
    end
  end
end

iex> import Hygiene
iex> result = 100              # the caller's own `result`
iex> double_it(5)             # => 10
iex> result                  # => 100, untouched. Hygiene kept them apart.

The result inside quote and the result at the call site are different variables despite sharing a name, because the quoted one carries the macro's hidden context (recall the third element of {:result, [], Context}). When you genuinely want to break hygiene and reach into the caller's scope, you opt in explicitly with var!:

defmodule Leaky do
  # Deliberately unhygienic: `var!(answer)` binds in the CALLER's scope.
  defmacro define_answer do
    quote do
      var!(answer) = 42
    end
  end
end

iex> import Leaky
iex> define_answer()
iex> answer                  # => 42, because var! escaped hygiene on purpose

This is a deliberate, conservative default: clean by design, escapable when you mean it. A common companion is bind_quoted, which evaluates unquote expressions exactly once before splicing them - avoiding the classic bug where an argument with side effects gets duplicated because it appears twice in the template:

defmacro log(message) do
  quote bind_quoted: [message: message] do
    # `message` is evaluated once, here, even if used several times below.
    IO.puts("LOG: " <> message)
    IO.puts("len: " <> Integer.to_string(byte_size(message)))
  end
end

The headline Elixir use of macros is use. When you write use GenServer, the compiler invokes GenServer.__using__/1, a macro that injects default callbacks and @behaviour declarations into your module. The entire "framework feel" of Phoenix, Ecto, and ExUnit is built on this __using__/use handshake.

LFE: the real Lisp deal

Elixir's AST is a faithful encoding of its syntax. LFE goes one step more direct: in a Lisp, the code you write is already the data structure the language manipulates. There is no encoding step. A program is a list. (+ 1 2) is a three-element list - the symbol +, then 1, then 2 - and quoting it with ' hands you exactly that list to take apart:

;; lfe - code and data are literally the same structure
'(+ 1 2)            ; => (+ 1 2)  -- a plain 3-element list
(car '(+ 1 2))      ; => +        -- the operator is just the head
(cdr '(+ 1 2))      ; => (1 2)    -- the operands are just the tail

That directness is why Lisp macros feel native rather than bolted-on. An LFE macro, defined with defmacro, is a compile-time function from code to code, and the template tool is quasiquotation: a backquote ` builds a template, a comma , splices in one value, and ,@ splices in a list of values. The mapping to Elixir is almost one-to-one - ` is quote, , is unquote, ,@ is unquote_splicing:

;; lfe - the same `unless`, as a Lisp macro
(defmacro unless (test . body)
  `(if ,test
       'ok
       (progn ,@body)))

The . body collects every remaining form into a list, and ,@body splices that whole list into the progn. Using it looks indistinguishable from a built-in, and macroexpand-1 shows precisely what the compiler will see:

;; lfe - inspect the expansion, one step
lfe> (macroexpand-1 '(unless (== x 1) (f) (g)) $ENV)
(if (== x 1) (quote ok) (progn (f) (g)))

Two differences from Elixir are worth stating precisely, because they are real and they cut both ways.

First, LFE macros are unhygienic, in the Common Lisp tradition. They do not automatically rename the variables they introduce, so a careless macro can capture a caller's binding. The flip side is total transparency and zero ceremony: what you write is what you get, and reaching into the caller's scope needs no special var! because nothing was hidden in the first place. The responsibility for avoiding capture sits with you. Notably, LFE has no gensym: the BEAM cannot mint a guaranteed-unique atom, and atoms are global, interned, and never reclaimed, so dynamically generating names would eventually exhaust the atom table. Instead LFE leans on lexically scoped variables and careful, distinctive naming to keep a macro's bindings out of the caller's way.

Second, LFE is a Lisp-2 (in fact "Lisp-2+"): functions and variables occupy separate namespaces, so a local variable named list does not shadow the list function. For macro authors this removes an entire class of accidental capture that single-namespace Lisps and most other languages have to think about.

For rule-based, pattern-style transformations LFE also offers defsyntax, modelled on Scheme's syntax-rules, when you want declarative templates rather than a procedural macro body. Either way, because LFE compiles to Core Erlang, every macro expands down to the same intermediate language the Erlang compiler itself uses - so all this power costs nothing at run time.

Side by side: the same macro, two homoiconic languages

It is clarifying to put Elixir and LFE's unless next to each other. Same idea, same compile-time mechanics, two notations for "code as data":

# elixir - hygienic, AST encoded as {op, meta, args} tuples
defmacro unless(condition, do: block) do
  quote do
    if !unquote(condition), do: unquote(block)
  end
end

;; lfe - unhygienic, AST IS the surface S-expression
(defmacro unless (test . body)
  `(if ,test 'ok (progn ,@body)))

The deep similarity is that in both, the body is captured as code and is only evaluated if the branch is taken - the thing a function can never do. The difference is philosophical: Elixir trades a little directness (its AST is an encoding) for automatic hygiene; LFE trades hygiene for the unmediated Lisp identity of code and data. Both compile to BEAM bytecode and both call straight into Erlang/OTP with no penalty.

Erlang: the honest baseline

It is worth a paragraph on where the family started, because both Elixir and LFE are reactions to it. Erlang has compile-time facilities, but they are blunt instruments. There is the C-style preprocessor with -define, which does textual substitution and cannot see the AST:

%% erlang - a preprocessor macro: textual, no AST awareness, ?-prefixed
-define(SQUARE(X), ((X) * (X))).

area(R) -> 3.14159 * ?SQUARE(R).

For genuine AST manipulation Erlang offers parse transforms - modules that receive and rewrite the parsed forms of another module at compile time. They are powerful (this is how tools like lager once injected logging) but they are global, fragile, and explicitly discouraged for everyday use. The gap between Erlang's ?MACRO substitution and Elixir/LFE's structured, expression-level macros is exactly the gap those two languages were built to close.

Gleam: the macros that aren't there

And then there is Gleam, which looks at all of the above and says: no.

Gleam - the BEAM's one statically typed member, with a sound, fully inferred Hindley-Milner type system - has no macros, no quote/unquote, no compile-time code generation, and no metaprogramming of any kind. This is not an unfinished feature. It is a deliberate, load-bearing design decision, and it follows directly from what Gleam is optimizing for.

// gleam - there is no macro system. This is ordinary code,
// and it does exactly and only what it says. No expansion, no surprises.
pub fn double(x: Int) -> Int {
  x * 2
}

The reasoning is consistent across Gleam's whole design:

So how does Gleam cover the ground that other languages use macros for? With ordinary language features and tooling instead. Higher-order functions and the use expression handle the "run-my-code-in-a-context" patterns that Elixir reaches for __using__ to do. Repetitive boilerplate (JSON encoders, for instance) is handled by code generators that run as a separate build step and emit plain, readable, type-checked Gleam source you can open and inspect - rather than by macros that expand invisibly inside the compiler. The generated code is just code; it is in your repository, it is checked like everything else, and it holds no secrets.

// gleam - what would be a derive-macro elsewhere is, by convention,
// generated ahead of time into ordinary source you can read and review.
pub fn user_to_json(user: User) -> String {
  // ... explicit, fully type-checked, no hidden expansion ...
  todo
}

The trade is real and Gleam owns it: you write more by hand, and you cannot grow the language upward with DSLs. In exchange you get a language with no hidden control flow, a type checker whose guarantees never leak, and a surface small enough to hold entirely in your head. For Gleam, that is the better deal.

Three philosophies, one runtime

Line the family up and the spectrum is sharp:

-- lua (Luerl) - Lua's own runtime "metaprogramming" is metatables,
-- a dynamic dispatch hook, not compile-time macros like Elixir/LFE.
local Vec = {}
Vec.__add = function(a, b) return { a[1] + b[1], a[2] + b[2] } end
local v = setmetatable({1, 2}, Vec)
local w = setmetatable({3, 4}, Vec)
local s = v + w   -- {4, 6}, via __add

Takeaway

Macros are a question every language has to answer, and the BEAM family answers it five different ways on one shared runtime. Elixir says: give me compile-time power, but keep me safe by default - hence hygienic macros over a tuple AST, quote as the template and unquote as the hole. LFE says: code and data are the same thing, so let me reshape the language without ceremony - the full Lisp tradition, unhygienic and unbounded. Gleam says: the most valuable thing I can offer is that what you read is what runs - and so it has no macros at all, and is stronger for the discipline.

Same VM. Same processes. Same OTP underneath. The difference is how much of the language each one lets you rewrite - and whether it thinks you should.