⟨⟩ Eubeamgua

← History

LFE: a Lisp on the BEAM

How Erlang co-creator Robert Virding gave the BEAM a real Lisp - S-expressions, homoiconic code-as-data, true compile-time macros, and zero-penalty OTP interop because it compiles straight to Core Erlang.

LFEErlang

Every language on the BEAM is, in some sense, an argument about syntax. They all share the same runtime - the same processes, the same garbage collector, the same OTP - so what differs is how you spell a program. Elixir spells it with Ruby-ish pipes; Gleam spells it with ML-style types; Erlang spells it with the prolog-descended syntax it was born with. LFE - Lisp Flavoured Erlang - spells it with parentheses.

But LFE is not a cosmetic reskin. By choosing Lisp, it brings two things to the BEAM that none of the others had at the start: homoiconicity (code written as the very data structures the language can manipulate) and real, unrestricted macros that run at compile time. And it does this while compiling to Core Erlang, so it talks to Erlang and OTP with no penalty at all. The whole thing comes from someone with impeccable credentials: Robert Virding, one of the three original co-inventors of Erlang.

A Lisp designed for the BEAM, not bolted on

There are many ways to put a Lisp on an existing runtime, and most of them involve a small impedance mismatch: the Lisp has its own data types, its own calling convention, and a translation layer to the host. LFE refuses that compromise. Virding - a long-time Lisp programmer who also wrote much of Erlang's early standard library and compiler - set out to build a Lisp whose data types, function calls, and pattern matching are Erlang's, so that interop is free rather than bridged.

He announced the first public release on the erlang-questions mailing list in March 2008, developed against Erlang/OTP R12B. That first cut was deliberately spare: no recursive letrec, no binaries, no receive, no try, and not even a Lisp shell - it was a Lisp syntax front-end to the Erlang compiler more than a complete environment. Over the following eight years it grew into a self-hosted, feature-complete language with a proper REPL, reaching its 1.0 release in March 2016 and continuing through the modern rebar3/hex.pm era (2.2.0 landed in January 2025).

The simplest program already shows the relationship. defmodule, export, and defun are forms - parenthesised lists - but the function they call is the same io:format/2 every Erlang module uses, complete with Erlang's ~n newline directive:

;; lfe - module, export list, and a function are all just S-expressions
(defmodule hello
  (export (run 0)))

(defun run ()
  (io:format "Hello, BEAM!~n"))

The equivalent Erlang is structurally identical - a module declaration, an export list of name/arity pairs, and a function - just written with a different grammar:

%% erlang - the same shape, the same io:format, different syntax
-module(hello).
-export([run/0]).

run() ->
    io:format("Hello, BEAM!~n").

Arity matters in both: on the BEAM a function's identity is its name/arity, which is why LFE exports list (run 0) exactly as Erlang exports run/0.

S-expressions: one syntax for everything

Lisp's defining trait is that there is essentially one syntactic rule. A program is a list whose first element is the operator and whose remaining elements are arguments. (+ 1 2) calls + on 1 and 2. Because grouping is explicit in the parentheses, there is no operator precedence to memorise and no statement/expression distinction - everything is an expression that yields a value.

;; lfe - prefix notation; the parens make evaluation order unambiguous
(+ 1 2 3)                          ; => 6   (+ takes any number of args)
(* (+ 1 2) 4)                      ; => 12  (inner form evaluates first)
(io:format "~p~n" (list (+ 40 2))) ; prints 42

That uniformity reaches into LFE's data types, which are Erlang's data types written as S-expressions. Tuples use #(...), lists are written with parentheses (and must be quoted with ' to be treated as data rather than a call), and maps use a (map ...) form:

;; lfe - Erlang terms, Lisp notation
#(ok 42)              ; a tagged 2-tuple, the classic result convention
'(1 2 3)              ; a quoted literal list (no ' would mean "call 1")
(list 1 2 3)          ; the same list, built at runtime
(cons 0 '(1 2 3))     ; => (0 1 2 3)
(map 'name "Ada" 'lang 'lfe)   ; a map literal

Pattern matching - the soul of Erlang programming - comes through unchanged. A defun can have multiple clauses whose heads are patterns, with optional guards introduced by when:

;; lfe - multi-clause function with pattern-matched heads and a guard
(defun classify
  ((n) (when (< n 0)) 'negative)
  ((0) 'zero)
  ((_n) 'positive))

;; destructure the ok/error result convention directly in clause heads
(defun handle
  (((tuple 'ok value))   value)
  (((tuple 'error reason)) (io:format "failed: ~p~n" (list reason))))

Compare the same logic in Erlang. The semantics are the same - try each clause top to bottom, run the first whose pattern (and guard) matches - only the punctuation differs:

%% erlang - the same multi-clause + guard idiom
classify(N) when N < 0 -> negative;
classify(0)            -> zero;
classify(N)            -> positive.

handle({ok, Value})     -> Value;
handle({error, Reason}) -> io:format("failed: ~p~n", [Reason]).

Homoiconicity: code is data

Here is where LFE diverges sharply from its siblings. In Lisp, the text you write - those nested lists of symbols, numbers, and tuples - is literally the same data structure the language manipulates at runtime. A program is a list. A function call is a list. This property is called homoiconicity, and it is what makes Lisp macros not just possible but natural.

In most languages, "code that writes code" means generating strings and hoping they parse, or building an abstract syntax tree through a verbose API. In LFE, the AST is the surface syntax. You can quote a fragment of program with ' and get back an ordinary list you can inspect and rearrange:

;; lfe - quoting a call yields a plain list you can manipulate as data
'(+ 1 2)              ; => (+ 1 2) -- a 3-element list: the symbol +, 1, 2
(car '(+ 1 2))       ; => +       -- the operator is just the head
(cdr '(+ 1 2))       ; => (1 2)   -- the arguments are just the tail

LFE is not the only BEAM language with this insight. Elixir is also homoiconic - but its AST is exposed as nested {operation, metadata, arguments} tuples rather than as the surface syntax itself, surfaced via quote:

# elixir - homoiconic too, but the AST is a tuple representation, not the source
quote do: 1 + 2
# => {:+, [context: Elixir, ...], [1, 2]}

The difference is one of directness. In LFE the program text and its data representation are the same thing; in Elixir the data representation is a faithful but distinct encoding. That directness is exactly the Lisp tradition LFE is faithful to.

It is worth noting one consequence Common Lisp programmers will recognise: LFE is a Lisp-2 (in fact a "Lisp-2+"), meaning functions and variables live in separate namespaces. A local variable named list does not shadow the list function - a deliberate choice in the Common Lisp lineage rather than Scheme's single-namespace design.

Real macros: extending the language itself

Because code is data, a macro in LFE is just a function that runs at compile time: it receives unevaluated code as data and returns new code to be compiled in its place. This is genuine compile-time metaprogramming - the thing plain Erlang never offered - and it is the headline reason to reach for LFE.

You define one with defmacro. The key tool is quasiquotation: a backquote ` builds a code template, a comma , splices in a single value, and ,@ splices in a list of values. Here is unless, a control form that expands into an if:

;; lfe - a macro: receives code, returns code, runs at compile time
(defmacro unless
  ((cons test body)
   `(if ,test
        'ok
        (progn ,@body))))

The (cons test body) pattern binds test to the first argument and captures every remaining form in body as a list, and ,@body splices that list into the progn. Now unless looks and behaves like a built-in:

;; lfe - using the macro; the body is NOT evaluated unless the test is false
(unless (== status 'ok)
  (io:format "something is wrong~n" '())
  (alert!))

The decisive point is that a macro controls whether and how often its arguments are evaluated - something a function fundamentally cannot do, because a function's arguments are evaluated before it is even called. That's why unless can choose not to run its body at all. You can see exactly what a macro produces with macroexpand-1, which is invaluable for debugging:

;; lfe - expand a macro one step to see the generated code
lfe> (macroexpand-1 '(unless x (f) (g)) $ENV)
(if x 'ok (progn (f) (g)))

For simpler, rule-based transformations LFE also offers defsyntax (and define-syntax), modelled on Scheme's syntax-rules. One caveat to be precise about: LFE macros are unhygienic, in the Common Lisp tradition - they do not automatically rename the variables they introduce. A careful author generates unique names (via the standard macro toolkit) to avoid accidentally capturing a caller's bindings. This is exactly the trade-off Common Lisp makes: maximum power, with the responsibility that comes with it.

This is a genuine point of contrast across the family. Elixir has macros too, but they are hygienic - variables they introduce can't clash with the call site by accident:

# elixir - hygienic macro: quote captures the AST, unquote injects values
defmacro unless(condition, do: block) do
  quote do
    if !unquote(condition), do: unquote(block)
  end
end

And Gleam sits at the opposite end of the spectrum entirely: it has no macros and no metaprogramming at all. Keeping the language small and predictable is an explicit design goal, so there is no quote, no unquote, no compile-time code generation - Gleam code does exactly and only what it literally says. Three philosophies, one runtime: LFE's unrestricted Lisp macros, Elixir's hygienic macros, Gleam's deliberate absence of them.

Compiling to Core Erlang

The reason LFE's interop is free rather than bridged is the compilation path. The LFE compiler runs in three passes - macro expansion, then linting, then code generation - and the linter and code generator only ever see LFE's small set of core forms. Code generation emits Core Erlang, the compact, explicitly functional intermediate language that the Erlang compiler itself uses internally. LFE then hands that Core Erlang to the back end of the standard Erlang compiler, which produces the final .beam bytecode.

LFE source  →  macro expansion  →  lint  →  Core Erlang  →  Erlang compiler back end  →  .beam

The payoff is total: LFE produces 100% Erlang-compatible modules. There is no foreign-function interface, no marshalling, no wrapper layer. An LFE module calling lists:foldl/3 is simply one BEAM module calling another, at exactly the speed Erlang gets:

;; lfe - calling straight into Erlang's stdlib with zero penalty
(defmodule math-utils
  (export (double 1) (sum 1)))

(defun double (x) (* x 2))

(defun sum (lst)
  (lists:foldl (lambda (x acc) (+ x acc)) 0 lst))

Because everything converges on the same bytecode and the same in-memory term representation, this cuts both ways: Erlang and Elixir can call LFE modules just as easily as LFE calls them. LFE code coexists with vanilla Erlang in the same release, and by extension with the entire BEAM ecosystem.

A practical bonus of this design: improvements to the Erlang compiler and to OTP flow straight through to LFE for free, because LFE rides the same back end rather than maintaining a parallel one.

Full OTP interop: a gen_server in parentheses

Compiling to Core Erlang means LFE inherits all of the BEAM's runtime gifts directly - lightweight processes, message passing, supervision trees, hot code loading, and the let-it-crash philosophy - and, crucially, the OTP behaviours that package them. There is nothing to reimplement; LFE just uses gen_server, supervisor, and friends.

The low-level concurrency primitives are all there, written as forms. (self) returns the current process id, spawn starts a process, ! sends a message, and receive matches on the mailbox:

;; lfe - spawn workers, collect their replies; pure BEAM concurrency
(defun run ()
  (let ((parent (self)))
    (lists:foreach
      (lambda (n)
        (spawn (lambda () (! parent (tuple n (* n n))))))
      (lists:seq 1 5))
    (collect 5 0)))

(defun collect
  ((0 sum) sum)
  ((count sum)
   (receive
     ((tuple _n square) (collect (- count 1) (+ sum square))))))
;; (run) => 55

But the idiomatic way to build a stateful server on the BEAM is the gen_server behaviour, and LFE declares it with a (behaviour gen_server) line, then implements the standard callbacks (init, handle_call for synchronous requests, handle_cast for asynchronous ones) as ordinary defuns:

;; lfe - a complete OTP gen_server, in S-expressions
(defmodule counter
  (behaviour gen_server)
  ;; public client API
  (export (start_link 0) (increment 0) (value 0))
  ;; gen_server callbacks
  (export (init 1) (handle_call 3) (handle_cast 2)))

;; --- client API: thin wrappers over the gen_server calls ---
(defun start_link ()
  (gen_server:start_link #(local counter) 'counter '() '()))

(defun increment ()
  (gen_server:cast 'counter 'increment))      ; fire-and-forget

(defun value ()
  (gen_server:call 'counter 'value))          ; synchronous request/reply

;; --- callbacks: run inside the server process ---
(defun init (_args)
  #(ok 0))                                     ; initial state = 0

(defun handle_call
  (('value _from state)
   `#(reply ,state ,state)))                   ; reply with the count

(defun handle_cast
  (('increment state)
   `#(noreply ,(+ state 1))))                  ; bump the count, no reply

Notice the return tuples - #(ok 0), #(reply ,state ,state), #(noreply ,(+ state 1)) - are exactly the tuples OTP's gen_server expects, here built with quasiquote so values can be spliced in with ,. The Erlang equivalent uses the same atoms and the same tuple shapes:

%% erlang - the same gen_server callbacks, same return tuples
init(_Args) ->
    {ok, 0}.

handle_call(value, _From, State) ->
    {reply, State, State}.

handle_cast(increment, State) ->
    {noreply, State + 1}.

That parallelism is the whole point. To OTP, the LFE module and the Erlang module are indistinguishable: both are BEAM modules implementing the gen_server contract. You can supervise the LFE counter from an Erlang supervisor, call it from Elixir, and upgrade it with hot code loading - all without LFE doing anything special, because at the bytecode level it is an Erlang module.

Where LFE fits in the family

Put the five mainstream BEAM languages side by side and LFE's niche is sharp:

If you want Lisp's signature superpower - treating programs as the malleable data they truly are, and growing the language upward with macros - LFE is the way to get it without giving up an ounce of the BEAM's concurrency or reliability. It is, fittingly, a Lisp built by someone who helped build the very runtime it runs on.

Takeaway

LFE answers a precise question: what does the BEAM look like through a Lisp lens? The answer is S-expressions for everything, code that is data, macros that genuinely extend the language at compile time, and a compiler that lowers all of it to Core Erlang so that OTP, the standard library, and the rest of the BEAM ecosystem are right there with zero friction. It is not Lisp grafted onto Erlang; it is Lisp and Erlang turning out to be the same thing, written two different ways.

Parentheses on the outside. The BEAM all the way down.