Bytecode VM — Architecture & Implementation Guide

The bytecode backend compiles a parsed Block AST into a register-based bytecode representation and executes it in a virtual machine. It is an alternative to the tree-walking interpreter and produces identical results for all valid Lua 5.5 programs.

The frontend (lexer, parser, optimizer) lives in the internal LuaInterpreterParser target; runtime types live in LuaInterpreterCore. This module (LuaInterpreterBytecode) depends on both.

Overview
Data Structures
Instruction Set — complete opcode listing
Compiler
Virtual Machine
Upvalues and Closures
Multi-Value Convention
Control Flow
For Loops
Table Construction
Public API

1. Overview

Source String
      │
      ▼
  ┌────────┐
  │  Lexer │
  └────────┘
      │  [Token]
      ▼
  ┌────────┐
  │ Parser │
  └────────┘
      │  Block (AST)
      ▼
  ┌───────────┐   (when optimizationEnabled == true)
  │ Optimizer │
  └───────────┘
      │  Block (transformed AST)
      ▼
  ┌──────────┐
  │ Compiler │  Compiler.swift
  └──────────┘
      │  Prototype (bytecode)
      ▼
  ┌─────┐
  │  VM │  BytecodeInterpreter.swift
  └─────┘
      │  [LuaValue]
      ▼
     Output

The Compiler translates a Block AST into a tree of Prototype objects — one per function body, nested. The VM (BytecodeInterpreter.runFrame) then executes a Prototype by dispatching on each Instruction in sequence.

2. Data Structures

`Prototype`

The static compiled representation of a single Lua function body. Multiple closures can share the same Prototype (e.g. a function created inside a loop).

public final class Prototype {
    var instructions: [Instruction]   // bytecode sequence
    var constants:    [LuaValue]      // literal pool (strings, floats, etc.)
    var protos:       [Prototype]     // nested function bodies
    var upvalueDescs: [UpvalueDescriptor] // how to capture upvalues at closure-creation time
    var paramCount:   Int             // number of fixed parameters
    var hasVarargs:   Bool            // whether ... is accepted
    var maxStack:     Int             // total registers needed (locals + max temporaries)
    var localNames:   [(name, startPC, endPC)] // debug info
    var upvalueNames: [String]        // debug info
    var toCloseRegs:  [Int]           // registers with the <close> attribute
}

`UpvalueCell`

A heap-allocated mutable box for a single LuaValue. Every register in a call frame is an UpvalueCell, so closures can capture mutable locals by reference without any special "close" step at capture time.

public final class UpvalueCell {
    var value: LuaValue
}

`LuaClosure`

The runtime closure: a Prototype plus the array of UpvalueCells captured from the enclosing scope when the closure instruction ran.

public final class LuaClosure {
    let proto:    Prototype
    let upvalues: [UpvalueCell]
}

`UpvalueDescriptor`

Describes how to obtain an upvalue cell when a closure is created (a closure instruction executes):

.inStack(reg: Int) — capture the enclosing frame's register cell at reg directly
.inUpvalue(idx: Int) — propagate an upvalue already held by the enclosing closure at idx

`CallFrame`

Per-call mutable state held only by the VM:

final class CallFrame {
    let closure:    LuaClosure
    var pc:         Int           // program counter — index into proto.instructions
    var registers:  [UpvalueCell] // register file, size = proto.maxStack
    var varargs:    [LuaValue]    // excess arguments beyond paramCount
    var top:        Int           // dynamic top-of-stack (updated by variable-produce ops)
    var toCloseRegs: [Int]        // registers to close on scope exit
}

3. Instruction Set

All instructions are cases of the Instruction enum. The naming convention is:

Symbol	Meaning
`dst`	Destination register
`src`	Source register
`tbl`	Register holding a table
`key`	Register holding a key
`kIdx`	Index into `proto.constants`
`upv`	Index into the closure's `upvalues` array

Multi-value operands (nArgs, nRet, n, count on setList / vararg / returnOp):

Value	Meaning
`≥ 0`	Exactly that many values
`-1`	Variable: use `frame.top` as the end of the argument/result span

Complete opcode listing

Every opcode is a case of Instruction in Sources/LuaInterpreterBytecode/Instruction.swift. Register operands index into the current frame's register file unless noted.

Opcode	Operands	Semantics
`loadNil`	`dst: Int`, `count: Int`	`regs[dst…dst+count-1] ← nil`
`loadBool`	`dst: Int`, `value: Bool`	`regs[dst] ← value`
`loadInt`	`dst: Int`, `value: Int64`	`regs[dst] ← .int(value)` (no constant pool)
`loadFloat`	`dst: Int`, `value: Double`	`regs[dst] ← .float(value)` (no constant pool)
`loadK`	`dst: Int`, `k: Int`	`regs[dst] ← constants[k]`
`move`	`dst: Int`, `src: Int`	`regs[dst] ← regs[src]`
`getUpval`	`dst: Int`, `upv: Int`	`regs[dst] ← closure.upvalues[upv].value`
`setUpval`	`src: Int`, `upv: Int`	`closure.upvalues[upv].value ← regs[src]`
`getGlobal`	`dst: Int`, `kIdx: Int`	`regs[dst] ← globals[constants[kIdx]]`
`setGlobal`	`src: Int`, `kIdx: Int`	`globals[constants[kIdx]] ← regs[src]`
`newTable`	`dst: Int`	`regs[dst] ← new LuaTable()`
`getTable`	`dst: Int`, `tbl: Int`, `key: Int`	`regs[dst] ← tableGet(regs[tbl], regs[key])`
`setTable`	`tbl: Int`, `key: Int`, `val: Int`	`tableSet(regs[tbl], regs[key], regs[val])`
`getField`	`dst: Int`, `tbl: Int`, `kIdx: Int`	`regs[dst] ← tableGet(regs[tbl], constants[kIdx])`
`setField`	`tbl: Int`, `kIdx: Int`, `val: Int`	`tableSet(regs[tbl], constants[kIdx], regs[val])`
`selfOp`	`dst: Int`, `tbl: Int`, `kIdx: Int`	`regs[dst+1] ← regs[tbl]`; `regs[dst] ← method` (for `obj:method()`)
`setList`	`tbl: Int`, `count: Int`	Fill array part of `regs[tbl]` from `regs[tbl+1…]`; `count = -1` uses `frame.top`
`add`	`dst: Int`, `b: Int`, `c: Int`	`regs[dst] ← regs[b] + regs[c]` (metamethods)
`sub`	`dst: Int`, `b: Int`, `c: Int`	`regs[dst] ← regs[b] - regs[c]`
`mul`	`dst: Int`, `b: Int`, `c: Int`	`regs[dst] ← regs[b] * regs[c]`
`div`	`dst: Int`, `b: Int`, `c: Int`	`regs[dst] ← regs[b] / regs[c]`
`idiv`	`dst: Int`, `b: Int`, `c: Int`	`regs[dst] ← regs[b] // regs[c]`
`mod`	`dst: Int`, `b: Int`, `c: Int`	`regs[dst] ← regs[b] % regs[c]`
`pow`	`dst: Int`, `b: Int`, `c: Int`	`regs[dst] ← regs[b] ^ regs[c]`
`unm`	`dst: Int`, `src: Int`	`regs[dst] ← -regs[src]`
`notOp`	`dst: Int`, `src: Int`	`regs[dst] ← !regs[src].isTruthy`
`lenOp`	`dst: Int`, `src: Int`	`regs[dst] ← #regs[src]` (`__len`)
`bnot`	`dst: Int`, `src: Int`	`regs[dst] ← ~regs[src]` (bitwise)
`band`	`dst: Int`, `b: Int`, `c: Int`	`regs[dst] ← regs[b] & regs[c]`
`bor`	`dst: Int`, `b: Int`, `c: Int`	`regs[dst] ← regs[b] \| regs[c]`
`bxor`	`dst: Int`, `b: Int`, `c: Int`	`regs[dst] ← regs[b] ~ regs[c]`
`shl`	`dst: Int`, `b: Int`, `c: Int`	`regs[dst] ← regs[b] << regs[c]`
`shr`	`dst: Int`, `b: Int`, `c: Int`	`regs[dst] ← regs[b] >> regs[c]`
`concat`	`dst: Int`, `from: Int`, `to: Int`	`regs[dst] ← regs[from] .. … .. regs[to]`
`jmp`	`offset: Int`	`pc += offset` (relative to instruction after `jmp`)
`test`	`src: Int`, `k: Bool`	If `regs[src].isTruthy != k`, skip next instruction (`jmp`)
`testSet`	`dst: Int`, `src: Int`, `k: Bool`	If `regs[src].isTruthy == k`: `regs[dst] ← regs[src]`, skip next
`eq`	`k: Bool`, `b: Int`, `c: Int`	If `(regs[b] == regs[c]) != k`, skip next `jmp`
`lt`	`k: Bool`, `b: Int`, `c: Int`	If `(regs[b] < regs[c]) != k`, skip next `jmp`
`le`	`k: Bool`, `b: Int`, `c: Int`	If `(regs[b] <= regs[c]) != k`, skip next `jmp`
`call`	`func_: Int`, `nArgs: Int`, `nRet: Int`	Call `regs[func_]`; args at `func_+1…`; `nArgs`/`nRet` may be `-1`
`tailCall`	`func_: Int`, `nArgs: Int`	Tail call; return merges into caller frame
`returnOp`	`base: Int`, `n: Int`	Return `regs[base…]`; `n = 0` none; `n = -1` through `frame.top - 1`
`return0`	(none)	Fast return with zero values
`vararg`	`dst: Int`, `n: Int`	Copy `...` into `regs[dst…]`; `n = 0` all; `n > 0` pad with nil
`forPrep`	`base: Int`, `skipOffset: Int`	Numeric `for`: init at `base`, limit `base+1`, step `base+2`, var `base+3`
`forLoop`	`base: Int`, `backOffset: Int`	Step numeric `for`; jump back if still in range
`tForCall`	`base: Int`, `nResults: Int`	Generic `for`: call iterator at `base`
`tForLoop`	`base: Int`, `backOffset: Int`	If `regs[base+2] != nil`, jump back to `tForCall`
`closure`	`dst: Int`, `protoIdx: Int`	`regs[dst] ←` closure for `proto.protos[protoIdx]`
`markTBC`	`reg: Int`	Mark `reg` as to-be-closed (`<close>`)
`closeUpvals`	`from: Int`	Run `__close` on all TBC registers ≥ `from`

Loads

Instruction	Effect
`loadNil(dst, count)`	`regs[dst…dst+count-1] = nil`
`loadBool(dst, value)`	`regs[dst] = value`
`loadInt(dst, value)`	`regs[dst] = Int64(value)`
`loadFloat(dst, value)`	`regs[dst] = Double(value)`
`loadK(dst, k)`	`regs[dst] = constants[k]`

Register Move

Instruction	Effect
`move(dst, src)`	`regs[dst] = regs[src]`

Upvalue Access

Instruction	Effect
`getUpval(dst, upv)`	`regs[dst] = closure.upvalues[upv].value`
`setUpval(src, upv)`	`closure.upvalues[upv].value = regs[src]`

Global Access

Globals are stored in the interpreter's Environment. The key is looked up in constants[kIdx].

Instruction	Effect
`getGlobal(dst, kIdx)`	`regs[dst] = globals[constants[kIdx]]`
`setGlobal(src, kIdx)`	`globals[constants[kIdx]] = regs[src]`

Table Operations

Instruction	Effect
`newTable(dst)`	`regs[dst] = new LuaTable()`
`getTable(dst, tbl, key)`	`regs[dst] = tableGet(regs[tbl], regs[key])`
`setTable(tbl, key, val)`	`tableSet(regs[tbl], regs[key], regs[val])`
`getField(dst, tbl, kIdx)`	`regs[dst] = tableGet(regs[tbl], constants[kIdx])`
`setField(tbl, kIdx, val)`	`tableSet(regs[tbl], constants[kIdx], regs[val])`
`selfOp(dst, tbl, kIdx)`	`regs[dst+1] = regs[tbl]; regs[dst] = tableGet(regs[tbl], constants[kIdx])`
`setList(tbl, count)`	Fill array part of `regs[tbl]` with `regs[tbl+1…]`; count=-1 uses `frame.top`

selfOp implements the method-call sugar t:f() — it copies the object into dst+1 (as self) and the method into dst (as the function), so a subsequent call can treat them as normal function + first argument.

Arithmetic

All binary operators use (dst, b, c) and dispatch through metamethods for non-numeric operands: add, sub, mul, div, idiv, mod, pow. Unary minus: unm(dst, src).

Bitwise

Binary (dst, b, c): band, bor, bxor, shl, shr. Unary: bnot(dst, src).

Other Unary

Instruction	Operands	Effect
`notOp`	`dst`, `src`	`regs[dst] = !regs[src].isTruthy`
`lenOp`	`dst`, `src`	`regs[dst] = #regs[src]` (respects `__len`)

Concatenation

Instruction	Effect
`concat(dst, from, to)`	`regs[dst] = regs[from] .. regs[from+1] .. … .. regs[to]`

Jumps and Conditionals

Instruction	Effect
`jmp(offset)`	`pc += offset` (relative to instruction after `jmp`)
`test(src, k)`	If `regs[src].isTruthy != k`, skip next instruction (a `jmp`)
`testSet(dst, src, k)`	If `regs[src].isTruthy == k`: `regs[dst] = regs[src]`, skip next
`eq(k, b, c)`	If `(regs[b] == regs[c]) != k`, skip next `jmp`
`lt(k, b, c)`	If `(regs[b] < regs[c]) != k`, skip next `jmp`
`le(k, b, c)`	If `(regs[b] <= regs[c]) != k`, skip next `jmp`

Comparisons always precede a jmp. The pattern is:

eq(k: true, b, c)   ; skip jmp if equal
jmp(to-false-branch)
... true branch ...

Calls and Returns

Instruction	Effect
`call(func_, nArgs, nRet)`	Call `regs[func_]` with args in `regs[func_+1…]`
`tailCall(func_, nArgs)`	Same but returns directly from the enclosing call
`returnOp(base, n)`	Return `regs[base…base+n-1]`; n=0 returns nothing; n=-1 uses `frame.top`
`return0`	Fast zero-value return

Varargs

Instruction	Effect
`vararg(dst, n)`	Place `...` into `regs[dst…]`; n=0 uses all (updates `frame.top`); n>0 pads to exactly n

Loops

Instruction	Effect
`forPrep(base, skipOffset)`	Coerce init/limit/step; skip if loop would not execute
`forLoop(base, backOffset)`	Advance loop variable; jump back if still in range
`tForCall(base, nResults)`	Call iterator `regs[base](state, control)` → results at `regs[base+3…]`
`tForLoop(base, backOffset)`	If `regs[base+2] != nil`, jump back to `tForCall`

Closures

Instruction	Effect
`closure(dst, protoIdx)`	Create closure from `proto.protos[protoIdx]`; capture upvalue cells per `upvalueDescs`

To-Be-Closed

Instruction	Operands	Effect
`markTBC`	`reg`	Mark register as to-be-closed after `<close>` local is initialized
`closeUpvals`	`from`	Close all to-be-closed registers ≥ `from` (calls `__close` metamethods)

4. Compiler

File: Sources/LuaInterpreterBytecode/Compiler.swift

The Compiler is a single-pass recursive visitor over the AST. It maintains:

locals: [LocalInfo] — currently in-scope locals, in order of declaration
numLocals: Int — count of current locals (= bottom of the temp region)
tempDepth: Int — number of currently live temporaries above the locals
proto: Prototype — the output being built
parent: Compiler? — enclosing function compiler (for upvalue resolution)

Register layout

┌──────────────────────────────────────────┐
│  locals[0..numLocals-1]  │  temps[0..tempDepth-1]  │  ...  │
└──────────────────────────────────────────┘
  reg 0                     reg numLocals

Temporaries are allocated with allocTemp() (returns numLocals + tempDepth, increments tempDepth) and freed with freeTempTo(savedDepth). At the top of every compileExpr call, ensureTemp(dst) is called to guarantee that subsequent allocTemp() calls won't return dst or anything below it — preventing sub-expression temporaries from overwriting the result register.

Variable resolution

resolveVar(name) searches in order:

Current locals array (innermost first) → .local(reg)
proto.upvalueNames (already registered) → .upvalue(idx)
resolveUpvalue(name, in: parent) (recursive upvalue creation) → .upvalue(idx)
Fallback → .global(name)

resolveUpvalue walks the parent chain. When a local is found in an enclosing scope, UpvalueDescriptor.inStack(reg) is added. When the upvalue is found in an already-enclosing upvalue list, UpvalueDescriptor.inUpvalue(idx) is used to propagate it up the chain.

Jump backpatching

Forward jumps are emitted with offset: 0 and their position saved. patch(pc, to: target) rewrites the offset to target - (pc + 1), since the VM increments pc before executing each instruction. Backward jumps (loops) compute the signed offset directly at emit time.

Multi-component function names

function a.b.c:d() has names = ["a","b","c"] and method = "d". The compiler:

Loads a into a temp register
Chains getField instructions for intermediate names (b, c)
Uses setField to assign the closure to the last key (d)

The base name (a) is recognized by the optimizer as a variable read, so the optimizer will not incorrectly eliminate a local a = {} declaration as unused.

5. Virtual Machine

File: Sources/LuaInterpreterBytecode/BytecodeInterpreter.swift (runFrame method)

The VM is a straightforward dispatch loop:

while pc < instructions.count:
    instr = instructions[pc]
    pc += 1
    switch instr { ... }

pc is incremented before the instruction executes, so all jump offsets are relative to the instruction after the jump. This matches the patch offset formula target - (jmpPC + 1).

Arithmetic and metamethods

Arithmetic instructions call shared helpers (luaAdd, luaSub, etc.) that:

Try the fast numeric path (int/int or float/float)
Coerce via toArithFloat for mixed types
Fall back to getMetamethod for non-numerics

Upvalue capture at closure creation

When closure(dst, protoIdx) executes, the VM walks innerProto.upvalueDescs:

.inStack(reg) → appends frame.registers[reg] (the actual cell, not a copy)
.inUpvalue(idx) → appends frame.closure.upvalues[idx]

Because all registers are already UpvalueCell references, "open upvalue" creation requires no additional heap allocation.

6. Upvalues and Closures

All locals are stored as UpvalueCell objects from the moment of allocation. When a closure captures a local, it receives a direct reference to that cell. Mutations through the outer name or any closure are immediately visible everywhere — no "open/close upvalue" transition is needed at runtime.

local x = 0
local function inc() x = x + 1 end
local function get() return x end
inc()   -- x cell modified
get()   -- reads same cell → 1

Both inc and get hold references to the same UpvalueCell for x. The cell lives as long as either closure is reachable.

7. Multi-Value Convention

Several instructions use a signed integer to express a variable count:

Value	Meaning
`>= 0`	Exactly that many values
`-1`	Variable: read/write from `frame.top`

call(func_, nArgs, nRet)

nArgs = -1: args are regs[func_+1 .. frame.top-1] (set by a prior variable-produce instruction)
nRet = -1: place all results starting at regs[func_], update frame.top
nRet >= 0: place exactly nRet results, reset frame.top to proto.maxStack

returnOp(base, n)

n = 0: return nothing
n = -1: return regs[base .. frame.top-1]
n >= 1: return regs[base .. base+n-1]

vararg(dst, n)

n = 0: place all varargs at regs[dst…], update frame.top
n > 0: place exactly n varargs (padding with nil), do not update frame.top

setList(tbl, count)

count >= 0: read exactly count items from regs[tbl+1…]
count = -1: read frame.top - (tbl+1) items

The frame.top chain is set up so that when a function call or vararg instruction produces variable results into a contiguous region, the subsequent instruction that consumes them (another call, setList, or returnOp) uses frame.top to know the actual count.

8. Control Flow

Logical `and` / `or`

a and b compiles to:

[compile a → dst]
test(src: dst, k: false)   ; if truthy, skip jmp (evaluate b)
jmp(to-end)                ; a is falsy: keep a, skip b
[compile b → dst]
[end:]

a or b compiles to:

[compile a → dst]
test(src: dst, k: true)    ; if falsy, skip jmp (evaluate b)
jmp(to-end)                ; a is truthy: keep a, skip b
[compile b → dst]
[end:]

Comparisons

if a < b then ... end

Compiles to:

[compile a → rA]
[compile b → rB]
lt(k: true, b: rA, c: rB)  ; if NOT (a < b): skip next jmp
jmp(to-else)               ; condition false → else
... then body ...
[else:]

The k flag inverts the test so the jmp is always a "skip this branch" jump.

`goto` and Labels

goto name emits a jmp(0) and records the patch location. When the label ::name:: is seen, all pending patches for that label are resolved. Labels defined before the goto are resolved immediately; forward labels are resolved as they appear.

9. For Loops

Numeric `for`

for i = start, limit, step do ... end

forPrep(base, skipOffset)  ; coerce values; skip to end if range empty
[loop body — uses regs[base+3] as i]
forLoop(base, backOffset)  ; advance, jump back if in range
[after loop:]

forLoop uses wrapping arithmetic (&+) for integer step to match Lua's overflow semantics.

Generic `for`

for k, v in iter, state, init do ... end

[set up base, base+1, base+2]
[loop top:]
tForCall(base, nResults)   ; call iter(state, control) → regs[base+3…]; update regs[base+2]
test(src: base+3, k: false); if first result is nil, skip jmp (exit)
jmp(exit)
[loop body]
jmp(loop top via tForLoop backpatch)
tForLoop(base, backOffset) ; if regs[base+2] != nil, jump back
[after loop:]

10. Table Construction

{a, b, key=val, c} compiles to:

newTable(dst) — create the table
Named/indexed fields: evaluate value, setField or setTable
Positional items: evaluate directly into regs[dst+1], regs[dst+2], … then setList(dst, count)
If the last positional item is a multi-value expression (function call, vararg), it is compiled with nRet=-1 and setList(dst, -1) consumes all results via frame.top

11. Public API

`BytecodeInterpreter`

import LuaInterpreterBytecode

let interp = BytecodeInterpreter()
interp.outputHandler = { print($0, terminator: "") }

try interp.execute("print('hello')")

Member	Description
`init(optimizationEnabled: Bool = true)`	Create interpreter; registers standard library
`globals: Environment`	Persistent global scope
`outputHandler: ((String) -> Void)?`	Custom output sink (nil → stdout)
`optimizationEnabled: Bool`	Whether to run the optimizer before compiling
`execute(_ source: String) throws -> [LuaValue]`	Parse, compile, and execute Lua source
`execute(block: Block) throws -> [LuaValue]`	Compile and execute a pre-parsed AST
`register(_ name: String, body: ([LuaValue]) throws -> [LuaValue])`	Register a native function
`callFunction(_ function: LuaValue, args: [LuaValue]) throws -> [LuaValue]`	Call a Lua value from Swift

`Lua` enum extensions

import LuaInterpreterBytecode

// One-shot execution
let results = try Lua.runBytecode("return 1 + 2")

// Persistent interpreter
let interp = Lua.createBytecodeInterpreter()
try interp.execute("x = 42")

Method	Description
`Lua.runBytecode(_ source: String) throws -> [LuaValue]`	Parse + execute in a fresh bytecode interpreter
`Lua.createBytecodeInterpreter(optimizationEnabled: Bool = true) -> BytecodeInterpreter`	Create a persistent bytecode interpreter

FilesExpand file tree

bytecode-vm.md

Latest commit

History