# Lux Compilation Strategy ## Vision Lux should compile to native code with zero-cost effects AND compile to JavaScript for frontend development - all while eliminating runtime overhead like Svelte does. ## Current State | Component | Status | |-----------|--------| | Interpreter | Full-featured, all language constructs | | C Backend | Complete (functions, closures, pattern matching, lists, RC) | | JS Backend | Complete (full language, browser & Node.js, DOM, TEA) | | JIT (Cranelift) | Integer arithmetic only, ~160x speedup | | Targets | Native (via C), JavaScript, JIT | ## Target Architecture ``` ┌──→ C Backend ──→ GCC/Clang ──→ Native Binary (servers, CLI) │ Lux Source ──→ HIR ──→ MIR ──→ JS Backend ──→ Optimized JS (no runtime, like Svelte) │ └──→ Wasm Backend ──→ WebAssembly (browsers, edge) ``` ## Compilation Targets Comparison ### What Other Languages Do | Language | Backend | Runtime | Effect Strategy | |----------|---------|---------|-----------------| | **Koka** | C (Perceus) | None | Evidence passing | | **Elm** | JS | Small runtime | No effects (pure) | | **Gleam** | Erlang/JS | BEAM/JS runtime | No effects | | **Svelte** | JS | **No runtime** | Compile-time reactivity | | **ReScript** | JS | Minimal | No effects | | **PureScript** | JS | Small runtime | Monad transformers | ### Key Insight: Svelte's No-Runtime Approach Svelte compiles components to vanilla JavaScript that directly manipulates the DOM. No virtual DOM diffing, no framework code shipped to the browser. ```svelte ``` Compiles to: ```javascript // Direct DOM manipulation, no runtime function create_fragment(ctx) { let button; return { c() { button = element("button"); button.textContent = ctx[0]; }, m(target, anchor) { insert(target, button, anchor); listen(button, "click", ctx[1]); }, p(ctx, dirty) { if (dirty & 1) set_data(button, ctx[0]); } }; } ``` **We can do the same for Lux effects on the frontend.** ## Phase 1: C Backend (Native Compilation) ### Goal Compile Lux to C code that can be compiled by GCC/Clang for native execution. ### Why C? 1. **Portability**: C compilers exist for every platform 2. **Performance**: Leverage decades of GCC/Clang optimizations 3. **No runtime**: Like Koka, compile effects away 4. **Proven path**: Koka, Nim, Chicken Scheme all do this successfully ### Implementation Plan #### Step 1.1: Basic C Code Generation - Integer arithmetic and comparisons - Function definitions and calls - If/else expressions - Let bindings #### Step 1.2: Data Structures - Records → C structs - ADTs → Tagged unions - Lists → Linked lists or arrays - Strings → UTF-8 byte arrays #### Step 1.3: Effect Compilation (Evidence Passing) Transform: ```lux fn greet(name: String): Unit with {Console} = Console.print("Hello, " + name) ``` To: ```c void greet(Evidence* ev, LuxString name) { LuxString msg = lux_string_concat("Hello, ", name); ev->console->print(ev, msg); } ``` #### Step 1.4: Memory Management (Perceus-style) - Compile-time reference counting insertion - Reuse analysis for in-place updates - No garbage collector needed ### File Structure ``` src/ codegen/ mod.rs # Code generation module c_backend.rs # C code generation c_runtime.h # Minimal C runtime header c_runtime.c # Runtime support (RC, strings, etc.) ``` ## Phase 2: JavaScript Backend (Frontend) ### Goal Compile Lux to optimized JavaScript with NO runtime, like Svelte. ### Effect Mapping | Lux Effect | JS Compilation | |------------|----------------| | `Console.print` | `console.log()` | | `Dom.getElementById` | `document.getElementById()` | | `Dom.addEventListener` | Direct event binding | | `Http.get` | `fetch()` with async/await | | `State.get/set` | Compile-time reactivity | ### No-Runtime Strategy Instead of shipping a runtime, compile effects to direct DOM manipulation: ```lux effect Dom { fn getElementById(id: String): Element fn setTextContent(el: Element, text: String): Unit fn addEventListener(el: Element, event: String, handler: fn(): Unit): Unit } fn counter(): Unit with {Dom} = { let btn = Dom.getElementById("btn") let count = 0 Dom.addEventListener(btn, "click", fn() => { count = count + 1 Dom.setTextContent(btn, toString(count)) }) } ``` Compiles to: ```javascript // No runtime needed - direct DOM calls function counter() { const btn = document.getElementById("btn"); let count = 0; btn.addEventListener("click", () => { count = count + 1; btn.textContent = String(count); }); } ``` ### Reactive State (Like Svelte) For reactive state, compile to fine-grained updates: ```lux effect Reactive { fn signal(initial: T): Signal fn derived(compute: fn(): T): Signal } let count = Reactive.signal(0) let doubled = Reactive.derived(fn() => count.get() * 2) ``` Compiles to: ```javascript // Compile-time reactivity, no virtual DOM let count = 0; let doubled; const count_subscribers = new Set(); function set_count(value) { count = value; doubled = count * 2; // Statically known dependency count_subscribers.forEach(fn => fn()); } ``` ## Phase 3: WebAssembly Backend ### Goal Compile to Wasm for browser and edge deployment. ### Strategy - Reuse C backend: `C → Emscripten → Wasm` - Or direct Wasm generation via Cranelift ## Phase 4: Zero-Cost Effects (Evidence Passing) ### Current Problem Effects use runtime handler lookup - O(n) per effect operation. ### Solution: Evidence Passing Transform effect operations to direct function calls at compile time. Before (runtime lookup): ``` performEffect("Console", "print", [msg]) → search handler stack → invoke handler ``` After (compile-time): ``` evidence.console.print(msg) → direct function call ``` ### Implementation ```rust // Transform AST to evidence-passing form fn transform_to_evidence(expr: Expr, effects: &[Effect]) -> Expr { match expr { Expr::EffectOp { effect, operation, args } => { // Transform Console.print(x) to ev.console.print(x) Expr::Call { func: Expr::Field { object: Expr::Field { object: Expr::Var("__evidence__"), field: effect.to_lowercase(), }, field: operation, }, args, } } // ... recurse on other expressions } } ``` ## Phase 5: FBIP (Functional But In-Place) ### Goal Detect when functional updates can be performed in-place. ### Example ```lux fn increment(tree: Tree): Tree = match tree { Leaf(n) => Leaf(n + 1), Node(l, r) => Node(increment(l), increment(r)) } ``` With FBIP analysis, if `tree` has refcount 1, reuse the memory: ```c Tree increment(Tree tree) { if (tree->tag == LEAF) { if (is_unique(tree)) { tree->leaf.value += 1; // In-place! return tree; } else { return make_leaf(tree->leaf.value + 1); } } // ... } ``` ## Performance Targets | Metric | Target | How | |--------|--------|-----| | Effect overhead | 0% | Evidence passing | | Memory allocation | Minimal | Perceus RC + FBIP | | JS bundle size | No runtime | Direct compilation | | Native vs C | <5% overhead | Good C codegen | ## Milestones ### C Backend - COMPLETE - [x] Integer/bool expressions → C - [x] Functions → C functions - [x] If/else → C conditionals - [x] Let bindings → C variables - [x] Basic main() generation - [x] Build with GCC/Clang - [x] Strings → C strings - [x] Pattern matching → Switch/if chains - [x] Lists → Linked structures - [x] Closures - [x] Reference counting (lists, boxed values) ### JavaScript Backend - COMPLETE - [x] Basic expressions → JS - [x] Functions → JS functions - [x] Effects → Direct DOM/API calls - [x] Standard library (String, List, Option, Result, Math, JSON) - [x] DOM effect (40+ operations) - [x] Html module (type-safe HTML) - [x] TEA runtime (Elm Architecture) - [x] Browser & Node.js support ### Remaining Work - [ ] Evidence passing for zero-cost effects - [ ] FBIP (Functional But In-Place) optimization - [ ] WASM backend (deprioritized) ## References - [Koka: Evidence Passing](https://github.com/koka-lang/koka) - [Perceus: Garbage Free Reference Counting](https://www.microsoft.com/en-us/research/publication/perceus-garbage-free-reference-counting-with-reuse/) - [FBIP: Functional But In-Place](https://www.microsoft.com/en-us/research/publication/fp2-fully-in-place-functional-programming/) - [Svelte: No Virtual DOM](https://svelte.dev/blog/virtual-dom-is-pure-overhead) - [Eff: Efficient Compilation of Handlers](https://dl.acm.org/doi/10.1145/3485479)