lux/docs/COMPILATION_STRATEGY.md

# Lux Compilation Strategy

## Vision

Lux should compile to native code with zero-cost effects AND compile to JavaScript for frontend development - all while eliminating runtime overhead like Svelte does.

## Current State

| Component | Status |
|-----------|--------|
| Interpreter | Full-featured, all language constructs |
| C Backend | Complete (functions, closures, pattern matching, lists, RC) |
| JS Backend | Complete (full language, browser & Node.js, DOM, TEA) |
| JIT (Cranelift) | Integer arithmetic only, ~160x speedup |
| Targets | Native (via C), JavaScript, JIT |

## Target Architecture

```
                         ┌──→ C Backend ──→ GCC/Clang ──→ Native Binary (servers, CLI)
                         │
Lux Source ──→ HIR ──→ MIR ──→ JS Backend ──→ Optimized JS (no runtime, like Svelte)
                         │
                         └──→ Wasm Backend ──→ WebAssembly (browsers, edge)
```

## Compilation Targets Comparison

### What Other Languages Do

| Language | Backend | Runtime | Effect Strategy |
|----------|---------|---------|-----------------|
| **Koka** | C (Perceus) | None | Evidence passing |
| **Elm** | JS | Small runtime | No effects (pure) |
| **Gleam** | Erlang/JS | BEAM/JS runtime | No effects |
| **Svelte** | JS | **No runtime** | Compile-time reactivity |
| **ReScript** | JS | Minimal | No effects |
| **PureScript** | JS | Small runtime | Monad transformers |

### Key Insight: Svelte's No-Runtime Approach

Svelte compiles components to vanilla JavaScript that directly manipulates the DOM. No virtual DOM diffing, no framework code shipped to the browser.

```svelte
<!-- Svelte component -->
<script>
  let count = 0;
</script>
<button on:click={() => count++}>{count}</button>
```

Compiles to:
```javascript
// Direct DOM manipulation, no runtime
function create_fragment(ctx) {
  let button;
  return {
    c() {
      button = element("button");
      button.textContent = ctx[0];
    },
    m(target, anchor) {
      insert(target, button, anchor);
      listen(button, "click", ctx[1]);
    },
    p(ctx, dirty) {
      if (dirty & 1) set_data(button, ctx[0]);
    }
  };
}
```

**We can do the same for Lux effects on the frontend.**

## Phase 1: C Backend (Native Compilation)

### Goal
Compile Lux to C code that can be compiled by GCC/Clang for native execution.

### Why C?
1. **Portability**: C compilers exist for every platform
2. **Performance**: Leverage decades of GCC/Clang optimizations
3. **No runtime**: Like Koka, compile effects away
4. **Proven path**: Koka, Nim, Chicken Scheme all do this successfully

### Implementation Plan

#### Step 1.1: Basic C Code Generation
- Integer arithmetic and comparisons
- Function definitions and calls
- If/else expressions
- Let bindings

#### Step 1.2: Data Structures
- Records → C structs
- ADTs → Tagged unions
- Lists → Linked lists or arrays
- Strings → UTF-8 byte arrays

#### Step 1.3: Effect Compilation (Evidence Passing)
Transform:
```lux
fn greet(name: String): Unit with {Console} =
    Console.print("Hello, " + name)
```

To:
```c
void greet(Evidence* ev, LuxString name) {
    LuxString msg = lux_string_concat("Hello, ", name);
    ev->console->print(ev, msg);
}
```

#### Step 1.4: Memory Management (Perceus-style)
- Compile-time reference counting insertion
- Reuse analysis for in-place updates
- No garbage collector needed

### File Structure
```
src/
  codegen/
    mod.rs          # Code generation module
    c_backend.rs    # C code generation
    c_runtime.h     # Minimal C runtime header
    c_runtime.c     # Runtime support (RC, strings, etc.)
```

## Phase 2: JavaScript Backend (Frontend)

### Goal
Compile Lux to optimized JavaScript with NO runtime, like Svelte.

### Effect Mapping

| Lux Effect | JS Compilation |
|------------|----------------|
| `Console.print` | `console.log()` |
| `Dom.getElementById` | `document.getElementById()` |
| `Dom.addEventListener` | Direct event binding |
| `Http.get` | `fetch()` with async/await |
| `State.get/set` | Compile-time reactivity |

### No-Runtime Strategy

Instead of shipping a runtime, compile effects to direct DOM manipulation:

```lux
effect Dom {
    fn getElementById(id: String): Element
    fn setTextContent(el: Element, text: String): Unit
    fn addEventListener(el: Element, event: String, handler: fn(): Unit): Unit
}

fn counter(): Unit with {Dom} = {
    let btn = Dom.getElementById("btn")
    let count = 0
    Dom.addEventListener(btn, "click", fn() => {
        count = count + 1
        Dom.setTextContent(btn, toString(count))
    })
}
```

Compiles to:
```javascript
// No runtime needed - direct DOM calls
function counter() {
  const btn = document.getElementById("btn");
  let count = 0;
  btn.addEventListener("click", () => {
    count = count + 1;
    btn.textContent = String(count);
  });
}
```

### Reactive State (Like Svelte)

For reactive state, compile to fine-grained updates:

```lux
effect Reactive {
    fn signal<T>(initial: T): Signal<T>
    fn derived<T>(compute: fn(): T): Signal<T>
}

let count = Reactive.signal(0)
let doubled = Reactive.derived(fn() => count.get() * 2)
```

Compiles to:
```javascript
// Compile-time reactivity, no virtual DOM
let count = 0;
let doubled;
const count_subscribers = new Set();

function set_count(value) {
  count = value;
  doubled = count * 2;  // Statically known dependency
  count_subscribers.forEach(fn => fn());
}
```

## Phase 3: WebAssembly Backend

### Goal
Compile to Wasm for browser and edge deployment.

### Strategy
- Reuse C backend: `C → Emscripten → Wasm`
- Or direct Wasm generation via Cranelift

## Phase 4: Zero-Cost Effects (Evidence Passing)

### Current Problem
Effects use runtime handler lookup - O(n) per effect operation.

### Solution: Evidence Passing
Transform effect operations to direct function calls at compile time.

Before (runtime lookup):
```
performEffect("Console", "print", [msg])
  → search handler stack
  → invoke handler
```

After (compile-time):
```
evidence.console.print(msg)
  → direct function call
```

### Implementation

```rust
// Transform AST to evidence-passing form
fn transform_to_evidence(expr: Expr, effects: &[Effect]) -> Expr {
    match expr {
        Expr::EffectOp { effect, operation, args } => {
            // Transform Console.print(x) to ev.console.print(x)
            Expr::Call {
                func: Expr::Field {
                    object: Expr::Field {
                        object: Expr::Var("__evidence__"),
                        field: effect.to_lowercase(),
                    },
                    field: operation,
                },
                args,
            }
        }
        // ... recurse on other expressions
    }
}
```

## Phase 5: FBIP (Functional But In-Place)

### Goal
Detect when functional updates can be performed in-place.

### Example
```lux
fn increment(tree: Tree): Tree =
    match tree {
        Leaf(n) => Leaf(n + 1),
        Node(l, r) => Node(increment(l), increment(r))
    }
```

With FBIP analysis, if `tree` has refcount 1, reuse the memory:
```c
Tree increment(Tree tree) {
    if (tree->tag == LEAF) {
        if (is_unique(tree)) {
            tree->leaf.value += 1;  // In-place!
            return tree;
        } else {
            return make_leaf(tree->leaf.value + 1);
        }
    }
    // ...
}
```

## Performance Targets

| Metric | Target | How |
|--------|--------|-----|
| Effect overhead | 0% | Evidence passing |
| Memory allocation | Minimal | Perceus RC + FBIP |
| JS bundle size | No runtime | Direct compilation |
| Native vs C | <5% overhead | Good C codegen |

## Milestones

### C Backend - COMPLETE
- [x] Integer/bool expressions → C
- [x] Functions → C functions
- [x] If/else → C conditionals
- [x] Let bindings → C variables
- [x] Basic main() generation
- [x] Build with GCC/Clang
- [x] Strings → C strings
- [x] Pattern matching → Switch/if chains
- [x] Lists → Linked structures
- [x] Closures
- [x] Reference counting (lists, boxed values)

### JavaScript Backend - COMPLETE
- [x] Basic expressions → JS
- [x] Functions → JS functions
- [x] Effects → Direct DOM/API calls
- [x] Standard library (String, List, Option, Result, Math, JSON)
- [x] DOM effect (40+ operations)
- [x] Html module (type-safe HTML)
- [x] TEA runtime (Elm Architecture)
- [x] Browser & Node.js support

### Remaining Work
- [ ] Evidence passing for zero-cost effects
- [ ] FBIP (Functional But In-Place) optimization
- [ ] WASM backend (deprioritized)

## References

- [Koka: Evidence Passing](https://github.com/koka-lang/koka)
- [Perceus: Garbage Free Reference Counting](https://www.microsoft.com/en-us/research/publication/perceus-garbage-free-reference-counting-with-reuse/)
- [FBIP: Functional But In-Place](https://www.microsoft.com/en-us/research/publication/fp2-fully-in-place-functional-programming/)
- [Svelte: No Virtual DOM](https://svelte.dev/blog/virtual-dom-is-pure-overhead)
- [Eff: Efficient Compilation of Handlers](https://dl.acm.org/doi/10.1145/3485479)