- JS_WASM_BACKEND_PLAN: Mark phases 1-5 complete, deprioritize WASM - LANGUAGE_COMPARISON: Update package manager status - OVERVIEW: Add completed features list - ROADMAP: Mark JS backend and package manager complete - Add PACKAGES.md documenting the package system Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
554 lines
14 KiB
Markdown
554 lines
14 KiB
Markdown
# Lux JavaScript/WASM Backend Plan
|
|
|
|
## Goal
|
|
|
|
Enable Lux to compile to JavaScript and WebAssembly, allowing:
|
|
1. **Browser execution** - Run Lux in web browsers
|
|
2. **Self-hosted website** - Build lux-lang.org in Lux itself (like Elm)
|
|
3. **Universal deployment** - Same code runs on server (native) and client (JS/WASM)
|
|
|
|
## Research Summary
|
|
|
|
### How Gleam Does It
|
|
|
|
[Gleam compiles to JavaScript](https://gleam.run/news/v0.16-gleam-compiles-to-javascript/) with these characteristics:
|
|
- **No runtime overhead** - Generated JS looks like human-written code
|
|
- **Promise-based concurrency** - Uses native JS promises, not Erlang actors
|
|
- **Good interop** - Gleam functions callable from JS/TS directly
|
|
- [30% performance improvement](https://gleam.run/news/gleam-javascript-gets-30-percent-faster/) in v1.11.0 through optimization
|
|
|
|
### How Elm Does It
|
|
|
|
[Elm compiles to JavaScript](https://guide.elm-lang.org/interop/) with:
|
|
- **Virtual DOM** - Efficient updates via diffing
|
|
- **Ports** - Explicit interop boundary with JavaScript
|
|
- **Small runtime** - Scheduler and virtual DOM
|
|
- **Dead code elimination** - Only includes used code
|
|
- **Name mangling** - Prefixes with underscore to avoid collisions
|
|
|
|
---
|
|
|
|
## Implementation Strategy
|
|
|
|
### Approach: Parallel to C Backend
|
|
|
|
Create `src/codegen/js_backend.rs` mirroring `c_backend.rs` structure:
|
|
|
|
```
|
|
src/codegen/
|
|
├── c_backend.rs # Existing: Lux → C → Native
|
|
├── js_backend.rs # New: Lux → JavaScript
|
|
└── wasm_backend.rs # Future: Lux → WASM (via C or direct)
|
|
```
|
|
|
|
### Type Mappings
|
|
|
|
| Lux Type | JavaScript Type |
|
|
|----------|-----------------|
|
|
| `Int` | `number` (BigInt for large values) |
|
|
| `Float` | `number` |
|
|
| `Bool` | `boolean` |
|
|
| `String` | `string` |
|
|
| `Unit` | `undefined` |
|
|
| `List<T>` | `Array` |
|
|
| `Option<T>` | `{tag: "Some", value: T} \| {tag: "None"}` |
|
|
| `Result<T, E>` | `{tag: "Ok", value: T} \| {tag: "Err", error: E}` |
|
|
| `Closure` | `function` (closure environment captured naturally) |
|
|
| `ADT` | `{tag: "VariantName", field0: ..., field1: ...}` |
|
|
|
|
### Code Generation Examples
|
|
|
|
#### Functions
|
|
|
|
```lux
|
|
fn add(a: Int, b: Int): Int = a + b
|
|
```
|
|
|
|
Generates:
|
|
```javascript
|
|
function add_lux(a, b) {
|
|
return a + b;
|
|
}
|
|
```
|
|
|
|
#### Closures
|
|
|
|
```lux
|
|
fn makeAdder(x: Int): (Int) -> Int = {
|
|
fn(y: Int): Int => x + y
|
|
}
|
|
```
|
|
|
|
Generates:
|
|
```javascript
|
|
function makeAdder_lux(x) {
|
|
return function(y) {
|
|
return x + y;
|
|
};
|
|
}
|
|
```
|
|
|
|
#### Pattern Matching
|
|
|
|
```lux
|
|
fn describe(opt: Option<Int>): String = {
|
|
match opt {
|
|
Some(n) => "Got " + toString(n),
|
|
None => "Nothing"
|
|
}
|
|
}
|
|
```
|
|
|
|
Generates:
|
|
```javascript
|
|
function describe_lux(opt) {
|
|
if (opt.tag === "Some") {
|
|
const n = opt.value;
|
|
return "Got " + String(n);
|
|
} else {
|
|
return "Nothing";
|
|
}
|
|
}
|
|
```
|
|
|
|
#### ADTs
|
|
|
|
```lux
|
|
type Tree =
|
|
| Leaf
|
|
| Node(Int, Tree, Tree)
|
|
```
|
|
|
|
Generates:
|
|
```javascript
|
|
// Constructor functions
|
|
function Leaf_lux() {
|
|
return { tag: "Leaf" };
|
|
}
|
|
|
|
function Node_lux(value, left, right) {
|
|
return { tag: "Node", field0: value, field1: left, field2: right };
|
|
}
|
|
```
|
|
|
|
#### Effects
|
|
|
|
Effects compile to async/await:
|
|
|
|
```lux
|
|
fn fetchData(): String with Http = {
|
|
Http.get("https://api.example.com/data")
|
|
}
|
|
```
|
|
|
|
Generates:
|
|
```javascript
|
|
async function fetchData_lux(handlers) {
|
|
return await handlers.Http.get("https://api.example.com/data");
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Implementation Status
|
|
|
|
| Phase | Status | Progress |
|
|
|-------|--------|----------|
|
|
| Phase 1: Core Language | COMPLETE | 100% |
|
|
| Phase 2: Standard Library | COMPLETE | 100% |
|
|
| Phase 3: Effects in JS | COMPLETE | 100% |
|
|
| Phase 4: Browser/DOM Support | COMPLETE | 100% |
|
|
| Phase 5: CLI Integration | COMPLETE | 100% |
|
|
| Phase 6: WASM Backend | NOT STARTED | 0% |
|
|
|
|
---
|
|
|
|
## Implementation Phases
|
|
|
|
### Phase 1: Core Language - COMPLETE
|
|
|
|
| Feature | Status | Notes |
|
|
|---------|--------|-------|
|
|
| Basic types (Int, Float, Bool, String) | DONE | Direct mapping |
|
|
| Arithmetic and comparison operators | DONE | |
|
|
| Functions and calls | DONE | |
|
|
| Let bindings | DONE | |
|
|
| If expressions | DONE | Ternary or if/else |
|
|
| Pattern matching (basic) | DONE | Tag checks, destructuring |
|
|
| ADT definitions and constructors | DONE | Object literals |
|
|
| Closures | DONE | Native JS closures |
|
|
| Lists | DONE | Map to Array |
|
|
|
|
**Milestone:** Can compile `fib.lux` to JS and run in Node.js - ACHIEVED
|
|
|
|
### Phase 2: Standard Library - COMPLETE
|
|
|
|
| Module | Status | JS Implementation |
|
|
|--------|--------|-------------------|
|
|
| Console | DONE | `console.log` |
|
|
| String | DONE | 20+ operations (length, concat, slice, etc.) |
|
|
| List | DONE | Array methods (map, filter, fold, etc.) |
|
|
| Math | DONE | `Math.*` (trig, pow, log, etc.) |
|
|
| Option | DONE | isSome, isNone, map, flatMap, unwrapOr |
|
|
| Result | DONE | isOk, isErr, map, mapErr, flatMap, toOption |
|
|
| JSON | DONE | parse, stringify, get, keys, values, etc. |
|
|
|
|
**Milestone:** Standard library examples work in browser - ACHIEVED
|
|
|
|
### Phase 3: Effects in JS - COMPLETE
|
|
|
|
| Effect | Status | JS Implementation |
|
|
|--------|--------|-------------------|
|
|
| Console | DONE | `console.log`, `prompt()` |
|
|
| Http | DONE | `fetch()` API with Ok/Err handling |
|
|
| Time | DONE | `Date.now()`, `setTimeout` |
|
|
| Random | DONE | `Math.random()`, int, bool, float |
|
|
| DOM | DONE | Full DOM manipulation API |
|
|
|
|
**Milestone:** HTTP requests work in browser - ACHIEVED
|
|
|
|
### Phase 4: Browser/DOM Support - COMPLETE
|
|
|
|
The `Dom` effect provides comprehensive browser manipulation:
|
|
|
|
```lux
|
|
// Available Dom operations:
|
|
Dom.querySelector(selector) // -> Option<Element>
|
|
Dom.querySelectorAll(selector) // -> List<Element>
|
|
Dom.getElementById(id) // -> Option<Element>
|
|
Dom.createElement(tag) // -> Element
|
|
Dom.createTextNode(text) // -> Element
|
|
Dom.appendChild(parent, child)
|
|
Dom.removeChild(parent, child)
|
|
Dom.setTextContent(el, text)
|
|
Dom.getTextContent(el)
|
|
Dom.setInnerHtml(el, html)
|
|
Dom.setAttribute(el, name, value)
|
|
Dom.getAttribute(el, name) // -> Option<String>
|
|
Dom.addClass(el, class)
|
|
Dom.removeClass(el, class)
|
|
Dom.hasClass(el, class) // -> Bool
|
|
Dom.setStyle(el, prop, value)
|
|
Dom.getValue(el) // For inputs
|
|
Dom.setValue(el, value)
|
|
Dom.addEventListener(el, event, handler)
|
|
Dom.focus(el)
|
|
Dom.blur(el)
|
|
Dom.scrollTo(x, y)
|
|
Dom.getBoundingClientRect(el)
|
|
Dom.getWindowSize()
|
|
```
|
|
|
|
**Html Module** for type-safe HTML construction:
|
|
|
|
```lux
|
|
// Element constructors
|
|
Html.div(attrs, children)
|
|
Html.span(attrs, children)
|
|
Html.button(attrs, children)
|
|
Html.input(attrs, children)
|
|
// ... 30+ HTML elements
|
|
|
|
// Attribute constructors
|
|
Html.class("name")
|
|
Html.id("id")
|
|
Html.href("url")
|
|
Html.onClick(handler)
|
|
Html.onInput(handler)
|
|
// ... many more
|
|
|
|
// Rendering
|
|
Html.render(node) // -> String (for SSR)
|
|
Html.renderToDom(node) // -> Element (for browser)
|
|
```
|
|
|
|
**TEA (The Elm Architecture) Runtime:**
|
|
|
|
```javascript
|
|
Lux.app({
|
|
init: initialModel,
|
|
update: (model, msg) => newModel,
|
|
view: (model, dispatch) => Html.div(...),
|
|
root: "#app"
|
|
});
|
|
```
|
|
|
|
| Feature | Status | Notes |
|
|
|---------|--------|-------|
|
|
| Basic DOM queries | DONE | querySelector, getElementById, querySelectorAll |
|
|
| Element creation | DONE | createElement, createTextNode, appendChild |
|
|
| Event handling | DONE | addEventListener with closures |
|
|
| Attribute manipulation | DONE | setAttribute, classList, styles |
|
|
| Form handling | DONE | getValue, setValue, isChecked |
|
|
| Html module | DONE | Type-safe HTML construction |
|
|
| TEA runtime | DONE | Elm-style app architecture |
|
|
| View dependency analysis | DONE | Svelte-style optimization hooks |
|
|
|
|
**Milestone:** Can build interactive web page in Lux - ACHIEVED
|
|
|
|
### Phase 5: CLI Integration - COMPLETE
|
|
|
|
```bash
|
|
# Compile to JavaScript
|
|
lux compile app.lux --target js -o app.js
|
|
|
|
# Compile and run in Node.js
|
|
lux compile app.lux --target js --run
|
|
```
|
|
|
|
### Phase 6: WASM Backend - DEPRIORITIZED
|
|
|
|
**Status:** Not planned. The JS backend fully serves the Elm/Gleam-style frontend use case.
|
|
|
|
**Rationale:** For typical web applications, JS compilation is superior:
|
|
- Seamless DOM interop (no WASM boundary overhead)
|
|
- Readable output for debugging
|
|
- Smaller bundle sizes
|
|
- Native event handling
|
|
- Direct JS ecosystem integration
|
|
|
|
Neither Elm nor Gleam compile to WASM—they target JS for these exact reasons.
|
|
|
|
**Future consideration:** WASM may be revisited for:
|
|
- Computation-heavy workloads (image processing, simulations, crypto)
|
|
- Sharing binary logic between native server and browser
|
|
- Porting performance-critical libraries
|
|
|
|
For now, this is out of scope.
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
### JS Backend Module Structure
|
|
|
|
```rust
|
|
// src/codegen/js_backend.rs
|
|
|
|
pub struct JsBackend {
|
|
output: String,
|
|
indent: usize,
|
|
functions: HashSet<String>,
|
|
name_counter: usize,
|
|
}
|
|
|
|
impl JsBackend {
|
|
pub fn new() -> Self { ... }
|
|
|
|
pub fn compile(program: &[Decl]) -> Result<String, JsGenError> { ... }
|
|
|
|
fn emit_decl(&mut self, decl: &Decl) -> Result<(), JsGenError> { ... }
|
|
fn emit_function(&mut self, func: &Function) -> Result<(), JsGenError> { ... }
|
|
fn emit_expr(&mut self, expr: &Expr) -> Result<String, JsGenError> { ... }
|
|
fn emit_pattern(&mut self, pattern: &Pattern, value: &str) -> Result<String, JsGenError> { ... }
|
|
fn emit_adt(&mut self, adt: &TypeDecl) -> Result<(), JsGenError> { ... }
|
|
|
|
// JS-specific
|
|
fn emit_runtime(&mut self) { ... } // Minimal runtime helpers
|
|
fn emit_effect_handlers(&mut self, effects: &[Effect]) { ... }
|
|
}
|
|
```
|
|
|
|
### Runtime (Minimal)
|
|
|
|
```javascript
|
|
// Lux JS Runtime (embedded in generated code)
|
|
const Lux = {
|
|
// Option helpers
|
|
Some: (value) => ({ tag: "Some", value }),
|
|
None: () => ({ tag: "None" }),
|
|
|
|
// Result helpers
|
|
Ok: (value) => ({ tag: "Ok", value }),
|
|
Err: (error) => ({ tag: "Err", error }),
|
|
|
|
// List helpers
|
|
Cons: (head, tail) => [head, ...tail],
|
|
Nil: () => [],
|
|
|
|
// Effect handler invoker
|
|
handle: async (computation, handlers) => {
|
|
return await computation(handlers);
|
|
}
|
|
};
|
|
```
|
|
|
|
---
|
|
|
|
## Browser Integration
|
|
|
|
### Entry Point
|
|
|
|
```lux
|
|
// app.lux
|
|
fn main(): Unit with Dom = {
|
|
let button = Dom.createElement("button")
|
|
Dom.setTextContent(button, "Click me!")
|
|
Dom.addEventListener(button, "click", fn(): Unit => {
|
|
Dom.setTextContent(button, "Clicked!")
|
|
})
|
|
let body = Dom.querySelector("body")
|
|
match body {
|
|
Some(el) => Dom.appendChild(el, button),
|
|
None => ()
|
|
}
|
|
}
|
|
```
|
|
|
|
Compiles to:
|
|
```javascript
|
|
async function main_lux(handlers) {
|
|
const button = handlers.Dom.createElement("button");
|
|
handlers.Dom.setTextContent(button, "Click me!");
|
|
handlers.Dom.addEventListener(button, "click", function() {
|
|
handlers.Dom.setTextContent(button, "Clicked!");
|
|
});
|
|
const body = handlers.Dom.querySelector("body");
|
|
if (body.tag === "Some") {
|
|
handlers.Dom.appendChild(body.value, button);
|
|
}
|
|
}
|
|
|
|
// Browser handler
|
|
const BrowserDom = {
|
|
createElement: (tag) => document.createElement(tag),
|
|
setTextContent: (el, text) => { el.textContent = text; },
|
|
addEventListener: (el, event, handler) => el.addEventListener(event, handler),
|
|
querySelector: (sel) => {
|
|
const el = document.querySelector(sel);
|
|
return el ? Lux.Some(el) : Lux.None();
|
|
},
|
|
appendChild: (parent, child) => parent.appendChild(child)
|
|
};
|
|
|
|
// Initialize
|
|
main_lux({ Dom: BrowserDom });
|
|
```
|
|
|
|
### HTML Integration
|
|
|
|
```html
|
|
<!DOCTYPE html>
|
|
<html>
|
|
<head>
|
|
<title>Lux App</title>
|
|
</head>
|
|
<body>
|
|
<script src="app.js"></script>
|
|
<script>
|
|
// Generated initialization code
|
|
Lux.main({ Dom: BrowserDom });
|
|
</script>
|
|
</body>
|
|
</html>
|
|
```
|
|
|
|
---
|
|
|
|
## Self-Hosting the Website
|
|
|
|
Once the JS backend is complete, the lux-lang.org website can be built in Lux:
|
|
|
|
```lux
|
|
// website/src/Main.lux
|
|
|
|
import Html.{div, h1, p, button, text}
|
|
import App.{Model, Msg, init, update, view}
|
|
|
|
fn main(): Unit with Dom = {
|
|
let model = init()
|
|
let root = Dom.querySelector("#app")
|
|
match root {
|
|
Some(el) => render(el, model),
|
|
None => Console.print("No #app element found")
|
|
}
|
|
}
|
|
|
|
fn render(root: Element, model: Model): Unit with Dom = {
|
|
let html = view(model)
|
|
Dom.setInnerHtml(root, html)
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Testing Strategy
|
|
|
|
### Unit Tests
|
|
|
|
```rust
|
|
#[test]
|
|
fn test_js_function_generation() {
|
|
let input = "fn add(a: Int, b: Int): Int = a + b";
|
|
let js = JsBackend::compile(input).unwrap();
|
|
assert!(js.contains("function add_lux(a, b)"));
|
|
assert!(js.contains("return a + b"));
|
|
}
|
|
```
|
|
|
|
### Integration Tests
|
|
|
|
Run generated JS in Node.js and compare output:
|
|
|
|
```rust
|
|
#[test]
|
|
fn test_fibonacci_js() {
|
|
let lux_code = include_str!("../../examples/fibonacci.lux");
|
|
let js_code = JsBackend::compile(lux_code).unwrap();
|
|
|
|
let output = Command::new("node")
|
|
.arg("-e")
|
|
.arg(&js_code)
|
|
.output()
|
|
.unwrap();
|
|
|
|
assert!(String::from_utf8_lossy(&output.stdout).contains("fib(10) = 55"));
|
|
}
|
|
```
|
|
|
|
### Browser Tests
|
|
|
|
Use Playwright/Puppeteer to test DOM manipulation:
|
|
|
|
```javascript
|
|
test('button click works', async ({ page }) => {
|
|
await page.goto('http://localhost:3000/test.html');
|
|
await page.click('button');
|
|
expect(await page.textContent('button')).toBe('Clicked!');
|
|
});
|
|
```
|
|
|
|
---
|
|
|
|
## Timeline
|
|
|
|
| Phase | Duration | Milestone |
|
|
|-------|----------|-----------|
|
|
| Phase 1: Core Language | 2-3 weeks | Fibonacci runs in Node.js |
|
|
| Phase 2: Standard Library | 1-2 weeks | Examples work in browser |
|
|
| Phase 3: Effects | 2 weeks | HTTP works in browser |
|
|
| Phase 4: DOM Support | 2-3 weeks | Interactive page in Lux |
|
|
| Phase 5: CLI Integration | 1 week | `lux compile --target js` |
|
|
| Phase 6: WASM | 3-4 weeks | WASM execution |
|
|
|
|
**Total: 11-15 weeks** for full JS/WASM support
|
|
|
|
---
|
|
|
|
## Success Criteria
|
|
|
|
1. **Correctness**: All existing tests pass when targeting JS
|
|
2. **Performance**: Within 2x of hand-written JS for benchmarks
|
|
3. **Size**: Generated JS is reasonable size (< 2x hand-written equivalent)
|
|
4. **Interop**: Easy to call Lux from JS and JS from Lux
|
|
5. **Self-hosting**: lux-lang.org runs entirely on Lux-compiled JS
|
|
|
|
---
|
|
|
|
## References
|
|
|
|
- [Gleam JS Compilation](https://gleam.run/news/v0.16-gleam-compiles-to-javascript/)
|
|
- [Elm JavaScript Interop](https://guide.elm-lang.org/interop/)
|
|
- [Koka JavaScript Backend](https://koka-lang.github.io/koka/doc/book.html)
|
|
- [PureScript Code Generation](https://github.com/purescript/purescript/tree/master/src/Language/PureScript/CodeGen)
|