lux/docs/IMPLEMENTATION_PLAN.md

Lux Language Implementation Plan

Current Status Summary

What's Working (147 tests passing)

Core Language:

• Lexer and parser for core syntax
• AST representation
• Type checker with Hindley-Milner inference
• Interpreter with REPL
• Tail call optimization (TCO)
• Pattern matching with exhaustiveness checking
• Algebraic data types (ADTs) with constructors
• Records and tuples
• Higher-order functions and closures
• Pipe operator (|>)
• Documentation comments

Effect System:

• Effect declarations
• Effect signatures on functions (with {Effect})
• Handler definitions
• run ... with syntax
• Effect inference within function bodies
• Effect subset checking (pure functions callable in effectful contexts)

Type Classes:

• Trait declarations
• Implementation blocks
• Basic trait method dispatch

Behavioral Properties:

• Property declarations (is pure, is idempotent, etc.)
• Property parsing and storage in AST

Tooling:

• REPL with history and autocomplete
• LSP server (diagnostics, hover, completions)
• Elm-style error messages

Example Programs Working

1. hello.lux - Basic effect usage
2. factorial.lux - Recursion
3. effects.lux - Custom effects and handlers
4. datatypes.lux - ADTs and pattern matching
5. functional.lux - Higher-order functions
6. traits.lux - Type classes
7. behavioral.lux - Behavioral properties
8. tailcall.lux - Tail call optimization
9. statemachine.lux - State machines with ADTs
10. pipelines.lux - Pipe operator

---

Missing Features Analysis

Priority 1: Critical Missing Features

1.1 Generic Type Parameters

Status: Parser supports Option<Int> syntax but type system doesn't fully support generics.

What's Missing:

• Type parameter declarations in type definitions (type List<T> = ...)
• Type application in type checking
• Generic function parameters (fn map<T, U>(f: fn(T): U, list: List<T>): List<U>)
• Type parameter constraints (where T: Eq)

Implementation Steps:

1. Add type parameter support to type definitions in AST
2. Implement kind checking (types vs type constructors)
3. Add type application rules to unification
4. Support polymorphic function instantiation

1.2 String Interpolation

Status: Not implemented. Currently requires manual toString() calls.

What's Missing:

• Parser support for "Hello, {name}!" syntax
• Type checking for interpolated expressions
• Runtime string building

Implementation Steps:

1. Add string interpolation tokens to lexer
2. Parse interpolated strings into AST (list of parts)
3. Desugar to string concatenation

1.3 Better Error Messages

Status: Basic error messages exist but can be improved.

What's Missing:

• Source code context in all errors
• Type diff display for mismatches
• Suggestions for common mistakes
• Error recovery in parser

Implementation Steps:

1. Ensure all errors include span information
2. Implement error recovery in parser
3. Add "did you mean?" suggestions

Priority 2: Effect System Completion

2.1 Effect Polymorphism

Status: Not implemented.

What's Missing:

• Functions generic over their effects
• Effect variables in type signatures
• Effect constraints

Example syntax:

fn withRetry<E>(action: fn(): T with E, attempts: Int): T with E = ...

Implementation Steps:

1. Add effect variables to effect representation
2. Implement effect unification with variables
3. Support effect quantification in type schemes

2.2 Built-in Effects

Status: Only Console effect is built-in.

Missing Effects:

• State<S> - get/put state
• Reader<R> - read-only environment
• Fail - early returns/exceptions
• Random - random number generation
• Time - current time, delays
• Async - async/await

Implementation Steps:

1. Define effect interfaces in prelude
2. Implement handlers in runtime
3. Add effect-specific type checking rules

2.3 Resumable Handlers

Status: Handlers exist but may not support continuation resumption.

What's Missing:

• resume keyword to continue computation
• Multi-shot continuations
• Proper effect handler semantics

Priority 3: Schema Evolution

3.1 Versioned Types

Status: Parser supports @v1 syntax but runtime doesn't use it.

What's Missing:

• Version tracking in type system
• Migration function generation
• Compatibility checking
• Codec generation

Implementation Steps:

1. Track version in type representation
2. Implement migration chain resolution
3. Add compatibility rules to type checker
4. Generate serialization code

Priority 4: Module System

4.1 Complete Import/Export

Status: Basic imports work but incomplete.

What's Missing:

• Re-exports
• Module aliases working properly
• Circular dependency detection (exists but needs testing)
• Package/namespace management

Implementation Steps:

1. Implement proper module resolution
2. Add re-export syntax
3. Support qualified names everywhere
4. Add package.lux for project configuration

Priority 5: Code Generation

5.1 Compile to Target

Status: Interpreter only.

Target Options:

1. WASM - Best for web and portable deployment
2. JavaScript - Easiest web integration
3. LLVM IR - Native performance
4. Custom bytecode - VM-based execution

Implementation Steps:

1. Design intermediate representation (IR)
2. Implement IR generation from AST
3. Implement backend for chosen target
4. Add optimization passes

Priority 6: Tooling

6.1 Package Manager

What's Needed:

• Package registry
• Dependency resolution
• Version management
• Build system integration

6.2 Standard Library

What's Needed:

• Collections (Map, Set, Array)
• String utilities
• Math functions
• File I/O
• Network I/O
• JSON/YAML parsing

6.3 Debugger

What's Needed:

• Breakpoints
• Step execution
• Variable inspection
• Stack traces

---

Recommended Implementation Order

Phase 1: Language Completeness (Essential)

1. Generic type parameters - Required for proper List, Option, etc.
2. String interpolation - Major usability improvement
3. Better error messages - Critical for adoption

Phase 2: Effect System Maturity

4. Built-in effects (State, Fail, Reader)
5. Effect polymorphism
6. Resumable handlers

Phase 3: Production Readiness

7. Complete module system
8. Standard library
9. Package manager

Phase 4: Performance

10. Code generation (WASM or JS first)
11. Optimization passes
12. Incremental compilation

Phase 5: Advanced Features

13. Schema evolution
14. Refinement types
15. SMT solver integration

---

Technical Debt

Known Issues Fixed During Testing

1. ✅ String concatenation with + operator wasn't type-checked correctly
2. ✅ ADT constructors weren't registered in type environment
3. ✅ ADT constructors weren't registered in interpreter environment
4. ✅ Handlers weren't accessible as values
5. ✅ Effect checking was too strict (required exact match instead of subset)
6. ✅ total keyword conflicts with variable names in examples

Remaining Technical Debt

1. Dead code in diagnostics, modules, parser (suppressed with #![allow(dead_code)])
2. Some test utilities not fully utilized
3. LSP server basic but could be expanded
4. Error recovery in parser incomplete

---

Feature Comparison with Other Languages

Feature	Lux	Koka	Haskell	Rust	TypeScript
Algebraic Effects	✅	✅	Via libs	❌	❌
Type Inference	✅	✅	✅	Partial	Partial
ADTs	✅	✅	✅	✅	Via unions
Pattern Matching	✅	✅	✅	✅	Limited
Generics	Partial	✅	✅	✅	✅
Type Classes	Basic	✅	✅	✅	❌
Effect Polymorphism	❌	✅	Via mtl	N/A	N/A
Schema Evolution	Planned	❌	❌	❌	❌
Refinement Types	Planned	❌	Via LH	❌	❌
Tail Call Opt	✅	✅	✅	Limited	❌
REPL	✅	✅	✅	Limited	✅
LSP	Basic	✅	✅	✅	✅

---

Unique Value Proposition

Lux differentiates itself through:

1. First-class algebraic effects - Making side effects explicit, testable, and composable
2. Schema evolution (planned) - Type-safe data migrations built into the language
3. Behavioral types (planned) - Compile-time verification of properties like purity and totality
4. Developer experience - Elm-style errors, REPL, LSP support

The combination of these features makes Lux particularly suited for:

• Building reliable backend services
• Applications with complex state management
• Systems requiring careful versioning and migration
• Projects where testing and verification are critical