lux/docs/IMPLEMENTATION_PLAN.md

# Lux Language Implementation Plan

## Current Status Summary

### What's Working (147 tests passing)

**Core Language:**
- Lexer and parser for core syntax
- AST representation
- Type checker with Hindley-Milner inference
- Interpreter with REPL
- Tail call optimization (TCO)
- Pattern matching with exhaustiveness checking
- Algebraic data types (ADTs) with constructors
- Records and tuples
- Higher-order functions and closures
- Pipe operator (`|>`)
- Documentation comments

**Effect System:**
- Effect declarations
- Effect signatures on functions (`with {Effect}`)
- Handler definitions
- `run ... with` syntax
- Effect inference within function bodies
- Effect subset checking (pure functions callable in effectful contexts)

**Type Classes:**
- Trait declarations
- Implementation blocks
- Basic trait method dispatch

**Behavioral Properties:**
- Property declarations (`is pure`, `is idempotent`, etc.)
- Property parsing and storage in AST

**Tooling:**
- REPL with history and autocomplete
- LSP server (diagnostics, hover, completions)
- Elm-style error messages

### Example Programs Working
1. `hello.lux` - Basic effect usage
2. `factorial.lux` - Recursion
3. `effects.lux` - Custom effects and handlers
4. `datatypes.lux` - ADTs and pattern matching
5. `functional.lux` - Higher-order functions
6. `traits.lux` - Type classes
7. `behavioral.lux` - Behavioral properties
8. `tailcall.lux` - Tail call optimization
9. `statemachine.lux` - State machines with ADTs
10. `pipelines.lux` - Pipe operator

---

## Missing Features Analysis

### Priority 1: Critical Missing Features

#### 1.1 Generic Type Parameters
**Status:** Parser supports `Option<Int>` syntax but type system doesn't fully support generics.

**What's Missing:**
- Type parameter declarations in type definitions (`type List<T> = ...`)
- Type application in type checking
- Generic function parameters (`fn map<T, U>(f: fn(T): U, list: List<T>): List<U>`)
- Type parameter constraints (`where T: Eq`)

**Implementation Steps:**
1. Add type parameter support to type definitions in AST
2. Implement kind checking (types vs type constructors)
3. Add type application rules to unification
4. Support polymorphic function instantiation

#### 1.2 String Interpolation
**Status:** Not implemented. Currently requires manual `toString()` calls.

**What's Missing:**
- Parser support for `"Hello, {name}!"` syntax
- Type checking for interpolated expressions
- Runtime string building

**Implementation Steps:**
1. Add string interpolation tokens to lexer
2. Parse interpolated strings into AST (list of parts)
3. Desugar to string concatenation

#### 1.3 Better Error Messages
**Status:** Basic error messages exist but can be improved.

**What's Missing:**
- Source code context in all errors
- Type diff display for mismatches
- Suggestions for common mistakes
- Error recovery in parser

**Implementation Steps:**
1. Ensure all errors include span information
2. Implement error recovery in parser
3. Add "did you mean?" suggestions

### Priority 2: Effect System Completion

#### 2.1 Effect Polymorphism
**Status:** Not implemented.

**What's Missing:**
- Functions generic over their effects
- Effect variables in type signatures
- Effect constraints

**Example syntax:**
```lux
fn withRetry<E>(action: fn(): T with E, attempts: Int): T with E = ...
```

**Implementation Steps:**
1. Add effect variables to effect representation
2. Implement effect unification with variables
3. Support effect quantification in type schemes

#### 2.2 Built-in Effects
**Status:** Only Console effect is built-in.

**Missing Effects:**
- `State<S>` - get/put state
- `Reader<R>` - read-only environment
- `Fail` - early returns/exceptions
- `Random` - random number generation
- `Time` - current time, delays
- `Async` - async/await

**Implementation Steps:**
1. Define effect interfaces in prelude
2. Implement handlers in runtime
3. Add effect-specific type checking rules

#### 2.3 Resumable Handlers
**Status:** Handlers exist but may not support continuation resumption.

**What's Missing:**
- `resume` keyword to continue computation
- Multi-shot continuations
- Proper effect handler semantics

### Priority 3: Schema Evolution

#### 3.1 Versioned Types
**Status:** Parser supports `@v1` syntax but runtime doesn't use it.

**What's Missing:**
- Version tracking in type system
- Migration function generation
- Compatibility checking
- Codec generation

**Implementation Steps:**
1. Track version in type representation
2. Implement migration chain resolution
3. Add compatibility rules to type checker
4. Generate serialization code

### Priority 4: Module System

#### 4.1 Complete Import/Export
**Status:** Basic imports work but incomplete.

**What's Missing:**
- Re-exports
- Module aliases working properly
- Circular dependency detection (exists but needs testing)
- Package/namespace management

**Implementation Steps:**
1. Implement proper module resolution
2. Add re-export syntax
3. Support qualified names everywhere
4. Add package.lux for project configuration

### Priority 5: Code Generation

#### 5.1 Compile to Target
**Status:** Interpreter only.

**Target Options:**
1. **WASM** - Best for web and portable deployment
2. **JavaScript** - Easiest web integration
3. **LLVM IR** - Native performance
4. **Custom bytecode** - VM-based execution

**Implementation Steps:**
1. Design intermediate representation (IR)
2. Implement IR generation from AST
3. Implement backend for chosen target
4. Add optimization passes

### Priority 6: Tooling

#### 6.1 Package Manager
**What's Needed:**
- Package registry
- Dependency resolution
- Version management
- Build system integration

#### 6.2 Standard Library
**What's Needed:**
- Collections (Map, Set, Array)
- String utilities
- Math functions
- File I/O
- Network I/O
- JSON/YAML parsing

#### 6.3 Debugger
**What's Needed:**
- Breakpoints
- Step execution
- Variable inspection
- Stack traces

---

## Recommended Implementation Order

### Phase 1: Language Completeness (Essential)
1. **Generic type parameters** - Required for proper List, Option, etc.
2. **String interpolation** - Major usability improvement
3. **Better error messages** - Critical for adoption

### Phase 2: Effect System Maturity
4. **Built-in effects** (State, Fail, Reader)
5. **Effect polymorphism**
6. **Resumable handlers**

### Phase 3: Production Readiness
7. **Complete module system**
8. **Standard library**
9. **Package manager**

### Phase 4: Performance
10. **Code generation** (WASM or JS first)
11. **Optimization passes**
12. **Incremental compilation**

### Phase 5: Advanced Features
13. **Schema evolution**
14. **Refinement types**
15. **SMT solver integration**

---

## Technical Debt

### Known Issues Fixed During Testing
1. ✅ String concatenation with `+` operator wasn't type-checked correctly
2. ✅ ADT constructors weren't registered in type environment
3. ✅ ADT constructors weren't registered in interpreter environment
4. ✅ Handlers weren't accessible as values
5. ✅ Effect checking was too strict (required exact match instead of subset)
6. ✅ `total` keyword conflicts with variable names in examples

### Remaining Technical Debt
1. Dead code in diagnostics, modules, parser (suppressed with `#![allow(dead_code)]`)
2. Some test utilities not fully utilized
3. LSP server basic but could be expanded
4. Error recovery in parser incomplete

---

## Feature Comparison with Other Languages

| Feature | Lux | Koka | Haskell | Rust | TypeScript |
|---------|-----|------|---------|------|------------|
| Algebraic Effects | ✅ | ✅ | Via libs | ❌ | ❌ |
| Type Inference | ✅ | ✅ | ✅ | Partial | Partial |
| ADTs | ✅ | ✅ | ✅ | ✅ | Via unions |
| Pattern Matching | ✅ | ✅ | ✅ | ✅ | Limited |
| Generics | Partial | ✅ | ✅ | ✅ | ✅ |
| Type Classes | Basic | ✅ | ✅ | ✅ | ❌ |
| Effect Polymorphism | ❌ | ✅ | Via mtl | N/A | N/A |
| Schema Evolution | Planned | ❌ | ❌ | ❌ | ❌ |
| Refinement Types | Planned | ❌ | Via LH | ❌ | ❌ |
| Tail Call Opt | ✅ | ✅ | ✅ | Limited | ❌ |
| REPL | ✅ | ✅ | ✅ | Limited | ✅ |
| LSP | Basic | ✅ | ✅ | ✅ | ✅ |

---

## Unique Value Proposition

Lux differentiates itself through:

1. **First-class algebraic effects** - Making side effects explicit, testable, and composable
2. **Schema evolution** (planned) - Type-safe data migrations built into the language
3. **Behavioral types** (planned) - Compile-time verification of properties like purity and totality
4. **Developer experience** - Elm-style errors, REPL, LSP support

The combination of these features makes Lux particularly suited for:
- Building reliable backend services
- Applications with complex state management
- Systems requiring careful versioning and migration
- Projects where testing and verification are critical