fix: make all example programs work correctly
- Add string concatenation support to + operator in typechecker - Register ADT constructors in both type environment and interpreter - Bind handlers as values so they can be referenced in run...with - Fix effect checking to use subset instead of exact match - Add built-in effects (Console, Fail, State) to run block contexts - Suppress dead code warnings in diagnostics, modules, parser Update all example programs with: - Expected output documented in comments - Proper run...with statements to execute code Add new example programs: - behavioral.lux: pure, idempotent, deterministic, commutative functions - pipelines.lux: pipe operator demonstrations - statemachine.lux: ADT-based state machines - tailcall.lux: tail call optimization examples - traits.lux: type classes and pattern matching Add documentation: - docs/IMPLEMENTATION_PLAN.md: feature roadmap and status - docs/PERFORMANCE_AND_TRADEOFFS.md: performance analysis Add benchmarks for performance testing. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
303
docs/IMPLEMENTATION_PLAN.md
Normal file
303
docs/IMPLEMENTATION_PLAN.md
Normal file
@@ -0,0 +1,303 @@
|
||||
# Lux Language Implementation Plan
|
||||
|
||||
## Current Status Summary
|
||||
|
||||
### What's Working (147 tests passing)
|
||||
|
||||
**Core Language:**
|
||||
- Lexer and parser for core syntax
|
||||
- AST representation
|
||||
- Type checker with Hindley-Milner inference
|
||||
- Interpreter with REPL
|
||||
- Tail call optimization (TCO)
|
||||
- Pattern matching with exhaustiveness checking
|
||||
- Algebraic data types (ADTs) with constructors
|
||||
- Records and tuples
|
||||
- Higher-order functions and closures
|
||||
- Pipe operator (`|>`)
|
||||
- Documentation comments
|
||||
|
||||
**Effect System:**
|
||||
- Effect declarations
|
||||
- Effect signatures on functions (`with {Effect}`)
|
||||
- Handler definitions
|
||||
- `run ... with` syntax
|
||||
- Effect inference within function bodies
|
||||
- Effect subset checking (pure functions callable in effectful contexts)
|
||||
|
||||
**Type Classes:**
|
||||
- Trait declarations
|
||||
- Implementation blocks
|
||||
- Basic trait method dispatch
|
||||
|
||||
**Behavioral Properties:**
|
||||
- Property declarations (`is pure`, `is idempotent`, etc.)
|
||||
- Property parsing and storage in AST
|
||||
|
||||
**Tooling:**
|
||||
- REPL with history and autocomplete
|
||||
- LSP server (diagnostics, hover, completions)
|
||||
- Elm-style error messages
|
||||
|
||||
### Example Programs Working
|
||||
1. `hello.lux` - Basic effect usage
|
||||
2. `factorial.lux` - Recursion
|
||||
3. `effects.lux` - Custom effects and handlers
|
||||
4. `datatypes.lux` - ADTs and pattern matching
|
||||
5. `functional.lux` - Higher-order functions
|
||||
6. `traits.lux` - Type classes
|
||||
7. `behavioral.lux` - Behavioral properties
|
||||
8. `tailcall.lux` - Tail call optimization
|
||||
9. `statemachine.lux` - State machines with ADTs
|
||||
10. `pipelines.lux` - Pipe operator
|
||||
|
||||
---
|
||||
|
||||
## Missing Features Analysis
|
||||
|
||||
### Priority 1: Critical Missing Features
|
||||
|
||||
#### 1.1 Generic Type Parameters
|
||||
**Status:** Parser supports `Option<Int>` syntax but type system doesn't fully support generics.
|
||||
|
||||
**What's Missing:**
|
||||
- Type parameter declarations in type definitions (`type List<T> = ...`)
|
||||
- Type application in type checking
|
||||
- Generic function parameters (`fn map<T, U>(f: fn(T): U, list: List<T>): List<U>`)
|
||||
- Type parameter constraints (`where T: Eq`)
|
||||
|
||||
**Implementation Steps:**
|
||||
1. Add type parameter support to type definitions in AST
|
||||
2. Implement kind checking (types vs type constructors)
|
||||
3. Add type application rules to unification
|
||||
4. Support polymorphic function instantiation
|
||||
|
||||
#### 1.2 String Interpolation
|
||||
**Status:** Not implemented. Currently requires manual `toString()` calls.
|
||||
|
||||
**What's Missing:**
|
||||
- Parser support for `"Hello, {name}!"` syntax
|
||||
- Type checking for interpolated expressions
|
||||
- Runtime string building
|
||||
|
||||
**Implementation Steps:**
|
||||
1. Add string interpolation tokens to lexer
|
||||
2. Parse interpolated strings into AST (list of parts)
|
||||
3. Desugar to string concatenation
|
||||
|
||||
#### 1.3 Better Error Messages
|
||||
**Status:** Basic error messages exist but can be improved.
|
||||
|
||||
**What's Missing:**
|
||||
- Source code context in all errors
|
||||
- Type diff display for mismatches
|
||||
- Suggestions for common mistakes
|
||||
- Error recovery in parser
|
||||
|
||||
**Implementation Steps:**
|
||||
1. Ensure all errors include span information
|
||||
2. Implement error recovery in parser
|
||||
3. Add "did you mean?" suggestions
|
||||
|
||||
### Priority 2: Effect System Completion
|
||||
|
||||
#### 2.1 Effect Polymorphism
|
||||
**Status:** Not implemented.
|
||||
|
||||
**What's Missing:**
|
||||
- Functions generic over their effects
|
||||
- Effect variables in type signatures
|
||||
- Effect constraints
|
||||
|
||||
**Example syntax:**
|
||||
```lux
|
||||
fn withRetry<E>(action: fn(): T with E, attempts: Int): T with E = ...
|
||||
```
|
||||
|
||||
**Implementation Steps:**
|
||||
1. Add effect variables to effect representation
|
||||
2. Implement effect unification with variables
|
||||
3. Support effect quantification in type schemes
|
||||
|
||||
#### 2.2 Built-in Effects
|
||||
**Status:** Only Console effect is built-in.
|
||||
|
||||
**Missing Effects:**
|
||||
- `State<S>` - get/put state
|
||||
- `Reader<R>` - read-only environment
|
||||
- `Fail` - early returns/exceptions
|
||||
- `Random` - random number generation
|
||||
- `Time` - current time, delays
|
||||
- `Async` - async/await
|
||||
|
||||
**Implementation Steps:**
|
||||
1. Define effect interfaces in prelude
|
||||
2. Implement handlers in runtime
|
||||
3. Add effect-specific type checking rules
|
||||
|
||||
#### 2.3 Resumable Handlers
|
||||
**Status:** Handlers exist but may not support continuation resumption.
|
||||
|
||||
**What's Missing:**
|
||||
- `resume` keyword to continue computation
|
||||
- Multi-shot continuations
|
||||
- Proper effect handler semantics
|
||||
|
||||
### Priority 3: Schema Evolution
|
||||
|
||||
#### 3.1 Versioned Types
|
||||
**Status:** Parser supports `@v1` syntax but runtime doesn't use it.
|
||||
|
||||
**What's Missing:**
|
||||
- Version tracking in type system
|
||||
- Migration function generation
|
||||
- Compatibility checking
|
||||
- Codec generation
|
||||
|
||||
**Implementation Steps:**
|
||||
1. Track version in type representation
|
||||
2. Implement migration chain resolution
|
||||
3. Add compatibility rules to type checker
|
||||
4. Generate serialization code
|
||||
|
||||
### Priority 4: Module System
|
||||
|
||||
#### 4.1 Complete Import/Export
|
||||
**Status:** Basic imports work but incomplete.
|
||||
|
||||
**What's Missing:**
|
||||
- Re-exports
|
||||
- Module aliases working properly
|
||||
- Circular dependency detection (exists but needs testing)
|
||||
- Package/namespace management
|
||||
|
||||
**Implementation Steps:**
|
||||
1. Implement proper module resolution
|
||||
2. Add re-export syntax
|
||||
3. Support qualified names everywhere
|
||||
4. Add package.lux for project configuration
|
||||
|
||||
### Priority 5: Code Generation
|
||||
|
||||
#### 5.1 Compile to Target
|
||||
**Status:** Interpreter only.
|
||||
|
||||
**Target Options:**
|
||||
1. **WASM** - Best for web and portable deployment
|
||||
2. **JavaScript** - Easiest web integration
|
||||
3. **LLVM IR** - Native performance
|
||||
4. **Custom bytecode** - VM-based execution
|
||||
|
||||
**Implementation Steps:**
|
||||
1. Design intermediate representation (IR)
|
||||
2. Implement IR generation from AST
|
||||
3. Implement backend for chosen target
|
||||
4. Add optimization passes
|
||||
|
||||
### Priority 6: Tooling
|
||||
|
||||
#### 6.1 Package Manager
|
||||
**What's Needed:**
|
||||
- Package registry
|
||||
- Dependency resolution
|
||||
- Version management
|
||||
- Build system integration
|
||||
|
||||
#### 6.2 Standard Library
|
||||
**What's Needed:**
|
||||
- Collections (Map, Set, Array)
|
||||
- String utilities
|
||||
- Math functions
|
||||
- File I/O
|
||||
- Network I/O
|
||||
- JSON/YAML parsing
|
||||
|
||||
#### 6.3 Debugger
|
||||
**What's Needed:**
|
||||
- Breakpoints
|
||||
- Step execution
|
||||
- Variable inspection
|
||||
- Stack traces
|
||||
|
||||
---
|
||||
|
||||
## Recommended Implementation Order
|
||||
|
||||
### Phase 1: Language Completeness (Essential)
|
||||
1. **Generic type parameters** - Required for proper List, Option, etc.
|
||||
2. **String interpolation** - Major usability improvement
|
||||
3. **Better error messages** - Critical for adoption
|
||||
|
||||
### Phase 2: Effect System Maturity
|
||||
4. **Built-in effects** (State, Fail, Reader)
|
||||
5. **Effect polymorphism**
|
||||
6. **Resumable handlers**
|
||||
|
||||
### Phase 3: Production Readiness
|
||||
7. **Complete module system**
|
||||
8. **Standard library**
|
||||
9. **Package manager**
|
||||
|
||||
### Phase 4: Performance
|
||||
10. **Code generation** (WASM or JS first)
|
||||
11. **Optimization passes**
|
||||
12. **Incremental compilation**
|
||||
|
||||
### Phase 5: Advanced Features
|
||||
13. **Schema evolution**
|
||||
14. **Refinement types**
|
||||
15. **SMT solver integration**
|
||||
|
||||
---
|
||||
|
||||
## Technical Debt
|
||||
|
||||
### Known Issues Fixed During Testing
|
||||
1. ✅ String concatenation with `+` operator wasn't type-checked correctly
|
||||
2. ✅ ADT constructors weren't registered in type environment
|
||||
3. ✅ ADT constructors weren't registered in interpreter environment
|
||||
4. ✅ Handlers weren't accessible as values
|
||||
5. ✅ Effect checking was too strict (required exact match instead of subset)
|
||||
6. ✅ `total` keyword conflicts with variable names in examples
|
||||
|
||||
### Remaining Technical Debt
|
||||
1. Dead code in diagnostics, modules, parser (suppressed with `#![allow(dead_code)]`)
|
||||
2. Some test utilities not fully utilized
|
||||
3. LSP server basic but could be expanded
|
||||
4. Error recovery in parser incomplete
|
||||
|
||||
---
|
||||
|
||||
## Feature Comparison with Other Languages
|
||||
|
||||
| Feature | Lux | Koka | Haskell | Rust | TypeScript |
|
||||
|---------|-----|------|---------|------|------------|
|
||||
| Algebraic Effects | ✅ | ✅ | Via libs | ❌ | ❌ |
|
||||
| Type Inference | ✅ | ✅ | ✅ | Partial | Partial |
|
||||
| ADTs | ✅ | ✅ | ✅ | ✅ | Via unions |
|
||||
| Pattern Matching | ✅ | ✅ | ✅ | ✅ | Limited |
|
||||
| Generics | Partial | ✅ | ✅ | ✅ | ✅ |
|
||||
| Type Classes | Basic | ✅ | ✅ | ✅ | ❌ |
|
||||
| Effect Polymorphism | ❌ | ✅ | Via mtl | N/A | N/A |
|
||||
| Schema Evolution | Planned | ❌ | ❌ | ❌ | ❌ |
|
||||
| Refinement Types | Planned | ❌ | Via LH | ❌ | ❌ |
|
||||
| Tail Call Opt | ✅ | ✅ | ✅ | Limited | ❌ |
|
||||
| REPL | ✅ | ✅ | ✅ | Limited | ✅ |
|
||||
| LSP | Basic | ✅ | ✅ | ✅ | ✅ |
|
||||
|
||||
---
|
||||
|
||||
## Unique Value Proposition
|
||||
|
||||
Lux differentiates itself through:
|
||||
|
||||
1. **First-class algebraic effects** - Making side effects explicit, testable, and composable
|
||||
2. **Schema evolution** (planned) - Type-safe data migrations built into the language
|
||||
3. **Behavioral types** (planned) - Compile-time verification of properties like purity and totality
|
||||
4. **Developer experience** - Elm-style errors, REPL, LSP support
|
||||
|
||||
The combination of these features makes Lux particularly suited for:
|
||||
- Building reliable backend services
|
||||
- Applications with complex state management
|
||||
- Systems requiring careful versioning and migration
|
||||
- Projects where testing and verification are critical
|
||||
Reference in New Issue
Block a user