blu/lux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 13 Wiki Activity

docs: update documentation to reflect actual implementation status

Browse Source 
Many features were documented as "missing" or "planned" but are actually
working: generics, string interpolation, File/HTTP/Random/Time effects,
JSON parsing, module system, and JIT CLI integration.

Updated IMPLEMENTATION_PLAN.md, OVERVIEW.md with accurate status.
Added ROADMAP.md (use-case-targeted) and LANGUAGE_COMPARISON.md.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Brandon Lucas
2026-02-13 21:12:24 -05:00
parent 730112a917
commit 26546f9a6a
4 changed files with 1012 additions and 96 deletions
Download Patch File Download Diff File Expand all files Collapse all files

208
docs/IMPLEMENTATION_PLAN.md
View File

@@ -2,7 +2,7 @@
## Current Status Summary ## Current Status Summary
### What's Working (147 tests passing) ### What's Working (150+ tests passing)
**Core Language:** **Core Language:**
- Lexer and parser for core syntax - Lexer and parser for core syntax
@@ -16,6 +16,9 @@
- Higher-order functions and closures - Higher-order functions and closures
- Pipe operator (`|>`) - Pipe operator (`|>`)
- Documentation comments - Documentation comments
- **Generic type parameters** (`fn map<T, U>`, `type List<T>`)
- **String interpolation** (`"Hello {name}!"`)
- **Escape sequences** (`\n`, `\t`, `\"`, `\{`, `\}`)
**Effect System:** **Effect System:**
- Effect declarations - Effect declarations
@@ -25,6 +28,13 @@
- Effect inference within function bodies - Effect inference within function bodies
- Effect subset checking (pure functions callable in effectful contexts) - Effect subset checking (pure functions callable in effectful contexts)
**Built-in Effects:**
- `Console` - print, readLine, readInt
- `File` - read, write, exists, delete, listDir, createDir
- `Http` - get, post
- `Random` - int, float, range, bool, element
- `Time` - now, sleep
**Type Classes:** **Type Classes:**
- Trait declarations - Trait declarations
- Implementation blocks - Implementation blocks
@@ -33,11 +43,34 @@
**Behavioral Properties:** **Behavioral Properties:**
- Property declarations (`is pure`, `is idempotent`, etc.) - Property declarations (`is pure`, `is idempotent`, etc.)
- Property parsing and storage in AST - Property parsing and storage in AST
- Pure function verification (no effects allowed)
**Module System:**
- Import/export with `pub` visibility
- Aliased imports (`import foo as bar`)
- Selective imports (`import foo.{a, b}`)
- Wildcard imports (`import foo.*`)
- Circular dependency detection
- Transitive imports
**JIT Compiler (Cranelift):**
- Integer arithmetic and comparisons
- Conditionals, let bindings, blocks
- Function calls including recursion
- ~160x speedup over interpreter for numeric code
- CLI integration via `lux compile <file>`
**Standard Library:**
- String operations (substring, length, split, trim, etc.)
- List operations (map, filter, fold, etc.)
- JSON parsing and serialization
- Math functions
**Tooling:** **Tooling:**
- REPL with history and autocomplete - REPL with history and autocomplete
- LSP server (diagnostics, hover, completions) - LSP server (diagnostics, hover, completions)
- Elm-style error messages - Watch mode for auto-recompilation
- Source-context error messages
### Example Programs Working ### Example Programs Working
1. `hello.lux` - Basic effect usage 1. `hello.lux` - Basic effect usage
@@ -50,6 +83,13 @@
8. `tailcall.lux` - Tail call optimization 8. `tailcall.lux` - Tail call optimization
9. `statemachine.lux` - State machines with ADTs 9. `statemachine.lux` - State machines with ADTs
10. `pipelines.lux` - Pipe operator 10. `pipelines.lux` - Pipe operator
11. `generics.lux` - Generic type parameters
12. `json.lux` - JSON parsing and serialization
13. `jit_test.lux` - JIT compilation demo
14. `guessing_game.lux` - Console input/output
15. `examples/modules/main.lux` - Module imports
16. `examples/modules/main_selective.lux` - Selective imports
17. `examples/modules/main_wildcard.lux` - Wildcard imports
--- ---
@@ -58,46 +98,42 @@
### Priority 1: Critical Missing Features ### Priority 1: Critical Missing Features
#### 1.1 Generic Type Parameters #### 1.1 Generic Type Parameters
**Status:** Parser supports `Option<Int>` syntax but type system doesn't fully support generics. **Status:** ✅ Complete
**What's Missing:** Generic type parameters are fully working:
- Type parameter declarations in type definitions (`type List<T> = ...`) - Type parameter declarations in type definitions (`type List<T> = ...`)
- Type application in type checking
- Generic function parameters (`fn map<T, U>(f: fn(T): U, list: List<T>): List<U>`) - Generic function parameters (`fn map<T, U>(f: fn(T): U, list: List<T>): List<U>`)
- Type application and instantiation
- See `examples/generics.lux` for working examples
**Still Missing (nice-to-have):**
- Type parameter constraints (`where T: Eq`) - Type parameter constraints (`where T: Eq`)
**Implementation Steps:**
1. Add type parameter support to type definitions in AST
2. Implement kind checking (types vs type constructors)
3. Add type application rules to unification
4. Support polymorphic function instantiation
#### 1.2 String Interpolation #### 1.2 String Interpolation
**Status:** Not implemented. Currently requires manual `toString()` calls. **Status:** ✅ Complete
**What's Missing:** String interpolation is fully working:
- Parser support for `"Hello, {name}!"` syntax - `"Hello, {name}!"` syntax
- Type checking for interpolated expressions - Nested expressions: `"Result: {1 + 2}"`
- Runtime string building - Escape sequences: `\{`, `\}`, `\n`, `\t`, `\"`, `\\`
**Implementation Steps:**
1. Add string interpolation tokens to lexer
2. Parse interpolated strings into AST (list of parts)
3. Desugar to string concatenation
#### 1.3 Better Error Messages #### 1.3 Better Error Messages
**Status:** Basic error messages exist but can be improved. **Status:** ⚠️ Partial
**What's Working:**
- Source code context with line/column numbers
- Caret pointing to error location
- Color-coded error output
**What's Missing:** **What's Missing:**
- Source code context in all errors
- Type diff display for mismatches - Type diff display for mismatches
- Suggestions for common mistakes - "Did you mean?" suggestions
- Error recovery in parser - Error recovery in parser
**Implementation Steps:** **Implementation Steps:**
1. Ensure all errors include span information 1. Add Levenshtein distance for suggestions
2. Implement error recovery in parser 2. Implement error recovery in parser
3. Add "did you mean?" suggestions 3. Add type diff visualization
### Priority 2: Effect System Completion ### Priority 2: Effect System Completion
@@ -120,20 +156,25 @@ fn withRetry<E>(action: fn(): T with E, attempts: Int): T with E = ...
3. Support effect quantification in type schemes 3. Support effect quantification in type schemes
#### 2.2 Built-in Effects #### 2.2 Built-in Effects
**Status:** Only Console effect is built-in. **Status:** ⚠️ Partial - Several working, some missing
**Working Effects:**
- `Console` - print, readLine, readInt
- `File` - read, write, exists, delete, listDir, createDir
- `Http` - get, post
- `Random` - int, float, range, bool, element
- `Time` - now, sleep
**Missing Effects:** **Missing Effects:**
- `State<S>` - get/put state - `State<S>` - get/put state (generic over state type)
- `Reader<R>` - read-only environment - `Reader<R>` - read-only environment
- `Fail` - early returns/exceptions - `Fail` - early returns/exceptions
- `Random` - random number generation
- `Time` - current time, delays
- `Async` - async/await - `Async` - async/await
**Implementation Steps:** **Implementation Steps:**
1. Define effect interfaces in prelude 1. Add generic effect support (`State<S>`)
2. Implement handlers in runtime 2. Implement `Fail` for error handling
3. Add effect-specific type checking rules 3. Add async/await pattern
#### 2.3 Resumable Handlers #### 2.3 Resumable Handlers
**Status:** Handlers exist but may not support continuation resumption. **Status:** Handlers exist but may not support continuation resumption.
@@ -163,36 +204,51 @@ fn withRetry<E>(action: fn(): T with E, attempts: Int): T with E = ...
### Priority 4: Module System ### Priority 4: Module System
#### 4.1 Complete Import/Export #### 4.1 Complete Import/Export
**Status:** Basic imports work but incomplete. **Status:** ✅ Complete.
**What's Missing:** The module system is fully functional with:
- Re-exports - `import foo/bar` - basic imports
- Module aliases working properly - `import foo/bar as alias` - aliased imports
- Circular dependency detection (exists but needs testing) - `import foo.{a, b}` - selective imports
- Package/namespace management - `import foo.*` - wildcard imports
- `pub fn` visibility for exports
- Circular dependency detection (tested)
- Module caching
- Transitive imports
- Type checking across modules
**Implementation Steps:** **Still Missing (nice-to-have):**
1. Implement proper module resolution - Re-exports (`pub import`)
2. Add re-export syntax - Package manifest (`package.lux`)
3. Support qualified names everywhere
4. Add package.lux for project configuration
### Priority 5: Code Generation ### Priority 5: Code Generation
#### 5.1 Compile to Target #### 5.1 Compile to Target
**Status:** Interpreter only. **Status:** ⚠️ Partial - JIT works for numeric code
**Target Options:** **What's Working:**
- `src/compiler.rs` - Cranelift JIT compiler (539 lines)
- Integer arithmetic, comparisons, boolean ops
- Conditionals, let bindings, function calls, blocks
- Recursive functions (fib(30) compiles and runs)
- CLI integration: `lux compile <file> [--benchmark]`
- ~160x speedup over interpreter for numeric code
**What's Missing in JIT:**
- Strings, floats, lists, records, tuples
- Pattern matching, ADTs
- Effects and handlers
**Target Options for Full Compilation:**
1. **WASM** - Best for web and portable deployment 1. **WASM** - Best for web and portable deployment
2. **JavaScript** - Easiest web integration 2. **JavaScript** - Easiest web integration
3. **LLVM IR** - Native performance 3. **Extend Cranelift** - Build on existing JIT
4. **Custom bytecode** - VM-based execution
**Implementation Steps:** **Implementation Steps:**
1. Design intermediate representation (IR) 1. ~~Expose JIT via CLI command~~ ✅ Done
2. Implement IR generation from AST 2. Add support for more types (strings, lists, etc.)
3. Implement backend for chosen target 3. Compile pattern matching
4. Add optimization passes 4. Handle effects (CPS transform or evidence passing)
### Priority 6: Tooling ### Priority 6: Tooling
@@ -224,29 +280,36 @@ fn withRetry<E>(action: fn(): T with E, attempts: Int): T with E = ...
## Recommended Implementation Order ## Recommended Implementation Order
### Phase 1: Language Completeness (Essential) ### Phase 1: Language Completeness (Essential)
1. **Generic type parameters** - Required for proper List, Option, etc. 1. ~~**Generic type parameters**~~ ✅ Done
2. **String interpolation** - Major usability improvement 2. ~~**String interpolation**~~ ✅ Done
3. **Better error messages** - Critical for adoption 3. **Better error messages** - Elm-quality suggestions
### Phase 2: Effect System Maturity ### Phase 2: Effect System Maturity
4. **Built-in effects** (State, Fail, Reader) 4. ~~**Built-in effects** (Console, File, Http, Random, Time)~~ ✅ Done
5. **Effect polymorphism** 5. **Effect polymorphism** - Generic effect parameters
6. **Resumable handlers** 6. **Fail effect** - Proper error handling pattern
7. **Resumable handlers** - Multi-shot continuations
### Phase 3: Production Readiness ### Phase 3: Production Readiness
7. **Complete module system** 8. ~~**Complete module system**~~ ✅ Done
8. **Standard library** 9. ~~**Standard library basics**~~ ✅ Done (String, List, JSON)
9. **Package manager** 10. **Package manager** - Dependency resolution
11. **Full compilation** - Extend JIT or add WASM/JS backend
### Phase 4: Performance ### Phase 4: Behavioral Types (Verification)
10. **Code generation** (WASM or JS first) 12. **Total function verification** - Termination checking
11. **Optimization passes** 13. **Idempotent verification** - Pattern-based analysis
12. **Incremental compilation** 14. **Where clause enforcement** - Constraint checking
### Phase 5: Advanced Features ### Phase 5: Schema Evolution (Data)
13. **Schema evolution** 15. **Type system version tracking**
14. **Refinement types** 16. **Auto-migration generation**
15. **SMT solver integration** 17. **Version-aware serialization**
### Phase 6: Advanced Features
18. **Refinement types** - SMT solver integration
19. **Async/await effect**
20. **Stream processing**
--- ---
@@ -276,12 +339,15 @@ fn withRetry<E>(action: fn(): T with E, attempts: Int): T with E = ...
| Type Inference | ✅ | ✅ | ✅ | Partial | Partial | | Type Inference | ✅ | ✅ | ✅ | Partial | Partial |
| ADTs | ✅ | ✅ | ✅ | ✅ | Via unions | | ADTs | ✅ | ✅ | ✅ | ✅ | Via unions |
| Pattern Matching | ✅ | ✅ | ✅ | ✅ | Limited | | Pattern Matching | ✅ | ✅ | ✅ | ✅ | Limited |
| Generics | Partial | ✅ | ✅ | ✅ | ✅ | | Generics | ✅ | ✅ | ✅ | ✅ | ✅ |
| Type Classes | Basic | ✅ | ✅ | ✅ | ❌ | | Type Classes | Basic | ✅ | ✅ | ✅ | ❌ |
| String Interpolation | ✅ | ✅ | ❌ | ❌ | ✅ |
| Effect Polymorphism | ❌ | ✅ | Via mtl | N/A | N/A | | Effect Polymorphism | ❌ | ✅ | Via mtl | N/A | N/A |
| Schema Evolution | Planned | ❌ | ❌ | ❌ | ❌ | | Schema Evolution | ⚠️ Parsing | ❌ | ❌ | ❌ | ❌ |
| Behavioral Types | ⚠️ Parsing | ❌ | ❌ | ❌ | ❌ |
| Refinement Types | Planned | ❌ | Via LH | ❌ | ❌ | | Refinement Types | Planned | ❌ | Via LH | ❌ | ❌ |
| Tail Call Opt | ✅ | ✅ | ✅ | Limited | ❌ | | Tail Call Opt | ✅ | ✅ | ✅ | Limited | ❌ |
| JIT Compilation | ⚠️ Numeric | ✅ | ✅ | N/A | N/A |
| REPL | ✅ | ✅ | ✅ | Limited | ✅ | | REPL | ✅ | ✅ | ✅ | Limited | ✅ |
| LSP | Basic | ✅ | ✅ | ✅ | ✅ | | LSP | Basic | ✅ | ✅ | ✅ | ✅ |

476
docs/LANGUAGE_COMPARISON.md Normal file
View File

@@ -0,0 +1,476 @@
# Lux Language Comparison & Market Analysis
*Analysis conducted February 2025, based on Stack Overflow 2025 Developer Survey and industry research.*
## Executive Summary
Lux occupies a unique position in the programming language landscape: it combines **algebraic effects** (like Koka), **Elm-style developer experience**, and targets **practical application development** rather than academic research. This document analyzes where Lux fits, what it's missing, and its potential.
---
## Part 1: The Current Language Landscape
### Most Admired Languages (Stack Overflow 2025)
| Rank | Language | Admiration % | Key Strength |
|------|----------|--------------|--------------|
| 1 | Rust | 72% | Memory safety without GC |
| 2 | Gleam | 70% | Type-safe BEAM |
| 3 | Elixir | 66% | Fault tolerance |
| 4 | Zig | 64% | Low-level control |
| 5 | Go | ~60% | Simplicity |
### Most Used Languages (Stack Overflow 2025)
| Language | Usage % | Primary Domain |
|----------|---------|----------------|
| JavaScript | 66% | Web frontend |
| Python | 58% | AI/ML, scripting |
| TypeScript | 44% | Web (typed) |
| SQL | 59% | Data |
| Go | Growing | Cloud infrastructure |
---
## Part 2: What Makes Languages Successful
Based on 2025 research, languages succeed through:
### 1. Ecosystem Strength
> "The success of a programming language is increasingly tied to its ecosystem, including frameworks, libraries, community support, and tooling."
**Winners:** Python (AI/ML libraries), JavaScript (npm), Rust (Cargo rated 71% admiration)
**Lux Status:** Limited. Built-in stdlib only. No package manager ecosystem.
### 2. Developer Experience
> "TypeScript has won... it catches an absurd number of bugs before they hit production."
**What developers want:**
- Instant feedback on errors
- IDE support (completions, hover, go-to-definition)
- Clear, helpful error messages
- Fast iteration cycles
**Lux Status:** Partial. Has REPL, basic LSP, but error messages need Elm-level polish.
### 3. "If It Compiles, It Works"
> "What I really love about Rust is that if it compiles it usually runs." — Senior embedded engineer
**Leaders:** Rust (borrow checker), Elm (no runtime exceptions), Gleam (same promise)
**Lux Status:** Strong potential. Effect tracking + exhaustive pattern matching should provide this guarantee.
### 4. AI Tooling Compatibility
> "Typed languages such as TypeScript pair especially well with AI-powered tooling because they provide structure the models can rely on."
**Lux Status:** Good potential. Static types + effect signatures = rich structure for AI code generation.
### 5. Simplicity
> "Go's entire language specification is about 50 pages. Compare that to C++ (over 1,000 pages!)."
**Lux Status:** Medium. Core language is simple, but effects/handlers add conceptual overhead.
---
## Part 3: Language Category Analysis
### Specialty Languages
#### Elm
**Domain:** Web frontends
**Why Developers Love It:**
- Zero runtime exceptions
- Famous error messages ("feels like taking part in a tightly choreographed dance")
- The Elm Architecture (TEA) enforces predictable state management
- "Good React code in 2025 looks suspiciously like Elm code from 2015"
**Limitations:**
- Web-only (compiles to JS)
- Small ecosystem
- Treats all users like novices
- Single maintainer, slow evolution
**What Lux Can Learn:**
- Error message quality is a killer feature
- Enforced architecture patterns build confidence
- No runtime exceptions is achievable and valuable
#### Elixir
**Domain:** Real-time systems, distributed applications
**Why Developers Love It:**
- "Let it crash" philosophy with supervisor trees
- 99.9999999% ("nine nines") uptime systems
- Battle-tested OTP framework
- Discord, Pinterest, WhatsApp scale
**Limitations:**
- Dynamic typing
- BEAM-specific
- Smaller job market
**What Lux Can Learn:**
- Fault tolerance matters for production systems
- Process isolation is valuable
- Effects could model supervision/restart semantics
#### Gleam
**Domain:** BEAM ecosystem with types
**Why Developers Love It:**
- Elm-like safety on BEAM runtime
- "If it compiles, you are good"
- Interop with Erlang/Elixir libraries
- 70% admiration (2nd highest in 2025!)
**Current State:**
- Team has considered algebraic effects but decided: "I am not yet convinced that effect handlers are simple enough of an API to fit in Gleam"
- Most users come from outside BEAM ecosystem
**Lux Opportunity:**
- Gleam explicitly rejected effects for complexity reasons
- Lux can prove effects can be simple enough
#### Nix
**Domain:** Reproducible builds and deployments
**Why Developers Love It:**
- Solves "works on my machine" problem
- Declarative system configuration
- Atomic rollbacks
**Limitations:**
- Steep learning curve
- Documentation criticized
- 60-180 second initialization overhead
- "Love/hate relationship"
**What Lux Can Learn:**
- Reproducibility is increasingly valued
- Developer experience can make or break adoption
- Nix shows a good idea poorly executed loses to worse ideas well executed
#### Koka
**Domain:** Research language for algebraic effects
**Why It Matters:**
- Most mature effect system implementation
- Evidence passing compilation to C
- Row-polymorphic effect types
**Limitations:**
- Academic focus
- Limited practical ecosystem
- Not targeting mainstream adoption
**Lux Opportunity:**
- Take Koka's effect system ideas
- Package them for practical developers
- Better error messages, tooling, ecosystem
### General Purpose Languages
#### Rust
**Domain:** Systems programming, performance-critical applications
**Why Developers Love It (72% admiration):**
- Memory safety without garbage collection
- "If it compiles, it works"
- Fearless concurrency
- Cargo (best-in-class package manager)
- Growing ecosystem (AWS, Microsoft, Google adoption)
**Limitations:**
- Steep learning curve (borrow checker)
- Long compile times
- Complex for simple tasks
**What Lux Can Learn:**
- The compiler as assistant, not adversary
- Great tooling (Cargo) drives adoption
- Safety guarantees are worth learning curves
#### Go
**Domain:** Cloud infrastructure, backend services
**Why Developers Love It:**
- "50-page language specification"
- Built-in concurrency (goroutines)
- Fast compilation
- Docker, Kubernetes, Terraform all written in Go
**Limitations:**
- No generics until recently
- Error handling verbose
- Limited expressiveness
**What Lux Can Learn:**
- Simplicity wins adoption
- Fast feedback loops matter
- Don't over-engineer
#### Python
**Domain:** AI/ML, scripting, data science
**Why Developers Love It (58% usage, #1 TIOBE):**
- Readable syntax, low barrier to entry
- Massive library ecosystem
- AI/ML dominance (41% of Python devs do ML)
- "You rarely build from scratch"
**Limitations:**
- Performance (hence Rust extensions)
- Dynamic typing errors at runtime
- GIL limits concurrency
**What Lux Can Learn:**
- Ecosystem size determines practical utility
- Low barrier to entry enables adoption
- Being "good enough" for many tasks beats "perfect" for few
#### TypeScript
**Domain:** Web development (frontend and backend)
**Why Developers Love It (43.6% usage):**
- Catches errors before runtime
- Better IDE support than JavaScript
- Team collaboration via type contracts
- AI tools work better with typed code
**Adoption:**
- 60M+ weekly downloads
- "TypeScript has won"
- 67% of JS devs now write more TS than JS
**What Lux Can Learn:**
- Gradual typing adoption path worked
- Types as documentation/contracts for teams
- Don't replace—extend and improve
#### Java
**Domain:** Enterprise, Android
**Why It Persists:**
- Massive existing codebase
- JVM ecosystem
- Enterprise tooling
**Limitations:**
- Verbose
- Slow to evolve
- "Legacy" perception
#### C/C++
**Domain:** Systems, embedded, games
**Why It Persists:**
- Maximum control
- Existing codebase
- Performance
**Limitations:**
- Memory safety issues
- Complexity (C++ especially)
- Being replaced by Rust in new projects
---
## Part 4: What Lux Has That Others Don't
### Unique Combination
| Feature | Koka | Elm | Rust | Go | Gleam | **Lux** |
|---------|------|-----|------|-----|-------|---------|
| Algebraic Effects | Yes | No | No | No | No | **Yes** |
| Effect Handlers | Yes | No | No | No | No | **Yes** |
| Elm-style Errors | No | Yes | Partial | No | Partial | **Goal** |
| No Runtime Exceptions | Partial | Yes | No | No | Yes | **Goal** |
| Practical Focus | No | Yes | Yes | Yes | Yes | **Yes** |
| Schema Evolution | No | No | No | No | No | **Planned** |
| Behavioral Types | No | No | No | No | No | **Planned** |
| JIT Compilation | No | No | N/A | N/A | No | **Yes** |
### Lux's Potential Differentiators
#### 1. Effects Without the PhD
Koka requires understanding row polymorphism and academic papers. Lux can make effects accessible:
```lux
// Clear: this function does database and logging
fn processOrder(order: Order): Receipt with {Database, Logger} = ...
// Testing: just swap the handlers
run processOrder(order) with { Database = mockDb, Logger = testLogger }
```
#### 2. Schema Evolution (Planned)
No other mainstream language has built-in versioned types:
```lux
type User @v1 { name: String }
type User @v2 { name: String, email: String }
// Compiler generates migrations, checks compatibility
```
#### 3. Behavioral Types (Planned)
Compile-time verification of properties:
```lux
fn sort(list: List<T>): List<T> is pure, is total
// Compiler verifies: no effects, always terminates
```
#### 4. Testing Without Mocking Frameworks
Effects make testing natural:
```lux
// Production
run app() with { Http = realHttp, Database = postgres }
// Test
run app() with { Http = mockHttp, Database = inMemoryDb }
```
---
## Part 5: What Lux Is Missing
### Critical Gaps (Blocking Adoption)
| Gap | Why It Matters | Priority |
|-----|----------------|----------|
| **Ecosystem/Packages** | "You rarely build from scratch" (Python's success) | P0 |
| **Generics** | Can't write reusable `List<T>` functions | P0 |
| **String Interpolation** | Basic usability | P1 |
| **File/Network IO** | Can't build real applications | P1 |
| **Elm-Quality Errors** | "Famous error messages" drive adoption | P1 |
| **Full Compilation** | JIT exists but limited | P2 |
### Developer Experience Gaps
| Gap | Best-in-Class Example | Impact |
|-----|----------------------|--------|
| Error message quality | Elm | High - drives word of mouth |
| IDE completions | TypeScript/Rust | High - productivity |
| Fast iteration | Go (fast compile) | Medium - developer happiness |
| Debugger | Most languages | Medium - debugging complex code |
| Documentation | Rust (docs.rs) | High - learning curve |
### Ecosystem Gaps
| Gap | Why It Matters |
|-----|----------------|
| No package registry | Can't share/reuse code |
| No HTTP library | Can't build web services |
| No database drivers | Can't build real backends |
| No JSON library | Can't build APIs |
| No testing framework | Can't ensure quality |
---
## Part 6: Target Use Cases
### Where Lux Could Excel
#### 1. Backend Services with Complex Business Logic
**Why:** Effect tracking shows exactly what each function does. Testing is trivial.
**Competition:** Go, Rust, Elixir, TypeScript
**Lux Advantage:** Explicit effects, easy mocking, schema evolution
#### 2. Financial/Healthcare Systems (High Reliability Required)
**Why:** Behavioral types (`is total`, `is idempotent`) provide guarantees.
**Competition:** Rust, Ada, formal methods
**Lux Advantage:** More accessible than formal methods, more guarantees than most languages
#### 3. Data Pipeline/ETL Systems
**Why:** Schema evolution handles changing data formats gracefully.
**Competition:** Python, Spark, custom solutions
**Lux Advantage:** Type-safe migrations, effect tracking for IO
#### 4. Teaching Functional Programming
**Why:** Effects are more intuitive than monads.
**Competition:** Elm, Haskell, OCaml
**Lux Advantage:** Practical focus, good errors, clear effect signatures
### Where Lux Should NOT Compete (Yet)
| Domain | Why Not | Better Choice |
|--------|---------|---------------|
| Web Frontend | No JS compilation | Elm, TypeScript |
| Systems/Embedded | Needs low-level control | Rust, Zig, C |
| AI/ML | No ecosystem | Python |
| Mobile | No compilation target | Kotlin, Swift |
| Quick Scripts | Overhead not worth it | Python, Bash |
---
## Part 7: Lessons from Language Adoption
### What Made Languages Succeed
| Language | Key Success Factor |
|----------|-------------------|
| Python | Ecosystem + simplicity + AI timing |
| TypeScript | Gradual adoption + IDE experience |
| Go | Simplicity + Google backing + cloud native |
| Rust | Safety guarantees + great tooling (Cargo) |
| Elixir | BEAM reliability + Phoenix framework |
### What Made Languages Fail to Reach Mainstream
| Language | Why Limited Adoption |
|----------|---------------------|
| Haskell | Complexity, academic perception |
| OCaml | Poor tooling historically |
| Elm | Single domain, single maintainer |
| Koka | Academic focus, no ecosystem |
| Nix | Terrible learning curve |
### The AI Factor (2025)
> "It will become harder for new languages to emerge. Previously, new languages could emerge from individuals or small teams evangelizing... A single well-written book could make an enormous difference. But while a few samples and a tutorial can be enough material to jump-start adoption among programmers, it's not enough for today's AIs."
**Implication for Lux:**
- Need substantial training data (examples, docs, tutorials)
- Type information helps AI generate correct code
- Effect signatures provide structure AI can use
---
## Part 8: Recommendations for Lux
### Immediate Priorities (Make It Usable)
1. **Generics** - Without `fn map<T, U>(f: fn(T): U, list: List<T>): List<U>`, the language is incomplete
2. **File/Network Effects** - Without IO, can't build real programs
3. **Error Message Quality** - Elm-level errors are table stakes for new languages
4. **String Interpolation** - Basic usability
### Medium-Term (Make It Attractive)
5. **Package Manager** - Learn from Cargo's success
6. **Standard HTTP Library** - Enable web backends
7. **Full JS Compilation** - Enable web deployment
8. **Comprehensive Documentation** - Examples, tutorials, cookbook
### Long-Term (Make It Unique)
9. **Schema Evolution** - This is genuinely novel
10. **Behavioral Types** - Compile-time property verification
11. **Effect-Aware Debugging** - Time-travel for effects
12. **AI Training Data** - Ensure AI tools can help Lux developers
### Marketing Positioning
**Don't say:** "A language with algebraic effects"
**Do say:** "A language where the compiler tells you exactly what your code does, and testing is trivial"
**Target audience:** Backend developers frustrated with:
- Hidden side effects
- Complex mocking frameworks
- Runtime surprises
- Schema migration pain
---
## Sources
- [Stack Overflow 2025 Developer Survey](https://survey.stackoverflow.co/2025/technology)
- [JetBrains State of Developer Ecosystem 2025](https://www.jetbrains.com/lp/devecosystem-2025/)
- [What Do People Love About Rust - Rust Blog](https://blog.rust-lang.org/2025/12/19/what-do-people-love-about-rust/)
- [Why Elm Code in 2025](https://cekrem.github.io/posts/why-i-hope-i-get-to-write-a-lot-of-elm-code-in-2025/)
- [Gleam Programming Language Introduction](https://alltechprojects.com/gleam-programming-language-introduction-2025/)
- [Gleam Algebraic Effects Discussion](https://github.com/gleam-lang/gleam/discussions/1740)
- [Why Elixir in 2025 - Medium](https://medium.com/beamworld/why-im-still-betting-on-elixir-in-2025-2ca26d4c52b4)
- [The Koka Programming Language](https://koka-lang.github.io/koka/doc/book.html)
- [Go Ecosystem 2025 - JetBrains](https://blog.jetbrains.com/go/2025/11/10/go-language-trends-ecosystem-2025/)
- [State of Python 2025 - JetBrains](https://blog.jetbrains.com/pycharm/2025/08/the-state-of-python-2025/)
- [TypeScript Industry Standard 2025](https://jeffbruchado.com.br/en/blog/typescript-industry-standard-2025-javascript)
- [Programming Language Adoption Trends](https://www.javacodegeeks.com/2025/11/adoption-and-decline-of-programming-languages-what-drives-programming-trends.html)

102
docs/OVERVIEW.md
View File

@@ -111,12 +111,51 @@ toString(42) // "42"
typeOf([1, 2, 3]) // "List" typeOf([1, 2, 3]) // "List"
``` ```
### Planned (Not Yet Implemented) ### Also Working
- **Schema Evolution**: Versioned types with automatic migrations ```lux
- **Behavioral Types**: Properties like `is pure`, `is idempotent` // Generic type parameters
- **Modules/Imports**: Code organization fn map<T, U>(f: fn(T): U, list: List<T>): List<U> = ...
- **Compilation**: Currently interpreter-only fn identity<T>(x: T): T = x
// String interpolation
let name = "Alice"
Console.print("Hello, {name}!") // Hello, Alice!
Console.print("1 + 2 = {1 + 2}") // 1 + 2 = 3
// File effects
let content = File.read("config.txt")
File.write("output.txt", "Hello!")
let exists = File.exists("file.lux")
// HTTP effects
let response = Http.get("https://api.example.com/data")
Http.post("https://api.example.com/submit", "{\"key\": \"value\"}")
// Random effects
let n = Random.int(1, 100)
let coin = Random.bool()
// Time effects
let now = Time.now()
Time.sleep(1000) // milliseconds
// JSON parsing
let obj = Json.parse("{\"name\": \"Alice\"}")
let str = Json.stringify(obj)
// Module system
import mymodule
import utils/helpers as h
import math.{sqrt, abs}
import prelude.*
```
### Planned (Not Yet Fully Implemented)
- **Schema Evolution**: Parsing works (`@v1`, `from @v1`), type system integration missing
- **Behavioral Types**: Parsing works (`is pure`, `is total`), verification beyond `pure` missing
- **Full Compilation**: JIT works for numeric code, strings/lists/ADTs missing
--- ---
@@ -168,12 +207,10 @@ Quick iteration with type inference and a REPL.
| Limitation | Description | | Limitation | Description |
|------------|-------------| |------------|-------------|
| **Interpreter Only** | No compilation to native/JS/WASM yet | | **Limited JIT** | Cranelift JIT works for numeric code only |
| **No Modules** | Can't split code across files | | **No Package Manager** | Can't share/publish packages yet |
| **Limited IO** | Only Console built-in, no file/network |
| **No Generics** | Polymorphic functions not fully implemented |
| **New Paradigm** | Effects require learning new concepts | | **New Paradigm** | Effects require learning new concepts |
| **Small Ecosystem** | No packages, libraries, or community | | **Small Ecosystem** | No community packages yet |
| **Early Stage** | Bugs likely, features incomplete | | **Early Stage** | Bugs likely, features incomplete |
--- ---
@@ -232,10 +269,9 @@ Quick iteration with type inference and a REPL.
### Not a Good Fit (Yet) ### Not a Good Fit (Yet)
- Production applications (too early) - Large production applications (early stage)
- Performance-critical code (interpreter) - Performance-critical code (JIT limited to numeric)
- Large codebases (no modules) - Web frontend development (no JS compilation)
- Web development (no JS compilation)
- Systems programming (no low-level control) - Systems programming (no low-level control)
--- ---
@@ -282,20 +318,36 @@ Source Code
│ Type Checker│ → Typed AST + Effect Tracking │ Type Checker│ → Typed AST + Effect Tracking
└─────────────┘ └─────────────┘
│ │
▼ ├─────────────────────────┐
┌─────────────┐ ▼ ▼
│ Interpreter │ → Values + Effect Handling ┌─────────────┐ ┌──────────────┐
└─────────────┘ │ Interpreter │ │ JIT Compiler │
│ (default) │ │ (Cranelift) │
└─────────────┘ └──────────────┘
│ │
▼ ▼
Values + Effects Native Code
(~160x speedup)
``` ```
--- ---
## Future Roadmap ## Future Roadmap
1. **Standard Library** - List, String, Option utilities **Complete:**
2. **Module System** - Import/export, namespaces - ✅ Standard Library (List, String, Option, Result, JSON)
3. **JavaScript Backend** - Run in browsers - ✅ Module System (imports, exports, aliases)
4. **Schema Evolution** - Versioned types - ✅ LSP Server (basic diagnostics, hover, completions)
5. **Behavioral Types** - is pure, is idempotent - ✅ Generics and String Interpolation
6. **LSP Server** - IDE support - ✅ File/HTTP/Random/Time Effects
7. **Package Manager** - Share code
**In Progress:**
1. **Behavioral Type Verification** - Total, idempotent, deterministic checking
2. **Schema Evolution** - Type system integration, auto-migration
3. **Error Message Quality** - Elm-style suggestions
**Planned:**
4. **HTTP Server Effect** - Build web APIs
5. **SQL Effect** - Database access
6. **Package Manager** - Share code
7. **JavaScript Backend** - Run in browsers

322
docs/ROADMAP.md Normal file
View File

@@ -0,0 +1,322 @@
# Lux Development Roadmap
*Targeting practical use cases in priority order.*
---
## Feature Status Summary
### Schema Evolution
| Component | Status |
|-----------|--------|
| Parser (`@v1`, `@v2`, `from @v1 = ...`) | ✅ Complete |
| AST representation | ✅ Complete |
| Runtime versioned values | ✅ Complete |
| Schema registry & compatibility checking | ✅ Complete |
| Basic migration execution | ✅ Complete |
| Type system integration | ⚠️ Partial (versions ignored in typechecker) |
| Auto-migration generation | ❌ Missing |
| Serialization/codec support | ❌ Missing |
### Behavioral Types
| Component | Status |
|-----------|--------|
| Parser (`is pure`, `is total`, etc.) | ✅ Complete |
| AST & PropertySet | ✅ Complete |
| Pure function checking (no effects) | ✅ Complete |
| Total verification | ❌ Missing |
| Idempotent verification | ❌ Missing |
| Deterministic verification | ❌ Missing |
| Where clause enforcement | ❌ Missing |
---
## Use Case 1: Backend Services with Complex Business Logic
**Value Proposition:** Effect tracking shows exactly what each function does. Testing is trivial.
### Phase 1.1: Make It Buildable (Current Blockers)
| Task | Priority | Effort | Status |
|------|----------|--------|--------|
| Generic type parameters | P0 | — | ✅ Complete |
| String interpolation | P1 | — | ✅ Complete |
| File effect (read/write) | P1 | — | ✅ Complete |
| HTTP effect (client) | P1 | — | ✅ Complete |
| JSON parsing/serialization | P1 | — | ✅ Complete |
### Phase 1.2: Database & Persistence
| Task | Priority | Effort | Status |
|------|----------|--------|--------|
| SQL effect (query, execute) | P1 | 2 weeks | ❌ Missing |
| Connection pooling | P2 | 1 week | ❌ Missing |
| Transaction effect | P2 | 1 week | ❌ Missing |
### Phase 1.3: Web Server Framework
| Task | Priority | Effort | Status |
|------|----------|--------|--------|
| HTTP server effect | P1 | 2 weeks | ❌ Missing |
| Routing DSL | P2 | 1 week | ❌ Missing |
| Middleware pattern | P2 | 1 week | ❌ Missing |
| Request/Response types | P1 | 3 days | ❌ Missing |
### Phase 1.4: Developer Experience
| Task | Priority | Effort | Status |
|------|----------|--------|--------|
| Elm-quality error messages | P1 | 2 weeks | ⚠️ Partial (context shown, suggestions missing) |
| Full JS compilation | P2 | 4 weeks | ⚠️ JIT for numeric code only |
| Hot reload / watch mode | P2 | — | ✅ Complete |
| Debugger improvements | P3 | 2 weeks | ✅ Basic |
| JIT CLI integration | P1 | — | ✅ Complete (`lux compile`) |
### Success Criteria for Use Case 1
- [ ] Can build a REST API with database access
- [ ] Can swap database handler for testing without mocks
- [ ] Effect signatures document all side effects
- [ ] Compile-time guarantees prevent common bugs
---
## Use Case 2: High-Reliability Systems
**Value Proposition:** Behavioral types provide compile-time guarantees (`is total`, `is idempotent`).
### Phase 2.1: Complete Behavioral Type Verification
| Task | Priority | Effort | Status |
|------|----------|--------|--------|
| Total function verification | P1 | 2 weeks | ❌ Missing |
| Idempotent verification | P1 | 2 weeks | ❌ Missing |
| Deterministic verification | P1 | 1 week | ❌ Missing |
| Where clause enforcement | P1 | 1 week | ❌ Missing |
**Implementation approach:**
- **Total:** Restrict to structural recursion, require termination proof for general recursion
- **Idempotent:** Pattern-based (setter patterns, specific effect combinations)
- **Deterministic:** Effect analysis (no Random, Time, or non-deterministic IO)
### Phase 2.2: Refinement Types (Stretch Goal)
| Task | Priority | Effort | Status |
|------|----------|--------|--------|
| Basic refinements (`Int where self > 0`) | P2 | 3 weeks | ❌ Missing |
| SMT solver integration (Z3) | P3 | 4 weeks | ❌ Missing |
| Contract checking | P3 | 2 weeks | ❌ Missing |
### Phase 2.3: Fault Tolerance Patterns
| Task | Priority | Effort | Status |
|------|----------|--------|--------|
| Retry effect with backoff | P2 | 1 week | ❌ Missing |
| Circuit breaker pattern | P2 | 1 week | ❌ Missing |
| Supervision trees (Elixir-style) | P3 | 3 weeks | ❌ Missing |
### Success Criteria for Use Case 2
- [ ] Compiler verifies `is total` functions always terminate
- [ ] Compiler verifies `is idempotent` functions are safe to retry
- [ ] Can build payment processing with idempotency guarantees
- [ ] Effect system enables fault isolation
---
## Use Case 3: Data Pipelines / ETL Systems
**Value Proposition:** Schema evolution handles changing data formats gracefully.
### Phase 3.1: Complete Schema Evolution
| Task | Priority | Effort | Status |
|------|----------|--------|--------|
| Type system version tracking | P1 | 1 week | ⚠️ Partial |
| Auto-migration generation | P1 | 2 weeks | ❌ Missing |
| Version compatibility errors | P1 | 1 week | ❌ Missing |
| Migration chain optimization | P2 | 1 week | ⚠️ Basic |
### Phase 3.2: Serialization Support
| Task | Priority | Effort | Status |
|------|----------|--------|--------|
| JSON codec generation | P1 | 2 weeks | ❌ Missing |
| Version-aware deserialization | P1 | 1 week | ❌ Missing |
| Binary format support | P3 | 2 weeks | ❌ Missing |
| Avro/Protobuf interop | P3 | 3 weeks | ❌ Missing |
### Phase 3.3: Stream Processing
| Task | Priority | Effort | Status |
|------|----------|--------|--------|
| Stream effect | P2 | 2 weeks | ❌ Missing |
| Batch processing patterns | P2 | 1 week | ❌ Missing |
| Backpressure handling | P3 | 2 weeks | ❌ Missing |
### Success Criteria for Use Case 3
- [ ] Can define versioned types with automatic migrations
- [ ] Compiler catches version mismatches
- [ ] Can deserialize old data formats automatically
- [ ] Can process data streams with effect tracking
---
## Use Case 4: Teaching Functional Programming
**Value Proposition:** Effects are more intuitive than monads for learning.
### Phase 4.1: Educational Materials
| Task | Priority | Effort | Status |
|------|----------|--------|--------|
| Beginner tutorial (effects intro) | P1 | 1 week | ⚠️ Basic |
| Interactive examples | P2 | 1 week | ❌ Missing |
| Comparison guides (vs Haskell monads) | P2 | 3 days | ❌ Missing |
| Video tutorials | P3 | 2 weeks | ❌ Missing |
### Phase 4.2: REPL Improvements
| Task | Priority | Effort | Status |
|------|----------|--------|--------|
| Step-by-step evaluation | P2 | 1 week | ❌ Missing |
| Effect visualization | P2 | 2 weeks | ❌ Missing |
| Type hole support (`_` placeholders) | P2 | 1 week | ❌ Missing |
### Phase 4.3: Error Message Excellence
| Task | Priority | Effort | Status |
|------|----------|--------|--------|
| Elm-style error catalog | P1 | 2 weeks | ❌ Missing |
| "Did you mean?" suggestions | P1 | 1 week | ❌ Missing |
| Example-based hints | P2 | 1 week | ❌ Missing |
| Beginner-friendly mode | P3 | 1 week | ❌ Missing |
### Success Criteria for Use Case 4
- [ ] New FP learner can understand effects in < 1 hour
- [ ] Error messages guide toward solutions
- [ ] Clear progression from pure functions to effects to handlers
---
## Cross-Cutting Concerns (All Use Cases)
### Ecosystem
| Task | Priority | Effort | Status |
|------|----------|--------|--------|
| Package manager (lux pkg) | P1 | 3 weeks | ⚠️ Basic |
| Package registry | P2 | 2 weeks | ❌ Missing |
| Dependency resolution | P2 | 2 weeks | ❌ Missing |
### Tooling
| Task | Priority | Effort | Status |
|------|----------|--------|--------|
| LSP completions | P1 | 1 week | ⚠️ Basic |
| LSP go-to-definition | P1 | 1 week | ⚠️ Partial |
| Formatter | P2 | — | ✅ Complete |
| Documentation generator | P2 | 1 week | ❌ Missing |
### Compilation
| Task | Priority | Effort | Status |
|------|----------|--------|--------|
| Extend JIT (strings, lists) | P1 | 2 weeks | ❌ Missing |
| JS backend | P2 | 4 weeks | ❌ Missing |
| WASM backend | P3 | 4 weeks | ❌ Missing |
---
## Recommended Implementation Order
### Quarter 1: Foundation ✅ COMPLETE
1. ~~**Generic type parameters**~~ ✅ Done
2. ~~**String interpolation**~~ ✅ Done
3. ~~**File effect**~~ ✅ Done
4. ~~**HTTP client effect**~~ ✅ Done
5. ~~**JSON support**~~ ✅ Done
6. **Elm-quality errors** — ⚠️ In progress
### Quarter 2: Backend Services (Use Case 1)
7. **HTTP server effect** — Build APIs
8. **SQL effect** — Database access
9. **Full JS compilation** — Deployment
10. **Package manager** — Code sharing
### Quarter 3: Reliability (Use Case 2)
11. **Behavioral type verification** — Total, idempotent, deterministic
12. **Where clause enforcement** — Type-level guarantees
13. **Schema evolution completion** — Version tracking in types
14. **Auto-migration generation** — Reduce boilerplate
### Quarter 4: Polish (Use Cases 3 & 4)
15. **Serialization codecs** — Data pipelines
16. **Educational materials** — Teaching
17. **Refinement types** — Advanced guarantees
18. **Stream processing** — ETL use case
---
## What's Already Done (Often Forgotten)
**Core Language:**
- ✅ Generic type parameters (`fn map<T, U>`, `type List<T>`)
- ✅ String interpolation (`"Hello {name}!"`)
- ✅ Pattern matching with exhaustiveness checking
- ✅ Algebraic data types
- ✅ Type inference (Hindley-Milner)
- ✅ Tail call optimization
**Effect System:**
- ✅ Effect declarations and handlers
- ✅ Console effect (print, readLine, readInt)
- ✅ File effect (read, write, exists, delete, listDir)
- ✅ HTTP effect (get, post)
- ✅ Random effect (int, float, range, bool)
- ✅ Time effect (now, sleep)
**Module System:**
- ✅ Imports, exports, aliases
- ✅ Selective imports (`import foo.{a, b}`)
- ✅ Wildcard imports (`import foo.*`)
- ✅ Circular dependency detection
**Standard Library:**
- ✅ String operations
- ✅ List operations
- ✅ JSON parsing/serialization
**Tooling:**
- ✅ JIT compiler (~160x speedup, CLI: `lux compile`)
- ✅ REPL with history
- ✅ Basic LSP server
- ✅ Formatter
- ✅ Watch mode
- ✅ Debugger (basic)
**Advanced (Parsing Only):**
- ✅ Schema evolution (parsing, runtime values)
- ✅ Behavioral types (parsing, pure checking only)
---
## Success Metrics
### Use Case 1: Backend Services
- Can build and deploy a production API
- 10+ companies using Lux for backends
### Use Case 2: High-Reliability
- Compiler catches idempotency violations
- Used in fintech/healthcare context
### Use Case 3: Data Pipelines
- Schema migrations happen automatically
- Zero data loss from version mismatches
### Use Case 4: Teaching
- Featured in FP courses
- "Effects finally make sense" testimonials