docs: update documentation to reflect actual implementation status
Many features were documented as "missing" or "planned" but are actually working: generics, string interpolation, File/HTTP/Random/Time effects, JSON parsing, module system, and JIT CLI integration. Updated IMPLEMENTATION_PLAN.md, OVERVIEW.md with accurate status. Added ROADMAP.md (use-case-targeted) and LANGUAGE_COMPARISON.md. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
322
docs/ROADMAP.md
Normal file
322
docs/ROADMAP.md
Normal file
@@ -0,0 +1,322 @@
|
||||
# Lux Development Roadmap
|
||||
|
||||
*Targeting practical use cases in priority order.*
|
||||
|
||||
---
|
||||
|
||||
## Feature Status Summary
|
||||
|
||||
### Schema Evolution
|
||||
| Component | Status |
|
||||
|-----------|--------|
|
||||
| Parser (`@v1`, `@v2`, `from @v1 = ...`) | ✅ Complete |
|
||||
| AST representation | ✅ Complete |
|
||||
| Runtime versioned values | ✅ Complete |
|
||||
| Schema registry & compatibility checking | ✅ Complete |
|
||||
| Basic migration execution | ✅ Complete |
|
||||
| Type system integration | ⚠️ Partial (versions ignored in typechecker) |
|
||||
| Auto-migration generation | ❌ Missing |
|
||||
| Serialization/codec support | ❌ Missing |
|
||||
|
||||
### Behavioral Types
|
||||
| Component | Status |
|
||||
|-----------|--------|
|
||||
| Parser (`is pure`, `is total`, etc.) | ✅ Complete |
|
||||
| AST & PropertySet | ✅ Complete |
|
||||
| Pure function checking (no effects) | ✅ Complete |
|
||||
| Total verification | ❌ Missing |
|
||||
| Idempotent verification | ❌ Missing |
|
||||
| Deterministic verification | ❌ Missing |
|
||||
| Where clause enforcement | ❌ Missing |
|
||||
|
||||
---
|
||||
|
||||
## Use Case 1: Backend Services with Complex Business Logic
|
||||
|
||||
**Value Proposition:** Effect tracking shows exactly what each function does. Testing is trivial.
|
||||
|
||||
### Phase 1.1: Make It Buildable (Current Blockers)
|
||||
|
||||
| Task | Priority | Effort | Status |
|
||||
|------|----------|--------|--------|
|
||||
| Generic type parameters | P0 | — | ✅ Complete |
|
||||
| String interpolation | P1 | — | ✅ Complete |
|
||||
| File effect (read/write) | P1 | — | ✅ Complete |
|
||||
| HTTP effect (client) | P1 | — | ✅ Complete |
|
||||
| JSON parsing/serialization | P1 | — | ✅ Complete |
|
||||
|
||||
### Phase 1.2: Database & Persistence
|
||||
|
||||
| Task | Priority | Effort | Status |
|
||||
|------|----------|--------|--------|
|
||||
| SQL effect (query, execute) | P1 | 2 weeks | ❌ Missing |
|
||||
| Connection pooling | P2 | 1 week | ❌ Missing |
|
||||
| Transaction effect | P2 | 1 week | ❌ Missing |
|
||||
|
||||
### Phase 1.3: Web Server Framework
|
||||
|
||||
| Task | Priority | Effort | Status |
|
||||
|------|----------|--------|--------|
|
||||
| HTTP server effect | P1 | 2 weeks | ❌ Missing |
|
||||
| Routing DSL | P2 | 1 week | ❌ Missing |
|
||||
| Middleware pattern | P2 | 1 week | ❌ Missing |
|
||||
| Request/Response types | P1 | 3 days | ❌ Missing |
|
||||
|
||||
### Phase 1.4: Developer Experience
|
||||
|
||||
| Task | Priority | Effort | Status |
|
||||
|------|----------|--------|--------|
|
||||
| Elm-quality error messages | P1 | 2 weeks | ⚠️ Partial (context shown, suggestions missing) |
|
||||
| Full JS compilation | P2 | 4 weeks | ⚠️ JIT for numeric code only |
|
||||
| Hot reload / watch mode | P2 | — | ✅ Complete |
|
||||
| Debugger improvements | P3 | 2 weeks | ✅ Basic |
|
||||
| JIT CLI integration | P1 | — | ✅ Complete (`lux compile`) |
|
||||
|
||||
### Success Criteria for Use Case 1
|
||||
- [ ] Can build a REST API with database access
|
||||
- [ ] Can swap database handler for testing without mocks
|
||||
- [ ] Effect signatures document all side effects
|
||||
- [ ] Compile-time guarantees prevent common bugs
|
||||
|
||||
---
|
||||
|
||||
## Use Case 2: High-Reliability Systems
|
||||
|
||||
**Value Proposition:** Behavioral types provide compile-time guarantees (`is total`, `is idempotent`).
|
||||
|
||||
### Phase 2.1: Complete Behavioral Type Verification
|
||||
|
||||
| Task | Priority | Effort | Status |
|
||||
|------|----------|--------|--------|
|
||||
| Total function verification | P1 | 2 weeks | ❌ Missing |
|
||||
| Idempotent verification | P1 | 2 weeks | ❌ Missing |
|
||||
| Deterministic verification | P1 | 1 week | ❌ Missing |
|
||||
| Where clause enforcement | P1 | 1 week | ❌ Missing |
|
||||
|
||||
**Implementation approach:**
|
||||
- **Total:** Restrict to structural recursion, require termination proof for general recursion
|
||||
- **Idempotent:** Pattern-based (setter patterns, specific effect combinations)
|
||||
- **Deterministic:** Effect analysis (no Random, Time, or non-deterministic IO)
|
||||
|
||||
### Phase 2.2: Refinement Types (Stretch Goal)
|
||||
|
||||
| Task | Priority | Effort | Status |
|
||||
|------|----------|--------|--------|
|
||||
| Basic refinements (`Int where self > 0`) | P2 | 3 weeks | ❌ Missing |
|
||||
| SMT solver integration (Z3) | P3 | 4 weeks | ❌ Missing |
|
||||
| Contract checking | P3 | 2 weeks | ❌ Missing |
|
||||
|
||||
### Phase 2.3: Fault Tolerance Patterns
|
||||
|
||||
| Task | Priority | Effort | Status |
|
||||
|------|----------|--------|--------|
|
||||
| Retry effect with backoff | P2 | 1 week | ❌ Missing |
|
||||
| Circuit breaker pattern | P2 | 1 week | ❌ Missing |
|
||||
| Supervision trees (Elixir-style) | P3 | 3 weeks | ❌ Missing |
|
||||
|
||||
### Success Criteria for Use Case 2
|
||||
- [ ] Compiler verifies `is total` functions always terminate
|
||||
- [ ] Compiler verifies `is idempotent` functions are safe to retry
|
||||
- [ ] Can build payment processing with idempotency guarantees
|
||||
- [ ] Effect system enables fault isolation
|
||||
|
||||
---
|
||||
|
||||
## Use Case 3: Data Pipelines / ETL Systems
|
||||
|
||||
**Value Proposition:** Schema evolution handles changing data formats gracefully.
|
||||
|
||||
### Phase 3.1: Complete Schema Evolution
|
||||
|
||||
| Task | Priority | Effort | Status |
|
||||
|------|----------|--------|--------|
|
||||
| Type system version tracking | P1 | 1 week | ⚠️ Partial |
|
||||
| Auto-migration generation | P1 | 2 weeks | ❌ Missing |
|
||||
| Version compatibility errors | P1 | 1 week | ❌ Missing |
|
||||
| Migration chain optimization | P2 | 1 week | ⚠️ Basic |
|
||||
|
||||
### Phase 3.2: Serialization Support
|
||||
|
||||
| Task | Priority | Effort | Status |
|
||||
|------|----------|--------|--------|
|
||||
| JSON codec generation | P1 | 2 weeks | ❌ Missing |
|
||||
| Version-aware deserialization | P1 | 1 week | ❌ Missing |
|
||||
| Binary format support | P3 | 2 weeks | ❌ Missing |
|
||||
| Avro/Protobuf interop | P3 | 3 weeks | ❌ Missing |
|
||||
|
||||
### Phase 3.3: Stream Processing
|
||||
|
||||
| Task | Priority | Effort | Status |
|
||||
|------|----------|--------|--------|
|
||||
| Stream effect | P2 | 2 weeks | ❌ Missing |
|
||||
| Batch processing patterns | P2 | 1 week | ❌ Missing |
|
||||
| Backpressure handling | P3 | 2 weeks | ❌ Missing |
|
||||
|
||||
### Success Criteria for Use Case 3
|
||||
- [ ] Can define versioned types with automatic migrations
|
||||
- [ ] Compiler catches version mismatches
|
||||
- [ ] Can deserialize old data formats automatically
|
||||
- [ ] Can process data streams with effect tracking
|
||||
|
||||
---
|
||||
|
||||
## Use Case 4: Teaching Functional Programming
|
||||
|
||||
**Value Proposition:** Effects are more intuitive than monads for learning.
|
||||
|
||||
### Phase 4.1: Educational Materials
|
||||
|
||||
| Task | Priority | Effort | Status |
|
||||
|------|----------|--------|--------|
|
||||
| Beginner tutorial (effects intro) | P1 | 1 week | ⚠️ Basic |
|
||||
| Interactive examples | P2 | 1 week | ❌ Missing |
|
||||
| Comparison guides (vs Haskell monads) | P2 | 3 days | ❌ Missing |
|
||||
| Video tutorials | P3 | 2 weeks | ❌ Missing |
|
||||
|
||||
### Phase 4.2: REPL Improvements
|
||||
|
||||
| Task | Priority | Effort | Status |
|
||||
|------|----------|--------|--------|
|
||||
| Step-by-step evaluation | P2 | 1 week | ❌ Missing |
|
||||
| Effect visualization | P2 | 2 weeks | ❌ Missing |
|
||||
| Type hole support (`_` placeholders) | P2 | 1 week | ❌ Missing |
|
||||
|
||||
### Phase 4.3: Error Message Excellence
|
||||
|
||||
| Task | Priority | Effort | Status |
|
||||
|------|----------|--------|--------|
|
||||
| Elm-style error catalog | P1 | 2 weeks | ❌ Missing |
|
||||
| "Did you mean?" suggestions | P1 | 1 week | ❌ Missing |
|
||||
| Example-based hints | P2 | 1 week | ❌ Missing |
|
||||
| Beginner-friendly mode | P3 | 1 week | ❌ Missing |
|
||||
|
||||
### Success Criteria for Use Case 4
|
||||
- [ ] New FP learner can understand effects in < 1 hour
|
||||
- [ ] Error messages guide toward solutions
|
||||
- [ ] Clear progression from pure functions to effects to handlers
|
||||
|
||||
---
|
||||
|
||||
## Cross-Cutting Concerns (All Use Cases)
|
||||
|
||||
### Ecosystem
|
||||
|
||||
| Task | Priority | Effort | Status |
|
||||
|------|----------|--------|--------|
|
||||
| Package manager (lux pkg) | P1 | 3 weeks | ⚠️ Basic |
|
||||
| Package registry | P2 | 2 weeks | ❌ Missing |
|
||||
| Dependency resolution | P2 | 2 weeks | ❌ Missing |
|
||||
|
||||
### Tooling
|
||||
|
||||
| Task | Priority | Effort | Status |
|
||||
|------|----------|--------|--------|
|
||||
| LSP completions | P1 | 1 week | ⚠️ Basic |
|
||||
| LSP go-to-definition | P1 | 1 week | ⚠️ Partial |
|
||||
| Formatter | P2 | — | ✅ Complete |
|
||||
| Documentation generator | P2 | 1 week | ❌ Missing |
|
||||
|
||||
### Compilation
|
||||
|
||||
| Task | Priority | Effort | Status |
|
||||
|------|----------|--------|--------|
|
||||
| Extend JIT (strings, lists) | P1 | 2 weeks | ❌ Missing |
|
||||
| JS backend | P2 | 4 weeks | ❌ Missing |
|
||||
| WASM backend | P3 | 4 weeks | ❌ Missing |
|
||||
|
||||
---
|
||||
|
||||
## Recommended Implementation Order
|
||||
|
||||
### Quarter 1: Foundation ✅ COMPLETE
|
||||
|
||||
1. ~~**Generic type parameters**~~ ✅ Done
|
||||
2. ~~**String interpolation**~~ ✅ Done
|
||||
3. ~~**File effect**~~ ✅ Done
|
||||
4. ~~**HTTP client effect**~~ ✅ Done
|
||||
5. ~~**JSON support**~~ ✅ Done
|
||||
6. **Elm-quality errors** — ⚠️ In progress
|
||||
|
||||
### Quarter 2: Backend Services (Use Case 1)
|
||||
|
||||
7. **HTTP server effect** — Build APIs
|
||||
8. **SQL effect** — Database access
|
||||
9. **Full JS compilation** — Deployment
|
||||
10. **Package manager** — Code sharing
|
||||
|
||||
### Quarter 3: Reliability (Use Case 2)
|
||||
|
||||
11. **Behavioral type verification** — Total, idempotent, deterministic
|
||||
12. **Where clause enforcement** — Type-level guarantees
|
||||
13. **Schema evolution completion** — Version tracking in types
|
||||
14. **Auto-migration generation** — Reduce boilerplate
|
||||
|
||||
### Quarter 4: Polish (Use Cases 3 & 4)
|
||||
|
||||
15. **Serialization codecs** — Data pipelines
|
||||
16. **Educational materials** — Teaching
|
||||
17. **Refinement types** — Advanced guarantees
|
||||
18. **Stream processing** — ETL use case
|
||||
|
||||
---
|
||||
|
||||
## What's Already Done (Often Forgotten)
|
||||
|
||||
**Core Language:**
|
||||
- ✅ Generic type parameters (`fn map<T, U>`, `type List<T>`)
|
||||
- ✅ String interpolation (`"Hello {name}!"`)
|
||||
- ✅ Pattern matching with exhaustiveness checking
|
||||
- ✅ Algebraic data types
|
||||
- ✅ Type inference (Hindley-Milner)
|
||||
- ✅ Tail call optimization
|
||||
|
||||
**Effect System:**
|
||||
- ✅ Effect declarations and handlers
|
||||
- ✅ Console effect (print, readLine, readInt)
|
||||
- ✅ File effect (read, write, exists, delete, listDir)
|
||||
- ✅ HTTP effect (get, post)
|
||||
- ✅ Random effect (int, float, range, bool)
|
||||
- ✅ Time effect (now, sleep)
|
||||
|
||||
**Module System:**
|
||||
- ✅ Imports, exports, aliases
|
||||
- ✅ Selective imports (`import foo.{a, b}`)
|
||||
- ✅ Wildcard imports (`import foo.*`)
|
||||
- ✅ Circular dependency detection
|
||||
|
||||
**Standard Library:**
|
||||
- ✅ String operations
|
||||
- ✅ List operations
|
||||
- ✅ JSON parsing/serialization
|
||||
|
||||
**Tooling:**
|
||||
- ✅ JIT compiler (~160x speedup, CLI: `lux compile`)
|
||||
- ✅ REPL with history
|
||||
- ✅ Basic LSP server
|
||||
- ✅ Formatter
|
||||
- ✅ Watch mode
|
||||
- ✅ Debugger (basic)
|
||||
|
||||
**Advanced (Parsing Only):**
|
||||
- ✅ Schema evolution (parsing, runtime values)
|
||||
- ✅ Behavioral types (parsing, pure checking only)
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
|
||||
### Use Case 1: Backend Services
|
||||
- Can build and deploy a production API
|
||||
- 10+ companies using Lux for backends
|
||||
|
||||
### Use Case 2: High-Reliability
|
||||
- Compiler catches idempotency violations
|
||||
- Used in fintech/healthcare context
|
||||
|
||||
### Use Case 3: Data Pipelines
|
||||
- Schema migrations happen automatically
|
||||
- Zero data loss from version mismatches
|
||||
|
||||
### Use Case 4: Teaching
|
||||
- Featured in FP courses
|
||||
- "Effects finally make sense" testimonials
|
||||
Reference in New Issue
Block a user