feat: rebuild website with full learning funnel
Website rebuilt from scratch based on analysis of 11 beloved language websites (Elm, Zig, Gleam, Swift, Kotlin, Haskell, OCaml, Crystal, Roc, Rust, Go). New website structure: - Homepage with hero, playground, three pillars, install guide - Language Tour with interactive lessons (hello world, types, effects) - Examples cookbook with categorized sidebar - API documentation index - Installation guide (Nix and source) - Sleek/noble design (black/gold, serif typography) Also includes: - New stdlib/json.lux module for JSON serialization - Enhanced stdlib/http.lux with middleware and routing - New string functions (charAt, indexOf, lastIndexOf, repeat) - LSP improvements (rename, signature help, formatting) - Package manager transitive dependency resolution - Updated documentation for effects and stdlib - New showcase example (task_manager.lux) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -53,6 +53,10 @@ Lux provides several built-in effects:
|
||||
| `Process` | `exec`, `env`, `args`, `cwd`, `exit` | System processes |
|
||||
| `Http` | `get`, `post`, `put`, `delete` | HTTP client |
|
||||
| `HttpServer` | `listen`, `accept`, `respond`, `stop` | HTTP server |
|
||||
| `Sql` | `open`, `openMemory`, `close`, `execute`, `query`, `queryOne`, `beginTx`, `commit`, `rollback` | SQLite database |
|
||||
| `Postgres` | `connect`, `close`, `execute`, `query`, `queryOne` | PostgreSQL database |
|
||||
| `Concurrent` | `spawn`, `await`, `yield`, `sleep`, `cancel`, `isRunning`, `taskCount` | Concurrent tasks |
|
||||
| `Channel` | `create`, `send`, `receive`, `tryReceive`, `close` | Inter-task communication |
|
||||
| `Test` | `assert`, `assertEqual`, `assertTrue`, `assertFalse` | Testing |
|
||||
|
||||
Example usage:
|
||||
|
||||
@@ -320,6 +320,114 @@ fn example(): Int with {Fail} = {
|
||||
}
|
||||
```
|
||||
|
||||
### Sql (SQLite)
|
||||
|
||||
```lux
|
||||
fn example(): Unit with {Sql, Console} = {
|
||||
let conn = Sql.open("mydb.sqlite") // Open database file
|
||||
// Or: let conn = Sql.openMemory() // In-memory database
|
||||
|
||||
// Execute statements (returns row count)
|
||||
Sql.execute(conn, "CREATE TABLE users (id INTEGER, name TEXT)")
|
||||
Sql.execute(conn, "INSERT INTO users VALUES (1, 'Alice')")
|
||||
|
||||
// Query returns list of rows
|
||||
let rows = Sql.query(conn, "SELECT * FROM users")
|
||||
|
||||
// Query for single row
|
||||
let user = Sql.queryOne(conn, "SELECT * FROM users WHERE id = 1")
|
||||
|
||||
// Transactions
|
||||
Sql.beginTx(conn)
|
||||
Sql.execute(conn, "UPDATE users SET name = 'Bob' WHERE id = 1")
|
||||
Sql.commit(conn) // Or: Sql.rollback(conn)
|
||||
|
||||
Sql.close(conn)
|
||||
}
|
||||
```
|
||||
|
||||
### Postgres (PostgreSQL)
|
||||
|
||||
```lux
|
||||
fn example(): Unit with {Postgres, Console} = {
|
||||
let conn = Postgres.connect("postgres://user:pass@localhost/mydb")
|
||||
|
||||
// Execute statements
|
||||
Postgres.execute(conn, "INSERT INTO users (name) VALUES ('Alice')")
|
||||
|
||||
// Query returns list of rows
|
||||
let rows = Postgres.query(conn, "SELECT * FROM users")
|
||||
|
||||
// Query for single row
|
||||
let user = Postgres.queryOne(conn, "SELECT * FROM users WHERE id = 1")
|
||||
|
||||
Postgres.close(conn)
|
||||
}
|
||||
```
|
||||
|
||||
### Concurrent (Parallel Tasks)
|
||||
|
||||
```lux
|
||||
fn example(): Unit with {Concurrent, Console} = {
|
||||
// Spawn concurrent tasks
|
||||
let task1 = Concurrent.spawn(fn(): Int => expensiveComputation(1))
|
||||
let task2 = Concurrent.spawn(fn(): Int => expensiveComputation(2))
|
||||
|
||||
// Do other work while tasks run
|
||||
Console.print("Tasks spawned, doing other work...")
|
||||
|
||||
// Wait for tasks to complete
|
||||
let result1 = Concurrent.await(task1)
|
||||
let result2 = Concurrent.await(task2)
|
||||
|
||||
Console.print("Results: " + toString(result1) + ", " + toString(result2))
|
||||
|
||||
// Check task status
|
||||
if Concurrent.isRunning(task1) then
|
||||
Concurrent.cancel(task1)
|
||||
|
||||
// Non-blocking sleep
|
||||
Concurrent.sleep(100) // 100ms
|
||||
|
||||
// Yield to allow other tasks to run
|
||||
Concurrent.yield()
|
||||
|
||||
// Get active task count
|
||||
let count = Concurrent.taskCount()
|
||||
}
|
||||
```
|
||||
|
||||
### Channel (Inter-Task Communication)
|
||||
|
||||
```lux
|
||||
fn example(): Unit with {Concurrent, Channel, Console} = {
|
||||
// Create a channel for communication
|
||||
let ch = Channel.create()
|
||||
|
||||
// Spawn producer task
|
||||
let producer = Concurrent.spawn(fn(): Unit => {
|
||||
Channel.send(ch, 1)
|
||||
Channel.send(ch, 2)
|
||||
Channel.send(ch, 3)
|
||||
Channel.close(ch)
|
||||
})
|
||||
|
||||
// Consumer receives values
|
||||
match Channel.receive(ch) {
|
||||
Some(value) => Console.print("Received: " + toString(value)),
|
||||
None => Console.print("Channel closed")
|
||||
}
|
||||
|
||||
// Non-blocking receive
|
||||
match Channel.tryReceive(ch) {
|
||||
Some(value) => Console.print("Got: " + toString(value)),
|
||||
None => Console.print("No value available")
|
||||
}
|
||||
|
||||
Concurrent.await(producer)
|
||||
}
|
||||
```
|
||||
|
||||
### Test
|
||||
|
||||
Native testing framework:
|
||||
@@ -360,6 +468,10 @@ fn main(): Unit with {Console} = {
|
||||
| Random | int, float, bool |
|
||||
| State | get, put |
|
||||
| Fail | fail |
|
||||
| Sql | open, openMemory, close, execute, query, queryOne, beginTx, commit, rollback |
|
||||
| Postgres | connect, close, execute, query, queryOne |
|
||||
| Concurrent | spawn, await, yield, sleep, cancel, isRunning, taskCount |
|
||||
| Channel | create, send, receive, tryReceive, close |
|
||||
| Test | assert, assertEqual, assertTrue, assertFalse |
|
||||
|
||||
## Next
|
||||
|
||||
449
docs/guide/12-behavioral-types.md
Normal file
449
docs/guide/12-behavioral-types.md
Normal file
@@ -0,0 +1,449 @@
|
||||
# Chapter 12: Behavioral Types
|
||||
|
||||
Lux's behavioral types let you make **compile-time guarantees** about function behavior. Unlike comments or documentation, these are actually verified by the compiler.
|
||||
|
||||
## Why Behavioral Types Matter
|
||||
|
||||
Consider these real-world scenarios:
|
||||
|
||||
1. **Payment processing**: You retry a failed charge. If the function isn't idempotent, you might charge the customer twice.
|
||||
|
||||
2. **Caching**: You cache a computation. If the function isn't deterministic, you'll serve stale/wrong results.
|
||||
|
||||
3. **Parallelization**: You run tasks in parallel. If they aren't pure, you'll have race conditions.
|
||||
|
||||
4. **Infinite loops**: A function never returns. If it was supposed to be total, you have a bug.
|
||||
|
||||
**Behavioral types catch these bugs at compile time.**
|
||||
|
||||
## The Five Properties
|
||||
|
||||
### 1. Pure (`is pure`)
|
||||
|
||||
A pure function has **no side effects**. It only depends on its inputs.
|
||||
|
||||
```lux
|
||||
// GOOD: No effects, just computation
|
||||
fn add(a: Int, b: Int): Int is pure = a + b
|
||||
|
||||
fn double(x: Int): Int is pure = x * 2
|
||||
|
||||
fn greet(name: String): String is pure = "Hello, " + name
|
||||
|
||||
// ERROR: Pure function cannot have effects
|
||||
fn impure(x: Int): Int is pure with {Console} =
|
||||
Console.print("x = " + toString(x)) // Compiler error!
|
||||
x
|
||||
```
|
||||
|
||||
**What the compiler checks:**
|
||||
- Function must have an empty effect set
|
||||
- No calls to effectful operations
|
||||
|
||||
**When to use `is pure`:**
|
||||
- Mathematical functions
|
||||
- Data transformations
|
||||
- Any function that should be cacheable
|
||||
|
||||
**Compiler optimizations enabled:**
|
||||
- Memoization (cache results)
|
||||
- Common subexpression elimination
|
||||
- Parallel execution
|
||||
- Dead code elimination (if result unused)
|
||||
|
||||
### 2. Total (`is total`)
|
||||
|
||||
A total function **always terminates** and **never fails**. It produces a value for every valid input.
|
||||
|
||||
```lux
|
||||
// GOOD: Always terminates (structural recursion)
|
||||
fn factorial(n: Int): Int is total =
|
||||
if n <= 1 then 1 else n * factorial(n - 1)
|
||||
|
||||
// GOOD: Non-recursive is always total
|
||||
fn max(a: Int, b: Int): Int is total =
|
||||
if a > b then a else b
|
||||
|
||||
// GOOD: List operations that terminate
|
||||
fn length<T>(list: List<T>): Int is total =
|
||||
match list {
|
||||
[] => 0,
|
||||
[_, ...rest] => 1 + length(rest) // Structurally decreasing
|
||||
}
|
||||
|
||||
// ERROR: Uses Fail effect
|
||||
fn divide(a: Int, b: Int): Int is total with {Fail} =
|
||||
if b == 0 then Fail.fail("division by zero") // Compiler error!
|
||||
else a / b
|
||||
|
||||
// ERROR: May not terminate (not structurally decreasing)
|
||||
fn collatz(n: Int): Int is total =
|
||||
if n == 1 then 1
|
||||
else if n % 2 == 0 then collatz(n / 2)
|
||||
else collatz(3 * n + 1) // Not structurally smaller!
|
||||
```
|
||||
|
||||
**What the compiler checks:**
|
||||
- No `Fail` effect used
|
||||
- Recursive calls must have at least one structurally decreasing argument
|
||||
|
||||
**When to use `is total`:**
|
||||
- Core business logic that must never crash
|
||||
- Mathematical functions
|
||||
- Data structure operations
|
||||
|
||||
**Compiler optimizations enabled:**
|
||||
- No exception handling overhead
|
||||
- Aggressive inlining
|
||||
- Removal of termination checks
|
||||
|
||||
### 3. Deterministic (`is deterministic`)
|
||||
|
||||
A deterministic function produces the **same output for the same input**, every time.
|
||||
|
||||
```lux
|
||||
// GOOD: Same input = same output
|
||||
fn hash(s: String): Int is deterministic =
|
||||
List.fold(String.chars(s), 0, fn(acc: Int, c: String): Int => acc * 31 + charCode(c))
|
||||
|
||||
fn formatDate(year: Int, month: Int, day: Int): String is deterministic =
|
||||
toString(year) + "-" + padZero(month) + "-" + padZero(day)
|
||||
|
||||
// ERROR: Random is non-deterministic
|
||||
fn generateId(): String is deterministic with {Random} =
|
||||
"id-" + toString(Random.int(0, 1000000)) // Compiler error!
|
||||
|
||||
// ERROR: Time is non-deterministic
|
||||
fn timestamp(): Int is deterministic with {Time} =
|
||||
Time.now() // Compiler error!
|
||||
```
|
||||
|
||||
**What the compiler checks:**
|
||||
- No `Random` effect
|
||||
- No `Time` effect
|
||||
|
||||
**When to use `is deterministic`:**
|
||||
- Hashing functions
|
||||
- Serialization/formatting
|
||||
- Test helpers
|
||||
|
||||
**Compiler optimizations enabled:**
|
||||
- Result caching
|
||||
- Parallel execution with consistent results
|
||||
- Test reproducibility
|
||||
|
||||
### 4. Idempotent (`is idempotent`)
|
||||
|
||||
An idempotent function satisfies: `f(f(x)) == f(x)`. Applying it multiple times has the same effect as applying it once.
|
||||
|
||||
```lux
|
||||
// GOOD: Pattern 1 - Constants
|
||||
fn alwaysZero(x: Int): Int is idempotent = 0
|
||||
|
||||
// GOOD: Pattern 2 - Identity
|
||||
fn identity<T>(x: T): T is idempotent = x
|
||||
|
||||
// GOOD: Pattern 3 - Projection
|
||||
fn getName(person: Person): String is idempotent = person.name
|
||||
|
||||
// GOOD: Pattern 4 - Clamping
|
||||
fn clampPositive(x: Int): Int is idempotent =
|
||||
if x < 0 then 0 else x
|
||||
|
||||
// GOOD: Pattern 5 - Absolute value
|
||||
fn abs(x: Int): Int is idempotent =
|
||||
if x < 0 then 0 - x else x
|
||||
|
||||
// ERROR: Not idempotent (increment changes value each time)
|
||||
fn increment(x: Int): Int is idempotent = x + 1 // f(f(1)) = 3, not 2
|
||||
|
||||
// If you're certain a function is idempotent but the compiler can't verify:
|
||||
fn normalize(s: String): String assume is idempotent =
|
||||
String.toLower(String.trim(s))
|
||||
```
|
||||
|
||||
**What the compiler checks:**
|
||||
- Pattern recognition: constants, identity, projections, clamping, abs
|
||||
|
||||
**When to use `is idempotent`:**
|
||||
- Setting configuration
|
||||
- Database upserts
|
||||
- API PUT/DELETE operations (REST semantics)
|
||||
- Retry-safe operations
|
||||
|
||||
**Real-world example - safe retries:**
|
||||
|
||||
```lux
|
||||
// Payment processing with safe retries
|
||||
fn chargeCard(amount: Int, cardId: String): Receipt
|
||||
is idempotent
|
||||
with {Payment, Logger} = {
|
||||
Logger.log("Charging card " + cardId)
|
||||
Payment.charge(amount, cardId)
|
||||
}
|
||||
|
||||
// Safe to retry because chargeCard is idempotent
|
||||
fn processWithRetry(amount: Int, cardId: String): Receipt with {Payment, Logger, Fail} = {
|
||||
let result = retry(3, fn(): Receipt => chargeCard(amount, cardId))
|
||||
match result {
|
||||
Ok(receipt) => receipt,
|
||||
Err(e) => Fail.fail("Payment failed after 3 attempts: " + e)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Commutative (`is commutative`)
|
||||
|
||||
A commutative function satisfies: `f(a, b) == f(b, a)`. The order of arguments doesn't matter.
|
||||
|
||||
```lux
|
||||
// GOOD: Addition is commutative
|
||||
fn add(a: Int, b: Int): Int is commutative = a + b
|
||||
|
||||
// GOOD: Multiplication is commutative
|
||||
fn multiply(a: Int, b: Int): Int is commutative = a * b
|
||||
|
||||
// GOOD: Min/max are commutative
|
||||
fn minimum(a: Int, b: Int): Int is commutative =
|
||||
if a < b then a else b
|
||||
|
||||
// ERROR: Subtraction is not commutative (3 - 2 != 2 - 3)
|
||||
fn subtract(a: Int, b: Int): Int is commutative = a - b // Compiler error!
|
||||
|
||||
// ERROR: Wrong number of parameters
|
||||
fn triple(a: Int, b: Int, c: Int): Int is commutative = a + b + c // Must have exactly 2
|
||||
```
|
||||
|
||||
**What the compiler checks:**
|
||||
- Must have exactly 2 parameters
|
||||
- Body must be a commutative operation (+, *, min, max, ==, !=, &&, ||)
|
||||
|
||||
**When to use `is commutative`:**
|
||||
- Mathematical operations
|
||||
- Set operations (union, intersection)
|
||||
- Merging/combining functions
|
||||
|
||||
**Compiler optimizations enabled:**
|
||||
- Argument reordering for efficiency
|
||||
- Parallel reduction
|
||||
- Algebraic simplifications
|
||||
|
||||
## Combining Properties
|
||||
|
||||
Properties can be combined for stronger guarantees:
|
||||
|
||||
```lux
|
||||
// Pure + deterministic + total = perfect for caching
|
||||
fn computeHash(data: String): Int
|
||||
is pure
|
||||
is deterministic
|
||||
is total = {
|
||||
List.fold(String.chars(data), 0, fn(acc: Int, c: String): Int =>
|
||||
acc * 31 + charCode(c)
|
||||
)
|
||||
}
|
||||
|
||||
// Pure + idempotent = safe transformation
|
||||
fn normalizeEmail(email: String): String
|
||||
is pure
|
||||
is idempotent = {
|
||||
String.toLower(String.trim(email))
|
||||
}
|
||||
|
||||
// Commutative + pure = parallel reduction friendly
|
||||
fn merge(a: Record, b: Record): Record
|
||||
is pure
|
||||
is commutative = {
|
||||
{ ...a, ...b } // Last wins, but both contribute
|
||||
}
|
||||
```
|
||||
|
||||
## Property Constraints in Where Clauses
|
||||
|
||||
You can require function arguments to have certain properties:
|
||||
|
||||
```lux
|
||||
// Higher-order function that requires a pure function
|
||||
fn map<T, U>(list: List<T>, f: fn(T): U is pure): List<U> is pure =
|
||||
match list {
|
||||
[] => [],
|
||||
[x, ...rest] => [f(x), ...map(rest, f)]
|
||||
}
|
||||
|
||||
// Only accepts idempotent functions - safe to retry
|
||||
fn retry<T>(times: Int, action: fn(): T is idempotent): Result<T, String> = {
|
||||
if times <= 0 then Err("No attempts left")
|
||||
else {
|
||||
match tryCall(action) {
|
||||
Ok(result) => Ok(result),
|
||||
Err(e) => retry(times - 1, action) // Safe because action is idempotent
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Only accepts deterministic functions - safe to cache
|
||||
fn memoize<K, V>(f: fn(K): V is deterministic): fn(K): V = {
|
||||
let cache = HashMap.new()
|
||||
fn(key: K): V => {
|
||||
match cache.get(key) {
|
||||
Some(v) => v,
|
||||
None => {
|
||||
let v = f(key)
|
||||
cache.set(key, v)
|
||||
v
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Usage:
|
||||
let cachedHash = memoize(computeHash) // OK: computeHash is deterministic
|
||||
let badCache = memoize(generateRandom) // ERROR: generateRandom is not deterministic
|
||||
```
|
||||
|
||||
## The `assume` Escape Hatch
|
||||
|
||||
Sometimes you know a function has a property but the compiler can't verify it. Use `assume`:
|
||||
|
||||
```lux
|
||||
// Compiler can't verify this is idempotent, but we know it is
|
||||
fn setUserStatus(userId: String, status: String): Unit
|
||||
assume is idempotent
|
||||
with {Database} = {
|
||||
Database.execute("UPDATE users SET status = ? WHERE id = ?", [status, userId])
|
||||
}
|
||||
|
||||
// Use assume sparingly - it bypasses compiler checks!
|
||||
// If you're wrong, you may have subtle bugs.
|
||||
```
|
||||
|
||||
**Warning**: `assume` tells the compiler to trust you. If you're wrong, the optimization or guarantee may be invalid.
|
||||
|
||||
## Compiler Optimizations
|
||||
|
||||
When the compiler knows behavioral properties, it can optimize aggressively:
|
||||
|
||||
| Property | Optimizations |
|
||||
|----------|---------------|
|
||||
| `is pure` | Memoization, CSE, dead code elimination, parallelization |
|
||||
| `is total` | No exception handling, aggressive inlining |
|
||||
| `is deterministic` | Result caching, parallel execution |
|
||||
| `is idempotent` | Retry optimization, duplicate call elimination |
|
||||
| `is commutative` | Argument reordering, parallel reduction |
|
||||
|
||||
### Example: Automatic Memoization
|
||||
|
||||
```lux
|
||||
fn expensiveComputation(n: Int): Int
|
||||
is pure
|
||||
is deterministic
|
||||
is total = {
|
||||
// Complex calculation...
|
||||
fib(n)
|
||||
}
|
||||
|
||||
// The compiler may automatically cache results because:
|
||||
// - pure: no side effects, so caching is safe
|
||||
// - deterministic: same input = same output
|
||||
// - total: will always return a value
|
||||
```
|
||||
|
||||
### Example: Safe Parallelization
|
||||
|
||||
```lux
|
||||
fn processItems(items: List<Item>): List<Result>
|
||||
is pure = {
|
||||
List.map(items, processItem)
|
||||
}
|
||||
|
||||
// If processItem is pure, the compiler can parallelize this automatically
|
||||
```
|
||||
|
||||
## Practical Examples
|
||||
|
||||
### Example 1: Financial Calculations
|
||||
|
||||
```lux
|
||||
// Interest calculation - pure, deterministic, total
|
||||
fn calculateInterest(principal: Int, rate: Float, years: Int): Float
|
||||
is pure
|
||||
is deterministic
|
||||
is total = {
|
||||
let r = rate / 100.0
|
||||
Float.fromInt(principal) * Math.pow(1.0 + r, Float.fromInt(years))
|
||||
}
|
||||
|
||||
// Transaction validation - pure, total
|
||||
fn validateTransaction(tx: Transaction): Result<Transaction, String>
|
||||
is pure
|
||||
is total = {
|
||||
if tx.amount <= 0 then Err("Amount must be positive")
|
||||
else if tx.from == tx.to then Err("Cannot transfer to self")
|
||||
else Ok(tx)
|
||||
}
|
||||
```
|
||||
|
||||
### Example 2: Data Processing Pipeline
|
||||
|
||||
```lux
|
||||
// Each step is pure and deterministic
|
||||
fn cleanData(raw: String): String is pure is deterministic =
|
||||
raw |> String.trim |> String.toLower
|
||||
|
||||
fn parseRecord(line: String): Result<Record, String> is pure is deterministic =
|
||||
match String.split(line, ",") {
|
||||
[name, age, email] => Ok({ name, age: parseInt(age), email }),
|
||||
_ => Err("Invalid format")
|
||||
}
|
||||
|
||||
fn validateRecord(record: Record): Bool is pure is deterministic is total =
|
||||
String.length(record.name) > 0 && record.age > 0
|
||||
|
||||
// Pipeline can be parallelized because all functions are pure + deterministic
|
||||
fn processFile(contents: String): List<Record> is pure is deterministic = {
|
||||
contents
|
||||
|> String.lines
|
||||
|> List.map(cleanData)
|
||||
|> List.map(parseRecord)
|
||||
|> List.filterMap(fn(r: Result<Record, String>): Option<Record> =>
|
||||
match r { Ok(v) => Some(v), Err(_) => None })
|
||||
|> List.filter(validateRecord)
|
||||
}
|
||||
```
|
||||
|
||||
### Example 3: Idempotent API Handlers
|
||||
|
||||
```lux
|
||||
// PUT /users/:id - idempotent by REST semantics
|
||||
fn handlePutUser(id: String, data: UserData): Response
|
||||
is idempotent
|
||||
with {Database, Logger} = {
|
||||
Logger.log("PUT /users/" + id)
|
||||
Database.upsert("users", id, data)
|
||||
Response.ok({ id, ...data })
|
||||
}
|
||||
|
||||
// DELETE /users/:id - idempotent by REST semantics
|
||||
fn handleDeleteUser(id: String): Response
|
||||
is idempotent
|
||||
with {Database, Logger} = {
|
||||
Logger.log("DELETE /users/" + id)
|
||||
Database.delete("users", id) // Safe to call multiple times
|
||||
Response.noContent()
|
||||
}
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
| Property | Meaning | Compiler Checks | Use Case |
|
||||
|----------|---------|-----------------|----------|
|
||||
| `is pure` | No effects | Empty effect set | Caching, parallelization |
|
||||
| `is total` | Always terminates | No Fail, structural recursion | Core logic |
|
||||
| `is deterministic` | Same in = same out | No Random/Time | Caching, testing |
|
||||
| `is idempotent` | f(f(x)) = f(x) | Pattern recognition | Retries, APIs |
|
||||
| `is commutative` | f(a,b) = f(b,a) | 2 params, commutative op | Math, merging |
|
||||
|
||||
## What's Next?
|
||||
|
||||
- [Chapter 13: Schema Evolution](./13-schema-evolution.md) - Version your data types
|
||||
- [Tutorials](../tutorials/README.md) - Practical projects
|
||||
573
docs/guide/13-schema-evolution.md
Normal file
573
docs/guide/13-schema-evolution.md
Normal file
@@ -0,0 +1,573 @@
|
||||
# Chapter 13: Schema Evolution
|
||||
|
||||
Data structures change over time. Fields get added, removed, or renamed. Types get split or merged. Without careful handling, these changes break systems—old data can't be read, services fail, migrations corrupt data.
|
||||
|
||||
Lux's **schema evolution** system makes these changes safe and automatic.
|
||||
|
||||
## The Problem
|
||||
|
||||
Consider a real scenario:
|
||||
|
||||
```lux
|
||||
// Version 1: Simple user
|
||||
type User {
|
||||
name: String
|
||||
}
|
||||
|
||||
// Later, you need email addresses
|
||||
type User {
|
||||
name: String,
|
||||
email: String // Breaking change! Old data doesn't have this.
|
||||
}
|
||||
```
|
||||
|
||||
In most languages, this breaks everything. Existing users in your database don't have email addresses. Deserializing old data fails. Services crash.
|
||||
|
||||
Lux solves this with **versioned types** and **automatic migrations**.
|
||||
|
||||
## Versioned Types
|
||||
|
||||
Add a version annotation to any type:
|
||||
|
||||
```lux
|
||||
// Version 1: Original definition
|
||||
type User @v1 {
|
||||
name: String
|
||||
}
|
||||
|
||||
// Version 2: Added email field
|
||||
type User @v2 {
|
||||
name: String,
|
||||
email: String,
|
||||
|
||||
// How to migrate from v1
|
||||
from @v1 = { name: old.name, email: "unknown@example.com" }
|
||||
}
|
||||
|
||||
// Version 3: Split name into first/last
|
||||
type User @v3 {
|
||||
firstName: String,
|
||||
lastName: String,
|
||||
email: String,
|
||||
|
||||
// How to migrate from v2
|
||||
from @v2 = {
|
||||
firstName: String.split(old.name, " ") |> List.head |> Option.getOrElse(""),
|
||||
lastName: String.split(old.name, " ") |> List.tail |> List.head |> Option.getOrElse(""),
|
||||
email: old.email
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The `@latest` alias always refers to the most recent version:
|
||||
|
||||
```lux
|
||||
type User @latest {
|
||||
firstName: String,
|
||||
lastName: String,
|
||||
email: String,
|
||||
|
||||
from @v2 = { ... }
|
||||
}
|
||||
|
||||
// These are equivalent:
|
||||
fn createUser(first: String, last: String, email: String): User@latest = ...
|
||||
fn createUser(first: String, last: String, email: String): User@v3 = ...
|
||||
```
|
||||
|
||||
## Migration Syntax
|
||||
|
||||
### Basic Migration
|
||||
|
||||
```lux
|
||||
type Config @v2 {
|
||||
theme: String,
|
||||
fontSize: Int,
|
||||
|
||||
// 'old' refers to the v1 value
|
||||
from @v1 = {
|
||||
theme: old.theme,
|
||||
fontSize: 14 // New field with default
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Computed Fields
|
||||
|
||||
```lux
|
||||
type Order @v2 {
|
||||
items: List<Item>,
|
||||
total: Int,
|
||||
itemCount: Int, // New computed field
|
||||
|
||||
from @v1 = {
|
||||
items: old.items,
|
||||
total: old.total,
|
||||
itemCount: List.length(old.items)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Removing Fields
|
||||
|
||||
When removing fields, simply don't include them in the new version:
|
||||
|
||||
```lux
|
||||
type Settings @v1 {
|
||||
theme: String,
|
||||
legacyMode: Bool, // To be removed
|
||||
volume: Int
|
||||
}
|
||||
|
||||
type Settings @v2 {
|
||||
theme: String,
|
||||
volume: Int,
|
||||
|
||||
// legacyMode is dropped - just don't migrate it
|
||||
from @v1 = {
|
||||
theme: old.theme,
|
||||
volume: old.volume
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Renaming Fields
|
||||
|
||||
```lux
|
||||
type Product @v1 {
|
||||
name: String,
|
||||
cost: Int // Old field name
|
||||
}
|
||||
|
||||
type Product @v2 {
|
||||
name: String,
|
||||
price: Int, // Renamed from 'cost'
|
||||
|
||||
from @v1 = {
|
||||
name: old.name,
|
||||
price: old.cost // Map old field to new name
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Complex Transformations
|
||||
|
||||
```lux
|
||||
type Address @v1 {
|
||||
fullAddress: String // "123 Main St, New York, NY 10001"
|
||||
}
|
||||
|
||||
type Address @v2 {
|
||||
street: String,
|
||||
city: String,
|
||||
state: String,
|
||||
zip: String,
|
||||
|
||||
from @v1 = {
|
||||
let parts = String.split(old.fullAddress, ", ")
|
||||
{
|
||||
street: List.get(parts, 0) |> Option.getOrElse(""),
|
||||
city: List.get(parts, 1) |> Option.getOrElse(""),
|
||||
state: List.get(parts, 2)
|
||||
|> Option.map(fn(s: String): String => String.split(s, " ") |> List.head |> Option.getOrElse(""))
|
||||
|> Option.getOrElse(""),
|
||||
zip: List.get(parts, 2)
|
||||
|> Option.map(fn(s: String): String => String.split(s, " ") |> List.last |> Option.getOrElse(""))
|
||||
|> Option.getOrElse("")
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Working with Versioned Values
|
||||
|
||||
The `Schema` module provides runtime operations for versioned values:
|
||||
|
||||
### Creating Versioned Values
|
||||
|
||||
```lux
|
||||
// Create a value tagged with a specific version
|
||||
let userV1 = Schema.versioned("User", 1, { name: "Alice" })
|
||||
let userV2 = Schema.versioned("User", 2, { name: "Alice", email: "alice@example.com" })
|
||||
```
|
||||
|
||||
### Checking Versions
|
||||
|
||||
```lux
|
||||
let user = Schema.versioned("User", 1, { name: "Alice" })
|
||||
let version = Schema.getVersion(user) // Returns 1
|
||||
|
||||
// Version-aware logic
|
||||
if version < 2 then
|
||||
Console.print("Legacy user format")
|
||||
else
|
||||
Console.print("Modern user format")
|
||||
```
|
||||
|
||||
### Migrating Values
|
||||
|
||||
```lux
|
||||
// Migrate to a specific version
|
||||
let userV1 = Schema.versioned("User", 1, { name: "Alice" })
|
||||
let userV2 = Schema.migrate(userV1, 2) // Uses declared migration
|
||||
|
||||
let version = Schema.getVersion(userV2) // Now 2
|
||||
|
||||
// Chain migrations (v1 -> v2 -> v3)
|
||||
let userV3 = Schema.migrate(userV1, 3) // Applies v1->v2, then v2->v3
|
||||
```
|
||||
|
||||
## Auto-Generated Migrations
|
||||
|
||||
For simple changes, Lux can **automatically generate** migrations:
|
||||
|
||||
```lux
|
||||
type Profile @v1 {
|
||||
name: String
|
||||
}
|
||||
|
||||
// Adding a field with a default? Migration is auto-generated
|
||||
type Profile @v2 {
|
||||
name: String,
|
||||
bio: String = "" // Default value provided
|
||||
}
|
||||
|
||||
// The compiler generates this for you:
|
||||
// from @v1 = { name: old.name, bio: "" }
|
||||
```
|
||||
|
||||
Auto-migration works for:
|
||||
- Adding fields with default values
|
||||
- Keeping existing fields unchanged
|
||||
|
||||
You must write explicit migrations for:
|
||||
- Field renaming
|
||||
- Field removal (to confirm intent)
|
||||
- Type changes
|
||||
- Computed/derived fields
|
||||
|
||||
## Practical Examples
|
||||
|
||||
### Example 1: API Response Versioning
|
||||
|
||||
```lux
|
||||
type ApiResponse @v1 {
|
||||
status: String,
|
||||
data: String
|
||||
}
|
||||
|
||||
type ApiResponse @v2 {
|
||||
status: String,
|
||||
data: String,
|
||||
meta: { timestamp: Int, version: String },
|
||||
|
||||
from @v1 = {
|
||||
status: old.status,
|
||||
data: old.data,
|
||||
meta: { timestamp: 0, version: "legacy" }
|
||||
}
|
||||
}
|
||||
|
||||
// Version-aware API client
|
||||
fn handleResponse(raw: ApiResponse@v1): ApiResponse@v2 = {
|
||||
Schema.migrate(Schema.versioned("ApiResponse", 1, raw), 2)
|
||||
}
|
||||
```
|
||||
|
||||
### Example 2: Database Record Evolution
|
||||
|
||||
```lux
|
||||
// Original schema
|
||||
type Customer @v1 {
|
||||
name: String,
|
||||
address: String
|
||||
}
|
||||
|
||||
// Split address into components
|
||||
type Customer @v2 {
|
||||
name: String,
|
||||
street: String,
|
||||
city: String,
|
||||
country: String,
|
||||
|
||||
from @v1 = {
|
||||
let parts = String.split(old.address, ", ")
|
||||
{
|
||||
name: old.name,
|
||||
street: List.get(parts, 0) |> Option.getOrElse(old.address),
|
||||
city: List.get(parts, 1) |> Option.getOrElse("Unknown"),
|
||||
country: List.get(parts, 2) |> Option.getOrElse("Unknown")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Load and migrate on read
|
||||
fn loadCustomer(id: String): Customer@v2 with {Database} = {
|
||||
let record = Database.query("SELECT * FROM customers WHERE id = ?", [id])
|
||||
let version = record.schema_version // Stored version
|
||||
|
||||
if version == 1 then
|
||||
let v1 = Schema.versioned("Customer", 1, {
|
||||
name: record.name,
|
||||
address: record.address
|
||||
})
|
||||
Schema.migrate(v1, 2)
|
||||
else
|
||||
{ name: record.name, street: record.street, city: record.city, country: record.country }
|
||||
}
|
||||
```
|
||||
|
||||
### Example 3: Configuration Files
|
||||
|
||||
```lux
|
||||
type AppConfig @v1 {
|
||||
debug: Bool,
|
||||
port: Int
|
||||
}
|
||||
|
||||
type AppConfig @v2 {
|
||||
debug: Bool,
|
||||
port: Int,
|
||||
logLevel: String, // New in v2
|
||||
|
||||
from @v1 = {
|
||||
debug: old.debug,
|
||||
port: old.port,
|
||||
logLevel: if old.debug then "debug" else "info"
|
||||
}
|
||||
}
|
||||
|
||||
type AppConfig @v3 {
|
||||
environment: String, // Replaces debug flag
|
||||
port: Int,
|
||||
logLevel: String,
|
||||
|
||||
from @v2 = {
|
||||
environment: if old.debug then "development" else "production",
|
||||
port: old.port,
|
||||
logLevel: old.logLevel
|
||||
}
|
||||
}
|
||||
|
||||
// Load config with automatic migration
|
||||
fn loadConfig(path: String): AppConfig@v3 with {File} = {
|
||||
let json = File.read(path)
|
||||
let parsed = Json.parse(json)
|
||||
let version = Json.getInt(parsed, "version") |> Option.getOrElse(1)
|
||||
|
||||
match version {
|
||||
1 => {
|
||||
let v1 = Schema.versioned("AppConfig", 1, {
|
||||
debug: Json.getBool(parsed, "debug") |> Option.getOrElse(false),
|
||||
port: Json.getInt(parsed, "port") |> Option.getOrElse(8080)
|
||||
})
|
||||
Schema.migrate(v1, 3)
|
||||
},
|
||||
2 => {
|
||||
let v2 = Schema.versioned("AppConfig", 2, {
|
||||
debug: Json.getBool(parsed, "debug") |> Option.getOrElse(false),
|
||||
port: Json.getInt(parsed, "port") |> Option.getOrElse(8080),
|
||||
logLevel: Json.getString(parsed, "logLevel") |> Option.getOrElse("info")
|
||||
})
|
||||
Schema.migrate(v2, 3)
|
||||
},
|
||||
_ => {
|
||||
// Already v3
|
||||
{
|
||||
environment: Json.getString(parsed, "environment") |> Option.getOrElse("production"),
|
||||
port: Json.getInt(parsed, "port") |> Option.getOrElse(8080),
|
||||
logLevel: Json.getString(parsed, "logLevel") |> Option.getOrElse("info")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Example 4: Event Sourcing
|
||||
|
||||
```lux
|
||||
// Event types evolve over time
|
||||
type UserCreated @v1 {
|
||||
userId: String,
|
||||
name: String,
|
||||
timestamp: Int
|
||||
}
|
||||
|
||||
type UserCreated @v2 {
|
||||
userId: String,
|
||||
name: String,
|
||||
email: String,
|
||||
createdAt: Int, // Renamed from timestamp
|
||||
|
||||
from @v1 = {
|
||||
userId: old.userId,
|
||||
name: old.name,
|
||||
email: "", // Not captured in v1
|
||||
createdAt: old.timestamp
|
||||
}
|
||||
}
|
||||
|
||||
// Process events regardless of version
|
||||
fn processEvent(event: UserCreated@v1 | UserCreated@v2): Unit with {Console} = {
|
||||
let normalized = Schema.migrate(event, 2) // Always work with v2
|
||||
Console.print("User created: " + normalized.name + " at " + toString(normalized.createdAt))
|
||||
}
|
||||
```
|
||||
|
||||
## Compile-Time Safety
|
||||
|
||||
The compiler catches schema evolution errors:
|
||||
|
||||
```lux
|
||||
type User @v2 {
|
||||
name: String,
|
||||
email: String
|
||||
|
||||
// ERROR: Migration references non-existent field
|
||||
from @v1 = { name: old.username, email: old.email }
|
||||
// ^^^^^^^^ 'username' does not exist in User@v1
|
||||
}
|
||||
```
|
||||
|
||||
```lux
|
||||
type User @v2 {
|
||||
name: String,
|
||||
email: String
|
||||
|
||||
// ERROR: Migration missing required field
|
||||
from @v1 = { name: old.name }
|
||||
// ^ Missing 'email' field
|
||||
}
|
||||
```
|
||||
|
||||
```lux
|
||||
type User @v2 {
|
||||
name: String,
|
||||
age: Int
|
||||
|
||||
// ERROR: Type mismatch in migration
|
||||
from @v1 = { name: old.name, age: old.birthYear }
|
||||
// ^^^^^^^^^^^^^ Expected Int, found String
|
||||
}
|
||||
```
|
||||
|
||||
## Compatibility Checking
|
||||
|
||||
Lux tracks compatibility between versions:
|
||||
|
||||
| Change Type | Backward Compatible | Forward Compatible |
|
||||
|-------------|--------------------|--------------------|
|
||||
| Add optional field (with default) | Yes | Yes |
|
||||
| Add required field | No | Yes (with migration) |
|
||||
| Remove field | Yes (with migration) | No |
|
||||
| Rename field | No | No (need migration) |
|
||||
| Change field type | No | No (need migration) |
|
||||
|
||||
The compiler warns about breaking changes:
|
||||
|
||||
```lux
|
||||
type User @v1 {
|
||||
name: String,
|
||||
email: String
|
||||
}
|
||||
|
||||
type User @v2 {
|
||||
name: String
|
||||
// Warning: Removing 'email' is a breaking change
|
||||
// Existing v2 consumers expect this field
|
||||
}
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Always Version Production Types
|
||||
|
||||
```lux
|
||||
// Good: Versioned from the start
|
||||
type Order @v1 {
|
||||
id: String,
|
||||
items: List<Item>,
|
||||
total: Int
|
||||
}
|
||||
|
||||
// Bad: Unversioned type is hard to evolve
|
||||
type Order {
|
||||
id: String,
|
||||
items: List<Item>,
|
||||
total: Int
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Keep Migrations Simple
|
||||
|
||||
```lux
|
||||
// Good: Simple, direct mapping
|
||||
from @v1 = {
|
||||
name: old.name,
|
||||
email: old.email |> Option.getOrElse("")
|
||||
}
|
||||
|
||||
// Avoid: Complex logic in migrations
|
||||
from @v1 = {
|
||||
name: old.name,
|
||||
email: {
|
||||
// Don't put complex business logic here
|
||||
let domain = inferDomainFromName(old.name)
|
||||
let local = String.toLower(String.replace(old.name, " ", "."))
|
||||
local + "@" + domain
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Test Migrations
|
||||
|
||||
```lux
|
||||
fn testUserMigration(): Unit with {Test} = {
|
||||
let v1User = Schema.versioned("User", 1, { name: "Alice" })
|
||||
let v2User = Schema.migrate(v1User, 2)
|
||||
|
||||
Test.assertEqual(v2User.name, "Alice")
|
||||
Test.assertEqual(v2User.email, "unknown@example.com")
|
||||
}
|
||||
```
|
||||
|
||||
### 4. Document Breaking Changes
|
||||
|
||||
```lux
|
||||
type User @v3 {
|
||||
// BREAKING: 'name' split into firstName/lastName
|
||||
// Migration: name.split(" ")[0] -> firstName, name.split(" ")[1] -> lastName
|
||||
firstName: String,
|
||||
lastName: String,
|
||||
email: String,
|
||||
|
||||
from @v2 = { ... }
|
||||
}
|
||||
```
|
||||
|
||||
## Schema Module Reference
|
||||
|
||||
| Function | Description |
|
||||
|----------|-------------|
|
||||
| `Schema.versioned(typeName, version, value)` | Create a versioned value |
|
||||
| `Schema.getVersion(value)` | Get the version of a value |
|
||||
| `Schema.migrate(value, targetVersion)` | Migrate to a target version |
|
||||
| `Schema.isCompatible(v1, v2)` | Check if versions are compatible |
|
||||
|
||||
## Summary
|
||||
|
||||
Schema evolution in Lux provides:
|
||||
|
||||
- **Versioned types** with `@v1`, `@v2`, `@latest` annotations
|
||||
- **Explicit migrations** with `from @vN = { ... }` syntax
|
||||
- **Automatic migrations** for simple field additions with defaults
|
||||
- **Runtime operations** via the `Schema` module
|
||||
- **Compile-time safety** catching migration errors early
|
||||
- **Migration chaining** for multi-step upgrades
|
||||
|
||||
This system ensures your data can evolve safely over time, without breaking existing code or losing information.
|
||||
|
||||
## What's Next?
|
||||
|
||||
- [Tutorials](../tutorials/README.md) - Build real projects
|
||||
- [Standard Library Reference](../stdlib/README.md) - Complete API docs
|
||||
Reference in New Issue
Block a user