feat: rebuild website with full learning funnel
Website rebuilt from scratch based on analysis of 11 beloved language websites (Elm, Zig, Gleam, Swift, Kotlin, Haskell, OCaml, Crystal, Roc, Rust, Go). New website structure: - Homepage with hero, playground, three pillars, install guide - Language Tour with interactive lessons (hello world, types, effects) - Examples cookbook with categorized sidebar - API documentation index - Installation guide (Nix and source) - Sleek/noble design (black/gold, serif typography) Also includes: - New stdlib/json.lux module for JSON serialization - Enhanced stdlib/http.lux with middleware and routing - New string functions (charAt, indexOf, lastIndexOf, repeat) - LSP improvements (rename, signature help, formatting) - Package manager transitive dependency resolution - Updated documentation for effects and stdlib - New showcase example (task_manager.lux) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
573
docs/guide/13-schema-evolution.md
Normal file
573
docs/guide/13-schema-evolution.md
Normal file
@@ -0,0 +1,573 @@
|
||||
# Chapter 13: Schema Evolution
|
||||
|
||||
Data structures change over time. Fields get added, removed, or renamed. Types get split or merged. Without careful handling, these changes break systems—old data can't be read, services fail, migrations corrupt data.
|
||||
|
||||
Lux's **schema evolution** system makes these changes safe and automatic.
|
||||
|
||||
## The Problem
|
||||
|
||||
Consider a real scenario:
|
||||
|
||||
```lux
|
||||
// Version 1: Simple user
|
||||
type User {
|
||||
name: String
|
||||
}
|
||||
|
||||
// Later, you need email addresses
|
||||
type User {
|
||||
name: String,
|
||||
email: String // Breaking change! Old data doesn't have this.
|
||||
}
|
||||
```
|
||||
|
||||
In most languages, this breaks everything. Existing users in your database don't have email addresses. Deserializing old data fails. Services crash.
|
||||
|
||||
Lux solves this with **versioned types** and **automatic migrations**.
|
||||
|
||||
## Versioned Types
|
||||
|
||||
Add a version annotation to any type:
|
||||
|
||||
```lux
|
||||
// Version 1: Original definition
|
||||
type User @v1 {
|
||||
name: String
|
||||
}
|
||||
|
||||
// Version 2: Added email field
|
||||
type User @v2 {
|
||||
name: String,
|
||||
email: String,
|
||||
|
||||
// How to migrate from v1
|
||||
from @v1 = { name: old.name, email: "unknown@example.com" }
|
||||
}
|
||||
|
||||
// Version 3: Split name into first/last
|
||||
type User @v3 {
|
||||
firstName: String,
|
||||
lastName: String,
|
||||
email: String,
|
||||
|
||||
// How to migrate from v2
|
||||
from @v2 = {
|
||||
firstName: String.split(old.name, " ") |> List.head |> Option.getOrElse(""),
|
||||
lastName: String.split(old.name, " ") |> List.tail |> List.head |> Option.getOrElse(""),
|
||||
email: old.email
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The `@latest` alias always refers to the most recent version:
|
||||
|
||||
```lux
|
||||
type User @latest {
|
||||
firstName: String,
|
||||
lastName: String,
|
||||
email: String,
|
||||
|
||||
from @v2 = { ... }
|
||||
}
|
||||
|
||||
// These are equivalent:
|
||||
fn createUser(first: String, last: String, email: String): User@latest = ...
|
||||
fn createUser(first: String, last: String, email: String): User@v3 = ...
|
||||
```
|
||||
|
||||
## Migration Syntax
|
||||
|
||||
### Basic Migration
|
||||
|
||||
```lux
|
||||
type Config @v2 {
|
||||
theme: String,
|
||||
fontSize: Int,
|
||||
|
||||
// 'old' refers to the v1 value
|
||||
from @v1 = {
|
||||
theme: old.theme,
|
||||
fontSize: 14 // New field with default
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Computed Fields
|
||||
|
||||
```lux
|
||||
type Order @v2 {
|
||||
items: List<Item>,
|
||||
total: Int,
|
||||
itemCount: Int, // New computed field
|
||||
|
||||
from @v1 = {
|
||||
items: old.items,
|
||||
total: old.total,
|
||||
itemCount: List.length(old.items)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Removing Fields
|
||||
|
||||
When removing fields, simply don't include them in the new version:
|
||||
|
||||
```lux
|
||||
type Settings @v1 {
|
||||
theme: String,
|
||||
legacyMode: Bool, // To be removed
|
||||
volume: Int
|
||||
}
|
||||
|
||||
type Settings @v2 {
|
||||
theme: String,
|
||||
volume: Int,
|
||||
|
||||
// legacyMode is dropped - just don't migrate it
|
||||
from @v1 = {
|
||||
theme: old.theme,
|
||||
volume: old.volume
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Renaming Fields
|
||||
|
||||
```lux
|
||||
type Product @v1 {
|
||||
name: String,
|
||||
cost: Int // Old field name
|
||||
}
|
||||
|
||||
type Product @v2 {
|
||||
name: String,
|
||||
price: Int, // Renamed from 'cost'
|
||||
|
||||
from @v1 = {
|
||||
name: old.name,
|
||||
price: old.cost // Map old field to new name
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Complex Transformations
|
||||
|
||||
```lux
|
||||
type Address @v1 {
|
||||
fullAddress: String // "123 Main St, New York, NY 10001"
|
||||
}
|
||||
|
||||
type Address @v2 {
|
||||
street: String,
|
||||
city: String,
|
||||
state: String,
|
||||
zip: String,
|
||||
|
||||
from @v1 = {
|
||||
let parts = String.split(old.fullAddress, ", ")
|
||||
{
|
||||
street: List.get(parts, 0) |> Option.getOrElse(""),
|
||||
city: List.get(parts, 1) |> Option.getOrElse(""),
|
||||
state: List.get(parts, 2)
|
||||
|> Option.map(fn(s: String): String => String.split(s, " ") |> List.head |> Option.getOrElse(""))
|
||||
|> Option.getOrElse(""),
|
||||
zip: List.get(parts, 2)
|
||||
|> Option.map(fn(s: String): String => String.split(s, " ") |> List.last |> Option.getOrElse(""))
|
||||
|> Option.getOrElse("")
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Working with Versioned Values
|
||||
|
||||
The `Schema` module provides runtime operations for versioned values:
|
||||
|
||||
### Creating Versioned Values
|
||||
|
||||
```lux
|
||||
// Create a value tagged with a specific version
|
||||
let userV1 = Schema.versioned("User", 1, { name: "Alice" })
|
||||
let userV2 = Schema.versioned("User", 2, { name: "Alice", email: "alice@example.com" })
|
||||
```
|
||||
|
||||
### Checking Versions
|
||||
|
||||
```lux
|
||||
let user = Schema.versioned("User", 1, { name: "Alice" })
|
||||
let version = Schema.getVersion(user) // Returns 1
|
||||
|
||||
// Version-aware logic
|
||||
if version < 2 then
|
||||
Console.print("Legacy user format")
|
||||
else
|
||||
Console.print("Modern user format")
|
||||
```
|
||||
|
||||
### Migrating Values
|
||||
|
||||
```lux
|
||||
// Migrate to a specific version
|
||||
let userV1 = Schema.versioned("User", 1, { name: "Alice" })
|
||||
let userV2 = Schema.migrate(userV1, 2) // Uses declared migration
|
||||
|
||||
let version = Schema.getVersion(userV2) // Now 2
|
||||
|
||||
// Chain migrations (v1 -> v2 -> v3)
|
||||
let userV3 = Schema.migrate(userV1, 3) // Applies v1->v2, then v2->v3
|
||||
```
|
||||
|
||||
## Auto-Generated Migrations
|
||||
|
||||
For simple changes, Lux can **automatically generate** migrations:
|
||||
|
||||
```lux
|
||||
type Profile @v1 {
|
||||
name: String
|
||||
}
|
||||
|
||||
// Adding a field with a default? Migration is auto-generated
|
||||
type Profile @v2 {
|
||||
name: String,
|
||||
bio: String = "" // Default value provided
|
||||
}
|
||||
|
||||
// The compiler generates this for you:
|
||||
// from @v1 = { name: old.name, bio: "" }
|
||||
```
|
||||
|
||||
Auto-migration works for:
|
||||
- Adding fields with default values
|
||||
- Keeping existing fields unchanged
|
||||
|
||||
You must write explicit migrations for:
|
||||
- Field renaming
|
||||
- Field removal (to confirm intent)
|
||||
- Type changes
|
||||
- Computed/derived fields
|
||||
|
||||
## Practical Examples
|
||||
|
||||
### Example 1: API Response Versioning
|
||||
|
||||
```lux
|
||||
type ApiResponse @v1 {
|
||||
status: String,
|
||||
data: String
|
||||
}
|
||||
|
||||
type ApiResponse @v2 {
|
||||
status: String,
|
||||
data: String,
|
||||
meta: { timestamp: Int, version: String },
|
||||
|
||||
from @v1 = {
|
||||
status: old.status,
|
||||
data: old.data,
|
||||
meta: { timestamp: 0, version: "legacy" }
|
||||
}
|
||||
}
|
||||
|
||||
// Version-aware API client
|
||||
fn handleResponse(raw: ApiResponse@v1): ApiResponse@v2 = {
|
||||
Schema.migrate(Schema.versioned("ApiResponse", 1, raw), 2)
|
||||
}
|
||||
```
|
||||
|
||||
### Example 2: Database Record Evolution
|
||||
|
||||
```lux
|
||||
// Original schema
|
||||
type Customer @v1 {
|
||||
name: String,
|
||||
address: String
|
||||
}
|
||||
|
||||
// Split address into components
|
||||
type Customer @v2 {
|
||||
name: String,
|
||||
street: String,
|
||||
city: String,
|
||||
country: String,
|
||||
|
||||
from @v1 = {
|
||||
let parts = String.split(old.address, ", ")
|
||||
{
|
||||
name: old.name,
|
||||
street: List.get(parts, 0) |> Option.getOrElse(old.address),
|
||||
city: List.get(parts, 1) |> Option.getOrElse("Unknown"),
|
||||
country: List.get(parts, 2) |> Option.getOrElse("Unknown")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Load and migrate on read
|
||||
fn loadCustomer(id: String): Customer@v2 with {Database} = {
|
||||
let record = Database.query("SELECT * FROM customers WHERE id = ?", [id])
|
||||
let version = record.schema_version // Stored version
|
||||
|
||||
if version == 1 then
|
||||
let v1 = Schema.versioned("Customer", 1, {
|
||||
name: record.name,
|
||||
address: record.address
|
||||
})
|
||||
Schema.migrate(v1, 2)
|
||||
else
|
||||
{ name: record.name, street: record.street, city: record.city, country: record.country }
|
||||
}
|
||||
```
|
||||
|
||||
### Example 3: Configuration Files
|
||||
|
||||
```lux
|
||||
type AppConfig @v1 {
|
||||
debug: Bool,
|
||||
port: Int
|
||||
}
|
||||
|
||||
type AppConfig @v2 {
|
||||
debug: Bool,
|
||||
port: Int,
|
||||
logLevel: String, // New in v2
|
||||
|
||||
from @v1 = {
|
||||
debug: old.debug,
|
||||
port: old.port,
|
||||
logLevel: if old.debug then "debug" else "info"
|
||||
}
|
||||
}
|
||||
|
||||
type AppConfig @v3 {
|
||||
environment: String, // Replaces debug flag
|
||||
port: Int,
|
||||
logLevel: String,
|
||||
|
||||
from @v2 = {
|
||||
environment: if old.debug then "development" else "production",
|
||||
port: old.port,
|
||||
logLevel: old.logLevel
|
||||
}
|
||||
}
|
||||
|
||||
// Load config with automatic migration
|
||||
fn loadConfig(path: String): AppConfig@v3 with {File} = {
|
||||
let json = File.read(path)
|
||||
let parsed = Json.parse(json)
|
||||
let version = Json.getInt(parsed, "version") |> Option.getOrElse(1)
|
||||
|
||||
match version {
|
||||
1 => {
|
||||
let v1 = Schema.versioned("AppConfig", 1, {
|
||||
debug: Json.getBool(parsed, "debug") |> Option.getOrElse(false),
|
||||
port: Json.getInt(parsed, "port") |> Option.getOrElse(8080)
|
||||
})
|
||||
Schema.migrate(v1, 3)
|
||||
},
|
||||
2 => {
|
||||
let v2 = Schema.versioned("AppConfig", 2, {
|
||||
debug: Json.getBool(parsed, "debug") |> Option.getOrElse(false),
|
||||
port: Json.getInt(parsed, "port") |> Option.getOrElse(8080),
|
||||
logLevel: Json.getString(parsed, "logLevel") |> Option.getOrElse("info")
|
||||
})
|
||||
Schema.migrate(v2, 3)
|
||||
},
|
||||
_ => {
|
||||
// Already v3
|
||||
{
|
||||
environment: Json.getString(parsed, "environment") |> Option.getOrElse("production"),
|
||||
port: Json.getInt(parsed, "port") |> Option.getOrElse(8080),
|
||||
logLevel: Json.getString(parsed, "logLevel") |> Option.getOrElse("info")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Example 4: Event Sourcing
|
||||
|
||||
```lux
|
||||
// Event types evolve over time
|
||||
type UserCreated @v1 {
|
||||
userId: String,
|
||||
name: String,
|
||||
timestamp: Int
|
||||
}
|
||||
|
||||
type UserCreated @v2 {
|
||||
userId: String,
|
||||
name: String,
|
||||
email: String,
|
||||
createdAt: Int, // Renamed from timestamp
|
||||
|
||||
from @v1 = {
|
||||
userId: old.userId,
|
||||
name: old.name,
|
||||
email: "", // Not captured in v1
|
||||
createdAt: old.timestamp
|
||||
}
|
||||
}
|
||||
|
||||
// Process events regardless of version
|
||||
fn processEvent(event: UserCreated@v1 | UserCreated@v2): Unit with {Console} = {
|
||||
let normalized = Schema.migrate(event, 2) // Always work with v2
|
||||
Console.print("User created: " + normalized.name + " at " + toString(normalized.createdAt))
|
||||
}
|
||||
```
|
||||
|
||||
## Compile-Time Safety
|
||||
|
||||
The compiler catches schema evolution errors:
|
||||
|
||||
```lux
|
||||
type User @v2 {
|
||||
name: String,
|
||||
email: String
|
||||
|
||||
// ERROR: Migration references non-existent field
|
||||
from @v1 = { name: old.username, email: old.email }
|
||||
// ^^^^^^^^ 'username' does not exist in User@v1
|
||||
}
|
||||
```
|
||||
|
||||
```lux
|
||||
type User @v2 {
|
||||
name: String,
|
||||
email: String
|
||||
|
||||
// ERROR: Migration missing required field
|
||||
from @v1 = { name: old.name }
|
||||
// ^ Missing 'email' field
|
||||
}
|
||||
```
|
||||
|
||||
```lux
|
||||
type User @v2 {
|
||||
name: String,
|
||||
age: Int
|
||||
|
||||
// ERROR: Type mismatch in migration
|
||||
from @v1 = { name: old.name, age: old.birthYear }
|
||||
// ^^^^^^^^^^^^^ Expected Int, found String
|
||||
}
|
||||
```
|
||||
|
||||
## Compatibility Checking
|
||||
|
||||
Lux tracks compatibility between versions:
|
||||
|
||||
| Change Type | Backward Compatible | Forward Compatible |
|
||||
|-------------|--------------------|--------------------|
|
||||
| Add optional field (with default) | Yes | Yes |
|
||||
| Add required field | No | Yes (with migration) |
|
||||
| Remove field | Yes (with migration) | No |
|
||||
| Rename field | No | No (need migration) |
|
||||
| Change field type | No | No (need migration) |
|
||||
|
||||
The compiler warns about breaking changes:
|
||||
|
||||
```lux
|
||||
type User @v1 {
|
||||
name: String,
|
||||
email: String
|
||||
}
|
||||
|
||||
type User @v2 {
|
||||
name: String
|
||||
// Warning: Removing 'email' is a breaking change
|
||||
// Existing v2 consumers expect this field
|
||||
}
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Always Version Production Types
|
||||
|
||||
```lux
|
||||
// Good: Versioned from the start
|
||||
type Order @v1 {
|
||||
id: String,
|
||||
items: List<Item>,
|
||||
total: Int
|
||||
}
|
||||
|
||||
// Bad: Unversioned type is hard to evolve
|
||||
type Order {
|
||||
id: String,
|
||||
items: List<Item>,
|
||||
total: Int
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Keep Migrations Simple
|
||||
|
||||
```lux
|
||||
// Good: Simple, direct mapping
|
||||
from @v1 = {
|
||||
name: old.name,
|
||||
email: old.email |> Option.getOrElse("")
|
||||
}
|
||||
|
||||
// Avoid: Complex logic in migrations
|
||||
from @v1 = {
|
||||
name: old.name,
|
||||
email: {
|
||||
// Don't put complex business logic here
|
||||
let domain = inferDomainFromName(old.name)
|
||||
let local = String.toLower(String.replace(old.name, " ", "."))
|
||||
local + "@" + domain
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Test Migrations
|
||||
|
||||
```lux
|
||||
fn testUserMigration(): Unit with {Test} = {
|
||||
let v1User = Schema.versioned("User", 1, { name: "Alice" })
|
||||
let v2User = Schema.migrate(v1User, 2)
|
||||
|
||||
Test.assertEqual(v2User.name, "Alice")
|
||||
Test.assertEqual(v2User.email, "unknown@example.com")
|
||||
}
|
||||
```
|
||||
|
||||
### 4. Document Breaking Changes
|
||||
|
||||
```lux
|
||||
type User @v3 {
|
||||
// BREAKING: 'name' split into firstName/lastName
|
||||
// Migration: name.split(" ")[0] -> firstName, name.split(" ")[1] -> lastName
|
||||
firstName: String,
|
||||
lastName: String,
|
||||
email: String,
|
||||
|
||||
from @v2 = { ... }
|
||||
}
|
||||
```
|
||||
|
||||
## Schema Module Reference
|
||||
|
||||
| Function | Description |
|
||||
|----------|-------------|
|
||||
| `Schema.versioned(typeName, version, value)` | Create a versioned value |
|
||||
| `Schema.getVersion(value)` | Get the version of a value |
|
||||
| `Schema.migrate(value, targetVersion)` | Migrate to a target version |
|
||||
| `Schema.isCompatible(v1, v2)` | Check if versions are compatible |
|
||||
|
||||
## Summary
|
||||
|
||||
Schema evolution in Lux provides:
|
||||
|
||||
- **Versioned types** with `@v1`, `@v2`, `@latest` annotations
|
||||
- **Explicit migrations** with `from @vN = { ... }` syntax
|
||||
- **Automatic migrations** for simple field additions with defaults
|
||||
- **Runtime operations** via the `Schema` module
|
||||
- **Compile-time safety** catching migration errors early
|
||||
- **Migration chaining** for multi-step upgrades
|
||||
|
||||
This system ensures your data can evolve safely over time, without breaking existing code or losing information.
|
||||
|
||||
## What's Next?
|
||||
|
||||
- [Tutorials](../tutorials/README.md) - Build real projects
|
||||
- [Standard Library Reference](../stdlib/README.md) - Complete API docs
|
||||
Reference in New Issue
Block a user