# Chapter 13: Schema Evolution

Data structures change over time. Fields get added, removed, or renamed. Types get split or merged. Without careful handling, these changes break systems—old data can't be read, services fail, migrations corrupt data.

Lux's **schema evolution** system makes these changes safe and automatic.

## The Problem

Consider a real scenario:

```lux
// Version 1: Simple user
type User {
    name: String
}

// Later, you need email addresses
type User {
    name: String,
    email: String   // Breaking change! Old data doesn't have this.
}
```

In most languages, this breaks everything. Existing users in your database don't have email addresses. Deserializing old data fails. Services crash.

Lux solves this with **versioned types** and **automatic migrations**.

## Versioned Types

Add a version annotation to any type:

```lux
// Version 1: Original definition
type User @v1 {
    name: String
}

// Version 2: Added email field
type User @v2 {
    name: String,
    email: String,

    // How to migrate from v1
    from @v1 = { name: old.name, email: "unknown@example.com" }
}

// Version 3: Split name into first/last
type User @v3 {
    firstName: String,
    lastName: String,
    email: String,

    // How to migrate from v2
    from @v2 = {
        firstName: String.split(old.name, " ") |> List.head |> Option.getOrElse(""),
        lastName: String.split(old.name, " ") |> List.tail |> List.head |> Option.getOrElse(""),
        email: old.email
    }
}
```

The `@latest` alias always refers to the most recent version:

```lux
type User @latest {
    firstName: String,
    lastName: String,
    email: String,

    from @v2 = { ... }
}

// These are equivalent:
fn createUser(first: String, last: String, email: String): User@latest = ...
fn createUser(first: String, last: String, email: String): User@v3 = ...
```

## Migration Syntax

### Basic Migration

```lux
type Config @v2 {
    theme: String,
    fontSize: Int,

    // 'old' refers to the v1 value
    from @v1 = {
        theme: old.theme,
        fontSize: 14  // New field with default
    }
}
```

### Computed Fields

```lux
type Order @v2 {
    items: List<Item>,
    total: Int,
    itemCount: Int,  // New computed field

    from @v1 = {
        items: old.items,
        total: old.total,
        itemCount: List.length(old.items)
    }
}
```

### Removing Fields

When removing fields, simply don't include them in the new version:

```lux
type Settings @v1 {
    theme: String,
    legacyMode: Bool,  // To be removed
    volume: Int
}

type Settings @v2 {
    theme: String,
    volume: Int,

    // legacyMode is dropped - just don't migrate it
    from @v1 = {
        theme: old.theme,
        volume: old.volume
    }
}
```

### Renaming Fields

```lux
type Product @v1 {
    name: String,
    cost: Int  // Old field name
}

type Product @v2 {
    name: String,
    price: Int,  // Renamed from 'cost'

    from @v1 = {
        name: old.name,
        price: old.cost  // Map old field to new name
    }
}
```

### Complex Transformations

```lux
type Address @v1 {
    fullAddress: String  // "123 Main St, New York, NY 10001"
}

type Address @v2 {
    street: String,
    city: String,
    state: String,
    zip: String,

    from @v1 = {
        let parts = String.split(old.fullAddress, ", ")
        {
            street: List.get(parts, 0) |> Option.getOrElse(""),
            city: List.get(parts, 1) |> Option.getOrElse(""),
            state: List.get(parts, 2)
                |> Option.map(fn(s: String): String => String.split(s, " ") |> List.head |> Option.getOrElse(""))
                |> Option.getOrElse(""),
            zip: List.get(parts, 2)
                |> Option.map(fn(s: String): String => String.split(s, " ") |> List.last |> Option.getOrElse(""))
                |> Option.getOrElse("")
        }
    }
}
```

## Working with Versioned Values

The `Schema` module provides runtime operations for versioned values:

### Creating Versioned Values

```lux
// Create a value tagged with a specific version
let userV1 = Schema.versioned("User", 1, { name: "Alice" })
let userV2 = Schema.versioned("User", 2, { name: "Alice", email: "alice@example.com" })
```

### Checking Versions

```lux
let user = Schema.versioned("User", 1, { name: "Alice" })
let version = Schema.getVersion(user)  // Returns 1

// Version-aware logic
if version < 2 then
    Console.print("Legacy user format")
else
    Console.print("Modern user format")
```

### Migrating Values

```lux
// Migrate to a specific version
let userV1 = Schema.versioned("User", 1, { name: "Alice" })
let userV2 = Schema.migrate(userV1, 2)  // Uses declared migration

let version = Schema.getVersion(userV2)  // Now 2

// Chain migrations (v1 -> v2 -> v3)
let userV3 = Schema.migrate(userV1, 3)  // Applies v1->v2, then v2->v3
```

## Auto-Generated Migrations

For simple changes, Lux can **automatically generate** migrations:

```lux
type Profile @v1 {
    name: String
}

// Adding a field with a default? Migration is auto-generated
type Profile @v2 {
    name: String,
    bio: String = ""  // Default value provided
}

// The compiler generates this for you:
// from @v1 = { name: old.name, bio: "" }
```

Auto-migration works for:
- Adding fields with default values
- Keeping existing fields unchanged

You must write explicit migrations for:
- Field renaming
- Field removal (to confirm intent)
- Type changes
- Computed/derived fields

## Practical Examples

### Example 1: API Response Versioning

```lux
type ApiResponse @v1 {
    status: String,
    data: String
}

type ApiResponse @v2 {
    status: String,
    data: String,
    meta: { timestamp: Int, version: String },

    from @v1 = {
        status: old.status,
        data: old.data,
        meta: { timestamp: 0, version: "legacy" }
    }
}

// Version-aware API client
fn handleResponse(raw: ApiResponse@v1): ApiResponse@v2 = {
    Schema.migrate(Schema.versioned("ApiResponse", 1, raw), 2)
}
```

### Example 2: Database Record Evolution

```lux
// Original schema
type Customer @v1 {
    name: String,
    address: String
}

// Split address into components
type Customer @v2 {
    name: String,
    street: String,
    city: String,
    country: String,

    from @v1 = {
        let parts = String.split(old.address, ", ")
        {
            name: old.name,
            street: List.get(parts, 0) |> Option.getOrElse(old.address),
            city: List.get(parts, 1) |> Option.getOrElse("Unknown"),
            country: List.get(parts, 2) |> Option.getOrElse("Unknown")
        }
    }
}

// Load and migrate on read
fn loadCustomer(id: String): Customer@v2 with {Database} = {
    let record = Database.query("SELECT * FROM customers WHERE id = ?", [id])
    let version = record.schema_version  // Stored version

    if version == 1 then
        let v1 = Schema.versioned("Customer", 1, {
            name: record.name,
            address: record.address
        })
        Schema.migrate(v1, 2)
    else
        { name: record.name, street: record.street, city: record.city, country: record.country }
}
```

### Example 3: Configuration Files

```lux
type AppConfig @v1 {
    debug: Bool,
    port: Int
}

type AppConfig @v2 {
    debug: Bool,
    port: Int,
    logLevel: String,  // New in v2

    from @v1 = {
        debug: old.debug,
        port: old.port,
        logLevel: if old.debug then "debug" else "info"
    }
}

type AppConfig @v3 {
    environment: String,  // Replaces debug flag
    port: Int,
    logLevel: String,

    from @v2 = {
        environment: if old.debug then "development" else "production",
        port: old.port,
        logLevel: old.logLevel
    }
}

// Load config with automatic migration
fn loadConfig(path: String): AppConfig@v3 with {File} = {
    let json = File.read(path)
    let parsed = Json.parse(json)
    let version = Json.getInt(parsed, "version") |> Option.getOrElse(1)

    match version {
        1 => {
            let v1 = Schema.versioned("AppConfig", 1, {
                debug: Json.getBool(parsed, "debug") |> Option.getOrElse(false),
                port: Json.getInt(parsed, "port") |> Option.getOrElse(8080)
            })
            Schema.migrate(v1, 3)
        },
        2 => {
            let v2 = Schema.versioned("AppConfig", 2, {
                debug: Json.getBool(parsed, "debug") |> Option.getOrElse(false),
                port: Json.getInt(parsed, "port") |> Option.getOrElse(8080),
                logLevel: Json.getString(parsed, "logLevel") |> Option.getOrElse("info")
            })
            Schema.migrate(v2, 3)
        },
        _ => {
            // Already v3
            {
                environment: Json.getString(parsed, "environment") |> Option.getOrElse("production"),
                port: Json.getInt(parsed, "port") |> Option.getOrElse(8080),
                logLevel: Json.getString(parsed, "logLevel") |> Option.getOrElse("info")
            }
        }
    }
}
```

### Example 4: Event Sourcing

```lux
// Event types evolve over time
type UserCreated @v1 {
    userId: String,
    name: String,
    timestamp: Int
}

type UserCreated @v2 {
    userId: String,
    name: String,
    email: String,
    createdAt: Int,  // Renamed from timestamp

    from @v1 = {
        userId: old.userId,
        name: old.name,
        email: "",  // Not captured in v1
        createdAt: old.timestamp
    }
}

// Process events regardless of version
fn processEvent(event: UserCreated@v1 | UserCreated@v2): Unit with {Console} = {
    let normalized = Schema.migrate(event, 2)  // Always work with v2
    Console.print("User created: " + normalized.name + " at " + toString(normalized.createdAt))
}
```

## Compile-Time Safety

The compiler catches schema evolution errors:

```lux
type User @v2 {
    name: String,
    email: String

    // ERROR: Migration references non-existent field
    from @v1 = { name: old.username, email: old.email }
    //                    ^^^^^^^^ 'username' does not exist in User@v1
}
```

```lux
type User @v2 {
    name: String,
    email: String

    // ERROR: Migration missing required field
    from @v1 = { name: old.name }
    //                           ^ Missing 'email' field
}
```

```lux
type User @v2 {
    name: String,
    age: Int

    // ERROR: Type mismatch in migration
    from @v1 = { name: old.name, age: old.birthYear }
    //                                ^^^^^^^^^^^^^ Expected Int, found String
}
```

## Compatibility Checking

Lux tracks compatibility between versions:

| Change Type | Backward Compatible | Forward Compatible |
|-------------|--------------------|--------------------|
| Add optional field (with default) | Yes | Yes |
| Add required field | No | Yes (with migration) |
| Remove field | Yes (with migration) | No |
| Rename field | No | No (need migration) |
| Change field type | No | No (need migration) |

The compiler warns about breaking changes:

```lux
type User @v1 {
    name: String,
    email: String
}

type User @v2 {
    name: String
    // Warning: Removing 'email' is a breaking change
    // Existing v2 consumers expect this field
}
```

## Best Practices

### 1. Always Version Production Types

```lux
// Good: Versioned from the start
type Order @v1 {
    id: String,
    items: List<Item>,
    total: Int
}

// Bad: Unversioned type is hard to evolve
type Order {
    id: String,
    items: List<Item>,
    total: Int
}
```

### 2. Keep Migrations Simple

```lux
// Good: Simple, direct mapping
from @v1 = {
    name: old.name,
    email: old.email |> Option.getOrElse("")
}

// Avoid: Complex logic in migrations
from @v1 = {
    name: old.name,
    email: {
        // Don't put complex business logic here
        let domain = inferDomainFromName(old.name)
        let local = String.toLower(String.replace(old.name, " ", "."))
        local + "@" + domain
    }
}
```

### 3. Test Migrations

```lux
fn testUserMigration(): Unit with {Test} = {
    let v1User = Schema.versioned("User", 1, { name: "Alice" })
    let v2User = Schema.migrate(v1User, 2)

    Test.assertEqual(v2User.name, "Alice")
    Test.assertEqual(v2User.email, "unknown@example.com")
}
```

### 4. Document Breaking Changes

```lux
type User @v3 {
    // BREAKING: 'name' split into firstName/lastName
    // Migration: name.split(" ")[0] -> firstName, name.split(" ")[1] -> lastName
    firstName: String,
    lastName: String,
    email: String,

    from @v2 = { ... }
}
```

## Schema Module Reference

| Function | Description |
|----------|-------------|
| `Schema.versioned(typeName, version, value)` | Create a versioned value |
| `Schema.getVersion(value)` | Get the version of a value |
| `Schema.migrate(value, targetVersion)` | Migrate to a target version |
| `Schema.isCompatible(v1, v2)` | Check if versions are compatible |

## Summary

Schema evolution in Lux provides:

- **Versioned types** with `@v1`, `@v2`, `@latest` annotations
- **Explicit migrations** with `from @vN = { ... }` syntax
- **Automatic migrations** for simple field additions with defaults
- **Runtime operations** via the `Schema` module
- **Compile-time safety** catching migration errors early
- **Migration chaining** for multi-step upgrades

This system ensures your data can evolve safely over time, without breaking existing code or losing information.

## What's Next?

- [Tutorials](../tutorials/README.md) - Build real projects
- [Standard Library Reference](../stdlib/README.md) - Complete API docs