lux/docs/OVERVIEW.md

# Lux Language Overview

## What is Lux?

Lux is a statically-typed functional programming language with **algebraic effects** as a first-class feature. It makes side effects explicit, trackable, and testable.

## What Can You Do With It?

### Currently Working

```lux
// Functions with type inference
fn factorial(n: Int): Int =
    if n <= 1 then 1 else n * factorial(n - 1)

// Higher-order functions
fn apply(f: fn(Int): Int, x: Int): Int = f(x)
fn double(x: Int): Int = x * 2
let result = apply(double, 21)  // 42

// Lambdas and closures
let add = fn(a: Int, b: Int): Int => a + b
let addFive = fn(x: Int): Int => add(5, x)

// Pattern matching
fn describe(n: Int): String =
    match n {
        0 => "zero",
        1 => "one",
        _ => "many"
    }

// Records
let person = { name: "Alice", age: 30 }
let age = person.age

// Tuples
let point = (10, 20)

// Lists
let numbers = [1, 2, 3, 4, 5]

// Pipe operator
let result = 5 |> double |> addOne  // (5 * 2) + 1 = 11

// Built-in effects (Console, Fail)
Console.print("Hello, world!")

// Custom effects
effect Logger {
    fn log(level: String, msg: String): Unit
}

// Effect handlers
handler consoleLogger: Logger {
    fn log(level, msg) = Console.print("[" + level + "] " + msg)
}

// Running with handlers
fn greet(name: String): Unit with {Logger} =
    Logger.log("info", "Hello, " + name)

run greet("Alice") with { Logger = consoleLogger }
```

### Standard Library (Built-in)

```lux
// List operations
List.map([1, 2, 3], fn(x: Int): Int => x * 2)    // [2, 4, 6]
List.filter([1, 2, 3, 4], fn(x: Int): Bool => x > 2)  // [3, 4]
List.fold([1, 2, 3], 0, fn(acc: Int, x: Int): Int => acc + x)  // 6
List.head([1, 2, 3])      // Some(1)
List.tail([1, 2, 3])      // Some([2, 3])
List.concat([1, 2], [3])  // [1, 2, 3]
List.reverse([1, 2, 3])   // [3, 2, 1]
List.length([1, 2, 3])    // 3
List.get([1, 2, 3], 0)    // Some(1)
List.range(0, 5)          // [0, 1, 2, 3, 4]

// String operations
String.split("a,b,c", ",")       // ["a", "b", "c"]
String.join(["a", "b"], "-")     // "a-b"
String.trim("  hello  ")         // "hello"
String.contains("hello", "ell")  // true
String.replace("hi", "i", "ey")  // "hey"
String.length("hello")           // 5
String.chars("hi")               // ['h', 'i']
String.lines("a\nb")             // ["a", "b"]

// Option operations
let x = Some(42)
let y = None
Option.map(x, fn(n: Int): Int => n * 2)  // Some(84)
Option.flatMap(x, fn(n: Int): Option<Int> => Some(n + 1))  // Some(43)
Option.getOrElse(y, 0)  // 0
Option.isSome(x)        // true
Option.isNone(y)        // true

// Result operations
let ok = Ok(42)
let err = Err("failed")
Result.map(ok, fn(n: Int): Int => n * 2)  // Ok(84)
Result.getOrElse(err, 0)  // 0
Result.isOk(ok)           // true
Result.isErr(err)         // true

// Utility functions
print("Hello")      // prints to stdout
toString(42)        // "42"
typeOf([1, 2, 3])   // "List"
```

### Also Working

```lux
// Generic type parameters
fn map<T, U>(f: fn(T): U, list: List<T>): List<U> = ...
fn identity<T>(x: T): T = x

// String interpolation
let name = "Alice"
Console.print("Hello, {name}!")  // Hello, Alice!
Console.print("1 + 2 = {1 + 2}") // 1 + 2 = 3

// File effects
let content = File.read("config.txt")
File.write("output.txt", "Hello!")
let exists = File.exists("file.lux")

// HTTP client effects
let response = Http.get("https://api.example.com/data")
Http.post("https://api.example.com/submit", "{\"key\": \"value\"}")

// HTTP server effects
HttpServer.listen(8080)
let req = HttpServer.accept()  // Returns { method, path, body, headers }
HttpServer.respond(200, "Hello, World!")
HttpServer.stop()

// Random effects
let n = Random.int(1, 100)
let coin = Random.bool()

// Time effects
let now = Time.now()
Time.sleep(1000)  // milliseconds

// JSON parsing
let obj = Json.parse("{\"name\": \"Alice\"}")
let str = Json.stringify(obj)

// Module system
import mymodule
import utils/helpers as h
import math.{sqrt, abs}
import prelude.*
```

### Also Working

```lux
// Behavioral types with compile-time verification
fn factorial(n: Int): Int is pure is deterministic is total =
    if n <= 1 then 1 else n * factorial(n - 1)

fn add(a: Int, b: Int): Int is commutative = a + b

fn absolute(x: Int): Int is idempotent =
    if x < 0 then 0 - x else x

// Schema evolution with version tracking
fn processV2(data: Int @v2): Int = data * 2
let value: Int @v2 = 42
let result = processV2(value)

// Version constraints
fn processModern(x: Int @v2+): Int = x  // v2 or later
fn processAny(x: Int @latest): Int = x  // any version
```

### Planned (Not Yet Fully Implemented)

- **Auto-migration Generation**: Migration bodies stored, execution pending

---

## Primary Use Cases

### 1. Learning Effect Systems
Lux is an excellent educational tool for understanding algebraic effects without the complexity of Haskell's monad transformers or the academic syntax of languages like Koka.

### 2. Testable Application Code
Effects make dependencies explicit. Swap handlers for testing:

```lux
// Production
run app() with { Database = postgres, Http = realHttp }

// Testing
run app() with { Database = mockDb, Http = mockHttp }
```

### 3. Domain Modeling
Explicit effects document what code can do:

```lux
fn processOrder(order: Order): Receipt with {Database, Email, Logger}
// ^ The signature tells you exactly what side effects this function performs
```

### 4. Prototyping
Quick iteration with type inference and a REPL.

---

## Pros and Cons

### Pros

| Advantage | Description |
|-----------|-------------|
| **Explicit Effects** | Function signatures show what side effects are possible |
| **Testability** | Swap effect handlers for mocking—no dependency injection frameworks |
| **Type Safety** | Static types catch errors at compile time |
| **Type Inference** | Write less type annotations, compiler figures it out |
| **Clean Syntax** | ML-family inspired, minimal boilerplate |
| **Pattern Matching** | Destructure data elegantly |
| **Immutable by Default** | Easier to reason about |
| **REPL** | Interactive development |

### Cons

| Limitation | Description |
|------------|-------------|
| **No Package Manager** | Can't share/publish packages yet |
| **New Paradigm** | Effects require learning new concepts |
| **Small Ecosystem** | No community packages yet |
| **Early Stage** | Bugs likely, features incomplete |

---

## Complexity Assessment

### Conceptual Complexity

| Concept | Difficulty | Notes |
|---------|------------|-------|
| Basic syntax | Easy | Similar to other ML-family languages |
| Functions | Easy | Standard functional style |
| Pattern matching | Easy | If you know any FP language |
| Type system | Medium | Hindley-Milner inference helps |
| Effects | Medium | New concept, but simpler than monads |
| Handlers | Medium | Requires understanding of continuations |

### Comparison to Other Languages

| Language | Complexity | Comparison to Lux |
|----------|------------|-------------------|
| Python | Simpler | No types, no effect tracking |
| TypeScript | Similar | Lux has effects, TS has larger ecosystem |
| Elm | Similar | Both pure FP, Lux has general effects |
| Haskell | More Complex | Monads harder than algebraic effects |
| Koka | Similar | Koka more academic, Lux more practical syntax |
| Rust | More Complex | Ownership adds significant complexity |

### Learning Curve

**Beginner** (1-2 hours):
- Basic expressions, functions, let bindings
- If/else, pattern matching
- REPL usage

**Intermediate** (1-2 days):
- Custom types and records
- Higher-order functions
- Built-in effects (Console)

**Advanced** (1 week):
- Custom effect definitions
- Effect handlers
- Understanding when to use effects vs. regular functions

---

## When to Use Lux

### Good Fit

- Learning algebraic effects
- Prototyping with explicit effect tracking
- Small tools where testability matters
- Teaching functional programming concepts

### Not a Good Fit (Yet)

- Large production applications (early stage)
- Performance-critical code (C backend working, but no advanced optimizations)
- Web frontend development (no JS compilation)
- Systems programming (no low-level control)

---

## Example Session

```
$ cargo run
Lux v0.1.0
Type :help for help, :quit to exit

lux> let x = 42
lux> x * 2
84
lux> fn greet(name: String): Unit with {Console} = Console.print("Hello, " + name)
lux> greet("World")
Hello, World
()
lux> let nums = [1, 2, 3]
lux> nums
[1, 2, 3]
lux> :quit
```

---

## Architecture

```
Source Code
    │
    ▼
┌─────────┐
│  Lexer  │  → Tokens
└─────────┘
    │
    ▼
┌─────────┐
│ Parser  │  → AST
└─────────┘
    │
    ▼
┌─────────────┐
│ Type Checker│  → Typed AST + Effect Tracking
└─────────────┘
    │
    ├─────────────────────────┐
    ▼                         ▼
┌─────────────┐       ┌──────────────┐
│ Interpreter │       │  C Backend   │
│ (default)   │       │  (compile)   │
└─────────────┘       └──────────────┘
    │                         │
    ▼                         ▼
Values + Effects        C Code → GCC/Clang
```

---

## Future Roadmap

**Complete:**
- ✅ Standard Library (List, String, Option, Result, Math, JSON)
- ✅ Module System (imports, exports, aliases, selective imports)
- ✅ LSP Server (diagnostics, hover, completions)
- ✅ Generics and String Interpolation
- ✅ File/HTTP/Random/Time/Process Effects
- ✅ HTTP Server Effect (listen, accept, respond, stop)
- ✅ Test Effect (native testing framework)
- ✅ Console.readLine and Console.readInt
- ✅ C Backend (basic functions, Console.print)
- ✅ C Backend closures and pattern matching
- ✅ C Backend lists (all 16 operations)
- ✅ C Backend reference counting (lists, boxed values)
- ✅ Watch mode / hot reload
- ✅ Formatter

**In Progress:**
1. **Schema Evolution** - Type system integration, auto-migration
2. **Error Message Quality** - Context lines shown, suggestions partial
3. **Memory Management** - RC working for lists/boxed, closures/ADTs pending

**Planned:**
4. **SQL Effect** - Database access
5. **Package Manager** - Share code (manifest parsing exists)
6. **JavaScript Backend** - Run in browsers
7. **Behavioral Type Verification** - Total, idempotent, deterministic checking