lux/docs/TESTING_DESIGN.md

# Lux Testing Framework Design

## Overview

Lux needs a native testing framework that leverages its effect system. The key insight: **testing is an effect** - assertions, test discovery, and reporting are all side effects.

## Survey of Testing Approaches

### 1. Rust Style (`#[test]` attributes)
```rust
#[test]
fn test_addition() {
    assert_eq!(2 + 2, 4);
}
```
**Pros:** Familiar, built-in, zero runtime overhead
**Cons:** Requires attribute/macro system

### 2. Elm Style (describe/test/expect)
```elm
suite = describe "Math"
    [ test "addition" <| \_ -> Expect.equal 4 (2 + 2)
    , test "subtraction" <| \_ -> Expect.equal 0 (2 - 2)
    ]
```
**Pros:** Readable, hierarchical
**Cons:** Verbose, needs custom runner

### 3. Zig Style (inline test blocks)
```zig
test "addition" {
    try std.testing.expect(2 + 2 == 4);
}
```
**Pros:** Inline with code, comptime evaluated
**Cons:** Needs language support

### 4. Property-Based (QuickCheck/Hypothesis)
```haskell
prop_reverse :: [Int] -> Bool
prop_reverse xs = reverse (reverse xs) == xs
```
**Pros:** Finds edge cases automatically
**Cons:** Complex to implement

### 5. Expect Tests (OCaml/Jane Street)
```ocaml
let%expect_test "hello" =
  print_endline "Hello";
  [%expect {| Hello |}]
```
**Pros:** Self-updating, snapshot testing
**Cons:** Requires tooling

## Recommended: Effect-Based Testing

Since Lux has effects, testing should be an effect. This is idiomatic and enables powerful features.

### Design

```lux
// Built-in Test effect
effect Test {
    fn assert(condition: Bool, message: String): Unit
    fn assertEqual<T>(expected: T, actual: T): Unit
    fn assertNotEqual<T>(a: T, b: T): Unit
    fn fail(message: String): Unit
    fn skip(reason: String): Unit
    fn todo(description: String): Unit
}

// Test declaration syntax (new keyword)
test "addition works" {
    Test.assertEqual(4, 2 + 2)
}

test "string concatenation" {
    let result = "hello" + " " + "world"
    Test.assertEqual("hello world", result)
}

// Grouped tests
describe "List operations" {
    test "map doubles values" {
        let input = [1, 2, 3]
        let output = List.map(input, fn(x) => x * 2)
        Test.assertEqual([2, 4, 6], output)
    }

    test "filter keeps matches" {
        let input = [1, 2, 3, 4, 5]
        let output = List.filter(input, fn(x) => x > 2)
        Test.assertEqual([3, 4, 5], output)
    }
}
```

### Why This Design?

1. **Idiomatic**: Testing is just another effect
2. **Mockable**: Can provide different Test handlers for different scenarios
3. **Parallel-safe**: Effect isolation prevents test interference
4. **Extensible**: Users can define custom assertion effects
5. **Property testing**: Can add `Test.forAll` for property-based testing

### Test Runner

```bash
# Run all tests in a file
lux test examples/math_test.lux

# Run all tests in directory
lux test tests/

# Run specific test
lux test tests/math.lux --filter "addition"

# Watch mode
lux test --watch
```

### Alternative: Annotation-Based (Simpler)

If we want simpler implementation:

```lux
// Functions prefixed with test_ are tests
fn test_addition(): Unit with {Test} = {
    Test.assertEqual(4, 2 + 2)
}

fn test_strings(): Unit with {Test} = {
    Test.assertEqual("hello", "hel" + "lo")
}
```

Run with:
```bash
lux test file.lux  # Finds all test_* functions
```

## Chosen Approach: Hybrid

Combine both for flexibility:

### Phase 1: Function-Based (Simple, Immediate)

```lux
// Any function starting with test_ is a test
fn test_addition(): Unit with {Test} =
    Test.assertEqual(4, 2 + 2)

fn test_factorial(): Unit with {Test} = {
    Test.assertEqual(1, factorial(0))
    Test.assertEqual(1, factorial(1))
    Test.assertEqual(120, factorial(5))
}

// Helper for multiple assertions
fn test_list_operations(): Unit with {Test} = {
    // Each assertion is independent
    Test.assertEqual([2, 4], List.map([1, 2], fn(x) => x * 2))
    Test.assertEqual([2, 3], List.filter([1, 2, 3], fn(x) => x > 1))
    Test.assertEqual(6, List.fold([1, 2, 3], 0, fn(a, b) => a + b))
}
```

### Phase 2: Block Syntax (Later)

```lux
test "descriptive name" {
    Test.assertEqual(4, 2 + 2)
}

describe "module name" {
    test "feature 1" { ... }
    test "feature 2" { ... }
}
```

## Implementation Plan

### Step 1: Test Effect

Add built-in `Test` effect to the type system:

```rust
// In types.rs, add Test effect
EffectDef {
    name: "Test".to_string(),
    operations: vec![
        EffectOpDef { name: "assert", params: vec![("cond", Bool), ("msg", String)], return_type: Unit },
        EffectOpDef { name: "assertEqual", params: vec![("expected", T), ("actual", T)], return_type: Unit },
        EffectOpDef { name: "fail", params: vec![("msg", String)], return_type: Unit },
    ],
}
```

### Step 2: Test Handler (Interpreter)

```rust
// In interpreter.rs
fn handle_test_effect(&mut self, op: &str, args: &[Value]) -> Result<Value, RuntimeError> {
    match op {
        "assert" => {
            let cond = args[0].as_bool()?;
            let msg = args[1].as_string()?;
            if !cond {
                self.test_failures.push(TestFailure { message: msg, .. });
            }
            Ok(Value::Unit)
        }
        "assertEqual" => {
            let expected = &args[0];
            let actual = &args[1];
            if expected != actual {
                self.test_failures.push(TestFailure {
                    message: format!("Expected {:?}, got {:?}", expected, actual),
                    ..
                });
            }
            Ok(Value::Unit)
        }
        // ...
    }
}
```

### Step 3: Test Discovery

```rust
// Find all test_* functions in a file
fn discover_tests(program: &Program) -> Vec<&FunctionDecl> {
    program.declarations.iter().filter_map(|d| {
        if let Declaration::Function(f) = d {
            if f.name.name.starts_with("test_") {
                return Some(f);
            }
        }
        None
    }).collect()
}
```

### Step 4: Test Runner CLI

```bash
lux test file.lux
```

Output:
```
Running tests in file.lux...

  test_addition ... OK
  test_factorial ... OK
  test_strings ... FAIL
    Expected: "hello world"
    Actual:   "helloworld"
    at file.lux:15

Results: 2 passed, 1 failed
```

## Property-Based Testing (Future)

```lux
// Properties are functions that return Bool
fn prop_reverse_twice<T>(xs: List<T>): Bool =
    List.reverse(List.reverse(xs)) == xs

fn prop_sort_idempotent(xs: List<Int>): Bool =
    List.sort(List.sort(xs)) == List.sort(xs)

// Test runner generates random inputs
test "reverse is involution" {
    Test.forAll(prop_reverse_twice)
}
```

## Expect Tests (Future)

```lux
test "json output" {
    let data = { name: "Alice", age: 30 }
    let json = Json.stringify(data)

    // This gets auto-updated by the test runner
    Test.expect(json, |||
        {"name":"Alice","age":30}
    |||)
}
```

## Summary

| Phase | Feature | Effort |
|-------|---------|--------|
| 1 | Test effect + test_* discovery | 1-2 days |
| 2 | Test runner CLI (`lux test`) | 1 day |
| 3 | Nice output formatting | 0.5 day |
| 4 | `test "name" {}` syntax | 1 day |
| 5 | `describe` blocks | 0.5 day |
| 6 | Property-based testing | 2-3 days |
| 7 | Expect/snapshot tests | 2 days |

**Recommendation:** Start with Phase 1-3 (function-based tests with CLI) for immediate value.