Files
lux/docs/STRESS_TEST_FINDINGS.md
Brandon Lucas 07a35f1829 docs: add stress test findings with identified bugs and gaps
Documents findings from systematic testing of parser, typechecker, and REPL:
- 5 critical bugs (record equality, invalid escapes, unknown effects, etc.)
- Parser missing features (hex literals, list patterns, guard clauses)
- Poor error messages for common mistakes
- REPL issues (:type doesn't show type, interpolation syntax)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-14 02:20:41 -05:00

174 lines
5.5 KiB
Markdown

# Lux Stress Test Findings
This document summarizes issues found during systematic stress testing of the Lux parser, typechecker, and REPL.
## Critical Bugs
### 1. Record Equality Returns False for Equal Records
```lux
let r1 = { x: 1, y: 2 }
let r2 = { x: 1, y: 2 }
r1 == r2 // Returns false! Should be true
```
**Impact**: High - breaks expected semantics for record comparison
**Location**: Likely in interpreter's equality logic for `Value::Record`
### 2. Invalid Escape Sequences Silently Accepted
```lux
"\z\q\w" // Returns "zqw" - backslashes silently dropped
"\x41" // Returns "x41" - hex escapes not supported but no error
"\u0041" // Returns "u0041" - unicode escapes not supported but no error
```
**Impact**: High - users expect invalid escapes to error
**Fix**: Parser should reject unknown escape sequences
### 3. Unknown Effects Silently Accepted in Declarations
```lux
fn test(): Unit with {CompletelyFakeEffect} = () // No error!
```
**Impact**: Medium - typos in effect names not caught until call site
**Fix**: Validate effect names during function declaration checking
### 4. No Forward References for Mutual Recursion
```lux
fn isEven(n: Int): Bool = if n == 0 then true else isOdd(n - 1) // Error: isOdd undefined
fn isOdd(n: Int): Bool = if n == 0 then false else isEven(n - 1)
```
**Impact**: Medium - common pattern not supported
**Fix**: Two-pass type checking or explicit forward declarations
### 5. Circular Type Definitions Silently Fail
```lux
type A = B
type B = A
// No output, no error
```
**Impact**: Medium - should produce clear error about circular definition
## Parser Issues
### Missing Features
| Feature | Status | Notes |
|---------|--------|-------|
| Hex literals (`0x10`) | Not supported | Generic parse error |
| Binary literals (`0b101`) | Not supported | Generic parse error |
| Octal literals (`0o77`) | Not supported | Generic parse error |
| Scientific notation (`1e10`) | Not supported | Generic parse error |
| Leading dot floats (`.5`) | Not supported | Generic parse error |
| Trailing dot floats (`1.`) | Not supported | Generic parse error |
| Underscore separators (`1_000_000`) | **Supported** | Works correctly |
| List patterns in match (`[a, b, c]`) | Not supported | Parse error |
| Guard clauses (`x if x > 0 =>`) | Not working | Shows redundant pattern error |
### Poor Error Messages
| Input | Current Error | Better Error |
|-------|---------------|--------------|
| `[1 2 3]` | "Expected declaration" | "Missing comma between list elements" |
| `{ x 1 }` | "Undefined variable: x" | "Missing colon in record field" |
| `funtion foo()` | "Expected declaration" | "Unknown keyword 'funtion'. Did you mean 'fn'?" |
| `(1 + 2` | No error | "Unclosed parenthesis" |
| `{ x: 1` | No error | "Unclosed brace" |
| `[1,, 2]` | No output | "Unexpected comma" |
### Identifier Issues
- Double underscore identifiers (`__name`) not allowed
- Unicode identifiers (`café`) partially work but may have issues
## Type Checker Issues
### Good Error Messages
- Type mismatches: Clear "Cannot unify X with Y"
- Undefined variables: Includes "Did you mean..." suggestions
- Wrong argument count: Clear arity mismatch message
- Non-exhaustive match: Lists uncovered patterns
- Effect mismatches: Shows expected vs actual effects
- Undefined record field: Clear "Record has no field"
- Calling non-function: Clear type mismatch
### Issues
- `:type 42` shows "(type checking passed)" but not the actual type
- No type inference for function parameters - must be annotated
## Runtime Issues
### Division Handling
- Integer `/0`: Proper "Division by zero" error
- Integer `%0`: Proper "Modulo by zero" error
- Float `/0.0`: Returns `inf` (IEEE 754 compliant)
### Recursion
- Deep recursion (10,000 calls): Works! Good tail call optimization
- Infinite recursion: Eventually times out (expected)
### Missing Stdlib Functions
- `String.get(s, i)` - not implemented
- `String.charAt(s, i)` - not implemented
## REPL Issues
### Working Well
- State persistence between lines
- Function redefinition
- Multi-line input
- `:help`, `:quit`, `:load` commands
- Tab completion (noted in help)
### Issues
- `:type expr` doesn't show the type, just "(type checking passed)"
- No `:ast` or `:tokens` debugging commands
- String interpolation syntax is `{x}` not `${x}` (unexpected)
- `"Hello {name}"` works
- `"Hello ${name}"` outputs `"Hello $<value>"` (keeps literal `$`)
## Comparison & Equality
| Type | `==` Works? |
|------|-------------|
| Int | Yes |
| Float | Yes |
| Bool | Yes |
| String | Yes |
| List | Yes |
| Tuple | Yes |
| Option/Result | Yes |
| **Record** | **NO (BUG)** |
| Function | No (expected) |
## Short-Circuit Evaluation
Works correctly:
- `true || crash()` - doesn't call `crash()`
- `false && crash()` - doesn't call `crash()`
- `if true then 1 else crash()` - doesn't call `crash()`
## Recommendations
### High Priority
1. Fix record equality
2. Error on invalid escape sequences
3. Validate effect names at declaration time
### Medium Priority
4. Add forward reference support for mutual recursion
5. Improve parser error messages with context
6. Add list pattern matching support
7. Error on circular type definitions
### Low Priority
8. Add hex/binary/octal literal support
9. Add scientific notation support
10. Fix `:type` command to show actual type
11. Document actual string interpolation syntax
## Test Commands Used
```bash
# Run stress tests
./target/release/lux test examples/test_math.lux
# Test specific edge case
echo 'let r1 = {x: 1}; let r2 = {x: 1}; r1 == r2' | ./target/release/lux
```