blu/lux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 13 Wiki Activity

fix: correct benchmark documentation with honest measurements

Browse Source 
Previous benchmark claims were incorrect:
- Claimed Lux "beats Rust and Zig" - this was false
- C backend has bugs and wasn't actually working
- Comparison used unfair optimization flags

Actual measurements (fib 35):
- C (gcc -O3): 0.028s
- Rust (-C opt-level=3 -C lto): 0.041s
- Zig (ReleaseFast): 0.046s
- Lux (interpreter): 0.254s

Lux is ~9x slower than C, which is expected for a
tree-walking interpreter. This is honest and comparable
to other interpreted languages without JIT.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Brandon Lucas
2026-02-16 05:03:36 -05:00
parent dfcfda1f48
commit 0cf8f2a4a2
2 changed files with 178 additions and 196 deletions
Download Patch File Download Diff File Expand all files Collapse all files

207
benchmarks/RESULTS.md
View File

@@ -1,148 +1,127 @@
# Lux Language Benchmark Results
Generated: Sat Feb 14 2026
Generated: Feb 16 2026
## Environment
- **Platform**: Linux x86_64
- **Lux**: Compiled to native via C (gcc -O2)
- **Rust**: rustc 1.92.0 with -O
- **C**: gcc -O2
- **Go**: go 1.25.5
- **Node.js**: v16.20.2 (V8 JIT)
- **Bun**: 1.3.5 (JavaScriptCore)
- **Python**: 3.13.5
- **Platform**: Linux x86_64 (NixOS)
- **Lux**: Tree-walking interpreter (Rust-based)
- **C**: gcc with -O3
- **Rust**: rustc with -C opt-level=3 -C lto
- **Zig**: zig with -O ReleaseFast
## Current Status
**Important**: Lux currently runs as an **interpreted language**. The C compilation backend exists but has bugs that prevent it from working on all programs. The numbers below reflect interpreter performance.
## Summary
Lux compiles to native code via C and achieves performance comparable to Rust and C, while being significantly faster than interpreted/JIT languages.
| Benchmark | C (gcc -O3) | Rust | Zig | **Lux (interp)** | Ratio |
|-----------|-------------|------|-----|------------------|-------|
| Fibonacci (35) | 0.028s | 0.041s | 0.046s | **0.254s** | ~9x slower than C |
| Benchmark | Lux | Rust | C | Go | Node.js | Bun | Python |
|-----------|-----|------|---|-----|---------|-----|--------|
| Fibonacci (fib 35) | 0.015s | 0.018s | 0.014s | 0.041s | 0.110s | 0.065s | 0.928s |
| Prime Counting (10k) | 0.002s | 0.002s | 0.001s | 0.002s | 0.034s | 0.012s | 0.023s |
| Sum Loop (10M) | 0.004s | 0.002s | 0.004s | 0.009s | 0.042s | 0.023s | 0.384s |
| Ackermann (3,10) | 0.020s | 0.029s | 0.020s | 0.107s | 0.207s | 0.121s | 5.716s |
| Selection Sort (1k) | 0.003s | 0.002s | 0.001s | 0.002s | 0.039s | 0.021s | 0.032s |
| List Operations (10k) | 0.002s | - | - | - | 0.030s | 0.016s | - |
### Honest Assessment
### Performance Rankings (Average)
Lux as an interpreter is approximately:
- **9x slower than C** (gcc -O3)
- **6x slower than Rust** (with full optimizations)
- **5.5x slower than Zig** (ReleaseFast)
- **Comparable to other interpreted languages** (faster than Python, similar to Lua)
1. **C** - Baseline (fastest)
2. **Rust** - ~1.0-1.5x of C
3. **Lux** - ~1.0-1.5x of C (matches Rust)
4. **Go** - ~2-5x of C
5. **Bun** - ~10-20x of C
6. **Node.js** - ~15-30x of C
7. **Python** - ~30-300x of C
This is expected for a tree-walking interpreter. The focus of Lux is on:
1. **Developer experience** - effect system, type safety, good error messages
2. **Correctness** - not raw performance
3. **Future compilation** - the C backend will eventually provide native performance
## Benchmark Details
### 1. Fibonacci (fib 35)
### Fibonacci (fib 35)
**Tests**: Recursive function calls
| Language | Time (s) | vs Lux |
|----------|----------|--------|
| C | 0.014 | 0.93x |
| Lux | 0.015 | 1.00x |
| Rust | 0.018 | 1.20x |
| Go | 0.041 | 2.73x |
| Bun | 0.065 | 4.33x |
| Node.js | 0.110 | 7.33x |
| Python | 0.928 | 61.87x |
```lux
fn fib(n: Int): Int = {
if n <= 1 then n
else fib(n - 1) + fib(n - 2)
}
```
Lux matches C and beats Rust in this recursive function call benchmark.
| Language | Time | Notes |
|----------|------|-------|
| C (gcc -O3) | 0.028s | Baseline |
| Rust (-C opt-level=3 -C lto) | 0.041s | ~1.5x slower than C |
| Zig (ReleaseFast) | 0.046s | ~1.6x slower than C |
| **Lux (interpreter)** | 0.254s | ~9x slower than C |
### 2. Prime Counting (up to 10000)
**Tests**: Loops and conditionals
**Analysis**: Lux's interpreter performance is typical for a tree-walking interpreter. The overhead comes from:
- AST traversal
- Dynamic dispatch
- No JIT compilation
- Reference counting
| Language | Time (s) | vs Lux |
|----------|----------|--------|
| C | 0.001 | 0.50x |
| Lux | 0.002 | 1.00x |
| Rust | 0.002 | 1.00x |
| Go | 0.002 | 1.00x |
| Bun | 0.012 | 6.00x |
| Python | 0.023 | 11.50x |
| Node.js | 0.034 | 17.00x |
## Why Lux is Slower (For Now)
Lux matches Rust and Go for tight loop-based code.
### Tree-Walking Interpreter
Lux currently uses a tree-walking interpreter written in Rust. This means:
- Every expression is evaluated by traversing the AST
- No machine code generation
- No JIT compilation
- Every operation goes through interpreter dispatch
### 3. Sum Loop (10 million iterations)
**Tests**: Tight numeric loop (tail-recursive in Lux)
### C Backend Status
Lux has a C compilation backend (`lux compile`) that generates C code, but it currently has bugs:
- Some standard library functions have issues in generated code
- Not all programs compile successfully
- When working, it would provide C-level performance
| Language | Time (s) | vs Lux |
|----------|----------|--------|
| Rust | 0.002 | 0.50x |
| C | 0.004 | 1.00x |
| Lux | 0.004 | 1.00x |
| Go | 0.009 | 2.25x |
| Bun | 0.023 | 5.75x |
| Node.js | 0.042 | 10.50x |
| Python | 0.384 | 96.00x |
## Future Performance Improvements
Lux's tail-call optimization achieves C-level performance.
Planned improvements that would make Lux faster:
### 4. Ackermann (3, 10)
**Tests**: Deep recursion (stack-heavy)
1. **Fix C backend** - Enable native compilation for all programs
2. **Bytecode VM** - Intermediate representation faster than tree-walking
3. **JIT compilation** - Runtime code generation for hot paths
4. **Optimization passes** - Inlining, constant folding, etc.
| Language | Time (s) | vs Lux |
|----------|----------|--------|
| C | 0.020 | 1.00x |
| Lux | 0.020 | 1.00x |
| Rust | 0.029 | 1.45x |
| Go | 0.107 | 5.35x |
| Bun | 0.121 | 6.05x |
| Node.js | 0.207 | 10.35x |
| Python | 5.716 | 285.80x |
## Running Benchmarks
Lux matches C and beats Rust in deep recursion, demonstrating excellent function call overhead.
```bash
# Enter nix development environment
nix develop
### 5. Selection Sort (1000 elements)
**Tests**: Sorting algorithm simulation
# Run Lux benchmark (interpreter)
time cargo run --release -- benchmarks/fib.lux
| Language | Time (s) | vs Lux |
|----------|----------|--------|
| C | 0.001 | 0.33x |
| Go | 0.002 | 0.67x |
| Rust | 0.002 | 0.67x |
| Lux | 0.003 | 1.00x |
| Bun | 0.021 | 7.00x |
| Python | 0.032 | 10.67x |
| Node.js | 0.039 | 13.00x |
# Compare with other languages
nix-shell -p gcc rustc zig --run '
gcc -O3 benchmarks/fib.c -o /tmp/fib_c && time /tmp/fib_c
rustc -C opt-level=3 -C lto benchmarks/fib.rs -o /tmp/fib_rust && time /tmp/fib_rust
zig build-exe benchmarks/fib.zig -O ReleaseFast && time ./fib
'
```
### 6. List Operations (10000 elements)
**Tests**: map/filter/fold on functional lists with closures
## Comparison Context
| Language | Time (s) | vs Lux |
|----------|----------|--------|
| Lux | 0.002 | 1.00x |
| Bun | 0.016 | 8.00x |
| Node.js | 0.030 | 15.00x |
For context, here's how other interpreted languages perform on similar benchmarks:
This benchmark showcases Lux's functional programming capabilities with FBIP optimization:
- **20,006 allocations, 20,006 frees** (no memory leaks)
- **2 FBIP reuses, 0 copies** (efficient memory reuse)
| Language | Typical fib(35) time | Type |
|----------|---------------------|------|
| C | ~0.03s | Compiled |
| Rust | ~0.04s | Compiled |
| Zig | ~0.05s | Compiled |
| Go | ~0.05s | Compiled |
| Java (JIT warmed) | ~0.05s | JIT Compiled |
| **Lux** | ~0.25s | Interpreted |
| Lua (LuaJIT) | ~0.15s | JIT Compiled |
| JavaScript (V8) | ~0.20s | JIT Compiled |
| Python | ~3.0s | Interpreted |
| Ruby | ~1.5s | Interpreted |
## Key Observations
Lux performs well for an interpreter without JIT compilation.
1. **Native Performance**: Lux consistently matches or beats Rust and C across benchmarks
2. **Functional Efficiency**: Despite functional patterns (recursion, immutability), Lux compiles to efficient imperative code
3. **Deep Recursion**: Lux excels at Ackermann, matching C and beating Rust by 45%
4. **vs JavaScript**: Lux is **7-15x faster than Node.js** and **4-8x faster than Bun**
5. **vs Python**: Lux is **10-285x faster than Python**
6. **vs Go**: Lux is **2-5x faster than Go** in most benchmarks
7. **Zero Memory Leaks**: Reference counting ensures all allocations are freed
## Note on Previous Benchmark Claims
## Compilation Strategy
Earlier versions of this document made claims about Lux "beating Rust and Zig." Those claims were incorrect:
- The C backend was not actually working
- The benchmarks were not run fairly
- The comparison methodology was flawed
Lux uses a sophisticated compilation pipeline:
1. Parse Lux source code
2. Type inference and checking
3. Generate optimized C code with:
- Reference counting for memory management
- FBIP (Functional But In-Place) optimization
- Tail-call optimization
- Closure conversion
4. Compile C code with gcc -O2
This approach combines the ergonomics of a high-level functional language with the performance of systems languages.
This document now reflects honest, reproducible measurements.