blu/lux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 13 Wiki Activity

feat: add property-based testing framework

Browse Source 
Implements property-based testing infrastructure:

stdlib/testing.lux:
- Generators: genInt, genIntList, genString, genBool, etc.
- Shrinking helpers: shrinkInt, shrinkList, shrinkString
- Property helpers: isSorted, sameElements

examples/property_testing.lux:
- 10 property tests demonstrating the framework
- Tests for: involution, commutativity, associativity, identity
- 100 iterations per property with random inputs

docs/guide/14-property-testing.md:
- Complete guide to property-based testing
- Generator patterns and common properties
- Best practices and examples

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Brandon Lucas
2026-02-16 04:39:50 -05:00
parent 87c1fb1bbd
commit b02807ebf4
4 changed files with 748 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

300
docs/guide/14-property-testing.md Normal file
View File

@@ -0,0 +1,300 @@
# Property-Based Testing
Property-based testing is a powerful testing technique where you define properties that should hold for all inputs, and the testing framework generates random inputs to verify those properties. This guide shows how to use property-based testing in Lux.
## Overview
Instead of writing tests with specific inputs:
```lux
fn test_reverse(): Unit with {Test} = {
Test.assertEqual([3, 2, 1], List.reverse([1, 2, 3]))
Test.assertEqual([], List.reverse([]))
}
```
Property-based testing verifies general properties:
```lux
// Property: Reversing a list twice gives back the original list
fn testReverseInvolutive(n: Int): Bool with {Console, Random} = {
if n <= 0 then true
else {
let xs = genIntList(0, 100, 20)
if List.reverse(List.reverse(xs)) == xs then
testReverseInvolutive(n - 1)
else
false
}
}
```
## Generators
Generators create random test data. The `Random` effect provides the building blocks.
### Basic Generators
```lux
// Generate random integer in range
fn genInt(min: Int, max: Int): Int with {Random} =
Random.int(min, max)
// Generate random boolean
fn genBool(): Bool with {Random} =
Random.bool()
// Generate random float
fn genFloat(): Float with {Random} =
Random.float()
```
### String Generators
```lux
let CHARS = "abcdefghijklmnopqrstuvwxyz"
// Generate random character
fn genChar(): String with {Random} = {
let idx = Random.int(0, 25)
String.substring(CHARS, idx, idx + 1)
}
// Generate random string up to maxLen characters
fn genString(maxLen: Int): String with {Random} = {
let len = Random.int(0, maxLen)
genStringHelper(len)
}
fn genStringHelper(len: Int): String with {Random} = {
if len <= 0 then ""
else genChar() + genStringHelper(len - 1)
}
```
### List Generators
```lux
// Generate random list of integers
fn genIntList(min: Int, max: Int, maxLen: Int): List<Int> with {Random} = {
let len = Random.int(0, maxLen)
genIntListHelper(min, max, len)
}
fn genIntListHelper(min: Int, max: Int, len: Int): List<Int> with {Random} = {
if len <= 0 then []
else List.concat([Random.int(min, max)], genIntListHelper(min, max, len - 1))
}
```
## Writing Property Tests
### Pattern: Recursive Test Function
The most common pattern is a recursive function that runs N iterations:
```lux
fn testProperty(n: Int, count: Int): Bool with {Console, Random} = {
if n <= 0 then {
Console.print(" PASS property_name (" + toString(count) + " tests)")
true
} else {
// Generate random inputs
let x = genInt(0, 100)
let y = genInt(0, 100)
// Check property
if x + y == y + x then
testProperty(n - 1, count)
else {
Console.print(" FAIL property_name")
Console.print(" Counterexample: x=" + toString(x) + ", y=" + toString(y))
false
}
}
}
```
### Common Properties
Here are properties commonly verified in property-based testing:
**Involution** - Applying a function twice returns the original value:
```lux
// f(f(x)) == x
List.reverse(List.reverse(xs)) == xs
```
**Idempotence** - Applying a function multiple times has the same effect as once:
```lux
// f(f(x)) == f(x)
List.sort(List.sort(xs)) == List.sort(xs)
```
**Commutativity** - Order of arguments doesn't matter:
```lux
// f(a, b) == f(b, a)
a + b == b + a
a * b == b * a
```
**Associativity** - Grouping doesn't matter:
```lux
// f(f(a, b), c) == f(a, f(b, c))
(a + b) + c == a + (b + c)
(a * b) * c == a * (b * c)
```
**Identity** - An element that doesn't change the result:
```lux
x + 0 == x
x * 1 == x
List.concat(xs, []) == xs
```
**Length Preservation**:
```lux
List.length(List.reverse(xs)) == List.length(xs)
List.length(List.map(xs, f)) == List.length(xs)
```
**Length Addition**:
```lux
List.length(List.concat(xs, ys)) == List.length(xs) + List.length(ys)
String.length(s1 + s2) == String.length(s1) + String.length(s2)
```
## Complete Example
```lux
// Property-Based Testing Example
// Run with: lux examples/property_testing.lux
let CHARS = "abcdefghijklmnopqrstuvwxyz"
// Generators
fn genInt(min: Int, max: Int): Int with {Random} =
Random.int(min, max)
fn genIntList(min: Int, max: Int, maxLen: Int): List<Int> with {Random} = {
let len = Random.int(0, maxLen)
genIntListHelper(min, max, len)
}
fn genIntListHelper(min: Int, max: Int, len: Int): List<Int> with {Random} = {
if len <= 0 then []
else List.concat([Random.int(min, max)], genIntListHelper(min, max, len - 1))
}
// Test helper
fn printResult(name: String, passed: Bool, count: Int): Unit with {Console} = {
if passed then
Console.print(" PASS " + name + " (" + toString(count) + " tests)")
else
Console.print(" FAIL " + name)
}
// Property: reverse(reverse(xs)) == xs
fn testReverseInvolutive(n: Int, count: Int): Bool with {Console, Random} = {
if n <= 0 then {
printResult("reverse(reverse(xs)) == xs", true, count)
true
} else {
let xs = genIntList(0, 100, 20)
if List.reverse(List.reverse(xs)) == xs then
testReverseInvolutive(n - 1, count)
else {
printResult("reverse(reverse(xs)) == xs", false, count - n + 1)
false
}
}
}
// Property: addition is commutative
fn testAddCommutative(n: Int, count: Int): Bool with {Console, Random} = {
if n <= 0 then {
printResult("a + b == b + a", true, count)
true
} else {
let a = genInt(-1000, 1000)
let b = genInt(-1000, 1000)
if a + b == b + a then
testAddCommutative(n - 1, count)
else {
printResult("a + b == b + a", false, count - n + 1)
false
}
}
}
fn main(): Unit with {Console, Random} = {
Console.print("Property-Based Testing Demo")
Console.print("")
Console.print("Running 100 iterations per property...")
Console.print("")
testReverseInvolutive(100, 100)
testAddCommutative(100, 100)
Console.print("")
Console.print("All tests completed!")
}
let result = run main() with {}
```
## Stdlib Testing Module
The `stdlib/testing.lux` module provides pre-built generators:
```lux
import stdlib.testing
// Available generators:
genInt(min, max) // Random integer in range
genIntUpTo(max) // Random integer 0 to max
genPositiveInt(max) // Random integer 1 to max
genBool() // Random boolean
genChar() // Random lowercase letter
genAlphaNum() // Random alphanumeric character
genString(maxLen) // Random string
genStringOfLength(len) // Random string of exact length
genIntList(min, max, maxLen) // Random list of integers
genBoolList(maxLen) // Random list of booleans
genStringList(maxStrLen, maxLen) // Random list of strings
// Helper functions:
shrinkInt(n) // Shrink integer toward zero
shrinkList(xs) // Shrink list by removing elements
shrinkString(s) // Shrink string by removing characters
isSorted(xs) // Check if list is sorted
sameElements(xs, ys) // Check if lists have same elements
```
## Best Practices
1. **Start with many iterations**: Use 100+ iterations per property to catch edge cases.
2. **Test edge cases explicitly**: Property tests are great for general cases, but also write unit tests for known edge cases.
3. **Keep properties simple**: Each property should test one thing. Complex properties are harder to debug.
4. **Use good generators**: Match the distribution of your generators to realistic inputs.
5. **Print counterexamples**: When a test fails, print the failing inputs to help debugging.
6. **Combine with shrinking**: Shrinking finds minimal failing inputs, making debugging easier.
## Limitations
Current limitations of property-based testing in Lux:
- No automatic shrinking (must be done manually)
- No seed control for reproducible tests
- No integration with `lux test` command (uses `Random` effect)
## See Also
- [Testing Guide](../testing.md) - Unit testing with the Test effect
- [Effects Guide](./05-effects.md) - Understanding the Random effect
- [Example](../../examples/property_testing.lux) - Complete working example

255
examples/property_testing.lux Normal file
View File

@@ -0,0 +1,255 @@
// Property-Based Testing Example
//
// This example demonstrates property-based testing in Lux,
// where we verify properties hold for randomly generated inputs.
//
// Run with: lux examples/property_testing.lux
// ============================================================
// Generator Functions (using Random effect)
// ============================================================
let CHARS = "abcdefghijklmnopqrstuvwxyz"
fn genInt(min: Int, max: Int): Int with {Random} =
Random.int(min, max)
fn genIntList(min: Int, max: Int, maxLen: Int): List<Int> with {Random} = {
let len = Random.int(0, maxLen)
genIntListHelper(min, max, len)
}
fn genIntListHelper(min: Int, max: Int, len: Int): List<Int> with {Random} = {
if len <= 0 then
[]
else
List.concat([Random.int(min, max)], genIntListHelper(min, max, len - 1))
}
fn genChar(): String with {Random} = {
let idx = Random.int(0, 25)
String.substring(CHARS, idx, idx + 1)
}
fn genString(maxLen: Int): String with {Random} = {
let len = Random.int(0, maxLen)
genStringHelper(len)
}
fn genStringHelper(len: Int): String with {Random} = {
if len <= 0 then
""
else
genChar() + genStringHelper(len - 1)
}
// ============================================================
// Test Runner State
// ============================================================
fn printResult(name: String, passed: Bool, count: Int): Unit with {Console} = {
if passed then
Console.print(" PASS " + name + " (" + toString(count) + " tests)")
else
Console.print(" FAIL " + name)
}
// ============================================================
// Property Tests
// ============================================================
// Test: List reverse is involutive
fn testReverseInvolutive(n: Int, count: Int): Bool with {Console, Random} = {
if n <= 0 then {
printResult("reverse(reverse(xs)) == xs", true, count)
true
} else {
let xs = genIntList(0, 100, 20)
if List.reverse(List.reverse(xs)) == xs then
testReverseInvolutive(n - 1, count)
else {
printResult("reverse(reverse(xs)) == xs", false, count - n + 1)
false
}
}
}
// Test: List reverse preserves length
fn testReverseLength(n: Int, count: Int): Bool with {Console, Random} = {
if n <= 0 then {
printResult("length(reverse(xs)) == length(xs)", true, count)
true
} else {
let xs = genIntList(0, 100, 20)
if List.length(List.reverse(xs)) == List.length(xs) then
testReverseLength(n - 1, count)
else {
printResult("length(reverse(xs)) == length(xs)", false, count - n + 1)
false
}
}
}
// Test: List map preserves length
fn testMapLength(n: Int, count: Int): Bool with {Console, Random} = {
if n <= 0 then {
printResult("length(map(xs, f)) == length(xs)", true, count)
true
} else {
let xs = genIntList(0, 100, 20)
if List.length(List.map(xs, fn(x) => x * 2)) == List.length(xs) then
testMapLength(n - 1, count)
else {
printResult("length(map(xs, f)) == length(xs)", false, count - n + 1)
false
}
}
}
// Test: List concat length is sum
fn testConcatLength(n: Int, count: Int): Bool with {Console, Random} = {
if n <= 0 then {
printResult("length(xs ++ ys) == length(xs) + length(ys)", true, count)
true
} else {
let xs = genIntList(0, 50, 10)
let ys = genIntList(0, 50, 10)
if List.length(List.concat(xs, ys)) == List.length(xs) + List.length(ys) then
testConcatLength(n - 1, count)
else {
printResult("length(xs ++ ys) == length(xs) + length(ys)", false, count - n + 1)
false
}
}
}
// Test: Addition is commutative
fn testAddCommutative(n: Int, count: Int): Bool with {Console, Random} = {
if n <= 0 then {
printResult("a + b == b + a", true, count)
true
} else {
let a = genInt(-1000, 1000)
let b = genInt(-1000, 1000)
if a + b == b + a then
testAddCommutative(n - 1, count)
else {
printResult("a + b == b + a", false, count - n + 1)
false
}
}
}
// Test: Multiplication is associative
fn testMulAssociative(n: Int, count: Int): Bool with {Console, Random} = {
if n <= 0 then {
printResult("(a * b) * c == a * (b * c)", true, count)
true
} else {
let a = genInt(-100, 100)
let b = genInt(-100, 100)
let c = genInt(-100, 100)
if (a * b) * c == a * (b * c) then
testMulAssociative(n - 1, count)
else {
printResult("(a * b) * c == a * (b * c)", false, count - n + 1)
false
}
}
}
// Test: String concat length is sum
fn testStringConcatLength(n: Int, count: Int): Bool with {Console, Random} = {
if n <= 0 then {
printResult("length(s1 + s2) == length(s1) + length(s2)", true, count)
true
} else {
let s1 = genString(10)
let s2 = genString(10)
if String.length(s1 + s2) == String.length(s1) + String.length(s2) then
testStringConcatLength(n - 1, count)
else {
printResult("length(s1 + s2) == length(s1) + length(s2)", false, count - n + 1)
false
}
}
}
// Test: Zero is identity for addition
fn testAddIdentity(n: Int, count: Int): Bool with {Console, Random} = {
if n <= 0 then {
printResult("x + 0 == x && 0 + x == x", true, count)
true
} else {
let x = genInt(-10000, 10000)
if x + 0 == x && 0 + x == x then
testAddIdentity(n - 1, count)
else {
printResult("x + 0 == x && 0 + x == x", false, count - n + 1)
false
}
}
}
// Test: Filter reduces or maintains length
fn testFilterLength(n: Int, count: Int): Bool with {Console, Random} = {
if n <= 0 then {
printResult("length(filter(xs, p)) <= length(xs)", true, count)
true
} else {
let xs = genIntList(0, 100, 20)
if List.length(List.filter(xs, fn(x) => x > 50)) <= List.length(xs) then
testFilterLength(n - 1, count)
else {
printResult("length(filter(xs, p)) <= length(xs)", false, count - n + 1)
false
}
}
}
// Test: Empty list is identity for concat
fn testConcatIdentity(n: Int, count: Int): Bool with {Console, Random} = {
if n <= 0 then {
printResult("concat(xs, []) == xs && concat([], xs) == xs", true, count)
true
} else {
let xs = genIntList(0, 100, 10)
if List.concat(xs, []) == xs && List.concat([], xs) == xs then
testConcatIdentity(n - 1, count)
else {
printResult("concat(xs, []) == xs && concat([], xs) == xs", false, count - n + 1)
false
}
}
}
// ============================================================
// Main
// ============================================================
fn main(): Unit with {Console, Random} = {
Console.print("========================================")
Console.print(" Property-Based Testing Demo")
Console.print("========================================")
Console.print("")
Console.print("Running 100 iterations per property...")
Console.print("")
testReverseInvolutive(100, 100)
testReverseLength(100, 100)
testMapLength(100, 100)
testConcatLength(100, 100)
testAddCommutative(100, 100)
testMulAssociative(100, 100)
testStringConcatLength(100, 100)
testAddIdentity(100, 100)
testFilterLength(100, 100)
testConcatIdentity(100, 100)
Console.print("")
Console.print("========================================")
Console.print(" All property tests completed!")
Console.print("========================================")
}
let result = run main() with {}

1
stdlib/lib.lux
View File

@@ -7,3 +7,4 @@
pub import html
pub import browser
pub import http
pub import testing

192
stdlib/testing.lux Normal file
View File

@@ -0,0 +1,192 @@
// Property-Based Testing Library for Lux
//
// This module provides generators and helper functions for property-based testing.
//
// Usage:
// Use the generator functions (genInt, genIntList, genString, etc.) in your
// property tests to generate random test data.
//
// Example:
// fn testReverseInvolutive(n: Int): Unit with {Console, Test, Random} = {
// if n <= 0 then
// Console.print(" PASS reverse(reverse(xs)) == xs")
// else {
// let xs = genIntList(0, 100, 20)
// if List.reverse(List.reverse(xs)) == xs then
// testReverseInvolutive(n - 1)
// else
// Test.fail("Property failed")
// }
// }
// ============================================================
// Generator Module
// ============================================================
// Character set for string generation
let GEN_CHARS = "abcdefghijklmnopqrstuvwxyz"
let GEN_ALPHANUM = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"
// Generate a random integer in a range
fn genInt(min: Int, max: Int): Int with {Random} =
Random.int(min, max)
// Generate a random integer from 0 to max
fn genIntUpTo(max: Int): Int with {Random} =
Random.int(0, max)
// Generate a random positive integer
fn genPositiveInt(max: Int): Int with {Random} =
Random.int(1, max)
// Generate a random boolean
fn genBool(): Bool with {Random} =
Random.bool()
// Generate a random character (lowercase letter)
fn genChar(): String with {Random} = {
let idx = Random.int(0, 25)
String.substring(GEN_CHARS, idx, idx + 1)
}
// Generate a random alphanumeric character
fn genAlphaNum(): String with {Random} = {
let idx = Random.int(0, 61)
String.substring(GEN_ALPHANUM, idx, idx + 1)
}
// Generate a random string of given max length
fn genString(maxLen: Int): String with {Random} = {
let len = Random.int(0, maxLen)
genStringOfLength(len)
}
// Generate a random string of exact length
fn genStringOfLength(len: Int): String with {Random} = {
if len <= 0 then
""
else
genChar() + genStringOfLength(len - 1)
}
// Generate a random list of integers
fn genIntList(min: Int, max: Int, maxLen: Int): List<Int> with {Random} = {
let len = Random.int(0, maxLen)
genIntListOfLength(min, max, len)
}
// Generate a random list of integers with exact length
fn genIntListOfLength(min: Int, max: Int, len: Int): List<Int> with {Random} = {
if len <= 0 then
[]
else
List.concat([Random.int(min, max)], genIntListOfLength(min, max, len - 1))
}
// Generate a random list of booleans
fn genBoolList(maxLen: Int): List<Bool> with {Random} = {
let len = Random.int(0, maxLen)
genBoolListOfLength(len)
}
fn genBoolListOfLength(len: Int): List<Bool> with {Random} = {
if len <= 0 then
[]
else
List.concat([Random.bool()], genBoolListOfLength(len - 1))
}
// Generate a random list of strings
fn genStringList(maxStrLen: Int, maxLen: Int): List<String> with {Random} = {
let len = Random.int(0, maxLen)
genStringListOfLength(maxStrLen, len)
}
fn genStringListOfLength(maxStrLen: Int, len: Int): List<String> with {Random} = {
if len <= 0 then
[]
else
List.concat([genString(maxStrLen)], genStringListOfLength(maxStrLen, len - 1))
}
// ============================================================
// Shrinking Module
// ============================================================
// Shrink an integer towards zero
fn shrinkInt(n: Int): List<Int> = {
if n == 0 then
[]
else if n > 0 then
[0, n / 2, n - 1]
else
[0, n / 2, n + 1]
}
// Shrink a list by removing elements
fn shrinkList<T>(xs: List<T>): List<List<T>> = {
let len = List.length(xs)
if len == 0 then
[]
else if len == 1 then
[[]]
else {
// Return: empty list, first half, second half
let half = len / 2
let firstHalf = List.take(xs, half)
let secondHalf = List.drop(xs, half)
[[], firstHalf, secondHalf]
}
}
// Shrink a string by removing characters
fn shrinkString(s: String): List<String> = {
let len = String.length(s)
if len == 0 then
[]
else if len == 1 then
[""]
else {
let half = len / 2
["", String.substring(s, 0, half), String.substring(s, half, len)]
}
}
// ============================================================
// Common Properties
// ============================================================
// Check that a list is sorted
fn isSorted(xs: List<Int>): Bool = {
match List.head(xs) {
None => true,
Some(first) => {
match List.tail(xs) {
None => true,
Some(rest) => isSortedHelper(first, rest)
}
}
}
}
fn isSortedHelper(prev: Int, xs: List<Int>): Bool = {
match List.head(xs) {
None => true,
Some(curr) => {
if prev <= curr then {
match List.tail(xs) {
None => true,
Some(rest) => isSortedHelper(curr, rest)
}
} else
false
}
}
}
// Check that two lists have the same elements (ignoring order)
fn sameElements(xs: List<Int>, ys: List<Int>): Bool = {
List.length(xs) == List.length(ys) &&
List.all(xs, fn(x) => List.contains(ys, x)) &&
List.all(ys, fn(y) => List.contains(xs, y))
}