lux/docs/guide/11-databases.md

Working with Databases

Lux includes built-in support for SQL databases through the Sql effect. This guide covers how to connect to databases, execute queries, handle transactions, and best practices for database operations.

Quick Start

fn main(): Unit with {Console, Sql} = {
    // Open an in-memory SQLite database
    let db = Sql.openMemory()

    // Create a table
    Sql.execute(db, "CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)")

    // Insert data
    Sql.execute(db, "INSERT INTO users (name) VALUES ('Alice')")
    Sql.execute(db, "INSERT INTO users (name) VALUES ('Bob')")

    // Query data
    let users = Sql.query(db, "SELECT * FROM users")
    Console.print("Found users: " ++ toString(users))

    // Clean up
    Sql.close(db)
}

run main() with {}

Connecting to Databases

In-Memory Database

For testing and temporary data:

let db = Sql.openMemory()
// Database exists only in memory, lost when closed

File-Based Database

For persistent storage:

let db = Sql.open("./data/app.db")
// Creates file if it doesn't exist

Connection Lifecycle

Always close connections when done:

fn withDatabase<T>(path: String, f: fn(SqlConn): T with Sql): T with Sql = {
    let db = Sql.open(path)
    let result = f(db)
    Sql.close(db)
    result
}

Executing Queries

Non-Returning Queries (INSERT, UPDATE, DELETE, CREATE)

Use Sql.execute for statements that don't return rows:

// Create table
Sql.execute(db, "CREATE TABLE IF NOT EXISTS posts (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    title TEXT NOT NULL,
    content TEXT,
    created_at TEXT DEFAULT CURRENT_TIMESTAMP
)")

// Insert
Sql.execute(db, "INSERT INTO posts (title, content) VALUES ('Hello', 'World')")

// Update
Sql.execute(db, "UPDATE posts SET title = 'Updated' WHERE id = 1")

// Delete
Sql.execute(db, "DELETE FROM posts WHERE id = 1")

Queries that Return Rows (SELECT)

Use Sql.query to get all matching rows:

// Returns List<SqlRow>
let users = Sql.query(db, "SELECT * FROM users")

// Each row is a record-like structure
for user in users {
    Console.print("User: " ++ user.name)
}

Use Sql.queryOne to get a single row (or None):

// Returns Option<SqlRow>
let maybeUser = Sql.queryOne(db, "SELECT * FROM users WHERE id = 1")

match maybeUser {
    Some(user) => Console.print("Found: " ++ user.name),
    None => Console.print("User not found")
}

Transactions

Transactions ensure multiple operations succeed or fail together.

Basic Transaction

// Start transaction
Sql.beginTx(db)

// Do operations
Sql.execute(db, "INSERT INTO accounts (name, balance) VALUES ('Alice', 1000)")
Sql.execute(db, "INSERT INTO accounts (name, balance) VALUES ('Bob', 1000)")

// Commit changes
Sql.commit(db)

Rollback on Error

Sql.beginTx(db)

let result = transferFunds(db, fromId, toId, amount)

match result {
    Ok(_) => Sql.commit(db),
    Err(_) => Sql.rollback(db)  // Undo all changes
}

Transaction Helper

Here's a pattern for safe transactions:

fn transaction<T>(db: SqlConn, f: fn(): T with Sql): Result<T, String> with Sql = {
    Sql.beginTx(db)

    // Execute the function
    // In a real implementation, you'd catch errors here
    let result = f()

    Sql.commit(db)
    Ok(result)
}

Working with Results

Query results are returned as List<SqlRow> where each row behaves like a record:

let rows = Sql.query(db, "SELECT id, name, age FROM users")

for row in rows {
    // Access columns by name
    let id = row.id       // Int
    let name = row.name   // String
    let age = row.age     // Int or null

    Console.print("{name} (age {age})")
}

Handling NULL values

NULL values from the database are represented as options:

let row = Sql.queryOne(db, "SELECT email FROM users WHERE id = 1")

match row {
    Some(r) => {
        match r.email {
            Some(email) => Console.print("Email: " ++ email),
            None => Console.print("No email set")
        }
    },
    None => Console.print("User not found")
}

SQL Injection Prevention

IMPORTANT: The current API passes SQL strings directly. For production use, always:

1. Validate and sanitize user input
2. Use parameterized queries (when available)
3. Never concatenate user input into SQL strings

// DANGEROUS - Never do this!
let query = "SELECT * FROM users WHERE name = '" ++ userInput ++ "'"

// SAFER - Validate input first
fn safeUserLookup(db: SqlConn, userId: Int): Option<SqlRow> with Sql = {
    // Integers are safe to interpolate
    Sql.queryOne(db, "SELECT * FROM users WHERE id = " ++ toString(userId))
}

// For strings, escape quotes at minimum
fn escapeString(s: String): String = {
    String.replace(s, "'", "''")
}

Common Patterns

Repository Pattern

Encapsulate database operations:

type User = { id: Int, name: String, email: Option<String> }

fn createUser(db: SqlConn, name: String): Int with Sql = {
    Sql.execute(db, "INSERT INTO users (name) VALUES ('" ++ escapeString(name) ++ "')")
    // Get last inserted ID
    let row = Sql.queryOne(db, "SELECT last_insert_rowid() as id")
    match row {
        Some(r) => r.id,
        None => -1
    }
}

fn findUserById(db: SqlConn, id: Int): Option<User> with Sql = {
    let row = Sql.queryOne(db, "SELECT * FROM users WHERE id = " ++ toString(id))
    match row {
        Some(r) => Some({ id: r.id, name: r.name, email: r.email }),
        None => None
    }
}

fn findAllUsers(db: SqlConn): List<User> with Sql = {
    let rows = Sql.query(db, "SELECT * FROM users ORDER BY name")
    List.map(rows, fn(r) => { id: r.id, name: r.name, email: r.email })
}

Testing with In-Memory Database

fn testUserRepository(): Unit with {Test, Sql} = {
    // Each test gets a fresh database
    let db = Sql.openMemory()

    // Set up schema
    Sql.execute(db, "CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)")

    // Test
    let id = createUser(db, "Test User")
    Test.assertTrue(id > 0, "Should return valid ID")

    let user = findUserById(db, id)
    Test.assertTrue(Option.isSome(user), "Should find created user")

    Sql.close(db)
}

Handler for Testing (Mock Database)

The effect system lets you swap database implementations:

// Production handler uses real SQLite
handler realDatabase(): Sql {
    // Uses actual rusqlite implementation
}

// Test handler uses in-memory storage
handler mockDatabase(): Sql {
    let storage = ref []

    fn execute(conn, sql) = {
        // Parse and simulate SQL
    }

    fn query(conn, sql) = {
        // Return mock data
        []
    }
}

// Run with mock database in tests
run userService() with {
    Sql -> mockDatabase()
}

API Reference

Types

type SqlConn      // Opaque connection handle
type SqlRow       // Row result with named column access

Operations

Function	Type	Description
Sql.open(path)	String -> SqlConn	Open database file
Sql.openMemory()	() -> SqlConn	Open in-memory database
Sql.close(conn)	SqlConn -> Unit	Close connection
Sql.execute(conn, sql)	(SqlConn, String) -> Int	Execute statement, return affected rows
Sql.query(conn, sql)	(SqlConn, String) -> List<SqlRow>	Query, return all rows
Sql.queryOne(conn, sql)	(SqlConn, String) -> Option<SqlRow>	Query, return first row
Sql.beginTx(conn)	SqlConn -> Unit	Begin transaction
Sql.commit(conn)	SqlConn -> Unit	Commit transaction
Sql.rollback(conn)	SqlConn -> Unit	Rollback transaction

Error Handling

Database operations can fail. In the current implementation, errors cause runtime failures. Future versions will support returning Result types:

// Future API (not yet implemented)
fn safeQuery(db: SqlConn, sql: String): Result<List<SqlRow>, SqlError> with Sql = {
    Sql.tryQuery(db, sql)
}

Performance Tips

1. Batch inserts in transactions - Much faster than individual inserts:

   Sql.beginTx(db)
   for item in items {
       Sql.execute(db, "INSERT INTO data (value) VALUES (" ++ toString(item) ++ ")")
   }
   Sql.commit(db)
   

2. Use queryOne for single results - More efficient than query when you only need one row

3. Create indexes for frequently queried columns:

   Sql.execute(db, "CREATE INDEX idx_users_email ON users(email)")
   

4. Close connections when done to free resources

PostgreSQL Support

Lux also provides native PostgreSQL support through the Postgres effect. This is ideal for production applications requiring a full-featured relational database.

Connecting to PostgreSQL

fn main(): Unit with {Console, Postgres} = {
    // Connect using a connection string
    let connStr = "host=localhost user=myuser password=mypass dbname=mydb"
    let conn = Postgres.connect(connStr)

    Console.print("Connected to PostgreSQL!")

    // ... use the connection ...

    Postgres.close(conn)
}

PostgreSQL Operations

The PostgreSQL API mirrors the SQLite API:

// Execute non-returning queries
Postgres.execute(conn, "CREATE TABLE users (id SERIAL PRIMARY KEY, name TEXT)")
Postgres.execute(conn, "INSERT INTO users (name) VALUES ('Alice')")

// Query multiple rows
let users = Postgres.query(conn, "SELECT * FROM users")
for user in users {
    Console.print("User: " ++ user.name)
}

// Query single row
match Postgres.queryOne(conn, "SELECT * FROM users WHERE id = 1") {
    Some(user) => Console.print("Found: " ++ user.name),
    None => Console.print("Not found")
}

PostgreSQL Transactions

// Start transaction
Postgres.beginTx(conn)

// Make changes
Postgres.execute(conn, "UPDATE accounts SET balance = balance - 100 WHERE id = 1")
Postgres.execute(conn, "UPDATE accounts SET balance = balance + 100 WHERE id = 2")

// Commit or rollback
Postgres.commit(conn)
// Or: Postgres.rollback(conn)

PostgreSQL API Reference

Function	Type	Description
Postgres.connect(connStr)	String -> Int	Connect to PostgreSQL, returns connection ID
Postgres.close(conn)	Int -> Unit	Close connection
Postgres.execute(conn, sql)	(Int, String) -> Int	Execute statement, return affected rows
Postgres.query(conn, sql)	(Int, String) -> List<Row>	Query, return all rows
Postgres.queryOne(conn, sql)	(Int, String) -> Option<Row>	Query, return first row
Postgres.beginTx(conn)	Int -> Unit	Begin transaction
Postgres.commit(conn)	Int -> Unit	Commit transaction
Postgres.rollback(conn)	Int -> Unit	Rollback transaction

When to Use SQLite vs PostgreSQL

Feature	SQLite	PostgreSQL
Setup	No setup needed	Requires server
Deployment	Single file	Server process
Concurrency	Limited	Excellent
Scale	Small-medium	Large
Features	Basic	Full RDBMS
Use case	Embedded, testing	Production

Limitations

• No connection pooling yet
• No prepared statements / parameterized queries yet
• Limited type mapping (basic Int, String, Float)

See Also

• Effects Guide - Understanding the effect system
• Testing Guide - Writing tests with mock handlers
• Roadmap - Planned database features