# Working with Databases

Lux includes built-in support for SQL databases through the `Sql` effect. This guide covers how to connect to databases, execute queries, handle transactions, and best practices for database operations.

## Quick Start

```lux
fn main(): Unit with {Console, Sql} = {
    // Open an in-memory SQLite database
    let db = Sql.openMemory()

    // Create a table
    Sql.execute(db, "CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)")

    // Insert data
    Sql.execute(db, "INSERT INTO users (name) VALUES ('Alice')")
    Sql.execute(db, "INSERT INTO users (name) VALUES ('Bob')")

    // Query data
    let users = Sql.query(db, "SELECT * FROM users")
    Console.print("Found users: " ++ toString(users))

    // Clean up
    Sql.close(db)
}

run main() with {}
```

## Connecting to Databases

### In-Memory Database

For testing and temporary data:

```lux
let db = Sql.openMemory()
// Database exists only in memory, lost when closed
```

### File-Based Database

For persistent storage:

```lux
let db = Sql.open("./data/app.db")
// Creates file if it doesn't exist
```

### Connection Lifecycle

Always close connections when done:

```lux
fn withDatabase<T>(path: String, f: fn(SqlConn): T with Sql): T with Sql = {
    let db = Sql.open(path)
    let result = f(db)
    Sql.close(db)
    result
}
```

## Executing Queries

### Non-Returning Queries (INSERT, UPDATE, DELETE, CREATE)

Use `Sql.execute` for statements that don't return rows:

```lux
// Create table
Sql.execute(db, "CREATE TABLE IF NOT EXISTS posts (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    title TEXT NOT NULL,
    content TEXT,
    created_at TEXT DEFAULT CURRENT_TIMESTAMP
)")

// Insert
Sql.execute(db, "INSERT INTO posts (title, content) VALUES ('Hello', 'World')")

// Update
Sql.execute(db, "UPDATE posts SET title = 'Updated' WHERE id = 1")

// Delete
Sql.execute(db, "DELETE FROM posts WHERE id = 1")
```

### Queries that Return Rows (SELECT)

Use `Sql.query` to get all matching rows:

```lux
// Returns List<SqlRow>
let users = Sql.query(db, "SELECT * FROM users")

// Each row is a record-like structure
for user in users {
    Console.print("User: " ++ user.name)
}
```

Use `Sql.queryOne` to get a single row (or None):

```lux
// Returns Option<SqlRow>
let maybeUser = Sql.queryOne(db, "SELECT * FROM users WHERE id = 1")

match maybeUser {
    Some(user) => Console.print("Found: " ++ user.name),
    None => Console.print("User not found")
}
```

## Transactions

Transactions ensure multiple operations succeed or fail together.

### Basic Transaction

```lux
// Start transaction
Sql.beginTx(db)

// Do operations
Sql.execute(db, "INSERT INTO accounts (name, balance) VALUES ('Alice', 1000)")
Sql.execute(db, "INSERT INTO accounts (name, balance) VALUES ('Bob', 1000)")

// Commit changes
Sql.commit(db)
```

### Rollback on Error

```lux
Sql.beginTx(db)

let result = transferFunds(db, fromId, toId, amount)

match result {
    Ok(_) => Sql.commit(db),
    Err(_) => Sql.rollback(db)  // Undo all changes
}
```

### Transaction Helper

Here's a pattern for safe transactions:

```lux
fn transaction<T>(db: SqlConn, f: fn(): T with Sql): Result<T, String> with Sql = {
    Sql.beginTx(db)

    // Execute the function
    // In a real implementation, you'd catch errors here
    let result = f()

    Sql.commit(db)
    Ok(result)
}
```

## Working with Results

Query results are returned as `List<SqlRow>` where each row behaves like a record:

```lux
let rows = Sql.query(db, "SELECT id, name, age FROM users")

for row in rows {
    // Access columns by name
    let id = row.id       // Int
    let name = row.name   // String
    let age = row.age     // Int or null

    Console.print("{name} (age {age})")
}
```

### Handling NULL values

NULL values from the database are represented as options:

```lux
let row = Sql.queryOne(db, "SELECT email FROM users WHERE id = 1")

match row {
    Some(r) => {
        match r.email {
            Some(email) => Console.print("Email: " ++ email),
            None => Console.print("No email set")
        }
    },
    None => Console.print("User not found")
}
```

## SQL Injection Prevention

**IMPORTANT**: The current API passes SQL strings directly. For production use, always:

1. Validate and sanitize user input
2. Use parameterized queries (when available)
3. Never concatenate user input into SQL strings

```lux
// DANGEROUS - Never do this!
let query = "SELECT * FROM users WHERE name = '" ++ userInput ++ "'"

// SAFER - Validate input first
fn safeUserLookup(db: SqlConn, userId: Int): Option<SqlRow> with Sql = {
    // Integers are safe to interpolate
    Sql.queryOne(db, "SELECT * FROM users WHERE id = " ++ toString(userId))
}

// For strings, escape quotes at minimum
fn escapeString(s: String): String = {
    String.replace(s, "'", "''")
}
```

## Common Patterns

### Repository Pattern

Encapsulate database operations:

```lux
type User = { id: Int, name: String, email: Option<String> }

fn createUser(db: SqlConn, name: String): Int with Sql = {
    Sql.execute(db, "INSERT INTO users (name) VALUES ('" ++ escapeString(name) ++ "')")
    // Get last inserted ID
    let row = Sql.queryOne(db, "SELECT last_insert_rowid() as id")
    match row {
        Some(r) => r.id,
        None => -1
    }
}

fn findUserById(db: SqlConn, id: Int): Option<User> with Sql = {
    let row = Sql.queryOne(db, "SELECT * FROM users WHERE id = " ++ toString(id))
    match row {
        Some(r) => Some({ id: r.id, name: r.name, email: r.email }),
        None => None
    }
}

fn findAllUsers(db: SqlConn): List<User> with Sql = {
    let rows = Sql.query(db, "SELECT * FROM users ORDER BY name")
    List.map(rows, fn(r) => { id: r.id, name: r.name, email: r.email })
}
```

### Testing with In-Memory Database

```lux
fn testUserRepository(): Unit with {Test, Sql} = {
    // Each test gets a fresh database
    let db = Sql.openMemory()

    // Set up schema
    Sql.execute(db, "CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)")

    // Test
    let id = createUser(db, "Test User")
    Test.assertTrue(id > 0, "Should return valid ID")

    let user = findUserById(db, id)
    Test.assertTrue(Option.isSome(user), "Should find created user")

    Sql.close(db)
}
```

### Handler for Testing (Mock Database)

The effect system lets you swap database implementations:

```lux
// Production handler uses real SQLite
handler realDatabase(): Sql {
    // Uses actual rusqlite implementation
}

// Test handler uses in-memory storage
handler mockDatabase(): Sql {
    let storage = ref []

    fn execute(conn, sql) = {
        // Parse and simulate SQL
    }

    fn query(conn, sql) = {
        // Return mock data
        []
    }
}

// Run with mock database in tests
run userService() with {
    Sql -> mockDatabase()
}
```

## API Reference

### Types

```lux
type SqlConn      // Opaque connection handle
type SqlRow       // Row result with named column access
```

### Operations

| Function | Type | Description |
|----------|------|-------------|
| `Sql.open(path)` | `String -> SqlConn` | Open database file |
| `Sql.openMemory()` | `() -> SqlConn` | Open in-memory database |
| `Sql.close(conn)` | `SqlConn -> Unit` | Close connection |
| `Sql.execute(conn, sql)` | `(SqlConn, String) -> Int` | Execute statement, return affected rows |
| `Sql.query(conn, sql)` | `(SqlConn, String) -> List<SqlRow>` | Query, return all rows |
| `Sql.queryOne(conn, sql)` | `(SqlConn, String) -> Option<SqlRow>` | Query, return first row |
| `Sql.beginTx(conn)` | `SqlConn -> Unit` | Begin transaction |
| `Sql.commit(conn)` | `SqlConn -> Unit` | Commit transaction |
| `Sql.rollback(conn)` | `SqlConn -> Unit` | Rollback transaction |

## Error Handling

Database operations can fail. In the current implementation, errors cause runtime failures. Future versions will support returning `Result` types:

```lux
// Future API (not yet implemented)
fn safeQuery(db: SqlConn, sql: String): Result<List<SqlRow>, SqlError> with Sql = {
    Sql.tryQuery(db, sql)
}
```

## Performance Tips

1. **Batch inserts in transactions** - Much faster than individual inserts:
   ```lux
   Sql.beginTx(db)
   for item in items {
       Sql.execute(db, "INSERT INTO data (value) VALUES (" ++ toString(item) ++ ")")
   }
   Sql.commit(db)
   ```

2. **Use `queryOne` for single results** - More efficient than `query` when you only need one row

3. **Create indexes** for frequently queried columns:
   ```lux
   Sql.execute(db, "CREATE INDEX idx_users_email ON users(email)")
   ```

4. **Close connections** when done to free resources

## PostgreSQL Support

Lux also provides native PostgreSQL support through the `Postgres` effect. This is ideal for production applications requiring a full-featured relational database.

### Connecting to PostgreSQL

```lux
fn main(): Unit with {Console, Postgres} = {
    // Connect using a connection string
    let connStr = "host=localhost user=myuser password=mypass dbname=mydb"
    let conn = Postgres.connect(connStr)

    Console.print("Connected to PostgreSQL!")

    // ... use the connection ...

    Postgres.close(conn)
}
```

### PostgreSQL Operations

The PostgreSQL API mirrors the SQLite API:

```lux
// Execute non-returning queries
Postgres.execute(conn, "CREATE TABLE users (id SERIAL PRIMARY KEY, name TEXT)")
Postgres.execute(conn, "INSERT INTO users (name) VALUES ('Alice')")

// Query multiple rows
let users = Postgres.query(conn, "SELECT * FROM users")
for user in users {
    Console.print("User: " ++ user.name)
}

// Query single row
match Postgres.queryOne(conn, "SELECT * FROM users WHERE id = 1") {
    Some(user) => Console.print("Found: " ++ user.name),
    None => Console.print("Not found")
}
```

### PostgreSQL Transactions

```lux
// Start transaction
Postgres.beginTx(conn)

// Make changes
Postgres.execute(conn, "UPDATE accounts SET balance = balance - 100 WHERE id = 1")
Postgres.execute(conn, "UPDATE accounts SET balance = balance + 100 WHERE id = 2")

// Commit or rollback
Postgres.commit(conn)
// Or: Postgres.rollback(conn)
```

### PostgreSQL API Reference

| Function | Type | Description |
|----------|------|-------------|
| `Postgres.connect(connStr)` | `String -> Int` | Connect to PostgreSQL, returns connection ID |
| `Postgres.close(conn)` | `Int -> Unit` | Close connection |
| `Postgres.execute(conn, sql)` | `(Int, String) -> Int` | Execute statement, return affected rows |
| `Postgres.query(conn, sql)` | `(Int, String) -> List<Row>` | Query, return all rows |
| `Postgres.queryOne(conn, sql)` | `(Int, String) -> Option<Row>` | Query, return first row |
| `Postgres.beginTx(conn)` | `Int -> Unit` | Begin transaction |
| `Postgres.commit(conn)` | `Int -> Unit` | Commit transaction |
| `Postgres.rollback(conn)` | `Int -> Unit` | Rollback transaction |

### When to Use SQLite vs PostgreSQL

| Feature | SQLite | PostgreSQL |
|---------|--------|------------|
| Setup | No setup needed | Requires server |
| Deployment | Single file | Server process |
| Concurrency | Limited | Excellent |
| Scale | Small-medium | Large |
| Features | Basic | Full RDBMS |
| Use case | Embedded, testing | Production |

## Limitations

- No connection pooling yet
- No prepared statements / parameterized queries yet
- Limited type mapping (basic Int, String, Float)

## See Also

- [Effects Guide](./05-effects.md) - Understanding the effect system
- [Testing Guide](../testing.md) - Writing tests with mock handlers
- [Roadmap](../ROADMAP.md) - Planned database features