feat: add PostgreSQL driver with Postgres effect
Implements full PostgreSQL support through the Postgres effect: - connect(connStr): Connect to PostgreSQL database - close(conn): Close connection - execute(conn, sql): Execute INSERT/UPDATE/DELETE, return affected rows - query(conn, sql): Execute SELECT, return all rows as records - queryOne(conn, sql): Execute SELECT, return first row as Option - beginTx(conn): Start transaction - commit(conn): Commit transaction - rollback(conn): Rollback transaction Includes: - Connection tracking with connection IDs - Row mapping to Lux records with field access - Transaction support - Example: examples/postgres_demo.lux - Documentation in docs/guide/11-databases.md Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
450
docs/guide/11-databases.md
Normal file
450
docs/guide/11-databases.md
Normal file
@@ -0,0 +1,450 @@
|
||||
# Working with Databases
|
||||
|
||||
Lux includes built-in support for SQL databases through the `Sql` effect. This guide covers how to connect to databases, execute queries, handle transactions, and best practices for database operations.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```lux
|
||||
fn main(): Unit with {Console, Sql} = {
|
||||
// Open an in-memory SQLite database
|
||||
let db = Sql.openMemory()
|
||||
|
||||
// Create a table
|
||||
Sql.execute(db, "CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)")
|
||||
|
||||
// Insert data
|
||||
Sql.execute(db, "INSERT INTO users (name) VALUES ('Alice')")
|
||||
Sql.execute(db, "INSERT INTO users (name) VALUES ('Bob')")
|
||||
|
||||
// Query data
|
||||
let users = Sql.query(db, "SELECT * FROM users")
|
||||
Console.print("Found users: " ++ toString(users))
|
||||
|
||||
// Clean up
|
||||
Sql.close(db)
|
||||
}
|
||||
|
||||
run main() with {}
|
||||
```
|
||||
|
||||
## Connecting to Databases
|
||||
|
||||
### In-Memory Database
|
||||
|
||||
For testing and temporary data:
|
||||
|
||||
```lux
|
||||
let db = Sql.openMemory()
|
||||
// Database exists only in memory, lost when closed
|
||||
```
|
||||
|
||||
### File-Based Database
|
||||
|
||||
For persistent storage:
|
||||
|
||||
```lux
|
||||
let db = Sql.open("./data/app.db")
|
||||
// Creates file if it doesn't exist
|
||||
```
|
||||
|
||||
### Connection Lifecycle
|
||||
|
||||
Always close connections when done:
|
||||
|
||||
```lux
|
||||
fn withDatabase<T>(path: String, f: fn(SqlConn): T with Sql): T with Sql = {
|
||||
let db = Sql.open(path)
|
||||
let result = f(db)
|
||||
Sql.close(db)
|
||||
result
|
||||
}
|
||||
```
|
||||
|
||||
## Executing Queries
|
||||
|
||||
### Non-Returning Queries (INSERT, UPDATE, DELETE, CREATE)
|
||||
|
||||
Use `Sql.execute` for statements that don't return rows:
|
||||
|
||||
```lux
|
||||
// Create table
|
||||
Sql.execute(db, "CREATE TABLE IF NOT EXISTS posts (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
title TEXT NOT NULL,
|
||||
content TEXT,
|
||||
created_at TEXT DEFAULT CURRENT_TIMESTAMP
|
||||
)")
|
||||
|
||||
// Insert
|
||||
Sql.execute(db, "INSERT INTO posts (title, content) VALUES ('Hello', 'World')")
|
||||
|
||||
// Update
|
||||
Sql.execute(db, "UPDATE posts SET title = 'Updated' WHERE id = 1")
|
||||
|
||||
// Delete
|
||||
Sql.execute(db, "DELETE FROM posts WHERE id = 1")
|
||||
```
|
||||
|
||||
### Queries that Return Rows (SELECT)
|
||||
|
||||
Use `Sql.query` to get all matching rows:
|
||||
|
||||
```lux
|
||||
// Returns List<SqlRow>
|
||||
let users = Sql.query(db, "SELECT * FROM users")
|
||||
|
||||
// Each row is a record-like structure
|
||||
for user in users {
|
||||
Console.print("User: " ++ user.name)
|
||||
}
|
||||
```
|
||||
|
||||
Use `Sql.queryOne` to get a single row (or None):
|
||||
|
||||
```lux
|
||||
// Returns Option<SqlRow>
|
||||
let maybeUser = Sql.queryOne(db, "SELECT * FROM users WHERE id = 1")
|
||||
|
||||
match maybeUser {
|
||||
Some(user) => Console.print("Found: " ++ user.name),
|
||||
None => Console.print("User not found")
|
||||
}
|
||||
```
|
||||
|
||||
## Transactions
|
||||
|
||||
Transactions ensure multiple operations succeed or fail together.
|
||||
|
||||
### Basic Transaction
|
||||
|
||||
```lux
|
||||
// Start transaction
|
||||
Sql.beginTx(db)
|
||||
|
||||
// Do operations
|
||||
Sql.execute(db, "INSERT INTO accounts (name, balance) VALUES ('Alice', 1000)")
|
||||
Sql.execute(db, "INSERT INTO accounts (name, balance) VALUES ('Bob', 1000)")
|
||||
|
||||
// Commit changes
|
||||
Sql.commit(db)
|
||||
```
|
||||
|
||||
### Rollback on Error
|
||||
|
||||
```lux
|
||||
Sql.beginTx(db)
|
||||
|
||||
let result = transferFunds(db, fromId, toId, amount)
|
||||
|
||||
match result {
|
||||
Ok(_) => Sql.commit(db),
|
||||
Err(_) => Sql.rollback(db) // Undo all changes
|
||||
}
|
||||
```
|
||||
|
||||
### Transaction Helper
|
||||
|
||||
Here's a pattern for safe transactions:
|
||||
|
||||
```lux
|
||||
fn transaction<T>(db: SqlConn, f: fn(): T with Sql): Result<T, String> with Sql = {
|
||||
Sql.beginTx(db)
|
||||
|
||||
// Execute the function
|
||||
// In a real implementation, you'd catch errors here
|
||||
let result = f()
|
||||
|
||||
Sql.commit(db)
|
||||
Ok(result)
|
||||
}
|
||||
```
|
||||
|
||||
## Working with Results
|
||||
|
||||
Query results are returned as `List<SqlRow>` where each row behaves like a record:
|
||||
|
||||
```lux
|
||||
let rows = Sql.query(db, "SELECT id, name, age FROM users")
|
||||
|
||||
for row in rows {
|
||||
// Access columns by name
|
||||
let id = row.id // Int
|
||||
let name = row.name // String
|
||||
let age = row.age // Int or null
|
||||
|
||||
Console.print("{name} (age {age})")
|
||||
}
|
||||
```
|
||||
|
||||
### Handling NULL values
|
||||
|
||||
NULL values from the database are represented as options:
|
||||
|
||||
```lux
|
||||
let row = Sql.queryOne(db, "SELECT email FROM users WHERE id = 1")
|
||||
|
||||
match row {
|
||||
Some(r) => {
|
||||
match r.email {
|
||||
Some(email) => Console.print("Email: " ++ email),
|
||||
None => Console.print("No email set")
|
||||
}
|
||||
},
|
||||
None => Console.print("User not found")
|
||||
}
|
||||
```
|
||||
|
||||
## SQL Injection Prevention
|
||||
|
||||
**IMPORTANT**: The current API passes SQL strings directly. For production use, always:
|
||||
|
||||
1. Validate and sanitize user input
|
||||
2. Use parameterized queries (when available)
|
||||
3. Never concatenate user input into SQL strings
|
||||
|
||||
```lux
|
||||
// DANGEROUS - Never do this!
|
||||
let query = "SELECT * FROM users WHERE name = '" ++ userInput ++ "'"
|
||||
|
||||
// SAFER - Validate input first
|
||||
fn safeUserLookup(db: SqlConn, userId: Int): Option<SqlRow> with Sql = {
|
||||
// Integers are safe to interpolate
|
||||
Sql.queryOne(db, "SELECT * FROM users WHERE id = " ++ toString(userId))
|
||||
}
|
||||
|
||||
// For strings, escape quotes at minimum
|
||||
fn escapeString(s: String): String = {
|
||||
String.replace(s, "'", "''")
|
||||
}
|
||||
```
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### Repository Pattern
|
||||
|
||||
Encapsulate database operations:
|
||||
|
||||
```lux
|
||||
type User = { id: Int, name: String, email: Option<String> }
|
||||
|
||||
fn createUser(db: SqlConn, name: String): Int with Sql = {
|
||||
Sql.execute(db, "INSERT INTO users (name) VALUES ('" ++ escapeString(name) ++ "')")
|
||||
// Get last inserted ID
|
||||
let row = Sql.queryOne(db, "SELECT last_insert_rowid() as id")
|
||||
match row {
|
||||
Some(r) => r.id,
|
||||
None => -1
|
||||
}
|
||||
}
|
||||
|
||||
fn findUserById(db: SqlConn, id: Int): Option<User> with Sql = {
|
||||
let row = Sql.queryOne(db, "SELECT * FROM users WHERE id = " ++ toString(id))
|
||||
match row {
|
||||
Some(r) => Some({ id: r.id, name: r.name, email: r.email }),
|
||||
None => None
|
||||
}
|
||||
}
|
||||
|
||||
fn findAllUsers(db: SqlConn): List<User> with Sql = {
|
||||
let rows = Sql.query(db, "SELECT * FROM users ORDER BY name")
|
||||
List.map(rows, fn(r) => { id: r.id, name: r.name, email: r.email })
|
||||
}
|
||||
```
|
||||
|
||||
### Testing with In-Memory Database
|
||||
|
||||
```lux
|
||||
fn testUserRepository(): Unit with {Test, Sql} = {
|
||||
// Each test gets a fresh database
|
||||
let db = Sql.openMemory()
|
||||
|
||||
// Set up schema
|
||||
Sql.execute(db, "CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)")
|
||||
|
||||
// Test
|
||||
let id = createUser(db, "Test User")
|
||||
Test.assertTrue(id > 0, "Should return valid ID")
|
||||
|
||||
let user = findUserById(db, id)
|
||||
Test.assertTrue(Option.isSome(user), "Should find created user")
|
||||
|
||||
Sql.close(db)
|
||||
}
|
||||
```
|
||||
|
||||
### Handler for Testing (Mock Database)
|
||||
|
||||
The effect system lets you swap database implementations:
|
||||
|
||||
```lux
|
||||
// Production handler uses real SQLite
|
||||
handler realDatabase(): Sql {
|
||||
// Uses actual rusqlite implementation
|
||||
}
|
||||
|
||||
// Test handler uses in-memory storage
|
||||
handler mockDatabase(): Sql {
|
||||
let storage = ref []
|
||||
|
||||
fn execute(conn, sql) = {
|
||||
// Parse and simulate SQL
|
||||
}
|
||||
|
||||
fn query(conn, sql) = {
|
||||
// Return mock data
|
||||
[]
|
||||
}
|
||||
}
|
||||
|
||||
// Run with mock database in tests
|
||||
run userService() with {
|
||||
Sql -> mockDatabase()
|
||||
}
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### Types
|
||||
|
||||
```lux
|
||||
type SqlConn // Opaque connection handle
|
||||
type SqlRow // Row result with named column access
|
||||
```
|
||||
|
||||
### Operations
|
||||
|
||||
| Function | Type | Description |
|
||||
|----------|------|-------------|
|
||||
| `Sql.open(path)` | `String -> SqlConn` | Open database file |
|
||||
| `Sql.openMemory()` | `() -> SqlConn` | Open in-memory database |
|
||||
| `Sql.close(conn)` | `SqlConn -> Unit` | Close connection |
|
||||
| `Sql.execute(conn, sql)` | `(SqlConn, String) -> Int` | Execute statement, return affected rows |
|
||||
| `Sql.query(conn, sql)` | `(SqlConn, String) -> List<SqlRow>` | Query, return all rows |
|
||||
| `Sql.queryOne(conn, sql)` | `(SqlConn, String) -> Option<SqlRow>` | Query, return first row |
|
||||
| `Sql.beginTx(conn)` | `SqlConn -> Unit` | Begin transaction |
|
||||
| `Sql.commit(conn)` | `SqlConn -> Unit` | Commit transaction |
|
||||
| `Sql.rollback(conn)` | `SqlConn -> Unit` | Rollback transaction |
|
||||
|
||||
## Error Handling
|
||||
|
||||
Database operations can fail. In the current implementation, errors cause runtime failures. Future versions will support returning `Result` types:
|
||||
|
||||
```lux
|
||||
// Future API (not yet implemented)
|
||||
fn safeQuery(db: SqlConn, sql: String): Result<List<SqlRow>, SqlError> with Sql = {
|
||||
Sql.tryQuery(db, sql)
|
||||
}
|
||||
```
|
||||
|
||||
## Performance Tips
|
||||
|
||||
1. **Batch inserts in transactions** - Much faster than individual inserts:
|
||||
```lux
|
||||
Sql.beginTx(db)
|
||||
for item in items {
|
||||
Sql.execute(db, "INSERT INTO data (value) VALUES (" ++ toString(item) ++ ")")
|
||||
}
|
||||
Sql.commit(db)
|
||||
```
|
||||
|
||||
2. **Use `queryOne` for single results** - More efficient than `query` when you only need one row
|
||||
|
||||
3. **Create indexes** for frequently queried columns:
|
||||
```lux
|
||||
Sql.execute(db, "CREATE INDEX idx_users_email ON users(email)")
|
||||
```
|
||||
|
||||
4. **Close connections** when done to free resources
|
||||
|
||||
## PostgreSQL Support
|
||||
|
||||
Lux also provides native PostgreSQL support through the `Postgres` effect. This is ideal for production applications requiring a full-featured relational database.
|
||||
|
||||
### Connecting to PostgreSQL
|
||||
|
||||
```lux
|
||||
fn main(): Unit with {Console, Postgres} = {
|
||||
// Connect using a connection string
|
||||
let connStr = "host=localhost user=myuser password=mypass dbname=mydb"
|
||||
let conn = Postgres.connect(connStr)
|
||||
|
||||
Console.print("Connected to PostgreSQL!")
|
||||
|
||||
// ... use the connection ...
|
||||
|
||||
Postgres.close(conn)
|
||||
}
|
||||
```
|
||||
|
||||
### PostgreSQL Operations
|
||||
|
||||
The PostgreSQL API mirrors the SQLite API:
|
||||
|
||||
```lux
|
||||
// Execute non-returning queries
|
||||
Postgres.execute(conn, "CREATE TABLE users (id SERIAL PRIMARY KEY, name TEXT)")
|
||||
Postgres.execute(conn, "INSERT INTO users (name) VALUES ('Alice')")
|
||||
|
||||
// Query multiple rows
|
||||
let users = Postgres.query(conn, "SELECT * FROM users")
|
||||
for user in users {
|
||||
Console.print("User: " ++ user.name)
|
||||
}
|
||||
|
||||
// Query single row
|
||||
match Postgres.queryOne(conn, "SELECT * FROM users WHERE id = 1") {
|
||||
Some(user) => Console.print("Found: " ++ user.name),
|
||||
None => Console.print("Not found")
|
||||
}
|
||||
```
|
||||
|
||||
### PostgreSQL Transactions
|
||||
|
||||
```lux
|
||||
// Start transaction
|
||||
Postgres.beginTx(conn)
|
||||
|
||||
// Make changes
|
||||
Postgres.execute(conn, "UPDATE accounts SET balance = balance - 100 WHERE id = 1")
|
||||
Postgres.execute(conn, "UPDATE accounts SET balance = balance + 100 WHERE id = 2")
|
||||
|
||||
// Commit or rollback
|
||||
Postgres.commit(conn)
|
||||
// Or: Postgres.rollback(conn)
|
||||
```
|
||||
|
||||
### PostgreSQL API Reference
|
||||
|
||||
| Function | Type | Description |
|
||||
|----------|------|-------------|
|
||||
| `Postgres.connect(connStr)` | `String -> Int` | Connect to PostgreSQL, returns connection ID |
|
||||
| `Postgres.close(conn)` | `Int -> Unit` | Close connection |
|
||||
| `Postgres.execute(conn, sql)` | `(Int, String) -> Int` | Execute statement, return affected rows |
|
||||
| `Postgres.query(conn, sql)` | `(Int, String) -> List<Row>` | Query, return all rows |
|
||||
| `Postgres.queryOne(conn, sql)` | `(Int, String) -> Option<Row>` | Query, return first row |
|
||||
| `Postgres.beginTx(conn)` | `Int -> Unit` | Begin transaction |
|
||||
| `Postgres.commit(conn)` | `Int -> Unit` | Commit transaction |
|
||||
| `Postgres.rollback(conn)` | `Int -> Unit` | Rollback transaction |
|
||||
|
||||
### When to Use SQLite vs PostgreSQL
|
||||
|
||||
| Feature | SQLite | PostgreSQL |
|
||||
|---------|--------|------------|
|
||||
| Setup | No setup needed | Requires server |
|
||||
| Deployment | Single file | Server process |
|
||||
| Concurrency | Limited | Excellent |
|
||||
| Scale | Small-medium | Large |
|
||||
| Features | Basic | Full RDBMS |
|
||||
| Use case | Embedded, testing | Production |
|
||||
|
||||
## Limitations
|
||||
|
||||
- No connection pooling yet
|
||||
- No prepared statements / parameterized queries yet
|
||||
- Limited type mapping (basic Int, String, Float)
|
||||
|
||||
## See Also
|
||||
|
||||
- [Effects Guide](./05-effects.md) - Understanding the effect system
|
||||
- [Testing Guide](../testing.md) - Writing tests with mock handlers
|
||||
- [Roadmap](../ROADMAP.md) - Planned database features
|
||||
Reference in New Issue
Block a user