lux/docs/SQL_DESIGN_ANALYSIS.md

SQL in Lux: Built-in Effect vs Package

Executive Summary

This document analyzes whether SQL database access should be a built-in language feature (as it currently is) or a separate package. After comparing approaches across 12+ languages, the recommendation is:

Keep SQL as a built-in effect, but refactor the implementation to be more modular.

Current Implementation

Lux currently implements SQL as a built-in effect:

fn main(): Unit with {Console, Sql} = {
    let db = Sql.openMemory()
    Sql.execute(db, "CREATE TABLE users (...)")
    let users = Sql.query(db, "SELECT * FROM users")
    Sql.close(db)
}

The implementation uses rusqlite (SQLite) compiled directly into the Lux binary.

How Other Languages Handle Database Access

Languages with Built-in Database Support

Language	Approach	Notes
Python	sqlite3 in stdlib	Most languages have SQLite in stdlib
Ruby	sqlite3 gem + AR are common	ActiveRecord is de facto standard
Go	database/sql interface in stdlib	Drivers are packages
Elixir	Ecto as separate package	But universally used
PHP	PDO in core	Multiple backends

Languages with Package-Only Database Support

Language	Approach	Notes
Rust	rusqlite, diesel, sqlx packages	No stdlib database
Node.js	pg, mysql2, better-sqlite3	Packages only
Haskell	postgresql-simple, persistent	Packages only
OCaml	caqti, postgresql-ocaml	Packages only

Analysis of Each Approach

Go's Model: Interface in Stdlib + Driver Packages

import (
    "database/sql"
    _ "github.com/lib/pq"  // PostgreSQL driver
)

db, _ := sql.Open("postgres", "...")
rows, _ := db.Query("SELECT * FROM users")

Pros:

• Standard interface for all databases
• Type-safe at compile time
• Drivers are swappable

Cons:

• Requires understanding interfaces
• Need external packages for actual database

Python's Model: SQLite in Stdlib

import sqlite3
conn = sqlite3.connect('example.db')
c = conn.cursor()
c.execute('SELECT * FROM users')

Pros:

• Zero dependencies for getting started
• Great for learning/prototyping
• Always available

Cons:

• Other databases need packages
• stdlib vs package API differences

Rust's Model: Everything is Packages

use rusqlite::{Connection, Result};

fn main() -> Result<()> {
    let conn = Connection::open("test.db")?;
    conn.execute("CREATE TABLE users (...)", [])?;
    Ok(())
}

Pros:

• Minimal core language
• Best-in-class implementations
• Clear ownership

Cons:

• Cargo.toml management
• Version conflicts possible
• Learning curve for package ecosystem

Elixir's Model: Strong Package Ecosystem

# Ecto is technically a package but universally used
Repo.all(from u in User, where: u.age > 18)

Pros:

• Best API emerges naturally
• Core team can focus on language
• Community ownership

Cons:

• Package can become outdated
• Multiple competing solutions

Arguments For Built-in SQL

1. Effect System Integration

The most compelling argument: SQL fits naturally into Lux's effect system.

// The effect signature documents database access
fn fetchUser(id: Int): User with {Sql} = { ... }

// Handlers enable testing without mocks
handler testDatabase(): Sql { ... }

This is harder to achieve with packages - they'd need to integrate deeply with the effect system.

2. Zero-Dependency Getting Started

New users can immediately:

• Follow tutorials that use databases
• Build real applications
• Learn effects with practical examples

lux run database_example.lux
# Just works - no package installation

3. Guaranteed API Stability

Built-in effects have stable, documented APIs. Package APIs can change between versions.

4. Teaching Functional Effects

SQL is an excellent teaching example for effects:

• Clear side effects (I/O to database)
• Handler swapping for testing
• Transaction scoping

5. Practical Utility

90%+ of real applications need database access. Making it trivial benefits most users.

Arguments For SQL as Package

1. Smaller Binary Size

rusqlite adds significant binary size (~2-3MB). Package-based approach lets users opt-in.

2. Database Backend Choice

Currently locked to SQLite. A package ecosystem could offer:

• lux-sqlite
• lux-postgres
• lux-mysql
• lux-mongodb

3. Faster Core Language Evolution

Core team focuses on language; community builds integrations.

4. Better Specialization

Dedicated package maintainers might build better database tooling than core team.

5. Multiple Competing Implementations

Competition drives quality. The best SQL package wins adoption.

Comparison Matrix

Factor	Built-in	Package
Effect integration	Excellent	Needs design work
Learning curve	Low	Medium
Binary size	Larger	User controls
Database options	Limited	Unlimited
API stability	Guaranteed	Version-dependent
Getting started	Instant	Requires install
Testing story	Built-in handlers	Package-specific
Maintenance burden	Core team	Community

Recommendation

Keep SQL as Built-in Effect, With Changes

Rationale:

1. Effect system is Lux's differentiator - SQL showcases it perfectly
2. Practicality matters - 90% of apps need databases
3. Teaching value - SQL is ideal for learning effects
4. Handler testing - Built-in integration enables powerful testing

Proposed Architecture

Core Lux
├── Sql effect (interface only)
│   ├── open/close
│   ├── execute/query
│   └── transaction operations
│
└── Default SQLite handler (built-in)
    └── Uses rusqlite

Future packages (optional)
├── lux-postgres  -- PostgreSQL handler
├── lux-mysql     -- MySQL handler
└── lux-redis     -- Redis (key-value, not Sql)

Specific Changes to Consider

1. Make SQLite compilation optional

   # Cargo.toml
   [features]
   default = ["sqlite"]
   sqlite = ["rusqlite"]
   

2. Define stable Sql effect interface

   effect Sql {
       fn open(path: String): SqlConn
       fn close(conn: SqlConn): Unit
       fn execute(conn: SqlConn, sql: String): Int
       fn query(conn: SqlConn, sql: String): List<SqlRow>
       // ...
   }
   

3. Allow package handlers to implement Sql

   // In lux-postgres package
   handler postgresHandler(connStr: String): Sql { ... }
   
   // Usage
   run myApp() with {
       Sql -> postgresHandler("postgres://...")
   }
   

4. Add connection pooling to core Important for production, should be standard.

Comparison to Similar Decisions

Console Effect

Console is built-in. Nobody questions this because:

• Universally needed
• Simple interface
• Hard to get wrong

SQL is similar but more complex.

HTTP Effect

HTTP client is built-in in Lux. This was the right call because:

• Most apps need HTTP
• Complex to implement well
• Effect system integration important

SQL follows same reasoning.

File Effect

File I/O is built-in. Same rationale applies.

What Other Effect-System Languages Do

Language	Database	Built-in?
Koka	No database support	N/A
Eff	No database support	N/A
Frank	No database support	N/A
Unison	Abilities + packages	Both

Lux is pioneering practical effects. Built-in SQL makes sense.

Conclusion

SQL should remain a built-in effect in Lux because:

1. It demonstrates the power of effects for real-world use
2. It enables the handler-based testing story
3. It removes friction for most applications
4. It serves as a teaching example for effects

However, the implementation should evolve to:

• Support multiple database backends via handlers
• Make SQLite optional for minimal binaries
• Provide connection pooling
• Add parameterized query support

This hybrid approach gives users the best of both worlds: immediate productivity with built-in SQLite, and flexibility through package-provided handlers for other databases.

---

Future Work

1. Parameterized queries - Critical for SQL injection prevention
2. Connection pooling - Required for production servers
3. PostgreSQL handler - Most requested database
4. Migration support - Schema evolution tooling
5. Type-safe queries - Compile-time SQL checking (ambitious)