diff --git a/CLAUDE.md b/CLAUDE.md index 45ec433..2ee2c35 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -71,6 +71,41 @@ nix develop --command cargo test # All tests pass (currently 381) | `lux serve` | `lux s` | Static file server | | `lux compile` | `lux c` | Compile to binary | +## Documenting Lux Language Errors + +When working on any major task that involves writing Lux code, **document every language error, limitation, or surprising behavior** you encounter. This log is optimized for LLM consumption so future sessions can avoid repeating mistakes. + +**File:** Maintain an `ISSUES.md` in the relevant project directory (e.g., `~/src/blu-site/ISSUES.md`). + +**Format for each entry:** +```markdown +## Issue N: + +**Category**: Parser limitation | Type checker gap | Missing feature | Runtime error | Documentation gap +**Severity**: High | Medium | Low +**Status**: Open | **Fixed** (commit hash or version) + +<1-2 sentence description of the problem> + +**Reproduction:** +```lux +// Minimal code that triggers the issue +``` + +**Error message:** `` + +**Workaround:** + +**Fix:** +``` + +**Rules:** +- Add new issues as you encounter them during any task +- When a previously documented issue gets fixed, update its status to **Fixed** and note the commit/version +- Remove entries that are no longer relevant (e.g., the feature was redesigned entirely) +- Keep the summary table at the bottom of ISSUES.md in sync with the entries +- Do NOT duplicate issues already documented -- check existing entries first + ## Code Quality - Fix all compiler warnings before committing diff --git a/PACKAGES.md b/PACKAGES.md new file mode 100644 index 0000000..2826339 --- /dev/null +++ b/PACKAGES.md @@ -0,0 +1,367 @@ +# Lux Package Ecosystem Plan + +## Current State + +### Stdlib (built-in) +| Module | Coverage | +|--------|----------| +| String | Comprehensive (split, join, trim, indexOf, replace, etc.) | +| List | Good (map, filter, fold, head, tail, concat, range, find, any, all, take, drop) | +| Option | Basic (map, flatMap, getOrElse, isSome, isNone) | +| Result | Basic (map, flatMap, getOrElse, isOk, isErr) | +| Math | Basic (abs, min, max, sqrt, pow, floor, ceil, round) | +| Json | Comprehensive (parse, stringify, get, typed extractors, constructors) | +| File | Good (read, write, append, exists, delete, readDir, isDir, mkdir) | +| Console | Good (print, read, readLine, readInt) | +| Process | Good (exec, execStatus, env, args, exit, cwd) | +| Http | Basic (get, post, put, delete, setHeader) | +| HttpServer | Basic (listen, accept, respond) | +| Time | Minimal (now, sleep) | +| Random | Basic (int, float, bool) | +| Sql | Good (SQLite: open, query, execute, transactions) | +| Postgres | Good (connect, query, execute, transactions) | +| Schema | Niche (versioned data migration) | +| Test | Good (assert, assertEqual, assertTrue) | +| Concurrent | Experimental (spawn, await, yield, cancel) | +| Channel | Experimental (create, send, receive) | + +### Registry (pkgs.lux) - 3 packages +| Package | Version | Notes | +|---------|---------|-------| +| json | 1.0.0 | Wraps stdlib Json with convenience functions (getPath, getString, etc.) | +| http-client | 0.1.0 | Wraps stdlib Http with JSON helpers, URL encoding | +| testing | 0.1.0 | Wraps stdlib Test with describe/it structure | + +--- + +## Gap Analysis + +### What's Missing vs Other Languages + +Compared to ecosystems like Rust/cargo, Go, Python, Elm, Gleam: + +| Category | Gap | Impact | Notes | +|----------|-----|--------|-------| +| **Collections** | No HashMap, Set, Queue, Stack | Critical | List-of-pairs with O(n) lookup is the only option | +| **Sorting** | No List.sort or List.sortBy | High | Must implement insertion sort manually | +| **Date/Time** | Only `Time.now()` (epoch ms), no parsing/formatting | High | blu-site does string-based date formatting manually | +| **Markdown** | No markdown parser | High | blu-site has 300+ lines of hand-rolled markdown | +| **XML/RSS** | No XML generation | High | Can't generate RSS feeds or sitemaps | +| **Regex** | No pattern matching on strings | High | Character-by-character scanning required | +| **Path** | No file path utilities | Medium | basename/dirname manually reimplemented | +| **YAML/TOML** | No config file parsing (beyond JSON) | Medium | Frontmatter parsing is manual | +| **Template** | No string templating | Medium | HTML built via raw string concatenation | +| **URL** | No URL parsing/encoding | Medium | http-client has basic urlEncode but no parser | +| **Crypto** | No hashing (SHA256, etc.) | Medium | Can't do checksums, content hashing | +| **Base64** | No encoding/decoding | Low | Needed for data URIs, some auth | +| **CSV** | No CSV parsing | Low | Common data format | +| **UUID** | No UUID generation | Low | Useful for IDs | +| **Logging** | No structured logging | Low | Just Console.print | +| **CLI** | No argument parsing library | Low | Manual arg handling | + +### What Should Be Stdlib vs Package + +**Should be stdlib additions** (too fundamental to be packages): +- HashMap / Map type (requires runtime support) +- List.sort / List.sortBy (fundamental operation) +- Better Time module (date parsing, formatting) +- Regex (needs runtime/C support for performance) +- Path module (cross-platform file path handling) + +**Should be packages** (application-level, opinionated, composable): +- markdown +- xml +- rss/atom +- frontmatter +- template +- csv +- crypto +- ssg (static site generator framework) + +--- + +## Priority Package Plans + +Ordered by what unblocks blu-site fixes first, then general ecosystem value. + +--- + +### Package 1: `markdown` (Priority: HIGHEST) + +**Why:** The 300-line markdown parser in blu-site's main.lux is general-purpose code that belongs in a reusable package. It's also the most complex part of blu-site and has known bugs (e.g., `### ` inside list items renders literally). + +**Scope:** +``` +markdown/ + lux.toml + lib.lux # Public API: parse, parseInline + src/ + inline.lux # Inline parsing (bold, italic, links, images, code) + block.lux # Block parsing (headings, lists, code blocks, blockquotes, hr) + types.lux # AST types (optional - could emit HTML directly) +``` + +**Public API:** +```lux +// Convert markdown string to HTML string +pub fn toHtml(markdown: String): String + +// Convert inline markdown only (no blocks) +pub fn inlineToHtml(text: String): String + +// Escape HTML entities +pub fn escapeHtml(s: String): String +``` + +**Improvements over current blu-site code:** +- Fix heading-inside-list-item rendering (`- ### Title` should work) +- Support nested lists (currently flat only) +- Support reference-style links `[text][ref]` +- Handle edge cases (empty lines in code blocks, nested blockquotes) +- Proper HTML entity escaping in more contexts + +**Depends on:** Nothing (pure string processing) + +**Estimated size:** ~400-500 lines of Lux + +--- + +### Package 2: `xml` (Priority: HIGH) + +**Why:** Needed for RSS/Atom feed generation, sitemap.xml, and robots.txt generation. General-purpose XML builder that doesn't try to parse XML (which would need regex), just emits it. + +**Scope:** +``` +xml/ + lux.toml + lib.lux # Public API: element, document, serialize +``` + +**Public API:** +```lux +type XmlNode = + | Element(String, List, List) + | Text(String) + | CData(String) + | Comment(String) + | Declaration(String, String) // version, encoding + +type XmlAttr = + | Attr(String, String) + +// Build an XML element +pub fn element(tag: String, attrs: List, children: List): XmlNode + +// Build a text node (auto-escapes) +pub fn text(content: String): XmlNode + +// Build a CDATA section +pub fn cdata(content: String): XmlNode + +// Serialize XML tree to string +pub fn serialize(node: XmlNode): String + +// Serialize with XML declaration header +pub fn document(version: String, encoding: String, root: XmlNode): String + +// Convenience: self-closing element +pub fn selfClosing(tag: String, attrs: List): XmlNode +``` + +**Depends on:** Nothing + +**Estimated size:** ~150-200 lines + +--- + +### Package 3: `rss` (Priority: HIGH) + +**Why:** Directly needed for blu-site's #6 priority fix (add RSS feed). Builds on `xml` package. + +**Scope:** +``` +rss/ + lux.toml # depends on xml + lib.lux # Public API: feed, item, toXml, toAtom +``` + +**Public API:** +```lux +type FeedInfo = + | FeedInfo(String, String, String, String, String) + // title, link, description, language, lastBuildDate + +type FeedItem = + | FeedItem(String, String, String, String, String, String) + // title, link, description, pubDate, guid, categories (comma-separated) + +// Generate RSS 2.0 XML string +pub fn toRss(info: FeedInfo, items: List): String + +// Generate Atom 1.0 XML string +pub fn toAtom(info: FeedInfo, items: List): String +``` + +**Depends on:** `xml` + +**Estimated size:** ~100-150 lines + +--- + +### Package 4: `frontmatter` (Priority: HIGH) + +**Why:** blu-site has ~50 lines of fragile frontmatter parsing. This is a common need for any content-driven Lux project. The current parser uses `String.indexOf(line, ": ")` which breaks on values containing `: `. + +**Scope:** +``` +frontmatter/ + lux.toml + lib.lux # Public API: parse +``` + +**Public API:** +```lux +type FrontmatterResult = + | FrontmatterResult(List<(String, String)>, String) + // key-value pairs, remaining body + +// Parse frontmatter from a string (--- delimited YAML-like header) +pub fn parse(content: String): FrontmatterResult + +// Get a value by key from parsed frontmatter +pub fn get(pairs: List<(String, String)>, key: String): Option + +// Get a value or default +pub fn getOrDefault(pairs: List<(String, String)>, key: String, default: String): String + +// Parse a space-separated tag string into a list +pub fn parseTags(tagString: String): List +``` + +**Improvements over current blu-site code:** +- Handle values with `: ` in them (only split on first `: `) +- Handle multi-line values (indented continuation) +- Handle quoted values with embedded newlines +- Strip quotes from values consistently + +**Depends on:** Nothing + +**Estimated size:** ~100-150 lines + +--- + +### Package 5: `path` (Priority: MEDIUM) + +**Why:** blu-site manually implements `basename` and `dirname`. Any file-processing Lux program needs these. Tiny but universally useful. + +**Scope:** +``` +path/ + lux.toml + lib.lux +``` + +**Public API:** +```lux +// Get filename from path: "/foo/bar.txt" -> "bar.txt" +pub fn basename(p: String): String + +// Get directory from path: "/foo/bar.txt" -> "/foo" +pub fn dirname(p: String): String + +// Get file extension: "file.txt" -> "txt", "file" -> "" +pub fn extension(p: String): String + +// Remove file extension: "file.txt" -> "file" +pub fn stem(p: String): String + +// Join path segments: join("foo", "bar") -> "foo/bar" +pub fn join(a: String, b: String): String + +// Normalize path: "foo//bar/../baz" -> "foo/baz" +pub fn normalize(p: String): String + +// Check if path is absolute +pub fn isAbsolute(p: String): Bool +``` + +**Depends on:** Nothing + +**Estimated size:** ~80-120 lines + +--- + +### Package 6: `sitemap` (Priority: MEDIUM) + +**Why:** Directly needed for blu-site's #9 priority fix. Simple package that generates sitemap.xml. + +**Scope:** +``` +sitemap/ + lux.toml # depends on xml + lib.lux +``` + +**Public API:** +```lux +type SitemapEntry = + | SitemapEntry(String, String, String, String) + // url, lastmod (ISO date), changefreq, priority + +// Generate sitemap.xml string +pub fn generate(entries: List): String + +// Generate a simple robots.txt pointing to the sitemap +pub fn robotsTxt(sitemapUrl: String): String +``` + +**Depends on:** `xml` + +**Estimated size:** ~50-70 lines + +--- + +### Package 7: `ssg` (Priority: LOW - future) + +**Why:** Once markdown, frontmatter, rss, sitemap, and path packages exist, the remaining logic in blu-site's main.lux is generic SSG framework code: read content dirs, parse posts, sort by date, generate section indexes, generate tag pages, copy static assets. This could be extracted into a framework package that other Lux users could use to build their own static sites. + +**This should wait** until the foundation packages above are stable and battle-tested through blu-site usage. + +--- + +## Non-Package Stdlib Improvements Needed + +These gaps are too fundamental to be packages and should be added to the Lux language itself: + +### HashMap (Critical) +Every package above that needs key-value lookups (frontmatter, xml attributes, etc.) is working around the lack of HashMap with `List<(String, String)>`. This is O(n) per lookup and makes code verbose. A stdlib `Map` module would transform the ecosystem. + +### List.sort / List.sortBy (High) +blu-site implements insertion sort manually. Every content-driven app needs sorting. This should be a stdlib function. + +### Time.format / Time.parse (High) +blu-site manually parses "2025-01-15" by substring extraction and maps month numbers to names. A proper date/time library (even just ISO 8601 parsing and basic formatting) would help every package above. + +--- + +## Implementation Order + +``` +Phase 1 (unblock blu-site fixes): + 1. markdown - extract from blu-site, fix bugs, publish + 2. frontmatter - extract from blu-site, improve robustness + 3. path - tiny, universally useful + 4. xml - needed by rss and sitemap + +Phase 2 (complete blu-site features): + 5. rss - depends on xml + 6. sitemap - depends on xml + +Phase 3 (ecosystem growth): + 7. template - string templating (mustache-like) + 8. csv - data processing + 9. cli - argument parsing + 10. ssg - framework extraction from blu-site +``` + +Each package should be developed in its own directory under `~/src/`, published to the git.qrty.ink registry, and tested by integrating it into blu-site.