# Lux Package Ecosystem Plan ## Current State ### Stdlib (built-in) | Module | Coverage | |--------|----------| | String | Comprehensive (split, join, trim, indexOf, replace, etc.) | | List | Good (map, filter, fold, head, tail, concat, range, find, any, all, take, drop) | | Option | Basic (map, flatMap, getOrElse, isSome, isNone) | | Result | Basic (map, flatMap, getOrElse, isOk, isErr) | | Math | Basic (abs, min, max, sqrt, pow, floor, ceil, round) | | Json | Comprehensive (parse, stringify, get, typed extractors, constructors) | | File | Good (read, write, append, exists, delete, readDir, isDir, mkdir) | | Console | Good (print, read, readLine, readInt) | | Process | Good (exec, execStatus, env, args, exit, cwd) | | Http | Basic (get, post, put, delete, setHeader) | | HttpServer | Basic (listen, accept, respond) | | Time | Minimal (now, sleep) | | Random | Basic (int, float, bool) | | Sql | Good (SQLite: open, query, execute, transactions) | | Postgres | Good (connect, query, execute, transactions) | | Schema | Niche (versioned data migration) | | Test | Good (assert, assertEqual, assertTrue) | | Concurrent | Experimental (spawn, await, yield, cancel) | | Channel | Experimental (create, send, receive) | ### Registry (pkgs.lux) - 3 packages | Package | Version | Notes | |---------|---------|-------| | json | 1.0.0 | Wraps stdlib Json with convenience functions (getPath, getString, etc.) | | http-client | 0.1.0 | Wraps stdlib Http with JSON helpers, URL encoding | | testing | 0.1.0 | Wraps stdlib Test with describe/it structure | --- ## Gap Analysis ### What's Missing vs Other Languages Compared to ecosystems like Rust/cargo, Go, Python, Elm, Gleam: | Category | Gap | Impact | Notes | |----------|-----|--------|-------| | **Collections** | No HashMap, Set, Queue, Stack | Critical | List-of-pairs with O(n) lookup is the only option | | **Sorting** | No List.sort or List.sortBy | High | Must implement insertion sort manually | | **Date/Time** | Only `Time.now()` (epoch ms), no parsing/formatting | High | blu-site does string-based date formatting manually | | **Markdown** | No markdown parser | High | blu-site has 300+ lines of hand-rolled markdown | | **XML/RSS** | No XML generation | High | Can't generate RSS feeds or sitemaps | | **Regex** | No pattern matching on strings | High | Character-by-character scanning required | | **Path** | No file path utilities | Medium | basename/dirname manually reimplemented | | **YAML/TOML** | No config file parsing (beyond JSON) | Medium | Frontmatter parsing is manual | | **Template** | No string templating | Medium | HTML built via raw string concatenation | | **URL** | No URL parsing/encoding | Medium | http-client has basic urlEncode but no parser | | **Crypto** | No hashing (SHA256, etc.) | Medium | Can't do checksums, content hashing | | **Base64** | No encoding/decoding | Low | Needed for data URIs, some auth | | **CSV** | No CSV parsing | Low | Common data format | | **UUID** | No UUID generation | Low | Useful for IDs | | **Logging** | No structured logging | Low | Just Console.print | | **CLI** | No argument parsing library | Low | Manual arg handling | ### What Should Be Stdlib vs Package **Should be stdlib additions** (too fundamental to be packages): - HashMap / Map type (requires runtime support) - List.sort / List.sortBy (fundamental operation) - Better Time module (date parsing, formatting) - Regex (needs runtime/C support for performance) - Path module (cross-platform file path handling) **Should be packages** (application-level, opinionated, composable): - markdown - xml - rss/atom - frontmatter - template - csv - crypto - ssg (static site generator framework) --- ## Priority Package Plans Ordered by what unblocks blu-site fixes first, then general ecosystem value. --- ### Package 1: `markdown` (Priority: HIGHEST) **Why:** The 300-line markdown parser in blu-site's main.lux is general-purpose code that belongs in a reusable package. It's also the most complex part of blu-site and has known bugs (e.g., `### ` inside list items renders literally). **Scope:** ``` markdown/ lux.toml lib.lux # Public API: parse, parseInline src/ inline.lux # Inline parsing (bold, italic, links, images, code) block.lux # Block parsing (headings, lists, code blocks, blockquotes, hr) types.lux # AST types (optional - could emit HTML directly) ``` **Public API:** ```lux // Convert markdown string to HTML string pub fn toHtml(markdown: String): String // Convert inline markdown only (no blocks) pub fn inlineToHtml(text: String): String // Escape HTML entities pub fn escapeHtml(s: String): String ``` **Improvements over current blu-site code:** - Fix heading-inside-list-item rendering (`- ### Title` should work) - Support nested lists (currently flat only) - Support reference-style links `[text][ref]` - Handle edge cases (empty lines in code blocks, nested blockquotes) - Proper HTML entity escaping in more contexts **Depends on:** Nothing (pure string processing) **Estimated size:** ~400-500 lines of Lux --- ### Package 2: `xml` (Priority: HIGH) **Why:** Needed for RSS/Atom feed generation, sitemap.xml, and robots.txt generation. General-purpose XML builder that doesn't try to parse XML (which would need regex), just emits it. **Scope:** ``` xml/ lux.toml lib.lux # Public API: element, document, serialize ``` **Public API:** ```lux type XmlNode = | Element(String, List, List) | Text(String) | CData(String) | Comment(String) | Declaration(String, String) // version, encoding type XmlAttr = | Attr(String, String) // Build an XML element pub fn element(tag: String, attrs: List, children: List): XmlNode // Build a text node (auto-escapes) pub fn text(content: String): XmlNode // Build a CDATA section pub fn cdata(content: String): XmlNode // Serialize XML tree to string pub fn serialize(node: XmlNode): String // Serialize with XML declaration header pub fn document(version: String, encoding: String, root: XmlNode): String // Convenience: self-closing element pub fn selfClosing(tag: String, attrs: List): XmlNode ``` **Depends on:** Nothing **Estimated size:** ~150-200 lines --- ### Package 3: `rss` (Priority: HIGH) **Why:** Directly needed for blu-site's #6 priority fix (add RSS feed). Builds on `xml` package. **Scope:** ``` rss/ lux.toml # depends on xml lib.lux # Public API: feed, item, toXml, toAtom ``` **Public API:** ```lux type FeedInfo = | FeedInfo(String, String, String, String, String) // title, link, description, language, lastBuildDate type FeedItem = | FeedItem(String, String, String, String, String, String) // title, link, description, pubDate, guid, categories (comma-separated) // Generate RSS 2.0 XML string pub fn toRss(info: FeedInfo, items: List): String // Generate Atom 1.0 XML string pub fn toAtom(info: FeedInfo, items: List): String ``` **Depends on:** `xml` **Estimated size:** ~100-150 lines --- ### Package 4: `frontmatter` (Priority: HIGH) **Why:** blu-site has ~50 lines of fragile frontmatter parsing. This is a common need for any content-driven Lux project. The current parser uses `String.indexOf(line, ": ")` which breaks on values containing `: `. **Scope:** ``` frontmatter/ lux.toml lib.lux # Public API: parse ``` **Public API:** ```lux type FrontmatterResult = | FrontmatterResult(List<(String, String)>, String) // key-value pairs, remaining body // Parse frontmatter from a string (--- delimited YAML-like header) pub fn parse(content: String): FrontmatterResult // Get a value by key from parsed frontmatter pub fn get(pairs: List<(String, String)>, key: String): Option // Get a value or default pub fn getOrDefault(pairs: List<(String, String)>, key: String, default: String): String // Parse a space-separated tag string into a list pub fn parseTags(tagString: String): List ``` **Improvements over current blu-site code:** - Handle values with `: ` in them (only split on first `: `) - Handle multi-line values (indented continuation) - Handle quoted values with embedded newlines - Strip quotes from values consistently **Depends on:** Nothing **Estimated size:** ~100-150 lines --- ### Package 5: `path` (Priority: MEDIUM) **Why:** blu-site manually implements `basename` and `dirname`. Any file-processing Lux program needs these. Tiny but universally useful. **Scope:** ``` path/ lux.toml lib.lux ``` **Public API:** ```lux // Get filename from path: "/foo/bar.txt" -> "bar.txt" pub fn basename(p: String): String // Get directory from path: "/foo/bar.txt" -> "/foo" pub fn dirname(p: String): String // Get file extension: "file.txt" -> "txt", "file" -> "" pub fn extension(p: String): String // Remove file extension: "file.txt" -> "file" pub fn stem(p: String): String // Join path segments: join("foo", "bar") -> "foo/bar" pub fn join(a: String, b: String): String // Normalize path: "foo//bar/../baz" -> "foo/baz" pub fn normalize(p: String): String // Check if path is absolute pub fn isAbsolute(p: String): Bool ``` **Depends on:** Nothing **Estimated size:** ~80-120 lines --- ### Package 6: `sitemap` (Priority: MEDIUM) **Why:** Directly needed for blu-site's #9 priority fix. Simple package that generates sitemap.xml. **Scope:** ``` sitemap/ lux.toml # depends on xml lib.lux ``` **Public API:** ```lux type SitemapEntry = | SitemapEntry(String, String, String, String) // url, lastmod (ISO date), changefreq, priority // Generate sitemap.xml string pub fn generate(entries: List): String // Generate a simple robots.txt pointing to the sitemap pub fn robotsTxt(sitemapUrl: String): String ``` **Depends on:** `xml` **Estimated size:** ~50-70 lines --- ### Package 7: `ssg` (Priority: LOW - future) **Why:** Once markdown, frontmatter, rss, sitemap, and path packages exist, the remaining logic in blu-site's main.lux is generic SSG framework code: read content dirs, parse posts, sort by date, generate section indexes, generate tag pages, copy static assets. This could be extracted into a framework package that other Lux users could use to build their own static sites. **This should wait** until the foundation packages above are stable and battle-tested through blu-site usage. --- ## Non-Package Stdlib Improvements Needed These gaps are too fundamental to be packages and should be added to the Lux language itself: ### HashMap (Critical) Every package above that needs key-value lookups (frontmatter, xml attributes, etc.) is working around the lack of HashMap with `List<(String, String)>`. This is O(n) per lookup and makes code verbose. A stdlib `Map` module would transform the ecosystem. ### List.sort / List.sortBy (High) blu-site implements insertion sort manually. Every content-driven app needs sorting. This should be a stdlib function. ### Time.format / Time.parse (High) blu-site manually parses "2025-01-15" by substring extraction and maps month numbers to names. A proper date/time library (even just ISO 8601 parsing and basic formatting) would help every package above. --- ## Implementation Order ``` Phase 1 (unblock blu-site fixes): 1. markdown - extract from blu-site, fix bugs, publish 2. frontmatter - extract from blu-site, improve robustness 3. path - tiny, universally useful 4. xml - needed by rss and sitemap Phase 2 (complete blu-site features): 5. rss - depends on xml 6. sitemap - depends on xml Phase 3 (ecosystem growth): 7. template - string templating (mustache-like) 8. csv - data processing 9. cli - argument parsing 10. ssg - framework extraction from blu-site ``` Each package should be developed in its own directory under `~/src/`, published to the git.qrty.ink registry, and tested by integrating it into blu-site.