commit b40ac9952465d98ac8eb410a0a0f5ad024e39dc0 Author: Brandon Lucas Date: Fri Feb 13 01:44:00 2026 -0500 Initial commit: Ultimate Notetaking, Sync & Backup System A NixOS-based system for managing personal data across three tiers: - Tier 1: Configuration (shareable via git) - Tier 2: Syncable data (nb + Syncthing) - Tier 3: Large data (self-hosted services + backup) Includes: - NixOS modules for nb, Syncthing, backup (restic) - Server modules for Forgejo, Immich, Jellyfin - Helper scripts (usync, ustatus) - Comprehensive documentation Co-Authored-By: Claude Opus 4.5 diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..4b9a797 --- /dev/null +++ b/.gitignore @@ -0,0 +1,28 @@ +# Nix +result +result-* +.direnv/ + +# Secrets (NEVER commit unencrypted secrets) +secrets/*.yaml +!secrets/secrets.yaml.example +*.key +*.pem +.age +.sops + +# Editor +*.swp +*.swo +*~ +.vim/ +.nvim/ +.vscode/ +.idea/ + +# OS +.DS_Store +Thumbs.db + +# Hardware configs (machine-specific) +hosts/*/hardware-configuration.nix diff --git a/README.md b/README.md new file mode 100644 index 0000000..2bbec39 --- /dev/null +++ b/README.md @@ -0,0 +1,292 @@ +# Ultimate Notetaking, Sync & Backup System + +A NixOS-based system for managing the three types of data in a computer: + +| Tier | Type | Examples | Sync Model | +|------|------|----------|------------| +| **1** | Configuration | flake.nix, dotfiles | Git (public, shareable) | +| **2** | Syncable Data | Notes, documents | Git (nb) + Syncthing | +| **3** | Large Data | Photos, videos, repos | Central backup, on-demand | + +## Quick Start + +```bash +# Clone this repo +git clone https://github.com/YOUR_USERNAME/ultimate-notetaking-sync-backup-system.git +cd ultimate-notetaking-sync-backup-system + +# Option 1: Just try the tools (no system changes) +nix develop + +# Option 2: Apply as a NixOS module +# Add to your flake.nix inputs, then import the module +``` + +## Philosophy + +1. **Config is code**: Your system configuration should be version-controlled and shareable +2. **Notes deserve versioning**: Use git-backed tools (nb) for notes, not just file sync +3. **Sync ≠ Backup**: Syncthing syncs; restic backs up. Use both. +4. **Self-host when practical**: Forgejo, Immich, Jellyfin over proprietary clouds +5. **Open source only**: Every tool in this stack is FOSS + +## Architecture + +``` +┌─────────────────────────────────────────────────────────────┐ +│ YOUR MACHINES │ +│ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ +│ │ Desktop │ │ Laptop │ │ Phone │ │ +│ └────┬────┘ └────┬────┘ └────┬────┘ │ +│ │ │ │ │ +│ └──────────────┼──────────────┘ │ +│ │ │ +│ ┌───────▼───────┐ │ +│ │ TIER 2 │ │ +│ │ nb (notes) │◄── git push/pull │ +│ │ Syncthing │◄── P2P sync (optional) │ +│ └───────────────┘ │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ YOUR SERVER │ +│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ +│ │ Forgejo │ │ Immich │ │ Jellyfin │ │ +│ │ (git/nb) │ │ (photos) │ │ (media) │ │ +│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │ +│ └────────────────┼────────────────┘ │ +│ │ │ +│ ┌──────▼──────┐ │ +│ │ restic │ │ +│ │ (backup) │ │ +│ └──────┬──────┘ │ +│ │ │ +│ ┌──────▼──────┐ │ +│ │ B2 / NAS │ │ +│ │ (offsite) │ │ +│ └─────────────┘ │ +└─────────────────────────────────────────────────────────────┘ +``` + +## What's Included + +### NixOS Modules + +- `modules/nb.nix` - nb notebook CLI installation and configuration +- `modules/syncthing.nix` - Declarative Syncthing setup +- `modules/neovim.nix` - Neovim with nb integration +- `modules/backup.nix` - restic backup timers +- `modules/server/` - Forgejo, Immich, Jellyfin configurations + +### Development Shell + +```bash +nix develop +``` + +Provides: `nb`, `syncthing`, `restic`, `rclone`, `jq`, and helper scripts. + +### Helper Scripts + +- `usync` - Unified sync command (syncs nb + triggers Syncthing scan) +- `ubackup` - Run restic backup manually +- `ustatus` - Show sync/backup status across all tiers + +## Usage + +### Tier 1: Configuration + +Your system config lives in this repo. Fork it, customize it, push to your own GitHub. + +```bash +# Rebuild your system +sudo nixos-rebuild switch --flake .#your-hostname + +# Update flake inputs +nix flake update +``` + +### Tier 2: Notes with nb + +```bash +# Create a note +nb add "My note title" + +# Edit with neovim +nb edit 1 + +# Sync to remote +nb sync + +# Search notes +nb search "keyword" + +# List notebooks +nb notebooks +``` + +Set up git remote for nb: +```bash +nb remote set git@your-forgejo:you/notes.git +``` + +### Tier 2: Documents with Syncthing + +Documents in `~/Documents/` sync automatically via Syncthing. + +```bash +# Check Syncthing status +syncthing cli show system + +# Force rescan +syncthing cli scan --folder documents +``` + +### Tier 3: Large Data + +Photos upload to Immich (mobile app or web). +Media is served via Jellyfin. +Manual files can be uploaded with rclone: + +```bash +# Upload to your server +rclone copy ~/Videos/project server:archive/videos/ + +# List remote files +rclone ls server:archive/ +``` + +## Customization + +### Adding Your Own Notebooks + +Edit `modules/nb.nix`: + +```nix +{ + programs.nb = { + notebooks = { + personal = { + remote = "git@forgejo:you/personal-notes.git"; + }; + work = { + remote = "git@forgejo:you/work-notes.git"; + }; + }; + }; +} +``` + +### Syncthing Folders + +Edit `modules/syncthing.nix`: + +```nix +{ + services.syncthing.settings.folders = { + "documents" = { + path = "~/Documents"; + devices = [ "laptop" "desktop" ]; + }; + "music" = { + path = "~/Music"; + devices = [ "laptop" "desktop" "server" ]; + }; + }; +} +``` + +### Backup Paths + +Edit `modules/backup.nix`: + +```nix +{ + services.restic.backups.default = { + paths = [ + "/home/you/Documents" + "/home/you/notes" + "/var/lib/important" + ]; + }; +} +``` + +## FAQ + +### Why nb instead of Obsidian/Notion/etc? + +- **Git-native**: Full version history, proper merges +- **Plain text**: Markdown files, no vendor lock-in +- **CLI-first**: Works over SSH, in tmux, everywhere +- **Scriptable**: Integrates with shell workflows +- **Encrypted option**: Built-in GPG encryption + +### Why not just Syncthing for everything? + +Syncthing is great for binary files but poor for text conflicts. When you edit notes on multiple devices, you want git-style merges, not `.sync-conflict` files scattered everywhere. + +### Is Syncthing buggy? + +No. See [our research](./docs/research/sync-tools-comparison.md). Common issues are: +- Treating it as a backup (it's not) +- Android battery killing the app (use Syncthing-Fork) +- Expecting cloud-style behavior from P2P sync + +### Why self-host? + +- **Control**: Your data on your hardware +- **Privacy**: No third-party access +- **Cost**: One-time hardware vs. monthly subscriptions +- **Learning**: Valuable sysadmin experience + +But cloud is fine too. This system works with GitHub, Backblaze B2, etc. + +## Directory Structure + +``` +. +├── flake.nix # Entry point +├── flake.lock +├── README.md +├── docs/ +│ ├── research/ +│ │ └── sync-tools-comparison.md +│ ├── ARCHITECTURE.md +│ └── LLM-CONTEXT.md # For AI assistants +├── modules/ +│ ├── nb.nix +│ ├── syncthing.nix +│ ├── neovim.nix +│ ├── backup.nix +│ └── server/ +│ ├── forgejo.nix +│ ├── immich.nix +│ └── jellyfin.nix +├── hosts/ +│ ├── desktop/ +│ │ └── configuration.nix +│ ├── laptop/ +│ │ └── configuration.nix +│ └── server/ +│ └── configuration.nix +├── home/ +│ └── default.nix # home-manager config +└── scripts/ + ├── usync + ├── ubackup + └── ustatus +``` + +## Contributing + +PRs welcome! Please read [ARCHITECTURE.md](./docs/ARCHITECTURE.md) first. + +## License + +MIT + +--- + +*Built with Nix, because reproducibility matters.* diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 0000000..ccb266b --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,267 @@ +# System Architecture + +## Design Principles + +### 1. Declarative Everything + +Every aspect of this system is declared in Nix. No imperative setup steps, no hidden state. + +``` +Desired State (Nix) → nixos-rebuild → Actual State +``` + +### 2. Separation of Concerns + +``` +┌──────────────────────────────────────────────────────────────┐ +│ CONFIGURATION LAYER │ +│ What: How machines should be configured │ +│ Where: This repo (flake.nix, modules/) │ +│ Managed by: Git + GitHub │ +│ Shareable: Yes (public) │ +└──────────────────────────────────────────────────────────────┘ + │ + ▼ +┌──────────────────────────────────────────────────────────────┐ +│ DATA LAYER │ +│ │ +│ ┌────────────────────┐ ┌────────────────────┐ │ +│ │ TIER 2: Sync │ │ TIER 3: Backup │ │ +│ │ │ │ │ │ +│ │ ~/notes/ (nb) │ │ Photos (Immich) │ │ +│ │ ~/Documents/ │ │ Media (Jellyfin) │ │ +│ │ │ │ Repos (Forgejo) │ │ +│ │ Managed by: │ │ │ │ +│ │ - nb (git sync) │ │ Managed by: │ │ +│ │ - Syncthing │ │ - Server apps │ │ +│ │ │ │ - restic backup │ │ +│ │ Multi-device: Yes │ │ Multi-device: No │ │ +│ │ Real-time: Yes │ │ (on-demand access)│ │ +│ └────────────────────┘ └────────────────────┘ │ +└──────────────────────────────────────────────────────────────┘ +``` + +### 3. No Vendor Lock-in + +Every component can be replaced: + +| Component | Could be replaced with | +|-----------|----------------------| +| nb | Obsidian + git, org-mode, plain vim + git | +| Syncthing | Unison, rclone bisync | +| Forgejo | Gitea, GitLab, bare git | +| Immich | PhotoPrism, any DAM | +| Jellyfin | Plex (non-FOSS), Kodi | +| restic | borg, duplicity | + +### 4. Offline-First + +All Tier 2 data is fully available offline: +- nb notebooks are local git repos +- Syncthing folders are local directories +- Sync happens when connectivity allows + +Tier 3 data is available via caching or on-demand download. + +## Data Flow + +### Tier 1: Configuration + +``` +┌─────────┐ git push ┌─────────┐ +│ Local │ ───────────────► │ GitHub │ +│ Repo │ ◄─────────────── │ (pub) │ +└────┬────┘ git pull └─────────┘ + │ + │ nixos-rebuild switch + ▼ +┌─────────┐ +│ Machine │ +│ State │ +└─────────┘ +``` + +### Tier 2: Notes (nb) + +``` +┌─────────┐ ┌─────────┐ +│ Device │ nb sync │ Forgejo │ +│ A │ ◄────────────────► │ (priv) │ +└─────────┘ └────┬────┘ + │ +┌─────────┐ nb sync ┌────▼────┐ +│ Device │ ◄────────────────► │ Forgejo │ +│ B │ │ (priv) │ +└─────────┘ └─────────┘ + +Conflict Resolution: Git 3-way merge +History: Full git history preserved +``` + +### Tier 2: Documents (Syncthing) + +``` +┌─────────┐ ┌─────────┐ +│ Device │ ◄──── P2P ───────► │ Device │ +│ A │ │ B │ +└────┬────┘ └────┬────┘ + │ │ + │ ┌─────────┐ │ + └────────►│ Device │◄─────────┘ + │ C │ + └─────────┘ + +Conflict Resolution: .sync-conflict files (manual) +History: Syncthing versioning (configurable) +``` + +### Tier 3: Large Data + +``` +┌─────────┐ upload ┌─────────┐ +│ Device │ ──────────────► │ Server │ +│ │ ◄────────────── │ Immich/ │ +└─────────┘ on-demand │ Jellyfin│ + └────┬────┘ + │ restic + ┌────▼────┐ + │ Backup │ + │ B2/NAS │ + └─────────┘ +``` + +## Module Dependency Graph + +``` + flake.nix + │ + ┌───────────────┼───────────────┐ + ▼ ▼ ▼ + hosts/ modules/ home/ + desktop (shared) default.nix + laptop │ + server │ + │ │ + ▼ ▼ + ┌──────────────────────────────────┐ + │ modules/ │ + │ ┌──────┐ ┌───────────┐ ┌─────┐ │ + │ │ nb │ │ syncthing │ │nvim │ │ + │ └──────┘ └───────────┘ └─────┘ │ + │ ┌────────┐ │ + │ │ backup │ │ + │ └────────┘ │ + │ │ + │ ┌─────────── server/ ────────┐ │ + │ │ forgejo immich jellyfin │ │ + │ └───────────────────────────┘ │ + └──────────────────────────────────┘ +``` + +## Security Model + +### Secrets + +``` +┌─────────────────────────────────────────┐ +│ sops-nix │ +│ │ +│ secrets.yaml (encrypted in repo) │ +│ │ │ +│ │ age/gpg key (not in repo) │ +│ ▼ │ +│ Decrypted at activation time │ +│ Placed in /run/secrets/ │ +│ Permissions set per-secret │ +└─────────────────────────────────────────┘ +``` + +### Network Security + +- All sync over TLS (Syncthing) or SSH (nb, Forgejo) +- Server services behind reverse proxy (Caddy/nginx) +- No ports exposed except 443 (HTTPS) +- Tailscale/WireGuard for private network (optional) + +## Backup Strategy + +``` +┌────────────────────────────────────────────────────────────┐ +│ 3-2-1 Backup Rule │ +│ │ +│ 3 copies: │ +│ ├── Primary: Local device │ +│ ├── Secondary: Home server │ +│ └── Tertiary: Offsite (B2/cloud) │ +│ │ +│ 2 media types: │ +│ ├── SSD (local) │ +│ └── Cloud object storage (offsite) │ +│ │ +│ 1 offsite: │ +│ └── Backblaze B2 / AWS S3 / etc. │ +└────────────────────────────────────────────────────────────┘ + +Backup Schedule (systemd timers): +├── Tier 2 (notes/docs): Daily, keep 30 days +├── Tier 3 (media): Weekly, keep 90 days +└── Server state: Daily, keep 14 days +``` + +## Scaling Considerations + +### Adding a New Machine + +1. Create `hosts//configuration.nix` +2. Add hardware-configuration.nix +3. Add to flake.nix outputs +4. Run `nixos-rebuild switch --flake .#` +5. nb automatically syncs when `nb sync` is run +6. Syncthing auto-discovers via device IDs in config + +### Adding a New User + +1. Add user to `hosts//configuration.nix` +2. Create home-manager config +3. Set up nb notebooks for user +4. Add Syncthing device ID + +### Multiple Servers + +The architecture supports multiple servers: + +``` +┌─────────┐ ┌─────────┐ ┌─────────┐ +│ Server │ │ Server │ │ Server │ +│ Home │ │ VPS │ │ NAS │ +│ (media) │ │ (Forgejo│ │(backup) │ +└─────────┘ └─────────┘ └─────────┘ +``` + +Each server has its own host config in `hosts/`. + +## Failure Modes + +| Failure | Impact | Recovery | +|---------|--------|----------| +| Device lost | Tier 2 data unavailable locally | Sync from other devices or Forgejo | +| Server down | Tier 3 unavailable, nb sync fails | Use local cache, fix server | +| Backup failure | No new backups | Alert (systemd OnFailure), investigate | +| Sync conflict | Manual resolution needed | Resolve .sync-conflict or git merge | +| Internet down | Can't sync | Work offline, sync when back | + +## Future Considerations + +### Potential Additions + +1. **Encrypted Tier 2**: Full-disk encryption + nb's GPG encryption +2. **Mobile support**: Termux + Unison, or Syncthing-Fork +3. **Real-time collaboration**: Automerge/CRDT-based notes +4. **Monitoring**: Prometheus + Grafana for sync/backup status + +### Not In Scope + +1. Windows support (NixOS only) +2. Proprietary services integration +3. Enterprise/team features +4. Real-time streaming replication diff --git a/docs/LLM-CONTEXT.md b/docs/LLM-CONTEXT.md new file mode 100644 index 0000000..07f11af --- /dev/null +++ b/docs/LLM-CONTEXT.md @@ -0,0 +1,240 @@ +# LLM Context Document + +This document provides context for AI assistants (Claude, GPT, Copilot, etc.) working with this codebase. + +## Project Overview + +**Name**: Ultimate Notetaking, Sync & Backup System +**Purpose**: NixOS-based declarative system for managing personal data across multiple machines +**Primary User**: Linux users, especially NixOS, who want full control over their data + +## Core Concepts + +### The Three-Tier Data Model + +``` +TIER 1: Configuration (Shareable) +├── What: System config, dotfiles, flake.nix +├── Where: This git repo (public on GitHub) +├── How: NixOS + home-manager + sops-nix +└── Philosophy: Anyone can clone and use this config + +TIER 2: Syncable Data (Private, Multi-device) +├── What: Notes, documents, writings +├── Where: ~/notes/ (nb), ~/Documents/ (Syncthing) +├── How: nb (git-backed) for text, Syncthing for binary +└── Philosophy: Edit anywhere, sync automatically + +TIER 3: Large Data (Backed up, On-demand) +├── What: Photos, videos, large repos +├── Where: Self-hosted servers (Immich, Jellyfin, Forgejo) +├── How: Upload to server, access on-demand, restic backup +└── Philosophy: Don't sync everything; backup everything +``` + +### Key Tools + +| Tool | Purpose | Tier | +|------|---------|------| +| NixOS | Declarative system configuration | 1 | +| home-manager | Declarative user configuration | 1 | +| sops-nix | Encrypted secrets in git | 1 | +| nb | CLI notebook with git sync | 2 | +| Syncthing | P2P file synchronization | 2 | +| Neovim | Primary editor | 1, 2 | +| Forgejo | Self-hosted git (GitHub alternative) | 3 | +| Immich | Self-hosted photos (Google Photos alternative) | 3 | +| Jellyfin | Self-hosted media (Plex alternative) | 3 | +| restic | Encrypted, deduplicated backups | 3 | +| rclone | Cloud storage Swiss Army knife | 3 | + +## File Structure + +``` +flake.nix # Nix flake entry point +├── inputs: nixpkgs, home-manager, sops-nix +├── outputs: nixosConfigurations, homeConfigurations, devShells +└── imports modules for each host + +modules/ +├── nb.nix # nb installation and config +├── syncthing.nix # Declarative Syncthing +├── neovim.nix # Neovim with nb integration +├── backup.nix # restic systemd timers +└── server/ # Server-only modules + ├── forgejo.nix + ├── immich.nix + └── jellyfin.nix + +hosts/ +├── desktop/ # Desktop-specific config +├── laptop/ # Laptop-specific config +└── server/ # Server-specific config + +home/ +└── default.nix # home-manager user config + +scripts/ +├── usync # Unified sync (nb + Syncthing) +├── ubackup # Manual backup trigger +└── ustatus # Status dashboard +``` + +## Common Tasks + +### User wants to add a new notebook + +1. Edit `modules/nb.nix` +2. Add notebook to `programs.nb.notebooks` +3. Run `nixos-rebuild switch` +4. Run `nb notebooks` to verify + +### User wants to add a Syncthing folder + +1. Edit `modules/syncthing.nix` +2. Add folder to `services.syncthing.settings.folders` +3. Add device IDs if new devices +4. Run `nixos-rebuild switch` + +### User wants to change backup paths + +1. Edit `modules/backup.nix` +2. Modify `services.restic.backups.default.paths` +3. Run `nixos-rebuild switch` + +### User wants to add a new host + +1. Create `hosts//configuration.nix` +2. Add to `flake.nix` outputs +3. Configure host-specific settings +4. Run `nixos-rebuild switch --flake .#` + +## Code Patterns + +### Module Structure + +All modules follow this pattern: + +```nix +{ config, lib, pkgs, ... }: + +with lib; + +let + cfg = config.services.ourService; +in { + options.services.ourService = { + enable = mkEnableOption "our service"; + + someOption = mkOption { + type = types.str; + default = "default-value"; + description = "Description of the option"; + }; + }; + + config = mkIf cfg.enable { + # Implementation + }; +} +``` + +### Home-manager Integration + +User-level config uses home-manager: + +```nix +{ config, pkgs, ... }: + +{ + programs.git = { + enable = true; + userName = "User Name"; + userEmail = "user@example.com"; + }; +} +``` + +### Secrets Management + +Secrets use sops-nix: + +```nix +{ + sops.secrets.syncthing-key = { + sopsFile = ./secrets/secrets.yaml; + owner = "user"; + }; +} +``` + +## Constraints + +1. **Open source only**: No proprietary tools +2. **NixOS-centric**: Must work with `nixos-rebuild` +3. **Flakes required**: All Nix code uses flakes +4. **No cloud dependencies**: Must work fully self-hosted (cloud optional) +5. **CLI-first**: GUI is optional, never required +6. **Markdown for notes**: Plain text, portable, version-control friendly + +## When Helping Users + +### DO: +- Suggest declarative (Nix) solutions +- Recommend nb for text/notes +- Recommend Syncthing for binary files +- Use systemd for services +- Use home-manager for user config +- Keep modules composable + +### DON'T: +- Suggest proprietary alternatives +- Hardcode paths (use options) +- Mix system and user config inappropriately +- Ignore the three-tier model +- Suggest Syncthing for notes (use nb instead) + +## Debugging + +### Common Issues + +| Symptom | Likely Cause | Solution | +|---------|--------------|----------| +| nb sync fails | SSH key not available | Check `ssh-add -l` | +| Syncthing not starting | Declarative config conflict | Check for GUI-made changes | +| Backup fails | Repository not initialized | Run `restic init` | +| Module not found | Not imported in flake.nix | Add to imports | + +### Useful Commands + +```bash +# Check Nix config +nix flake check + +# Show Syncthing status +syncthing cli show system + +# nb sync status +nb sync status + +# restic backup status +restic -r snapshots +``` + +## Extension Points + +Users commonly want to extend: + +1. **Additional notebooks**: Add to `modules/nb.nix` +2. **New Syncthing folders**: Add to `modules/syncthing.nix` +3. **Backup exclusions**: Modify `modules/backup.nix` +4. **Server services**: Add new modules under `modules/server/` +5. **Editor config**: Modify `modules/neovim.nix` + +## Related Documentation + +- [Architecture](./ARCHITECTURE.md) - Detailed system design +- [Sync Tools Comparison](./research/sync-tools-comparison.md) - Why we chose these tools +- [nb documentation](https://xwmx.github.io/nb/) +- [NixOS manual](https://nixos.org/manual/nixos/stable/) +- [home-manager manual](https://nix-community.github.io/home-manager/) diff --git a/docs/research/sync-tools-comparison.md b/docs/research/sync-tools-comparison.md new file mode 100644 index 0000000..daab350 --- /dev/null +++ b/docs/research/sync-tools-comparison.md @@ -0,0 +1,271 @@ +# File Synchronization Tools: Comprehensive Comparison + +*Research conducted: February 2026* + +## Executive Summary + +After extensive research, **Syncthing is not buggy—but it is complex and unforgiving**. Most reported "bugs" are actually: +1. Misunderstanding of how P2P sync works +2. Conflict handling that differs from user expectations +3. Android-specific issues (now largely resolved with forks) +4. Users treating it as a backup tool (it's not) + +For a NixOS-based notes/data sync system, the recommended approach is: +- **Tier 2 (Notes)**: `nb` with git sync to private Forgejo (explicit, versioned) +- **Tier 2 (Documents)**: Syncthing OR Unison (based on preference) +- **Tier 3 (Large files)**: rclone + restic to self-hosted or cloud storage + +--- + +## Tool-by-Tool Analysis + +### Syncthing + +**What it is**: Continuous, real-time, P2P file synchronization + +**Strengths**: +- Truly decentralized (no server required) +- Real-time sync via filesystem watching +- Handles file renames/moves efficiently +- Cross-platform with good GUI +- Battle-tested with large user base +- Declarative NixOS module available + +**Weaknesses**: +- **Conflict handling is file-level**: Creates `.sync-conflict-*` files that must be manually resolved +- **Not a backup**: Deletions propagate immediately; if you delete on one device, it's gone everywhere +- **Android app discontinued** (Dec 2024): Use [Syncthing-Fork](https://github.com/catfriend1/syncthing-android) instead +- **Complexity ceiling**: Multi-device topologies can create unexpected sync loops +- **No ownership/permission sync**: By design, to avoid requiring root +- **OOM issues in v2.0**: On low-memory devices (1GB RAM) with large file sets + +**Common Misconceptions**: +| Issue Reported | Actual Cause | +|----------------|--------------| +| "Files disappearing" | Deletions syncing correctly; user expected backup behavior | +| "Sync stuck at 99%" | Large files still transferring, or ignored files being counted | +| "Random conflicts" | Multiple devices modifying same file; Syncthing is doing its job | +| "Ignoring read-only" | Misconfigured folder type; "Receive Only" ≠ read-only filesystem | + +**Verdict**: Syncthing works well if you understand that it's **sync, not backup**. For notes that change frequently on multiple devices, conflicts will happen. + +**Sources**: +- [Syncthing GitHub Issues](https://github.com/syncthing/syncthing/issues) +- [Syncthing 2.0 OOM Discussion](https://forum.syncthing.net/t/syncthing-2-0-1-oom/24824) +- [Lobsters: "Do not use Syncthing"](https://lobste.rs/s/pacmpc/do_not_use_syncthing) +- [Syncthing Android Discontinuation](https://www.ghacks.net/2024/10/21/syncthing-for-android-is-being-discontinued-but-theres-an-alternative-app-you-can-switch-to/) + +--- + +### Unison + +**What it is**: Bidirectional file synchronizer, runs on-demand (not continuous) + +**Strengths**: +- **Explicit sync**: You run it when you want, see exactly what will happen +- **Proper conflict detection**: Shows both versions, lets you choose +- **SSH-native**: Works over SSH, no extra protocols +- **Version 2.52+ is OCaml-independent**: No more version matching hell +- **ACL/xattr support** (v2.53+): Can sync permissions and extended attributes +- **Hub-and-spoke works great**: Sync all devices to a NAS/server +- **Lower battery usage** than continuous sync + +**Weaknesses**: +- **Manual invocation**: No background daemon (by design, but requires discipline) +- **File rename detection is poor**: Rename = delete + create, potentially losing history +- **GUI is deprecated**: lablgtk maintenance uncertain; CLI is the future +- **Small maintainer base**: ~2.5 people, 0.1 FTE +- **Learning curve**: Profile configuration can be confusing + +**Best for**: Users who want explicit control over sync timing and conflict resolution + +**Sources**: +- [Unison GitHub](https://github.com/bcpierce00/unison) +- [Migration from Syncthing to Unison](https://codelearn.me/2025/12/29/from-syncthing-to-unison.html) +- [Syncthing Forum Comparison](https://forum.syncthing.net/t/syncthing-vs-rsync-vs-unison/6298) + +--- + +### rsync + +**What it is**: One-way file copying/mirroring tool + +**Strengths**: +- Rock solid, ubiquitous +- Efficient delta transfers +- Perfect for backups + +**Weaknesses**: +- **Unidirectional only**: Not a sync tool +- No conflict handling (overwrites target) + +**Best for**: Backups, deployments, one-way mirroring. Not for bidirectional sync. + +--- + +### rclone (with bisync) + +**What it is**: Cloud storage Swiss Army knife, with experimental bidirectional sync + +**Strengths**: +- Supports 70+ cloud backends +- `bisync` command provides bidirectional sync (as of v1.66) +- Checksum-based comparison available +- Lock files prevent concurrent corruption +- Great for cloud ↔ local sync + +**Weaknesses**: +- **bisync is "advanced"**: Documentation warns of potential data loss +- Designed for cloud storage, not LAN sync +- More complex ignore pattern syntax than Syncthing +- No real-time watching (cron-based) + +**Best for**: Syncing with cloud storage (S3, B2, Drive, etc.), Tier 3 backup + +**Sources**: +- [rclone bisync documentation](https://rclone.org/bisync/) + +--- + +### Mutagen + +**What it is**: Fast bidirectional sync for remote development + +**Strengths**: +- Designed for code sync to remote containers/servers +- Very fast (near-native Docker volume performance) +- Real-time filesystem watching +- Multiple sync modes (two-way-safe, one-way, etc.) +- SSH and Docker native + +**Weaknesses**: +- **Ephemeral by design**: Not meant for long-term sync +- Less configuration flexibility than Syncthing +- Primarily for developer workflows + +**Best for**: Remote development, Docker volumes, SSH workspaces. Overkill for notes. + +**Sources**: +- [Mutagen GitHub](https://github.com/mutagen-io/mutagen) +- [Mutagen Documentation](https://mutagen.io/documentation/synchronization/) + +--- + +### nb (Notebook CLI) + +**What it is**: CLI note-taking with built-in git sync + +**Strengths**: +- **Git-native**: Every notebook is a git repo +- Explicit sync via `nb sync` +- Full history, branching, proper merges +- Works with any git remote (GitHub, Gitea, Forgejo) +- Markdown by default, encryption optional +- Single portable shell script +- `$EDITOR` integration (neovim-friendly) + +**Weaknesses**: +- **Explicit sync only**: Must run `nb sync` (or set up cron/systemd) +- Git conflicts require git knowledge to resolve +- No GUI (by design) + +**Best for**: Notes and writing. This is the ideal Tier 2 solution for text content. + +**Sources**: +- [nb GitHub](https://github.com/xwmx/nb) +- [nb Documentation](https://xwmx.github.io/nb/) + +--- + +### CRDTs (Automerge, Yjs) + +**What it is**: Conflict-free Replicated Data Types for automatic merging + +**Strengths**: +- **No conflicts by design**: Concurrent edits automatically merge +- Real-time collaboration possible +- Local-first architecture +- Automerge 3.0 has 10x lower memory usage + +**Weaknesses**: +- **Application-level, not filesystem-level**: Requires app integration +- Not a drop-in replacement for file sync +- Larger file sizes than plain text +- Requires specialized tooling + +**Best for**: Future note-taking apps, real-time collaboration. Not ready for filesystem sync. + +**Sources**: +- [Automerge GitHub](https://github.com/automerge/automerge) +- [Local, First, Forever](https://tonsky.me/blog/crdt-filesync/) + +--- + +## Recommendation Matrix + +| Use Case | Recommended Tool | Reasoning | +|----------|------------------|-----------| +| **Notes (Markdown)** | `nb` with git | Explicit sync, full history, proper conflict resolution | +| **Documents (Office, misc)** | Syncthing OR Unison | Syncthing for "set and forget", Unison for control | +| **Code/Dotfiles** | Git + home-manager | Version control is the right tool | +| **Photos** | Immich + rclone | Specialized tool for media | +| **Large files (video, etc.)** | rclone to B2/S3/NAS | Cloud storage with proper backup | +| **Backup** | restic or borg | Deduplication, encryption, versioning | + +--- + +## Why "Syncthing is Buggy" is Usually Wrong + +The perception of bugginess typically comes from: + +1. **Expectation mismatch**: Users expect Dropbox behavior (cloud-centric) but get P2P behavior +2. **Conflict = failure to them**: Syncthing correctly identifies conflicts; users see this as a bug +3. **Android battery drain**: Real issue, but fixed in forks with battery optimization +4. **Propagated deletions**: Syncthing does what it's supposed to; users wanted a backup +5. **Complex topologies**: 5+ devices with selective sync creates edge cases + +**If you experienced "bugginess" with Syncthing, ask yourself**: +- Did you have proper backups outside of Syncthing? +- Were multiple devices editing the same files? +- Did you understand the difference between "Send Only" / "Receive Only" / "Send & Receive"? +- Was your Android device killing the app in the background? + +--- + +## Final Recommendation for This Project + +``` +┌─────────────────────────────────────────────────────────────┐ +│ TIER 1: Configuration │ +│ Tool: Git (public GitHub repo) │ +│ NixOS flake + home-manager, sops-nix for secrets │ +└─────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────┐ +│ TIER 2: Syncable Data │ +│ │ +│ Notes & Writing: │ +│ ├── Tool: nb (with git sync to private Forgejo) │ +│ ├── Why: Explicit sync, full history, proper merges │ +│ └── Conflict handling: Git-style (user resolves) │ +│ │ +│ Other Documents: │ +│ ├── Tool: Syncthing (if you want auto-sync) │ +│ │ OR Unison (if you want manual control) │ +│ ├── Why: Real-time for docs that change less frequently │ +│ └── Conflict handling: .sync-conflict files (Syncthing) │ +│ or interactive (Unison) │ +└─────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────┐ +│ TIER 3: Large Data │ +│ │ +│ Photos: Immich (self-hosted) │ +│ Media: Jellyfin (self-hosted) │ +│ Repos: Forgejo (self-hosted) │ +│ Backup: restic → B2/Backblaze/NAS │ +│ Access: On-demand download via rclone or web UI │ +└─────────────────────────────────────────────────────────────┘ +``` + +The key insight: **Use git for text, file-sync for binary, and specialized tools for media**. diff --git a/flake.nix b/flake.nix new file mode 100644 index 0000000..5789a4f --- /dev/null +++ b/flake.nix @@ -0,0 +1,141 @@ +{ + description = "Ultimate Notetaking, Sync & Backup System - NixOS configuration for managing personal data"; + + inputs = { + nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; + + home-manager = { + url = "github:nix-community/home-manager"; + inputs.nixpkgs.follows = "nixpkgs"; + }; + + sops-nix = { + url = "github:Mic92/sops-nix"; + inputs.nixpkgs.follows = "nixpkgs"; + }; + + # Optional: Neovim distribution + # nixvim = { + # url = "github:nix-community/nixvim"; + # inputs.nixpkgs.follows = "nixpkgs"; + # }; + }; + + outputs = { self, nixpkgs, home-manager, sops-nix, ... }@inputs: + let + # Supported systems + supportedSystems = [ "x86_64-linux" "aarch64-linux" ]; + + # Helper to generate outputs for all systems + forAllSystems = nixpkgs.lib.genAttrs supportedSystems; + + # Nixpkgs instantiated for each system + nixpkgsFor = forAllSystems (system: import nixpkgs { + inherit system; + config.allowUnfree = false; # FOSS only + }); + + # Shared modules for all hosts + sharedModules = [ + ./modules/nb.nix + ./modules/syncthing.nix + ./modules/backup.nix + sops-nix.nixosModules.sops + ]; + + # Create a NixOS configuration for a host + mkHost = { hostname, system ? "x86_64-linux", extraModules ? [] }: + nixpkgs.lib.nixosSystem { + inherit system; + specialArgs = { inherit inputs; }; + modules = sharedModules ++ [ + ./hosts/${hostname}/configuration.nix + home-manager.nixosModules.home-manager + { + home-manager.useGlobalPkgs = true; + home-manager.useUserPackages = true; + # home-manager.users.YOUR_USERNAME = import ./home; + } + ] ++ extraModules; + }; + + in { + # NixOS configurations + # Uncomment and customize for your hosts: + # + # nixosConfigurations = { + # desktop = mkHost { hostname = "desktop"; }; + # laptop = mkHost { hostname = "laptop"; }; + # server = mkHost { + # hostname = "server"; + # extraModules = [ + # ./modules/server/forgejo.nix + # ./modules/server/immich.nix + # ./modules/server/jellyfin.nix + # ]; + # }; + # }; + + # Development shell with all tools + devShells = forAllSystems (system: + let + pkgs = nixpkgsFor.${system}; + in { + default = pkgs.mkShell { + name = "unsbs-dev"; + + packages = with pkgs; [ + # Tier 2: Notes & Sync + nb # Notebook CLI + syncthing # File sync + unison # Alternative sync + + # Tier 3: Backup & Cloud + restic # Backup + rclone # Cloud storage + + # Development + git + neovim + jq + ripgrep + fd + + # Nix tools + nil # Nix LSP + nixpkgs-fmt # Nix formatter + ]; + + shellHook = '' + echo "Ultimate Notetaking, Sync & Backup System" + echo "" + echo "Available tools:" + echo " nb - Note-taking CLI" + echo " syncthing - File synchronization" + echo " unison - Alternative file sync" + echo " restic - Backup" + echo " rclone - Cloud storage" + echo "" + echo "Run 'nb help' to get started with notes." + ''; + }; + } + ); + + # Export modules for use in other flakes + nixosModules = { + nb = import ./modules/nb.nix; + syncthing = import ./modules/syncthing.nix; + backup = import ./modules/backup.nix; + default = { imports = sharedModules; }; + }; + + # Templates for creating new hosts + templates = { + default = { + path = ./templates/host; + description = "Template for a new host configuration"; + }; + }; + }; +} diff --git a/home/default.nix b/home/default.nix new file mode 100644 index 0000000..202742e --- /dev/null +++ b/home/default.nix @@ -0,0 +1,248 @@ +# Home Manager Configuration +# +# User-level configuration for dotfiles and user services. +# This is imported by home-manager in flake.nix. +# +# Customize for your user. + +{ config, pkgs, lib, ... }: + +{ + # ============================================================================ + # USER INFO + # ============================================================================ + + home.username = "youruser"; # Change this! + home.homeDirectory = "/home/youruser"; # Change this! + + # ============================================================================ + # PACKAGES + # ============================================================================ + + home.packages = with pkgs; [ + # CLI tools + ripgrep + fd + jq + bat + eza # ls replacement + fzf + htop + + # Development + git + gh # GitHub CLI + + # Notes & sync (also in system, but ensuring availability) + nb + ]; + + # ============================================================================ + # GIT + # ============================================================================ + + programs.git = { + enable = true; + userName = "Your Name"; # Change this! + userEmail = "you@example.com"; # Change this! + + extraConfig = { + init.defaultBranch = "main"; + pull.rebase = true; + push.autoSetupRemote = true; + + # Sign commits (optional) + # commit.gpgsign = true; + # gpg.format = "ssh"; + # user.signingkey = "~/.ssh/id_ed25519.pub"; + }; + + # Useful aliases + aliases = { + st = "status"; + co = "checkout"; + br = "branch"; + ci = "commit"; + lg = "log --oneline --graph --decorate"; + }; + }; + + # ============================================================================ + # SHELL + # ============================================================================ + + programs.zsh = { + enable = true; + autosuggestion.enable = true; + syntaxHighlighting.enable = true; + + shellAliases = { + # Nix + rebuild = "sudo nixos-rebuild switch --flake ~/.config/nixos"; + update = "nix flake update ~/.config/nixos"; + + # Notes + n = "nb"; + na = "nb add"; + ne = "nb edit"; + ns = "nb search"; + nsy = "nb sync"; + + # Files + ls = "eza"; + ll = "eza -la"; + tree = "eza --tree"; + + # Sync + sync = "nb sync && syncthing cli scan --all"; + }; + + initExtra = '' + # nb completion + eval "$(nb completions zsh)" + + # FZF integration + if command -v fzf &> /dev/null; then + source ${pkgs.fzf}/share/fzf/key-bindings.zsh + source ${pkgs.fzf}/share/fzf/completion.zsh + fi + ''; + }; + + # Alternative: Bash + # programs.bash = { + # enable = true; + # shellAliases = { ... }; + # }; + + # ============================================================================ + # NEOVIM + # ============================================================================ + + programs.neovim = { + enable = true; + defaultEditor = true; + viAlias = true; + vimAlias = true; + + plugins = with pkgs.vimPlugins; [ + # Essentials + plenary-nvim + telescope-nvim + nvim-treesitter.withAllGrammars + + # Git + vim-fugitive + gitsigns-nvim + + # LSP (for Nix editing) + nvim-lspconfig + + # Quality of life + comment-nvim + which-key-nvim + nvim-autopairs + + # Theme + catppuccin-nvim + ]; + + extraLuaConfig = '' + -- Basic settings + vim.opt.number = true + vim.opt.relativenumber = true + vim.opt.expandtab = true + vim.opt.tabstop = 2 + vim.opt.shiftwidth = 2 + vim.opt.smartindent = true + vim.opt.wrap = false + vim.opt.ignorecase = true + vim.opt.smartcase = true + + -- Theme + vim.cmd.colorscheme "catppuccin" + + -- Leader key + vim.g.mapleader = " " + + -- Telescope + local telescope = require('telescope.builtin') + vim.keymap.set('n', 'ff', telescope.find_files, { desc = 'Find files' }) + vim.keymap.set('n', 'fg', telescope.live_grep, { desc = 'Grep' }) + vim.keymap.set('n', 'fb', telescope.buffers, { desc = 'Buffers' }) + + -- Git signs + require('gitsigns').setup() + + -- Comments + require('Comment').setup() + + -- Which-key + require('which-key').setup() + + -- Autopairs + require('nvim-autopairs').setup() + + -- LSP for Nix + require('lspconfig').nil_ls.setup{} + ''; + }; + + # ============================================================================ + # TMUX + # ============================================================================ + + programs.tmux = { + enable = true; + terminal = "tmux-256color"; + prefix = "C-a"; + mouse = true; + + extraConfig = '' + # Split panes with | and - + bind | split-window -h + bind - split-window -v + + # Vim-style pane navigation + bind h select-pane -L + bind j select-pane -D + bind k select-pane -U + bind l select-pane -R + + # Status bar + set -g status-position top + set -g status-style 'bg=default fg=white' + ''; + }; + + # ============================================================================ + # DIRECTORIES + # ============================================================================ + + # Create standard directories + home.file = { + ".local/bin/.keep".text = ""; # Ensure bin directory exists + }; + + xdg = { + enable = true; + userDirs = { + enable = true; + createDirectories = true; + documents = "$HOME/Documents"; + download = "$HOME/Downloads"; + music = "$HOME/Music"; + pictures = "$HOME/Pictures"; + videos = "$HOME/Videos"; + }; + }; + + # ============================================================================ + # STATE VERSION + # ============================================================================ + + home.stateVersion = "24.05"; # Don't change after initial setup + + # Let home-manager manage itself + programs.home-manager.enable = true; +} diff --git a/hosts/example/configuration.nix b/hosts/example/configuration.nix new file mode 100644 index 0000000..299d8ae --- /dev/null +++ b/hosts/example/configuration.nix @@ -0,0 +1,202 @@ +# Example Host Configuration +# +# This is a template for your machine's configuration.nix. +# Copy this to hosts/your-hostname/configuration.nix and customize. +# +# To use: +# 1. Copy this file to hosts//configuration.nix +# 2. Generate hardware-configuration.nix: nixos-generate-config --show-hardware-config +# 3. Customize the settings below +# 4. Add to flake.nix outputs + +{ config, pkgs, lib, inputs, ... }: + +{ + imports = [ + ./hardware-configuration.nix # Generate with nixos-generate-config + ]; + + # ============================================================================ + # SYSTEM BASICS + # ============================================================================ + + networking.hostName = "example"; # Change this! + + # Bootloader (adjust for your system) + boot.loader.systemd-boot.enable = true; + boot.loader.efi.canTouchEfiVariables = true; + + # Networking + networking.networkmanager.enable = true; + + # Timezone and locale + time.timeZone = "America/New_York"; # Change this! + i18n.defaultLocale = "en_US.UTF-8"; + + # ============================================================================ + # USERS + # ============================================================================ + + users.users.youruser = { # Change this! + isNormalUser = true; + description = "Your Name"; + extraGroups = [ "wheel" "networkmanager" ]; + shell = pkgs.zsh; # Or bash, fish, etc. + }; + + # ============================================================================ + # TIER 1: CONFIGURATION + # ============================================================================ + + # Enable Nix flakes + nix.settings = { + experimental-features = [ "nix-command" "flakes" ]; + auto-optimise-store = true; + }; + + # Garbage collection + nix.gc = { + automatic = true; + dates = "weekly"; + options = "--delete-older-than 30d"; + }; + + # ============================================================================ + # TIER 2: NOTES (nb) + # ============================================================================ + + programs.nb = { + enable = true; + editor = "nvim"; + defaultExtension = "md"; + + # Configure your notebooks + notebooks = { + # personal = { + # remote = "git@forgejo.yourdomain.com:youruser/personal-notes.git"; + # }; + # work = { + # remote = "git@forgejo.yourdomain.com:youruser/work-notes.git"; + # }; + }; + }; + + # ============================================================================ + # TIER 2: SYNC (Syncthing) + # ============================================================================ + + services.syncthing-managed = { + enable = true; + user = "youruser"; # Change this! + + # Add your device IDs here + devices = { + # laptop = { + # id = "XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX"; + # }; + # desktop = { + # id = "YYYYYYY-YYYYYYY-YYYYYYY-YYYYYYY-YYYYYYY-YYYYYYY-YYYYYYY-YYYYYYY"; + # }; + }; + + # Configure folders to sync + folders = { + # documents = { + # path = "/home/youruser/Documents"; + # devices = [ "laptop" "desktop" ]; + # versioning = { type = "simple"; params.keep = "5"; }; + # }; + }; + }; + + # ============================================================================ + # TIER 3: BACKUP + # ============================================================================ + + services.backup = { + enable = true; + + # Configure your backup repository + # repository = "b2:your-bucket:backup"; + repository = "/mnt/backup"; # Local example + + # Paths to back up + paths = [ + "/home/youruser/Documents" + "/home/youruser/notes" + # Add more paths + ]; + + # Password file (create this manually or use sops-nix) + passwordFile = "/etc/restic-password"; # Create this! + + # For cloud storage, set environment file + # environmentFile = "/etc/restic-env"; + + # Schedule: 2 AM daily + schedule = "*-*-* 02:00:00"; + }; + + # ============================================================================ + # EDITOR + # ============================================================================ + + programs.neovim = { + enable = true; + defaultEditor = true; + viAlias = true; + vimAlias = true; + }; + + # ============================================================================ + # PACKAGES + # ============================================================================ + + environment.systemPackages = with pkgs; [ + # Shell essentials + git + ripgrep + fd + jq + htop + tmux + + # Development + neovim + + # Sync & backup (installed by modules, but explicit is fine) + syncthing + restic + rclone + ]; + + # ============================================================================ + # SECRETS (sops-nix) + # ============================================================================ + + # Uncomment and configure when you set up sops-nix + # + # sops = { + # defaultSopsFile = ../../secrets/secrets.yaml; + # age.keyFile = "/home/youruser/.config/sops/age/keys.txt"; + # + # secrets = { + # "restic-password" = {}; + # "syncthing-key" = { + # owner = "youruser"; + # }; + # }; + # }; + + # ============================================================================ + # SERVICES + # ============================================================================ + + services.openssh.enable = true; + + # ============================================================================ + # SYSTEM + # ============================================================================ + + system.stateVersion = "24.05"; # Don't change after initial install +} diff --git a/modules/backup.nix b/modules/backup.nix new file mode 100644 index 0000000..c6505f1 --- /dev/null +++ b/modules/backup.nix @@ -0,0 +1,297 @@ +# Backup Module +# +# Declarative restic backup configuration with systemd timers. +# +# Usage: +# services.backup.enable = true; +# services.backup.paths = [ "/home/user/Documents" "/home/user/notes" ]; +# services.backup.repository = "b2:mybucket:backup"; + +{ config, lib, pkgs, ... }: + +with lib; + +let + cfg = config.services.backup; +in { + options.services.backup = { + enable = mkEnableOption "automated backups with restic"; + + repository = mkOption { + type = types.str; + description = '' + Restic repository location. + Examples: + - Local: /mnt/backup + - SFTP: sftp:user@host:/path + - B2: b2:bucket-name:path + - S3: s3:s3.amazonaws.com/bucket-name + - REST: rest:http://host:8000/ + ''; + example = "b2:mybucket:backup"; + }; + + paths = mkOption { + type = types.listOf types.str; + default = []; + description = "Paths to back up."; + example = [ "/home/user/Documents" "/home/user/notes" "/var/lib/important" ]; + }; + + exclude = mkOption { + type = types.listOf types.str; + default = [ + "**/.git" + "**/node_modules" + "**/__pycache__" + "**/.cache" + "**/Cache" + "**/.thumbnails" + "**/Trash" + "**/.local/share/Trash" + ]; + description = "Patterns to exclude from backup."; + }; + + passwordFile = mkOption { + type = types.str; + description = "Path to file containing restic repository password."; + example = "/run/secrets/restic-password"; + }; + + environmentFile = mkOption { + type = types.nullOr types.str; + default = null; + description = '' + Path to file containing environment variables for cloud storage. + Should contain variables like B2_ACCOUNT_ID, B2_ACCOUNT_KEY, etc. + ''; + example = "/run/secrets/restic-env"; + }; + + schedule = mkOption { + type = types.str; + default = "*-*-* 02:00:00"; # 2 AM daily + description = "When to run backups (systemd calendar format)."; + example = "*-*-* 04:00:00"; + }; + + pruneSchedule = mkOption { + type = types.str; + default = "Sun *-*-* 03:00:00"; # Sunday 3 AM + description = "When to prune old backups."; + }; + + retention = { + daily = mkOption { + type = types.int; + default = 7; + description = "Number of daily backups to keep."; + }; + + weekly = mkOption { + type = types.int; + default = 4; + description = "Number of weekly backups to keep."; + }; + + monthly = mkOption { + type = types.int; + default = 6; + description = "Number of monthly backups to keep."; + }; + + yearly = mkOption { + type = types.int; + default = 2; + description = "Number of yearly backups to keep."; + }; + }; + + extraOptions = mkOption { + type = types.listOf types.str; + default = []; + description = "Extra options to pass to restic backup."; + example = [ "--verbose" "--exclude-caches" ]; + }; + + user = mkOption { + type = types.str; + default = "root"; + description = "User to run backups as."; + }; + + notifyOnFailure = mkOption { + type = types.bool; + default = true; + description = "Send notification on backup failure."; + }; + }; + + config = mkIf cfg.enable { + # Install restic + environment.systemPackages = [ pkgs.restic ]; + + # Backup service + systemd.services.restic-backup = { + description = "Restic Backup"; + after = [ "network-online.target" ]; + wants = [ "network-online.target" ]; + + serviceConfig = { + Type = "oneshot"; + User = cfg.user; + ExecStart = let + excludeArgs = concatMapStringsSep " " (e: "--exclude '${e}'") cfg.exclude; + pathArgs = concatStringsSep " " cfg.paths; + extraArgs = concatStringsSep " " cfg.extraOptions; + in pkgs.writeShellScript "restic-backup" '' + set -euo pipefail + + export RESTIC_REPOSITORY="${cfg.repository}" + export RESTIC_PASSWORD_FILE="${cfg.passwordFile}" + ${optionalString (cfg.environmentFile != null) "source ${cfg.environmentFile}"} + + echo "Starting backup at $(date)" + + ${pkgs.restic}/bin/restic backup \ + ${excludeArgs} \ + ${extraArgs} \ + ${pathArgs} + + echo "Backup completed at $(date)" + ''; + + # Security hardening + PrivateTmp = true; + ProtectSystem = "strict"; + ReadWritePaths = [ "/tmp" ]; + NoNewPrivileges = true; + }; + + # Retry on failure + unitConfig = { + OnFailure = mkIf cfg.notifyOnFailure [ "backup-notify-failure@%n.service" ]; + }; + }; + + # Backup timer + systemd.timers.restic-backup = { + description = "Restic Backup Timer"; + wantedBy = [ "timers.target" ]; + + timerConfig = { + OnCalendar = cfg.schedule; + Persistent = true; # Run missed backups + RandomizedDelaySec = "30min"; # Spread load + }; + }; + + # Prune service + systemd.services.restic-prune = { + description = "Restic Prune Old Backups"; + after = [ "network-online.target" ]; + wants = [ "network-online.target" ]; + + serviceConfig = { + Type = "oneshot"; + User = cfg.user; + ExecStart = pkgs.writeShellScript "restic-prune" '' + set -euo pipefail + + export RESTIC_REPOSITORY="${cfg.repository}" + export RESTIC_PASSWORD_FILE="${cfg.passwordFile}" + ${optionalString (cfg.environmentFile != null) "source ${cfg.environmentFile}"} + + echo "Pruning old backups at $(date)" + + ${pkgs.restic}/bin/restic forget \ + --keep-daily ${toString cfg.retention.daily} \ + --keep-weekly ${toString cfg.retention.weekly} \ + --keep-monthly ${toString cfg.retention.monthly} \ + --keep-yearly ${toString cfg.retention.yearly} \ + --prune + + echo "Prune completed at $(date)" + ''; + + PrivateTmp = true; + ProtectSystem = "strict"; + NoNewPrivileges = true; + }; + }; + + # Prune timer + systemd.timers.restic-prune = { + description = "Restic Prune Timer"; + wantedBy = [ "timers.target" ]; + + timerConfig = { + OnCalendar = cfg.pruneSchedule; + Persistent = true; + }; + }; + + # Failure notification service template + systemd.services."backup-notify-failure@" = mkIf cfg.notifyOnFailure { + description = "Backup Failure Notification"; + serviceConfig = { + Type = "oneshot"; + ExecStart = pkgs.writeShellScript "backup-notify-failure" '' + echo "Backup failed: %i at $(date)" | ${pkgs.util-linux}/bin/wall + # Add your notification method here: + # - Send email + # - Push notification + # - Slack webhook + # - etc. + ''; + }; + }; + + # Helper scripts + environment.systemPackages = [ + (pkgs.writeShellScriptBin "ubackup" '' + # Unified backup helper + case "''${1:-}" in + now) + echo "Starting backup..." + sudo systemctl start restic-backup + ;; + status) + echo "=== Recent backups ===" + export RESTIC_REPOSITORY="${cfg.repository}" + export RESTIC_PASSWORD_FILE="${cfg.passwordFile}" + ${optionalString (cfg.environmentFile != null) "source ${cfg.environmentFile}"} + ${pkgs.restic}/bin/restic snapshots --last 10 + ;; + restore) + if [ -z "''${2:-}" ]; then + echo "Usage: ubackup restore " + exit 1 + fi + export RESTIC_REPOSITORY="${cfg.repository}" + export RESTIC_PASSWORD_FILE="${cfg.passwordFile}" + ${optionalString (cfg.environmentFile != null) "source ${cfg.environmentFile}"} + ${pkgs.restic}/bin/restic restore "$2" --target "''${3:-.}" + ;; + check) + echo "Checking backup integrity..." + export RESTIC_REPOSITORY="${cfg.repository}" + export RESTIC_PASSWORD_FILE="${cfg.passwordFile}" + ${optionalString (cfg.environmentFile != null) "source ${cfg.environmentFile}"} + ${pkgs.restic}/bin/restic check + ;; + *) + echo "Usage: ubackup {now|status|restore|check}" + echo "" + echo "Commands:" + echo " now Run backup immediately" + echo " status Show recent backups" + echo " restore ID PATH Restore snapshot to path" + echo " check Verify backup integrity" + ;; + esac + '') + ]; + }; +} diff --git a/modules/nb.nix b/modules/nb.nix new file mode 100644 index 0000000..21ea860 --- /dev/null +++ b/modules/nb.nix @@ -0,0 +1,146 @@ +# nb - CLI Notebook Module +# +# This module installs and configures nb, a command-line notebook tool +# with git-backed versioning and sync. +# +# Usage in your configuration.nix: +# programs.nb.enable = true; +# programs.nb.notebooks.personal.remote = "git@forgejo:you/notes.git"; + +{ config, lib, pkgs, ... }: + +with lib; + +let + cfg = config.programs.nb; +in { + options.programs.nb = { + enable = mkEnableOption "nb notebook CLI"; + + package = mkOption { + type = types.package; + default = pkgs.nb; + defaultText = literalExpression "pkgs.nb"; + description = "The nb package to use."; + }; + + defaultNotebook = mkOption { + type = types.str; + default = "home"; + description = "The default notebook name."; + }; + + editor = mkOption { + type = types.str; + default = "nvim"; + description = "Editor to use for editing notes."; + }; + + defaultExtension = mkOption { + type = types.str; + default = "md"; + description = "Default file extension for new notes."; + }; + + autoSync = mkOption { + type = types.bool; + default = false; + description = '' + Whether to automatically sync notebooks after operations. + Note: This can slow down operations if network is slow. + ''; + }; + + colorTheme = mkOption { + type = types.str; + default = "blacklight"; + description = "Color theme for nb (see 'nb settings colors')."; + }; + + notebooks = mkOption { + type = types.attrsOf (types.submodule { + options = { + remote = mkOption { + type = types.nullOr types.str; + default = null; + description = "Git remote URL for syncing this notebook."; + example = "git@github.com:user/notes.git"; + }; + + encrypted = mkOption { + type = types.bool; + default = false; + description = "Whether to encrypt notes in this notebook."; + }; + }; + }); + default = {}; + description = "Notebooks to configure with their remotes."; + example = literalExpression '' + { + personal = { remote = "git@forgejo:user/personal.git"; }; + work = { remote = "git@forgejo:user/work.git"; encrypted = true; }; + } + ''; + }; + }; + + config = mkIf cfg.enable { + environment.systemPackages = [ + cfg.package + pkgs.git # nb requires git + pkgs.bat # Optional: better file viewing + pkgs.w3m # Optional: for nb browse + ]; + + # Set environment variables for nb + environment.variables = { + EDITOR = cfg.editor; + NB_DEFAULT_EXTENSION = cfg.defaultExtension; + NB_AUTO_SYNC = if cfg.autoSync then "1" else "0"; + NB_COLOR_THEME = cfg.colorTheme; + }; + + # Create activation script to set up notebooks + system.activationScripts.nb-setup = let + notebookSetup = concatStringsSep "\n" (mapAttrsToList (name: opts: '' + # Create notebook if it doesn't exist + if ! ${cfg.package}/bin/nb notebooks | grep -q "^${name}$"; then + echo "Creating nb notebook: ${name}" + ${cfg.package}/bin/nb notebooks add ${name} + fi + + ${optionalString (opts.remote != null) '' + # Set remote for notebook + echo "Setting remote for ${name}: ${opts.remote}" + ${cfg.package}/bin/nb ${name}:remote set ${opts.remote} 2>/dev/null || true + ''} + '') cfg.notebooks); + in stringAfter [ "users" ] '' + # Skip if running in a chroot or during initial install + if [ -d /home ]; then + ${notebookSetup} + fi + ''; + + # Optional: systemd timer for periodic sync + # Uncomment if you want automatic background sync + # + # systemd.user.services.nb-sync = { + # description = "Sync nb notebooks"; + # serviceConfig = { + # Type = "oneshot"; + # ExecStart = "${cfg.package}/bin/nb sync --all"; + # }; + # }; + # + # systemd.user.timers.nb-sync = { + # description = "Periodic nb sync"; + # wantedBy = [ "timers.target" ]; + # timerConfig = { + # OnCalendar = "*:0/15"; # Every 15 minutes + # Persistent = true; + # }; + # }; + }; +} diff --git a/modules/server/forgejo.nix b/modules/server/forgejo.nix new file mode 100644 index 0000000..c330551 --- /dev/null +++ b/modules/server/forgejo.nix @@ -0,0 +1,145 @@ +# Forgejo Module +# +# Self-hosted Git forge (GitHub/Gitea alternative). +# Used for hosting private nb notebook repositories. +# +# Usage: +# services.forgejo-managed.enable = true; +# services.forgejo-managed.domain = "git.yourdomain.com"; + +{ config, lib, pkgs, ... }: + +with lib; + +let + cfg = config.services.forgejo-managed; +in { + options.services.forgejo-managed = { + enable = mkEnableOption "managed Forgejo git forge"; + + domain = mkOption { + type = types.str; + description = "Domain name for Forgejo."; + example = "git.example.com"; + }; + + httpPort = mkOption { + type = types.port; + default = 3000; + description = "HTTP port for Forgejo (behind reverse proxy)."; + }; + + sshPort = mkOption { + type = types.port; + default = 2222; + description = "SSH port for git operations."; + }; + + stateDir = mkOption { + type = types.str; + default = "/var/lib/forgejo"; + description = "State directory for Forgejo data."; + }; + + enableLFS = mkOption { + type = types.bool; + default = true; + description = "Enable Git LFS support."; + }; + + disableRegistration = mkOption { + type = types.bool; + default = true; + description = "Disable open user registration."; + }; + + adminEmail = mkOption { + type = types.nullOr types.str; + default = null; + description = "Admin email address."; + }; + + appName = mkOption { + type = types.str; + default = "Forgejo"; + description = "Application name shown in UI."; + }; + }; + + config = mkIf cfg.enable { + services.forgejo = { + enable = true; + stateDir = cfg.stateDir; + + settings = { + DEFAULT = { + APP_NAME = cfg.appName; + }; + + server = { + DOMAIN = cfg.domain; + HTTP_PORT = cfg.httpPort; + ROOT_URL = "https://${cfg.domain}/"; + SSH_PORT = cfg.sshPort; + SSH_DOMAIN = cfg.domain; + START_SSH_SERVER = true; # Built-in SSH server + LFS_START_SERVER = cfg.enableLFS; + }; + + service = { + DISABLE_REGISTRATION = cfg.disableRegistration; + REQUIRE_SIGNIN_VIEW = false; # Allow viewing public repos + }; + + repository = { + DEFAULT_PRIVATE = "private"; # New repos are private by default + ENABLE_PUSH_CREATE_USER = true; # Allow creating repos via push + }; + + "repository.upload" = { + ENABLED = true; + FILE_MAX_SIZE = 100; # MB + MAX_FILES = 10; + }; + + session = { + PROVIDER = "file"; + COOKIE_SECURE = true; # Require HTTPS + }; + + log = { + LEVEL = "Info"; + }; + + # Security + security = { + INSTALL_LOCK = true; # Prevent web-based install + MIN_PASSWORD_LENGTH = 12; + }; + + mailer = mkIf (cfg.adminEmail != null) { + ENABLED = true; + FROM = cfg.adminEmail; + }; + }; + }; + + # Open SSH port + networking.firewall.allowedTCPPorts = [ cfg.sshPort ]; + + # Create backup for Forgejo data + services.backup.paths = mkIf config.services.backup.enable [ + cfg.stateDir + ]; + + # Reverse proxy with Caddy (optional, can use nginx) + # Uncomment if using Caddy: + # + # services.caddy = { + # enable = true; + # virtualHosts."${cfg.domain}".extraConfig = '' + # reverse_proxy localhost:${toString cfg.httpPort} + # ''; + # }; + }; +} diff --git a/modules/server/immich.nix b/modules/server/immich.nix new file mode 100644 index 0000000..27bd444 --- /dev/null +++ b/modules/server/immich.nix @@ -0,0 +1,93 @@ +# Immich Module +# +# Self-hosted photo and video backup (Google Photos alternative). +# For Tier 3 photo management. +# +# Note: Immich is complex and changes frequently. This module provides +# a starting point but may need updates. Check NixOS options for latest. +# +# Usage: +# services.immich-managed.enable = true; +# services.immich-managed.domain = "photos.yourdomain.com"; + +{ config, lib, pkgs, ... }: + +with lib; + +let + cfg = config.services.immich-managed; +in { + options.services.immich-managed = { + enable = mkEnableOption "managed Immich photo service"; + + domain = mkOption { + type = types.str; + description = "Domain name for Immich."; + example = "photos.example.com"; + }; + + port = mkOption { + type = types.port; + default = 2283; + description = "Port for Immich web interface."; + }; + + mediaLocation = mkOption { + type = types.str; + default = "/var/lib/immich"; + description = "Location for storing photos and videos."; + }; + + externalLibraryPaths = mkOption { + type = types.listOf types.str; + default = []; + description = "Additional paths for external photo libraries."; + example = [ "/mnt/photos/archive" ]; + }; + + enableMachineLearning = mkOption { + type = types.bool; + default = true; + description = "Enable ML features (face recognition, search)."; + }; + }; + + config = mkIf cfg.enable { + # Immich service (NixOS 24.05+) + services.immich = { + enable = true; + port = cfg.port; + mediaLocation = cfg.mediaLocation; + + # Machine learning (optional, resource-intensive) + machine-learning.enable = cfg.enableMachineLearning; + + # Settings + settings = { + # Add any Immich-specific settings here + # Check Immich docs for available options + }; + }; + + # Ensure media directory exists with correct permissions + systemd.tmpfiles.rules = [ + "d ${cfg.mediaLocation} 0755 immich immich -" + ] ++ (map (path: "d ${path} 0755 immich immich -") cfg.externalLibraryPaths); + + # Backup Immich data + services.backup.paths = mkIf config.services.backup.enable [ + cfg.mediaLocation + "/var/lib/immich" # Database and config + ]; + + # Memory recommendation + warnings = mkIf (cfg.enableMachineLearning && config.hardware.cpu.intel.updateMicrocode or false) [ + "Immich ML features benefit from GPU acceleration. Consider enabling CUDA or OpenCL." + ]; + + # Reverse proxy example (Caddy) + # services.caddy.virtualHosts."${cfg.domain}".extraConfig = '' + # reverse_proxy localhost:${toString cfg.port} + # ''; + }; +} diff --git a/modules/server/jellyfin.nix b/modules/server/jellyfin.nix new file mode 100644 index 0000000..1e94b3d --- /dev/null +++ b/modules/server/jellyfin.nix @@ -0,0 +1,91 @@ +# Jellyfin Module +# +# Self-hosted media server (Plex alternative). +# For Tier 3 media streaming. +# +# Usage: +# services.jellyfin-managed.enable = true; +# services.jellyfin-managed.domain = "media.yourdomain.com"; +# services.jellyfin-managed.mediaLibraries = [ "/mnt/media/movies" "/mnt/media/tv" ]; + +{ config, lib, pkgs, ... }: + +with lib; + +let + cfg = config.services.jellyfin-managed; +in { + options.services.jellyfin-managed = { + enable = mkEnableOption "managed Jellyfin media server"; + + domain = mkOption { + type = types.str; + description = "Domain name for Jellyfin."; + example = "media.example.com"; + }; + + port = mkOption { + type = types.port; + default = 8096; + description = "HTTP port for Jellyfin."; + }; + + mediaLibraries = mkOption { + type = types.listOf types.str; + default = []; + description = "Paths to media libraries."; + example = [ "/mnt/media/movies" "/mnt/media/tv" "/mnt/media/music" ]; + }; + + enableHardwareAcceleration = mkOption { + type = types.bool; + default = false; + description = "Enable hardware transcoding (requires compatible GPU)."; + }; + + openFirewall = mkOption { + type = types.bool; + default = true; + description = "Open firewall for Jellyfin ports."; + }; + }; + + config = mkIf cfg.enable { + services.jellyfin = { + enable = true; + openFirewall = cfg.openFirewall; + }; + + # Give jellyfin access to media directories + systemd.services.jellyfin.serviceConfig.SupplementaryGroups = [ + "render" # For hardware acceleration + "video" # For hardware acceleration + ]; + + # Ensure media directories have correct permissions + systemd.tmpfiles.rules = map (path: + "d ${path} 0755 jellyfin jellyfin -" + ) cfg.mediaLibraries; + + # Hardware acceleration (Intel VAAPI example) + hardware.graphics = mkIf cfg.enableHardwareAcceleration { + enable = true; + extraPackages = with pkgs; [ + intel-media-driver # For Intel + # nvidia-vaapi-driver # For NVIDIA + libva + ]; + }; + + # Note: For hardware acceleration, jellyfin user needs access to /dev/dri + users.users.jellyfin.extraGroups = mkIf cfg.enableHardwareAcceleration [ + "render" + "video" + ]; + + # Reverse proxy example + # services.caddy.virtualHosts."${cfg.domain}".extraConfig = '' + # reverse_proxy localhost:${toString cfg.port} + # ''; + }; +} diff --git a/modules/syncthing.nix b/modules/syncthing.nix new file mode 100644 index 0000000..4d2743c --- /dev/null +++ b/modules/syncthing.nix @@ -0,0 +1,269 @@ +# Syncthing Module +# +# Declarative Syncthing configuration for NixOS. +# This wraps the built-in syncthing module with sensible defaults. +# +# Usage: +# services.syncthing-managed.enable = true; +# services.syncthing-managed.user = "youruser"; +# services.syncthing-managed.devices.laptop.id = "DEVICE-ID-HERE"; +# services.syncthing-managed.folders.documents.path = "~/Documents"; + +{ config, lib, pkgs, ... }: + +with lib; + +let + cfg = config.services.syncthing-managed; +in { + options.services.syncthing-managed = { + enable = mkEnableOption "managed Syncthing file synchronization"; + + user = mkOption { + type = types.str; + description = "User to run Syncthing as."; + example = "alice"; + }; + + group = mkOption { + type = types.str; + default = "users"; + description = "Group to run Syncthing as."; + }; + + dataDir = mkOption { + type = types.str; + default = "/home/${cfg.user}"; + defaultText = literalExpression ''"/home/''${cfg.user}"''; + description = "Default directory for Syncthing data."; + }; + + configDir = mkOption { + type = types.str; + default = "/home/${cfg.user}/.config/syncthing"; + defaultText = literalExpression ''"/home/''${cfg.user}/.config/syncthing"''; + description = "Directory for Syncthing configuration."; + }; + + guiAddress = mkOption { + type = types.str; + default = "127.0.0.1:8384"; + description = "Address for the Syncthing web GUI."; + }; + + openFirewall = mkOption { + type = types.bool; + default = true; + description = "Whether to open the firewall for Syncthing."; + }; + + devices = mkOption { + type = types.attrsOf (types.submodule { + options = { + id = mkOption { + type = types.str; + description = "Device ID (from syncthing CLI or GUI)."; + example = "XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX"; + }; + + name = mkOption { + type = types.nullOr types.str; + default = null; + description = "Friendly name for the device."; + }; + + addresses = mkOption { + type = types.listOf types.str; + default = [ "dynamic" ]; + description = "Addresses to connect to this device."; + }; + + autoAcceptFolders = mkOption { + type = types.bool; + default = false; + description = "Automatically accept shared folders from this device."; + }; + }; + }); + default = {}; + description = "Syncthing devices to connect to."; + example = literalExpression '' + { + laptop = { + id = "XXXXXXX-..."; + name = "My Laptop"; + }; + server = { + id = "YYYYYYY-..."; + addresses = [ "tcp://server.local:22000" ]; + }; + } + ''; + }; + + folders = mkOption { + type = types.attrsOf (types.submodule ({ name, ... }: { + options = { + path = mkOption { + type = types.str; + description = "Local path to sync."; + example = "~/Documents"; + }; + + devices = mkOption { + type = types.listOf types.str; + default = attrNames cfg.devices; + defaultText = literalExpression "attrNames cfg.devices"; + description = "Devices to share this folder with."; + }; + + id = mkOption { + type = types.str; + default = name; + description = "Unique folder ID."; + }; + + type = mkOption { + type = types.enum [ "sendreceive" "sendonly" "receiveonly" ]; + default = "sendreceive"; + description = '' + Folder type: + - sendreceive: Full two-way sync (default) + - sendonly: Only send changes, ignore remote changes + - receiveonly: Only receive changes, don't send local changes + ''; + }; + + versioning = mkOption { + type = types.nullOr (types.submodule { + options = { + type = mkOption { + type = types.enum [ "simple" "staggered" "trashcan" "external" ]; + default = "simple"; + description = "Versioning type."; + }; + + params = mkOption { + type = types.attrsOf types.str; + default = { keep = "5"; }; + description = "Versioning parameters."; + }; + }; + }); + default = null; + description = "Versioning configuration for this folder."; + example = literalExpression '' + { + type = "staggered"; + params = { + cleanInterval = "3600"; + maxAge = "31536000"; + }; + } + ''; + }; + + ignorePerms = mkOption { + type = types.bool; + default = false; + description = "Ignore permission changes."; + }; + + rescanInterval = mkOption { + type = types.int; + default = 3600; + description = "How often to rescan the folder (seconds). 0 to disable."; + }; + + fsWatcherEnabled = mkOption { + type = types.bool; + default = true; + description = "Use filesystem watcher for real-time sync."; + }; + }; + })); + default = {}; + description = "Folders to synchronize."; + example = literalExpression '' + { + documents = { + path = "~/Documents"; + devices = [ "laptop" "desktop" ]; + versioning = { type = "simple"; params.keep = "5"; }; + }; + music = { + path = "~/Music"; + type = "receiveonly"; # Don't upload changes + }; + } + ''; + }; + + extraOptions = mkOption { + type = types.attrs; + default = {}; + description = "Extra options to pass to services.syncthing.settings."; + }; + }; + + config = mkIf cfg.enable { + services.syncthing = { + enable = true; + user = cfg.user; + group = cfg.group; + dataDir = cfg.dataDir; + configDir = cfg.configDir; + guiAddress = cfg.guiAddress; + + overrideDevices = true; # Declarative device management + overrideFolders = true; # Declarative folder management + + settings = { + devices = mapAttrs (name: device: { + inherit (device) id addresses autoAcceptFolders; + name = if device.name != null then device.name else name; + }) cfg.devices; + + folders = mapAttrs (name: folder: { + inherit (folder) path id type ignorePerms rescanInterval fsWatcherEnabled; + devices = folder.devices; + versioning = if folder.versioning != null then folder.versioning else {}; + }) cfg.folders; + + options = { + urAccepted = -1; # Disable usage reporting + relaysEnabled = true; + globalAnnounceEnabled = true; + localAnnounceEnabled = true; + } // cfg.extraOptions; + }; + }; + + # Open firewall if requested + networking.firewall = mkIf cfg.openFirewall { + allowedTCPPorts = [ 22000 ]; # Syncthing protocol + allowedUDPPorts = [ 22000 21027 ]; # Syncthing + discovery + }; + + # Convenience alias + environment.systemPackages = [ + (pkgs.writeShellScriptBin "st" '' + # Syncthing shortcut + case "$1" in + status) + ${pkgs.syncthing}/bin/syncthing cli show system + ;; + scan) + ${pkgs.syncthing}/bin/syncthing cli scan --all + ;; + errors) + ${pkgs.syncthing}/bin/syncthing cli errors + ;; + *) + echo "Usage: st {status|scan|errors}" + ;; + esac + '') + ]; + }; +} diff --git a/scripts/ustatus b/scripts/ustatus new file mode 100755 index 0000000..c911a05 --- /dev/null +++ b/scripts/ustatus @@ -0,0 +1,227 @@ +#!/usr/bin/env bash +# +# ustatus - Status dashboard for the Ultimate Notetaking System +# +# Shows status of all sync and backup systems. +# +# Usage: +# ustatus - Show full status +# ustatus brief - Show brief status + +set -euo pipefail + +# Colors +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +BOLD='\033[1m' +NC='\033[0m' + +status_ok() { + echo -e "${GREEN}●${NC} $1" +} + +status_warn() { + echo -e "${YELLOW}●${NC} $1" +} + +status_error() { + echo -e "${RED}●${NC} $1" +} + +header() { + echo -e "\n${BOLD}═══ $1 ═══${NC}" +} + +# Check if a command exists +has_cmd() { + command -v "$1" &> /dev/null +} + +brief_status() { + echo -e "${BOLD}Ultimate Notetaking System Status${NC}" + echo "" + + # nb + if has_cmd nb; then + notebook_count=$(nb notebooks --names 2>/dev/null | wc -l || echo 0) + status_ok "nb: $notebook_count notebooks" + else + status_error "nb: not installed" + fi + + # Syncthing + if has_cmd syncthing && syncthing cli show system &> /dev/null 2>&1; then + status_ok "Syncthing: running" + elif has_cmd syncthing; then + status_warn "Syncthing: not running" + else + status_error "Syncthing: not installed" + fi + + # Backup + if systemctl is-active restic-backup.timer &> /dev/null; then + status_ok "Backup: timer active" + elif has_cmd restic; then + status_warn "Backup: timer not active" + else + status_error "Backup: restic not installed" + fi +} + +full_status() { + echo -e "${BOLD}╔════════════════════════════════════════════════════════════╗${NC}" + echo -e "${BOLD}║ Ultimate Notetaking, Sync & Backup System ║${NC}" + echo -e "${BOLD}╚════════════════════════════════════════════════════════════╝${NC}" + + # ==================== TIER 1: Configuration ==================== + header "TIER 1: Configuration" + + # Nix/NixOS + if has_cmd nixos-rebuild; then + generation=$(nixos-rebuild list-generations 2>/dev/null | head -1 | awk '{print $1}' || echo "?") + status_ok "NixOS generation: $generation" + fi + + if has_cmd nix; then + nix_version=$(nix --version | head -1) + status_ok "Nix: $nix_version" + fi + + # Git + if has_cmd git; then + status_ok "Git: $(git --version | cut -d' ' -f3)" + fi + + # ==================== TIER 2: Notes ==================== + header "TIER 2: Notes (nb)" + + if has_cmd nb; then + nb_version=$(nb version 2>/dev/null || echo "unknown") + status_ok "nb version: $nb_version" + + echo "" + echo "Notebooks:" + nb_dir="${NB_DIR:-$HOME/.nb}" + for notebook in $(nb notebooks --names 2>/dev/null || true); do + if [ -d "$nb_dir/$notebook" ]; then + note_count=$(find "$nb_dir/$notebook" -name "*.md" 2>/dev/null | wc -l) + + # Check if has remote + if [ -d "$nb_dir/$notebook/.git" ]; then + remote=$(git -C "$nb_dir/$notebook" remote get-url origin 2>/dev/null || echo "") + if [ -n "$remote" ]; then + last_sync=$(git -C "$nb_dir/$notebook" log -1 --format="%ar" 2>/dev/null || echo "never") + echo -e " ${GREEN}●${NC} $notebook: $note_count notes, last sync: $last_sync" + else + echo -e " ${YELLOW}●${NC} $notebook: $note_count notes (no remote)" + fi + else + echo -e " ${YELLOW}●${NC} $notebook: $note_count notes (not git repo)" + fi + fi + done + else + status_error "nb not installed" + fi + + # ==================== TIER 2: Syncthing ==================== + header "TIER 2: File Sync (Syncthing)" + + if has_cmd syncthing; then + if syncthing cli show system &> /dev/null 2>&1; then + uptime=$(syncthing cli show system 2>/dev/null | jq -r '.uptime // 0' | awk '{printf "%dd %dh %dm", $1/86400, ($1%86400)/3600, ($1%3600)/60}') + status_ok "Syncthing running (uptime: $uptime)" + + echo "" + echo "Folders:" + syncthing cli show config 2>/dev/null | jq -r '.folders[] | "\(.label // .id)|\(.path)"' 2>/dev/null | while IFS='|' read -r label path; do + if [ -d "$path" ]; then + file_count=$(find "$path" -type f 2>/dev/null | wc -l) + echo -e " ${GREEN}●${NC} $label: $path ($file_count files)" + else + echo -e " ${YELLOW}●${NC} $label: $path (not found)" + fi + done || echo " Unable to list folders" + + echo "" + echo "Devices:" + syncthing cli show config 2>/dev/null | jq -r '.devices[] | "\(.name // .deviceID[:8])"' 2>/dev/null | while read -r device; do + echo " ● $device" + done || echo " Unable to list devices" + else + status_warn "Syncthing installed but not running" + fi + else + status_error "Syncthing not installed" + fi + + # ==================== TIER 3: Backup ==================== + header "TIER 3: Backup (restic)" + + if has_cmd restic; then + status_ok "restic: $(restic version | head -1)" + + # Check systemd timer + if systemctl is-active restic-backup.timer &> /dev/null; then + next_run=$(systemctl show restic-backup.timer --property=NextElapseUSecRealtime 2>/dev/null | cut -d= -f2) + status_ok "Backup timer active (next: $next_run)" + else + status_warn "Backup timer not active" + fi + + # Try to show recent snapshots (requires RESTIC_REPOSITORY to be set) + if [ -n "${RESTIC_REPOSITORY:-}" ]; then + echo "" + echo "Recent snapshots:" + restic snapshots --last 3 2>/dev/null | tail -n +3 | head -5 || echo " Unable to list snapshots" + fi + else + status_error "restic not installed" + fi + + # ==================== TIER 3: Services ==================== + header "TIER 3: Services" + + # Check common services + for service in forgejo immich jellyfin; do + if systemctl is-active "$service" &> /dev/null 2>&1; then + status_ok "$service: running" + elif systemctl is-enabled "$service" &> /dev/null 2>&1; then + status_warn "$service: enabled but not running" + else + echo -e " ${BLUE}○${NC} $service: not configured" + fi + done + + # ==================== Summary ==================== + header "Quick Actions" + echo " usync - Sync all (nb + Syncthing)" + echo " ubackup now - Run backup immediately" + echo " nb add - Create new note" + echo "" +} + +# Main +case "${1:-full}" in + brief|short|b) + brief_status + ;; + full|"") + full_status + ;; + -h|--help|help) + echo "ustatus - System status dashboard" + echo "" + echo "Usage:" + echo " ustatus Show full status" + echo " ustatus brief Show brief status" + echo " ustatus help Show this help" + ;; + *) + echo "Unknown option: $1" + echo "Run 'ustatus help' for usage" + exit 1 + ;; +esac diff --git a/scripts/usync b/scripts/usync new file mode 100755 index 0000000..28bdd1b --- /dev/null +++ b/scripts/usync @@ -0,0 +1,156 @@ +#!/usr/bin/env bash +# +# usync - Unified sync command for the Ultimate Notetaking System +# +# Syncs both nb notebooks and triggers Syncthing scans. +# +# Usage: +# usync - Sync everything +# usync nb - Sync only nb notebooks +# usync st - Sync only Syncthing folders +# usync status - Show sync status + +set -euo pipefail + +# Colors +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +NC='\033[0m' # No Color + +log_info() { + echo -e "${BLUE}[INFO]${NC} $1" +} + +log_success() { + echo -e "${GREEN}[OK]${NC} $1" +} + +log_warn() { + echo -e "${YELLOW}[WARN]${NC} $1" +} + +log_error() { + echo -e "${RED}[ERROR]${NC} $1" +} + +sync_nb() { + log_info "Syncing nb notebooks..." + + if ! command -v nb &> /dev/null; then + log_error "nb not found in PATH" + return 1 + fi + + # Get list of notebooks + notebooks=$(nb notebooks --names 2>/dev/null || echo "") + + if [ -z "$notebooks" ]; then + log_warn "No nb notebooks found" + return 0 + fi + + # Sync each notebook + for notebook in $notebooks; do + log_info " Syncing notebook: $notebook" + if nb "$notebook:sync" 2>/dev/null; then + log_success " $notebook synced" + else + log_warn " $notebook sync failed (no remote?)" + fi + done +} + +sync_syncthing() { + log_info "Triggering Syncthing scan..." + + if ! command -v syncthing &> /dev/null; then + log_error "syncthing not found in PATH" + return 1 + fi + + # Check if Syncthing is running + if ! syncthing cli show system &> /dev/null; then + log_warn "Syncthing is not running" + return 1 + fi + + # Trigger scan on all folders + if syncthing cli scan --all 2>/dev/null; then + log_success "Syncthing scan triggered" + else + log_error "Failed to trigger Syncthing scan" + return 1 + fi +} + +show_status() { + echo "=== Sync Status ===" + echo "" + + # nb status + echo "--- nb Notebooks ---" + if command -v nb &> /dev/null; then + nb notebooks 2>/dev/null || echo " No notebooks" + echo "" + echo "Last sync times:" + for notebook in $(nb notebooks --names 2>/dev/null); do + nb_dir="${NB_DIR:-$HOME/.nb}" + if [ -d "$nb_dir/$notebook/.git" ]; then + last_commit=$(git -C "$nb_dir/$notebook" log -1 --format="%ar" 2>/dev/null || echo "never") + echo " $notebook: $last_commit" + fi + done + else + echo " nb not installed" + fi + + echo "" + + # Syncthing status + echo "--- Syncthing ---" + if command -v syncthing &> /dev/null && syncthing cli show system &> /dev/null; then + syncthing cli show system 2>/dev/null | grep -E "(myID|startTime)" || true + echo "" + echo "Folders:" + syncthing cli show config 2>/dev/null | jq -r '.folders[] | " \(.label // .id): \(.path)"' 2>/dev/null || echo " Unable to get folder info" + else + echo " Syncthing not running" + fi +} + +# Main +case "${1:-all}" in + nb) + sync_nb + ;; + st|syncthing) + sync_syncthing + ;; + status) + show_status + ;; + all|"") + sync_nb + echo "" + sync_syncthing + echo "" + log_success "All sync operations complete" + ;; + -h|--help|help) + echo "usync - Unified sync command" + echo "" + echo "Usage:" + echo " usync Sync everything (nb + Syncthing)" + echo " usync nb Sync only nb notebooks" + echo " usync st Trigger Syncthing scan" + echo " usync status Show sync status" + echo " usync help Show this help" + ;; + *) + log_error "Unknown command: $1" + echo "Run 'usync help' for usage" + exit 1 + ;; +esac diff --git a/secrets/secrets.yaml.example b/secrets/secrets.yaml.example new file mode 100644 index 0000000..e6a1b8b --- /dev/null +++ b/secrets/secrets.yaml.example @@ -0,0 +1,38 @@ +# Example secrets file for sops-nix +# +# This file shows the structure of secrets.yaml. +# DO NOT put actual secrets in this file! +# +# To use: +# 1. Install sops and age +# 2. Create age key: age-keygen -o ~/.config/sops/age/keys.txt +# 3. Create .sops.yaml in repo root with your public key +# 4. Copy this to secrets.yaml and encrypt: sops secrets/secrets.yaml +# +# Example .sops.yaml: +# --- +# keys: +# - &admin age1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx +# creation_rules: +# - path_regex: secrets/secrets\.yaml$ +# key_groups: +# - age: +# - *admin + +# Restic backup password +restic-password: "your-secure-backup-password-here" + +# Syncthing API key (optional, for automation) +syncthing-api-key: "your-syncthing-api-key" + +# Cloud storage credentials +# For Backblaze B2: +b2-account-id: "your-b2-account-id" +b2-account-key: "your-b2-account-key" + +# For AWS S3: +# aws-access-key-id: "your-aws-key" +# aws-secret-access-key: "your-aws-secret" + +# Forgejo admin password (initial setup) +forgejo-admin-password: "your-forgejo-admin-password"