A NixOS-based system for managing personal data across three tiers: - Tier 1: Configuration (shareable via git) - Tier 2: Syncable data (nb + Syncthing) - Tier 3: Large data (self-hosted services + backup) Includes: - NixOS modules for nb, Syncthing, backup (restic) - Server modules for Forgejo, Immich, Jellyfin - Helper scripts (usync, ustatus) - Comprehensive documentation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
12 KiB
12 KiB
System Architecture
Design Principles
1. Declarative Everything
Every aspect of this system is declared in Nix. No imperative setup steps, no hidden state.
Desired State (Nix) → nixos-rebuild → Actual State
2. Separation of Concerns
┌──────────────────────────────────────────────────────────────┐
│ CONFIGURATION LAYER │
│ What: How machines should be configured │
│ Where: This repo (flake.nix, modules/) │
│ Managed by: Git + GitHub │
│ Shareable: Yes (public) │
└──────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────┐
│ DATA LAYER │
│ │
│ ┌────────────────────┐ ┌────────────────────┐ │
│ │ TIER 2: Sync │ │ TIER 3: Backup │ │
│ │ │ │ │ │
│ │ ~/notes/ (nb) │ │ Photos (Immich) │ │
│ │ ~/Documents/ │ │ Media (Jellyfin) │ │
│ │ │ │ Repos (Forgejo) │ │
│ │ Managed by: │ │ │ │
│ │ - nb (git sync) │ │ Managed by: │ │
│ │ - Syncthing │ │ - Server apps │ │
│ │ │ │ - restic backup │ │
│ │ Multi-device: Yes │ │ Multi-device: No │ │
│ │ Real-time: Yes │ │ (on-demand access)│ │
│ └────────────────────┘ └────────────────────┘ │
└──────────────────────────────────────────────────────────────┘
3. No Vendor Lock-in
Every component can be replaced:
| Component | Could be replaced with |
|---|---|
| nb | Obsidian + git, org-mode, plain vim + git |
| Syncthing | Unison, rclone bisync |
| Forgejo | Gitea, GitLab, bare git |
| Immich | PhotoPrism, any DAM |
| Jellyfin | Plex (non-FOSS), Kodi |
| restic | borg, duplicity |
4. Offline-First
All Tier 2 data is fully available offline:
- nb notebooks are local git repos
- Syncthing folders are local directories
- Sync happens when connectivity allows
Tier 3 data is available via caching or on-demand download.
Data Flow
Tier 1: Configuration
┌─────────┐ git push ┌─────────┐
│ Local │ ───────────────► │ GitHub │
│ Repo │ ◄─────────────── │ (pub) │
└────┬────┘ git pull └─────────┘
│
│ nixos-rebuild switch
▼
┌─────────┐
│ Machine │
│ State │
└─────────┘
Tier 2: Notes (nb)
┌─────────┐ ┌─────────┐
│ Device │ nb sync │ Forgejo │
│ A │ ◄────────────────► │ (priv) │
└─────────┘ └────┬────┘
│
┌─────────┐ nb sync ┌────▼────┐
│ Device │ ◄────────────────► │ Forgejo │
│ B │ │ (priv) │
└─────────┘ └─────────┘
Conflict Resolution: Git 3-way merge
History: Full git history preserved
Tier 2: Documents (Syncthing)
┌─────────┐ ┌─────────┐
│ Device │ ◄──── P2P ───────► │ Device │
│ A │ │ B │
└────┬────┘ └────┬────┘
│ │
│ ┌─────────┐ │
└────────►│ Device │◄─────────┘
│ C │
└─────────┘
Conflict Resolution: .sync-conflict files (manual)
History: Syncthing versioning (configurable)
Tier 3: Large Data
┌─────────┐ upload ┌─────────┐
│ Device │ ──────────────► │ Server │
│ │ ◄────────────── │ Immich/ │
└─────────┘ on-demand │ Jellyfin│
└────┬────┘
│ restic
┌────▼────┐
│ Backup │
│ B2/NAS │
└─────────┘
Module Dependency Graph
flake.nix
│
┌───────────────┼───────────────┐
▼ ▼ ▼
hosts/ modules/ home/
desktop (shared) default.nix
laptop │
server │
│ │
▼ ▼
┌──────────────────────────────────┐
│ modules/ │
│ ┌──────┐ ┌───────────┐ ┌─────┐ │
│ │ nb │ │ syncthing │ │nvim │ │
│ └──────┘ └───────────┘ └─────┘ │
│ ┌────────┐ │
│ │ backup │ │
│ └────────┘ │
│ │
│ ┌─────────── server/ ────────┐ │
│ │ forgejo immich jellyfin │ │
│ └───────────────────────────┘ │
└──────────────────────────────────┘
Security Model
Secrets
┌─────────────────────────────────────────┐
│ sops-nix │
│ │
│ secrets.yaml (encrypted in repo) │
│ │ │
│ │ age/gpg key (not in repo) │
│ ▼ │
│ Decrypted at activation time │
│ Placed in /run/secrets/ │
│ Permissions set per-secret │
└─────────────────────────────────────────┘
Network Security
- All sync over TLS (Syncthing) or SSH (nb, Forgejo)
- Server services behind reverse proxy (Caddy/nginx)
- No ports exposed except 443 (HTTPS)
- Tailscale/WireGuard for private network (optional)
Backup Strategy
┌────────────────────────────────────────────────────────────┐
│ 3-2-1 Backup Rule │
│ │
│ 3 copies: │
│ ├── Primary: Local device │
│ ├── Secondary: Home server │
│ └── Tertiary: Offsite (B2/cloud) │
│ │
│ 2 media types: │
│ ├── SSD (local) │
│ └── Cloud object storage (offsite) │
│ │
│ 1 offsite: │
│ └── Backblaze B2 / AWS S3 / etc. │
└────────────────────────────────────────────────────────────┘
Backup Schedule (systemd timers):
├── Tier 2 (notes/docs): Daily, keep 30 days
├── Tier 3 (media): Weekly, keep 90 days
└── Server state: Daily, keep 14 days
Scaling Considerations
Adding a New Machine
- Create
hosts/<hostname>/configuration.nix - Add hardware-configuration.nix
- Add to flake.nix outputs
- Run
nixos-rebuild switch --flake .#<hostname> - nb automatically syncs when
nb syncis run - Syncthing auto-discovers via device IDs in config
Adding a New User
- Add user to
hosts/<hostname>/configuration.nix - Create home-manager config
- Set up nb notebooks for user
- Add Syncthing device ID
Multiple Servers
The architecture supports multiple servers:
┌─────────┐ ┌─────────┐ ┌─────────┐
│ Server │ │ Server │ │ Server │
│ Home │ │ VPS │ │ NAS │
│ (media) │ │ (Forgejo│ │(backup) │
└─────────┘ └─────────┘ └─────────┘
Each server has its own host config in hosts/.
Failure Modes
| Failure | Impact | Recovery |
|---|---|---|
| Device lost | Tier 2 data unavailable locally | Sync from other devices or Forgejo |
| Server down | Tier 3 unavailable, nb sync fails | Use local cache, fix server |
| Backup failure | No new backups | Alert (systemd OnFailure), investigate |
| Sync conflict | Manual resolution needed | Resolve .sync-conflict or git merge |
| Internet down | Can't sync | Work offline, sync when back |
Future Considerations
Potential Additions
- Encrypted Tier 2: Full-disk encryption + nb's GPG encryption
- Mobile support: Termux + Unison, or Syncthing-Fork
- Real-time collaboration: Automerge/CRDT-based notes
- Monitoring: Prometheus + Grafana for sync/backup status
Not In Scope
- Windows support (NixOS only)
- Proprietary services integration
- Enterprise/team features
- Real-time streaming replication