Brandon Lucas
b40ac99524
A NixOS-based system for managing personal data across three tiers: - Tier 1: Configuration (shareable via git) - Tier 2: Syncable data (nb + Syncthing) - Tier 3: Large data (self-hosted services + backup) Includes: - NixOS modules for nb, Syncthing, backup (restic) - Server modules for Forgejo, Immich, Jellyfin - Helper scripts (usync, ustatus) - Comprehensive documentation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
272 lines
11 KiB
Markdown
272 lines
11 KiB
Markdown
# File Synchronization Tools: Comprehensive Comparison
|
|
|
|
*Research conducted: February 2026*
|
|
|
|
## Executive Summary
|
|
|
|
After extensive research, **Syncthing is not buggy—but it is complex and unforgiving**. Most reported "bugs" are actually:
|
|
1. Misunderstanding of how P2P sync works
|
|
2. Conflict handling that differs from user expectations
|
|
3. Android-specific issues (now largely resolved with forks)
|
|
4. Users treating it as a backup tool (it's not)
|
|
|
|
For a NixOS-based notes/data sync system, the recommended approach is:
|
|
- **Tier 2 (Notes)**: `nb` with git sync to private Forgejo (explicit, versioned)
|
|
- **Tier 2 (Documents)**: Syncthing OR Unison (based on preference)
|
|
- **Tier 3 (Large files)**: rclone + restic to self-hosted or cloud storage
|
|
|
|
---
|
|
|
|
## Tool-by-Tool Analysis
|
|
|
|
### Syncthing
|
|
|
|
**What it is**: Continuous, real-time, P2P file synchronization
|
|
|
|
**Strengths**:
|
|
- Truly decentralized (no server required)
|
|
- Real-time sync via filesystem watching
|
|
- Handles file renames/moves efficiently
|
|
- Cross-platform with good GUI
|
|
- Battle-tested with large user base
|
|
- Declarative NixOS module available
|
|
|
|
**Weaknesses**:
|
|
- **Conflict handling is file-level**: Creates `.sync-conflict-*` files that must be manually resolved
|
|
- **Not a backup**: Deletions propagate immediately; if you delete on one device, it's gone everywhere
|
|
- **Android app discontinued** (Dec 2024): Use [Syncthing-Fork](https://github.com/catfriend1/syncthing-android) instead
|
|
- **Complexity ceiling**: Multi-device topologies can create unexpected sync loops
|
|
- **No ownership/permission sync**: By design, to avoid requiring root
|
|
- **OOM issues in v2.0**: On low-memory devices (1GB RAM) with large file sets
|
|
|
|
**Common Misconceptions**:
|
|
| Issue Reported | Actual Cause |
|
|
|----------------|--------------|
|
|
| "Files disappearing" | Deletions syncing correctly; user expected backup behavior |
|
|
| "Sync stuck at 99%" | Large files still transferring, or ignored files being counted |
|
|
| "Random conflicts" | Multiple devices modifying same file; Syncthing is doing its job |
|
|
| "Ignoring read-only" | Misconfigured folder type; "Receive Only" ≠ read-only filesystem |
|
|
|
|
**Verdict**: Syncthing works well if you understand that it's **sync, not backup**. For notes that change frequently on multiple devices, conflicts will happen.
|
|
|
|
**Sources**:
|
|
- [Syncthing GitHub Issues](https://github.com/syncthing/syncthing/issues)
|
|
- [Syncthing 2.0 OOM Discussion](https://forum.syncthing.net/t/syncthing-2-0-1-oom/24824)
|
|
- [Lobsters: "Do not use Syncthing"](https://lobste.rs/s/pacmpc/do_not_use_syncthing)
|
|
- [Syncthing Android Discontinuation](https://www.ghacks.net/2024/10/21/syncthing-for-android-is-being-discontinued-but-theres-an-alternative-app-you-can-switch-to/)
|
|
|
|
---
|
|
|
|
### Unison
|
|
|
|
**What it is**: Bidirectional file synchronizer, runs on-demand (not continuous)
|
|
|
|
**Strengths**:
|
|
- **Explicit sync**: You run it when you want, see exactly what will happen
|
|
- **Proper conflict detection**: Shows both versions, lets you choose
|
|
- **SSH-native**: Works over SSH, no extra protocols
|
|
- **Version 2.52+ is OCaml-independent**: No more version matching hell
|
|
- **ACL/xattr support** (v2.53+): Can sync permissions and extended attributes
|
|
- **Hub-and-spoke works great**: Sync all devices to a NAS/server
|
|
- **Lower battery usage** than continuous sync
|
|
|
|
**Weaknesses**:
|
|
- **Manual invocation**: No background daemon (by design, but requires discipline)
|
|
- **File rename detection is poor**: Rename = delete + create, potentially losing history
|
|
- **GUI is deprecated**: lablgtk maintenance uncertain; CLI is the future
|
|
- **Small maintainer base**: ~2.5 people, 0.1 FTE
|
|
- **Learning curve**: Profile configuration can be confusing
|
|
|
|
**Best for**: Users who want explicit control over sync timing and conflict resolution
|
|
|
|
**Sources**:
|
|
- [Unison GitHub](https://github.com/bcpierce00/unison)
|
|
- [Migration from Syncthing to Unison](https://codelearn.me/2025/12/29/from-syncthing-to-unison.html)
|
|
- [Syncthing Forum Comparison](https://forum.syncthing.net/t/syncthing-vs-rsync-vs-unison/6298)
|
|
|
|
---
|
|
|
|
### rsync
|
|
|
|
**What it is**: One-way file copying/mirroring tool
|
|
|
|
**Strengths**:
|
|
- Rock solid, ubiquitous
|
|
- Efficient delta transfers
|
|
- Perfect for backups
|
|
|
|
**Weaknesses**:
|
|
- **Unidirectional only**: Not a sync tool
|
|
- No conflict handling (overwrites target)
|
|
|
|
**Best for**: Backups, deployments, one-way mirroring. Not for bidirectional sync.
|
|
|
|
---
|
|
|
|
### rclone (with bisync)
|
|
|
|
**What it is**: Cloud storage Swiss Army knife, with experimental bidirectional sync
|
|
|
|
**Strengths**:
|
|
- Supports 70+ cloud backends
|
|
- `bisync` command provides bidirectional sync (as of v1.66)
|
|
- Checksum-based comparison available
|
|
- Lock files prevent concurrent corruption
|
|
- Great for cloud ↔ local sync
|
|
|
|
**Weaknesses**:
|
|
- **bisync is "advanced"**: Documentation warns of potential data loss
|
|
- Designed for cloud storage, not LAN sync
|
|
- More complex ignore pattern syntax than Syncthing
|
|
- No real-time watching (cron-based)
|
|
|
|
**Best for**: Syncing with cloud storage (S3, B2, Drive, etc.), Tier 3 backup
|
|
|
|
**Sources**:
|
|
- [rclone bisync documentation](https://rclone.org/bisync/)
|
|
|
|
---
|
|
|
|
### Mutagen
|
|
|
|
**What it is**: Fast bidirectional sync for remote development
|
|
|
|
**Strengths**:
|
|
- Designed for code sync to remote containers/servers
|
|
- Very fast (near-native Docker volume performance)
|
|
- Real-time filesystem watching
|
|
- Multiple sync modes (two-way-safe, one-way, etc.)
|
|
- SSH and Docker native
|
|
|
|
**Weaknesses**:
|
|
- **Ephemeral by design**: Not meant for long-term sync
|
|
- Less configuration flexibility than Syncthing
|
|
- Primarily for developer workflows
|
|
|
|
**Best for**: Remote development, Docker volumes, SSH workspaces. Overkill for notes.
|
|
|
|
**Sources**:
|
|
- [Mutagen GitHub](https://github.com/mutagen-io/mutagen)
|
|
- [Mutagen Documentation](https://mutagen.io/documentation/synchronization/)
|
|
|
|
---
|
|
|
|
### nb (Notebook CLI)
|
|
|
|
**What it is**: CLI note-taking with built-in git sync
|
|
|
|
**Strengths**:
|
|
- **Git-native**: Every notebook is a git repo
|
|
- Explicit sync via `nb sync`
|
|
- Full history, branching, proper merges
|
|
- Works with any git remote (GitHub, Gitea, Forgejo)
|
|
- Markdown by default, encryption optional
|
|
- Single portable shell script
|
|
- `$EDITOR` integration (neovim-friendly)
|
|
|
|
**Weaknesses**:
|
|
- **Explicit sync only**: Must run `nb sync` (or set up cron/systemd)
|
|
- Git conflicts require git knowledge to resolve
|
|
- No GUI (by design)
|
|
|
|
**Best for**: Notes and writing. This is the ideal Tier 2 solution for text content.
|
|
|
|
**Sources**:
|
|
- [nb GitHub](https://github.com/xwmx/nb)
|
|
- [nb Documentation](https://xwmx.github.io/nb/)
|
|
|
|
---
|
|
|
|
### CRDTs (Automerge, Yjs)
|
|
|
|
**What it is**: Conflict-free Replicated Data Types for automatic merging
|
|
|
|
**Strengths**:
|
|
- **No conflicts by design**: Concurrent edits automatically merge
|
|
- Real-time collaboration possible
|
|
- Local-first architecture
|
|
- Automerge 3.0 has 10x lower memory usage
|
|
|
|
**Weaknesses**:
|
|
- **Application-level, not filesystem-level**: Requires app integration
|
|
- Not a drop-in replacement for file sync
|
|
- Larger file sizes than plain text
|
|
- Requires specialized tooling
|
|
|
|
**Best for**: Future note-taking apps, real-time collaboration. Not ready for filesystem sync.
|
|
|
|
**Sources**:
|
|
- [Automerge GitHub](https://github.com/automerge/automerge)
|
|
- [Local, First, Forever](https://tonsky.me/blog/crdt-filesync/)
|
|
|
|
---
|
|
|
|
## Recommendation Matrix
|
|
|
|
| Use Case | Recommended Tool | Reasoning |
|
|
|----------|------------------|-----------|
|
|
| **Notes (Markdown)** | `nb` with git | Explicit sync, full history, proper conflict resolution |
|
|
| **Documents (Office, misc)** | Syncthing OR Unison | Syncthing for "set and forget", Unison for control |
|
|
| **Code/Dotfiles** | Git + home-manager | Version control is the right tool |
|
|
| **Photos** | Immich + rclone | Specialized tool for media |
|
|
| **Large files (video, etc.)** | rclone to B2/S3/NAS | Cloud storage with proper backup |
|
|
| **Backup** | restic or borg | Deduplication, encryption, versioning |
|
|
|
|
---
|
|
|
|
## Why "Syncthing is Buggy" is Usually Wrong
|
|
|
|
The perception of bugginess typically comes from:
|
|
|
|
1. **Expectation mismatch**: Users expect Dropbox behavior (cloud-centric) but get P2P behavior
|
|
2. **Conflict = failure to them**: Syncthing correctly identifies conflicts; users see this as a bug
|
|
3. **Android battery drain**: Real issue, but fixed in forks with battery optimization
|
|
4. **Propagated deletions**: Syncthing does what it's supposed to; users wanted a backup
|
|
5. **Complex topologies**: 5+ devices with selective sync creates edge cases
|
|
|
|
**If you experienced "bugginess" with Syncthing, ask yourself**:
|
|
- Did you have proper backups outside of Syncthing?
|
|
- Were multiple devices editing the same files?
|
|
- Did you understand the difference between "Send Only" / "Receive Only" / "Send & Receive"?
|
|
- Was your Android device killing the app in the background?
|
|
|
|
---
|
|
|
|
## Final Recommendation for This Project
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ TIER 1: Configuration │
|
|
│ Tool: Git (public GitHub repo) │
|
|
│ NixOS flake + home-manager, sops-nix for secrets │
|
|
└─────────────────────────────────────────────────────────────┘
|
|
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ TIER 2: Syncable Data │
|
|
│ │
|
|
│ Notes & Writing: │
|
|
│ ├── Tool: nb (with git sync to private Forgejo) │
|
|
│ ├── Why: Explicit sync, full history, proper merges │
|
|
│ └── Conflict handling: Git-style (user resolves) │
|
|
│ │
|
|
│ Other Documents: │
|
|
│ ├── Tool: Syncthing (if you want auto-sync) │
|
|
│ │ OR Unison (if you want manual control) │
|
|
│ ├── Why: Real-time for docs that change less frequently │
|
|
│ └── Conflict handling: .sync-conflict files (Syncthing) │
|
|
│ or interactive (Unison) │
|
|
└─────────────────────────────────────────────────────────────┘
|
|
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ TIER 3: Large Data │
|
|
│ │
|
|
│ Photos: Immich (self-hosted) │
|
|
│ Media: Jellyfin (self-hosted) │
|
|
│ Repos: Forgejo (self-hosted) │
|
|
│ Backup: restic → B2/Backblaze/NAS │
|
|
│ Access: On-demand download via rclone or web UI │
|
|
└─────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
The key insight: **Use git for text, file-sync for binary, and specialized tools for media**.
|