flan/truenas-truecloud-patch
1
0
Fork 0
Code Issues 1 Pull Requests 1 Actions Packages Projects Releases 12 Wiki Activity

Retire the two modules independently; calm the README down

Browse Source 
The native-support check looked only for native B2 restic support and, on
finding it, set the kill switch and disabled the whole patch. That was fine when
providers were the only thing here. With nested-dataset snapshots in the patch it
is wrong: TrueNAS is likely to ship one capability long before the other, and a
single all-or-nothing kill switch would silently take a still-needed module down
with the superseded one.

apply.sh now treats the patch as two independent modules:

  providers  b2.py + restic.py + the UI credential dropdown
             native when B2RcloneRemote carries a real get_restic_config()

  nested     plugins/cloud/{snapshot,crud}.py + cloud_backup/sync.py (opt-in)
             native when the "no further nesting" validation is gone from
             plugins/cloud/crud.py

Each is detected and skipped on its own. The kill switch fires only once BOTH are
done (native, or nested was never enabled). The UI patch belongs to providers and
is skipped with it. The deferred middlewared restart now fires when ANY
still-needed module landed -- keying it off providers alone would have left a
freshly-patched nested module on disk and never loaded on a native-B2 box.
hook_status.json reports each module with an active flag and a reason.

README: dropped the warning boxes and the disclaimer's fear-bulleting in favour
of plain statements, documented the two-module design and the per-module
auto-disable, and added a Development section disclosing AI assistance. The one
caveat kept, as a plain sentence rather than a banner: the mount --bind staging
step has not yet been exercised by a live backup run.

67 tests, ruff and shellcheck clean.
This commit is contained in:
flan
2026-07-12 22:17:37 +00:00
parent a80de88078
commit eca6eb3f3b
4 changed files with 375 additions and 121 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+143 -59
View File
@@ -27,31 +27,41 @@ costs a fraction of the new Storj price.
---
## ⚠ Disclaimer — please read before installing
## Before you install
**This project is unofficial, unsupported, and not affiliated with iXsystems
or the TrueNAS project in any way.**
This project is unofficial and not affiliated with iXsystems. A few things worth
knowing:
By installing this patch you accept the following:
- It targets **internal middleware APIs** with no stability contract, so a
TrueNAS update can break it. Every patch is fail-safe: if it can't apply,
middlewared starts normally and the reason is logged to `apply.log`. Check the
log after an update.
- If you file a TrueNAS bug report, **remove the patch first** and reproduce on a
stock system.
- **Test your restores.** That is true of any backup, but it matters more here —
see [Verifying it works](#verifying-it-works).
- Provided as-is, no warranty. See LICENSE.
- **Unsupported configuration.** TrueNAS support staff are not obligated to
help with any issue on a system running this patch. If you file a bug report,
remove the patch first and reproduce the issue on an unmodified system.
The patch is two independent modules — **providers** (B2/S3) and **nested**
(snapshots on nested datasets) — and each retires on its own once TrueNAS ships
that capability natively. See [Native support](#if-truenas-adds-native-support).
- **May break on TrueNAS updates.** The patch targets internal middleware APIs
that are not part of any public contract. They can change at any time. When
they do, the patch silently degrades to Storj-only behaviour rather than
breaking TrueNAS — but you should check the log after each update.
## Development
- **Your backups are your responsibility.** Verify that your backup jobs
complete successfully and that restores work before relying on them for
disaster recovery.
Parts of this project were written with AI assistance (Claude). All of it is
reviewed and tested before release; the test suite and CI exist in large part to
make that review meaningful. Bugs are mine.
- **No warranty.** This software is provided as-is. See the LICENSE file.
```bash
pip install pytest ruff
ruff check patch tests
pytest tests
```
If TrueNAS adds native B2 or S3 support to TrueCloud Backup, the patch
detects it at boot, disables itself, and tells you to run `uninstall.sh` —
see [Native support](#if-truenas-adds-native-support) below.
CI runs shellcheck, `bash -n`, ruff, and pytest on Python 3.11–3.13. The tests
include a pass that `compile()`s the `*_BLOCK` strings in `patch/apply.sh` —
those are Python source appended into live `middlewared` modules, so a syntax
error there would break the box at boot.
---
@@ -77,29 +87,35 @@ restarting. Expect one middlewared restart shortly after every boot; the UI
and API are briefly unavailable while it happens, and running services are
not affected.
| Layer | What changes | Technique |
| Module | What changes | Technique |
|---|---|---|
| **Backend** | `B2RcloneRemote` gains `get_restic_config()` — skipped automatically if TrueNAS already provides one on the class. `restic.py` URL builder is fixed: strips the stray leading slash and converts the slash separator to a colon (`b2:bucket:path`), which is the format restic 0.16.x expects. URL wrapper is a no-op if the URL is already correctly formed. | File patch applied inside the overlayfs upper layer |
| **Nested snapshots** | `_truecloud_nested.py` is installed into `plugins/cloud/`, and `plugins/cloud/{snapshot,crud}.py` + `plugins/cloud_backup/sync.py` are patched so `snapshot = true` works on a dataset that has child datasets. See [below](#nested-dataset-snapshots). | New module + file patches inside the overlayfs upper layer |
| **UI** | The Angular bundle's `filterByProviders` binding is widened from `["STORJ_IX"]` to `["STORJ_IX","S3","B2"]` | In-place text replacement in the compiled JS chunk; original is backed up before patching |
| **providers** | `B2RcloneRemote` gains `get_restic_config()` — skipped automatically if TrueNAS already provides one on the class. `restic.py` URL builder is fixed: strips the stray leading slash and converts the slash separator to a colon (`b2:bucket:path`), which is the format restic 0.16.x expects. URL wrapper is a no-op if the URL is already correctly formed. | File patch applied inside the overlayfs upper layer |
| **providers** (UI) | The Angular bundle's `filterByProviders` binding is widened from `["STORJ_IX"]` to `["STORJ_IX","S3","B2"]` | In-place text replacement in the compiled JS chunk; original is backed up before patching |
| **nested** (opt-in) | `_truecloud_nested.py` is installed into `plugins/cloud/`, and `plugins/cloud/{snapshot,crud}.py` + `plugins/cloud_backup/sync.py` are patched so `snapshot = true` works on a dataset that has child datasets. See [below](#nested-dataset-snapshots). | New module + file patches inside the overlayfs upper layer |
All changes are **fail-safe**: if a patch cannot be applied (e.g. TrueNAS
restructured the relevant code), middlewared starts normally with Storj-only
support and the reason is logged to `apply.log` in your repo root.
restructured the relevant code), middlewared starts normally, the affected module
is simply inactive, and the reason is logged to `apply.log` in your repo root. The
two modules are independent — one failing or going native does not disable the
other.
## Nested-dataset snapshots
> **Opt-in, and off by default.** This feature changes how backups read their
> source data, so it is never enabled implicitly:
>
> ```bash
> bash install.sh --enable-nested-snapshots # turn it on
> bash install.sh --disable-nested-snapshots # turn it back off
> ```
>
> With neither flag, `install.sh` leaves the current setting alone — so a
> `git pull && bash install.sh` can never silently flip it. The B2/S3 provider
> patch is unaffected either way.
**Opt-in, off by default.** It changes how backups read their source data, so it
is never enabled implicitly:
```bash
bash install.sh --enable-nested-snapshots
bash install.sh --disable-nested-snapshots
```
With neither flag `install.sh` leaves the setting alone, so `git pull && bash
install.sh` won't flip it. The providers module is unaffected either way.
The planner and the snapshot lifecycle have been validated against a real
250-dataset pool. The `mount --bind` staging step has not yet been exercised by a
live backup run, so confirm your first backup actually contains child-dataset
data before relying on it — see [Verifying it works](#verifying-it-works).
TrueCloud Backup's **Take Snapshot** option makes restic read from a frozen ZFS
snapshot instead of live files. Without it the backup reads data *while apps are
@@ -118,7 +134,7 @@ That rules out **any pool running Apps** — every app is its own dataset, usual
with `config`/`pgdata` children of its own. On a typical box that is 100+ nested
datasets, so the feature is effectively unusable exactly where it matters most.
### Why stock refuses — and why you must not simply delete the check
### Why stock refuses
The guard is **correct**. `plugins/cloud/snapshot.py` already takes a *recursive*
ZFS snapshot — but it then points restic at the **parent** dataset's
@@ -130,14 +146,13 @@ through a parent's snapshot directory:
/mnt/Tap/apps/lidarr/config/.zfs/snapshot/<snap>/ -> the real data
```
So if you just remove the validation, restic walks a **near-empty tree, reports
SUCCESS, and uploads almost nothing.** You get a green backup job protecting no
data. iX gate the config rather than ship a backup that lies about succeeding.
So if you just remove the validation, restic walks a near-empty tree, reports
SUCCESS, and uploads almost nothing — a green backup job protecting no data. iX
gate the config rather than ship a backup that lies about succeeding.
> **⚠ If you are tempted to patch this yourself: deleting those four lines in
> `plugins/cloud/crud.py` is the obvious move and it is catastrophic.** The
> guard is load-bearing. It must be *replaced* with a working traversal, never
> merely removed.
That is worth spelling out, because deleting those four lines in
`plugins/cloud/crud.py` is the obvious "fix" and it is the wrong one. The guard
is load-bearing: it has to be *replaced* with a working traversal, not removed.
### What this patch does instead
@@ -171,10 +186,69 @@ its parent snapshot between runs. Stock's `.zfs/snapshot/<name>-<timestamp>/`
path changes every run, defeating restic's parent detection and forcing a full
re-scan each time.
**Expected log noise:** stock `restic_backup()` deletes the ZFS snapshot in its
own `finally`, which fails with `EBUSY` while the staging mounts pin it. You will
see one benign `Error deleting snapshot ...` warning per run; the patch then
unmounts and deletes the snapshot for real.
### Snapshot lifecycle
`zfs.snapshot.delete` defaults to **`recursive=False`**, and stock
`restic_backup()` calls it with no options. Stock is safe only because its
validation means a *recursive* snapshot never actually happens in the field.
Enabling nested datasets makes them real: on a 250-dataset pool,
`zfs snapshot -r` creates **250 snapshots**, and stock's delete removes only the
parent — orphaning **249 on every successful run** (measured, not theorised).
So the patch owns the whole lifecycle:
- **Sweeps the parent and every child**, and is idempotent against stock's
`finally` winning the race once the mounts are released.
- **Records the snapshot in a sidecar file before mounting anything**, so a
middlewared restart mid-backup cannot orphan the tree (this patch *schedules*
a restart at boot, so that is not hypothetical).
- **Reclaims the tree left by a crashed run** instead of overwriting the record.
- **Deletes the tree when staging fails** — sync.py's own `finally` deletes
*nothing* in that case, because its `snapshot` local never gets assigned.
- **Enumerates datasets *after* the snapshot, never before.** A list read
beforehand can miss a dataset created in the gap, which the recursive snapshot
*would* capture but the staging plan would not — a silent omission.
**Expected log noise:** stock's delete fails with `EBUSY` while the staging
mounts pin the snapshot. You will see one benign `Error deleting snapshot ...`
warning per run; the patch then unmounts and deletes the tree for real.
### Verifying it works
This feature exists because a backup can report SUCCESS while containing
nothing, so check the contents rather than the exit status:
```bash
# 1. Does the restic snapshot actually contain child-dataset data?
# Pick a path that lives in a CHILD dataset (e.g. an app's config).
midclt call cloud_backup.list_snapshots <task_id> | head
# 2. List a child-dataset path inside the newest restic snapshot.
# If this is empty, the staging tree did not work and you are backing up NOTHING.
midclt call cloud_backup.list_snapshot_directory <task_id> "<snapshot_id>" "/apps/lidarr/config"
```
You should see the app's real files (`lidarr.db`, `config.xml`, …). An empty
listing means the child datasets were not staged; disable the feature and open an
issue.
```bash
# 3. No snapshots may be left behind after a run.
zfs list -t snapshot -r <pool> | grep -c cloud_backup- # expect 0 between runs
# 4. No staging mounts may be left behind.
mount | grep truecloud-nested # expect no output
```
### Troubleshooting
| Symptom | Cause |
|---|---|
| `This option is only available for datasets that have no further nesting` | Feature not enabled. Run `install.sh --enable-nested-snapshots`, then restart middlewared. |
| Backup fails: `dataset '…' has no snapshot '…'; refusing to back up an incomplete tree` | Working as designed — a descendant dataset was not covered by the snapshot. The backup is refused rather than silently omitting that data. |
| Backup fails: `snapshot '…' cannot be read (Permission denied)` | The snapshot exists but is unreadable. Middleware runs as root, so this indicates a real permissions problem, not a missing snapshot. |
| `cloud_backup-*` snapshots accumulating | The sweep is not running. Check `apply.log` for the nested patch applying, and confirm `sync.py` carries the `TRUECLOUD_PATCH` block. |
| Stale mounts under `/run/truecloud-nested` | A crashed run. The next run tears them down; `uninstall.sh` also cleans them. |
## Supported providers after patching
@@ -332,25 +406,35 @@ and restores the original UI bundle from backup.
## If TrueNAS adds native support
`apply.sh` checks at every boot whether TrueNAS has shipped native B2 restic
support (by inspecting `B2RcloneRemote.__dict__`). If it has:
The patch is **two independent modules**, and each retires on its own — TrueNAS
is likely to ship one of these natively long before the other, and a module
going native must not take the other one down with it.
1. The kill switch (`disabled` file) is set — no patching on any future boot.
2. Any active overlays are unmounted immediately.
3. The following message is written to `apply.log`:
| Module | What it does | Detected as native when |
|---|---|---|
| **providers** | B2/S3 credentials for TrueCloud Backup (`b2.py`, `restic.py`, UI dropdown) | `B2RcloneRemote` carries a real `get_restic_config()` |
| **nested** | Snapshots on datasets with child datasets (`plugins/cloud/*`) | the *"no further nesting"* validation is gone from `plugins/cloud/crud.py` |
```
NOTICE: TrueNAS now provides native B2 restic support — truecloud-patch is no longer needed.
NOTICE: Setting kill switch; patching will be skipped on all future boots.
NOTICE: Run the following to fully remove the patch:
NOTICE: bash /mnt/tank/truenas-truecloud-patch/uninstall.sh
```
At every boot `apply.sh` checks both:
- **One module goes native** → that module is skipped and logged; the other keeps
working, and the patch stays installed.
- **Both are done** (native, or nested was never enabled) → the kill switch
(`disabled` file) is set, overlays are unmounted, and `apply.log` tells you to
run `uninstall.sh`.
So on a box using only the provider patch, native B2 support retires the whole
thing as before. On a box that also uses nested snapshots, native B2 support
retires *just* that half.
Check the log after any TrueNAS update:
```bash
cat /mnt/tank/truenas-truecloud-patch/apply.log | tail -20
tail -20 /mnt/tank/truenas-truecloud-patch/apply.log
```
`hook_status.json` reports each module separately (`module.providers`,
`module.nested_snapshots`) with an `active` flag and a reason.
**Scenarios where the auto-detect may not fire** (manual check needed):
| Scenario | What happens | Action |