flan
605231b39f
TrueNAS 26 deletes plugins/zfs_/ outright, taking the private zfs.dataset.query, zfs.snapshot.query and zfs.snapshot.delete with it. All three were on the nested module's critical path, so nested snapshots were BROKEN on 26. Snapshot deletion now resolves its namespace at runtime: pool.snapshot on 25.10 and 26, zfs.snapshot on 24.10 and 25.04. No single namespace spans every supported release. tools/compat.py checks the same list the runtime uses, so what CI verifies and what runs cannot drift apart. Enumeration does NOT move to pool.dataset.query / pool.snapshot.query, and that is the point of this commit. Those methods exist, are documented, and are covered by iX's deprecation policy — and they are not like-for-like replacements. They apply a visibility policy that hides ix-apps/*, .system/* and .ix-virt/*: 84 of 270 datasets on a real pool, including live application data. Staging from that view omits them silently, and plan_staging never sees them, so they do not even reach the skipped list. The snapshot query hides the same datasets' snapshots, so the sweep orphans one per hidden dataset on every run. So: read the truth from ZFS, make changes through middleware. zfs list cannot be filtered by policy and behaves identically on every release. A failing zfs list raises rather than returning an empty list — "no datasets" and "the command broke" must never look the same. No shipped release is affected: v0.6.1 and earlier use the private zfs.dataset.query, which returns all 270 datasets. The bug existed only in this port. Verified on a real TrueNAS 26.0.0-BETA.1 install: 274-snapshot recursive backup of a 292-dataset pool, zero orphaned snapshots, zero leaked mounts, and a byte-identical restore of a four-level-deep child dataset that pool.dataset.query hides.
218 lines
9.3 KiB
Markdown
218 lines
9.3 KiB
Markdown
# truenas-truecloud-patch
|
|
|
|
Extends TrueNAS SCALE's **TrueCloud Backup** to:
|
|
|
|
- back up to **Backblaze B2 and any S3-compatible provider**, not just Storj;
|
|
- snapshot **datasets that have child datasets** — which is every box running Apps.
|
|
|
|
**Requires TrueNAS SCALE 24.10 or newer.** TrueCloud Backup does not exist before
|
|
that, and `install.sh` will refuse.
|
|
|
|
> Storj raised the price of their TrueNAS-integrated tier from **$5/month to
|
|
> $50/month** in 2026. TrueCloud Backup is the only native TrueNAS feature that gives
|
|
> you pre-backup ZFS snapshots, restic dedup, scheduled tasks with UI progress, and
|
|
> dataset-lock integration. Running restic by hand loses all of it. This gets the
|
|
> feature back with storage you already pay for.
|
|
|
|
---
|
|
|
|
## Install
|
|
|
|
Clone it onto a **pool** (not the boot device — that is wiped on TrueNAS upgrades),
|
|
then run `install.sh` as root:
|
|
|
|
```bash
|
|
git clone https://github.com/sudolulo/truenas-truecloud-patch.git \
|
|
/mnt/tank/truenas-truecloud-patch # replace `tank` with your pool
|
|
cd /mnt/tank/truenas-truecloud-patch
|
|
sudo bash install.sh
|
|
```
|
|
|
|
That registers a PREINIT boot hook, patches middleware in a volatile overlay, and
|
|
restarts middlewared. **It survives TrueNAS updates** — the patch is re-applied at
|
|
every boot, never written to the system dataset.
|
|
|
|
Nested-dataset snapshots are **opt-in**:
|
|
|
|
```bash
|
|
sudo bash install.sh --enable-nested-snapshots
|
|
```
|
|
|
|
Then create a TrueCloud Backup task in the UI with a B2 or S3 credential, or from
|
|
[the CLI](docs/cli.md).
|
|
|
|
**Check it worked:**
|
|
|
|
```bash
|
|
sudo python3 patch/create_task.py verify
|
|
```
|
|
|
|
If something is wrong, the reason is in `apply.log` — start at
|
|
[Recovery](docs/recovery.md).
|
|
|
|
---
|
|
|
|
## TrueNAS compatibility
|
|
|
|
<!-- BEGIN COMPAT MATRIX (generated by tools/compat.py --matrix --markdown) -->
|
|
| TrueNAS | B2/S3 providers | Nested snapshots | Hardware-verified |
|
|
| --- | --- | --- | --- |
|
|
| 24.10.2.4 | ok | ok | — |
|
|
| 25.04.2.6 | ok | ok | — |
|
|
| 25.10.4 | ok | ok | nested + providers; 252-snapshot recursive backup of /mnt/Tap, 18m |
|
|
| 26.0.0-BETA.3 _(unreleased)_ | ok | ok | — |
|
|
| master _(unreleased)_ | **BROKEN** | **BROKEN** | — |
|
|
|
|
| verdict | meaning |
|
|
| --- | --- |
|
|
| **ok** | Every assumption the patch makes about middleware still holds. |
|
|
| **BROKEN** | middleware changed underneath the patch. `apply.sh` **refuses to apply that module** on this version and leaves TrueNAS stock, so backups keep working — without the module's feature. |
|
|
| **native** | TrueNAS does this itself now. The module retires; it is not a failure. |
|
|
|
|
"ok" means *the patch's assumptions hold*, checked automatically against iX's
|
|
source. It does not mean a human ran a backup on it — that is the
|
|
**Hardware-verified** column, which is filled in by hand and only by doing it.
|
|
<!-- END COMPAT MATRIX -->
|
|
|
|
The table is **regenerated daily by CI** against iXsystems' actual middleware source
|
|
— it is not a claim somebody typed once and forgot.
|
|
|
|
**On TrueNAS 26:** the patch was run on a real TrueNAS **26.0.0-BETA.1** install — a
|
|
274-snapshot recursive backup of a 292-dataset pool, followed by a byte-identical
|
|
restore of a four-level-deep child dataset. The *Hardware-verified* column tracks the
|
|
newest beta iX has tagged (currently BETA.3), so it does not carry that mark: a build
|
|
nobody has actually run a backup on does not get credit for one.
|
|
|
|
**TrueNAS 26: nested snapshots are not supported yet, and upgrading will not break
|
|
you.** 26 rewrites `cloud_backup` and deletes the ZFS methods this module calls. On
|
|
26 `apply.sh` finds that the assumptions no longer hold and **does not apply the
|
|
module**: TrueNAS is left stock, B2/S3 keeps working, nested datasets are simply not
|
|
covered, and the reason is named in `apply.log`. A broken backup is worse than a
|
|
missing feature. Details: [How it works](docs/how-it-works.md#truenas-26).
|
|
|
|
---
|
|
|
|
## Updating
|
|
|
|
```bash
|
|
cd /mnt/tank/truenas-truecloud-patch
|
|
sudo bash update.sh # newest release
|
|
sudo bash update.sh --check # what would change?
|
|
sudo bash update.sh --rollback # back to the previous version
|
|
```
|
|
|
|
`update.sh` refuses to run on a dirty checkout, pins you to a release tag, and keeps
|
|
the previous revision so a rollback is one command. **There is no auto-update**: this
|
|
patches system internals as root, and a bad commit reaching your box unattended would
|
|
detonate on the next reboot — v0.0.4 shipped exactly such a bug and took 54 apps
|
|
down. The manual step *is* the safety gate.
|
|
|
|
---
|
|
|
|
## Update alerts
|
|
|
|
When a newer release exists, the patch raises a **TrueNAS alert** (the bell in the
|
|
UI) telling you so. It's on by default and checks once a day.
|
|
|
|
```bash
|
|
bash install.sh --no-update-alerts # turn it off
|
|
bash install.sh --update-alerts # turn it back on
|
|
```
|
|
|
|
**It will not nag you about a README.** A release whose CHANGELOG contains only a
|
|
`### Docs` section changed no code, and raises nothing. Anything that touched the
|
|
system raises an INFO alert; a release with a `### Security` section raises a
|
|
WARNING. The CHANGELOG's own section headings are the signal, and a security fix
|
|
anywhere in the range escalates the whole span — a docs-only release on top of a
|
|
security fix still reports as security.
|
|
|
|
**Release candidates never alert.** They are invisible to `update.sh` and to the
|
|
alert, which both take the newest plain `vX.Y.Z` tag. That is what lets debugging
|
|
happen in `-rc` tags instead of in your notification bell — see
|
|
[Releasing](docs/releasing.md).
|
|
|
|
The changelog is read from whichever forge `origin` points at, derived from the
|
|
remote rather than hard-coded. That is not cosmetic: when the changelog cannot be
|
|
read, the alert deliberately fires **anyway** rather than risk hiding a security
|
|
fix — so a wrong URL would not silence the alert, it would make it fire on *every*
|
|
release, including the documentation-only ones this section promises to suppress.
|
|
|
|
### How it works, and why it's built this way
|
|
|
|
TrueNAS **cannot raise an alert from the CLI** — `midclt` exposes only
|
|
`alert.dismiss`, `alert.list`, `alert.list_categories`, `alert.list_policies` and
|
|
`alert.restore`. Alert *creation* is internal to middlewared, and none of its ~60
|
|
one-shot alert classes is generic enough to reuse. So the only way to get a real
|
|
alert is to register an `AlertSource`, which is what `patch/alert_source.py` does.
|
|
|
|
That is also the **least invasive** thing this patch does:
|
|
|
|
| | |
|
|
|---|---|
|
|
| providers module | **modifies** stock files (appends code to `b2.py`, `restic.py`) |
|
|
| nested module | **modifies** stock files (3 middleware modules) |
|
|
| **update alert** | **adds one file. Modifies nothing.** |
|
|
|
|
It's the native mechanism — the same one every built-in TrueNAS alert uses — and
|
|
TrueNAS polls it itself, so there is no cron job and no systemd timer.
|
|
|
|
- **Fail-safe.** Every error path returns `None`. It cannot take middlewared down.
|
|
- **Read-only.** `git ls-remote` plus an HTTPS fetch of the CHANGELOG. It never
|
|
writes to `.git`, so it cannot leave root-owned objects behind the way a
|
|
`git fetch` from middlewared (which runs as root) would.
|
|
- **Removed by `uninstall.sh`.**
|
|
|
|
It only *tells* you. It never updates anything — see
|
|
[Updating](#updating).
|
|
|
|
---
|
|
|
|
## Uninstall
|
|
|
|
```bash
|
|
bash /mnt/tank/truenas-truecloud-patch/uninstall.sh
|
|
```
|
|
|
|
Replace the path with your clone location. Removes the PREINIT hook,
|
|
unmounts the overlay (restoring the original backend files immediately),
|
|
and restores the original UI bundle from backup.
|
|
|
|
---
|
|
|
|
## Documentation
|
|
|
|
| | |
|
|
|---|---|
|
|
| [Nested-dataset snapshots](docs/nested-snapshots.md) | Why stock refuses, what this does instead, and **how to verify your backups actually contain the data** |
|
|
| [How it works](docs/how-it-works.md) | What is patched, how it survives updates, the boot sequence, and what happens when TrueNAS goes native |
|
|
| [Recovery](docs/recovery.md) | middlewared won't start, blank web UI, `verify` shows FAIL |
|
|
| [CLI](docs/cli.md) | Creating tasks with `create_task.py` |
|
|
| [Development](docs/releasing.md) | Tests, CI, and the release process |
|
|
|
|
## What's in the repo
|
|
|
|
| Path | What it is |
|
|
|---|---|
|
|
| `install.sh` | Register the boot hook, patch, restart middlewared. Also `--enable/--disable-nested-snapshots`. |
|
|
| `update.sh` | Fetch and apply a newer release. `--check`, `--rollback`, `--to`, `--main`. |
|
|
| `uninstall.sh` | Remove everything. |
|
|
| `recover.sh` | Emergency: kill switch + restart against stock files. |
|
|
| `patch/apply.sh` | The PREINIT script. Runs at **every boot**. |
|
|
| `patch/truecloud_nested.py` | Nested-dataset staging: plan, mount, verify, tear down, sweep snapshots. |
|
|
| `patch/create_task.py` | Create tasks with S3/B2 credentials; `verify` the patch state. |
|
|
| `tools/compat.py` | What the patch assumes about middlewared — and the checker. Run daily by CI *and* at every boot. |
|
|
| `release.sh` | Cut a release. Two stages, and the second is refused without the first. |
|
|
|
|
## Before you install
|
|
|
|
- This is **unofficial** and not affiliated with iXsystems.
|
|
- It patches **internal middleware APIs** with no stability contract. Every patch is
|
|
fail-safe: if it cannot apply, middlewared starts normally and the reason is logged.
|
|
- **Test your restores.** True of any backup; more so here. See [Verifying it
|
|
works](docs/nested-snapshots.md#verifying-it-works).
|
|
- Filing a TrueNAS bug? **Remove the patch first** and reproduce on a stock system.
|
|
- Provided as-is, no warranty. See LICENSE.
|
|
|
|
Parts of this project were written with AI assistance (Claude); all of it is reviewed
|
|
and tested before release. Bugs are mine.
|