Support ZFS snapshots on datasets with child datasets

TrueCloud Backup's "Take Snapshot" option is rejected on any path containing
child datasets:

  This option is only available for datasets that have no further nesting

That excludes every pool running Apps, where each app is its own dataset and
often has config/pgdata children. Without the option the backup reads live
files, so databases are captured mid-write and an app that continuously
rewrites its files can stall a run as restic chases a moving target.

The stock guard is correct and must not simply be removed. create_snapshot()
already takes a recursive ZFS snapshot, but points the backup tool at the
parent dataset's .zfs/snapshot/, and ZFS does not expose child datasets there:

  /mnt/Tap/.zfs/snapshot/<snap>/apps/                -> 0 entries
  /mnt/Tap/apps/lidarr/config/.zfs/snapshot/<snap>/  -> the real data

Deleting the check would make restic walk a near-empty tree, report success,
and upload almost nothing.

Implement the missing traversal instead. After the recursive snapshot is taken,
each descendant dataset's own .zfs/snapshot/<snap> is bind-mounted into a
staging tree mirroring the original layout, and the backup tool is pointed at
the staging root. The guard is relaxed only after that machinery is in place.

Safety properties:
- staging failure aborts the backup; a partial tree is never handed to restic
- a post-mount pass asserts every target is a mountpoint and the root is
  non-empty, so this cannot regress into the empty backup it exists to prevent
- apply.sh patches crud.py last, so a partial failure leaves the guard intact
  rather than exposing "guard removed, traversal missing"
- every injected block no-ops when _truecloud_nested is absent
- unmountable/locked datasets are skipped and reported, never dropped silently
- scoped to cloud_backup; cloudsync has no teardown wired in, so its guard stays

The staging root is stable per task, so restic can find its parent snapshot
between runs; stock's timestamped .zfs path changes every run and forces a
full re-scan.

Add CI (shellcheck, bash -n, ruff, pytest on 3.11-3.13), including tests that
compile the *_BLOCK strings, which are Python source appended to live
middlewared modules and were previously unchecked.

Also: sync stale version strings, untrack a committed .pyc, gitignore
__pycache__.
This commit is contained in:
flan
2026-07-12 21:20:22 +00:00
parent 4ded8cff3d
commit a572eb2164
14 changed files with 1113 additions and 15 deletions
+60
View File
@@ -0,0 +1,60 @@
name: CI
on:
push:
branches: [main, "feat/**", "fix/**"]
pull_request:
workflow_dispatch:
permissions:
contents: read
jobs:
shell:
name: shell (shellcheck + syntax)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: bash syntax check
run: |
fail=0
while IFS= read -r f; do
bash -n "$f" || { echo "::error file=$f::bash syntax error"; fail=1; }
done < <(find . -name '*.sh' -not -path './.git/*')
exit $fail
- name: shellcheck
uses: ludeeus/action-shellcheck@master
env:
SHELLCHECK_OPTS: -S warning -e SC1091
python:
name: python ${{ matrix.python }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
# TrueNAS SCALE middleware runs 3.11+; keep the patch importable across
# the versions it may be injected into.
python: ["3.11", "3.12", "3.13"]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python }}
- name: install dev deps
run: python -m pip install --upgrade pip pytest ruff
- name: ruff
run: ruff check patch tests
- name: pytest
run: pytest tests -v
- name: verify injected middleware blocks compile
# Belt-and-braces: the *_BLOCK strings are appended into live middlewared
# modules. A syntax error there would break the box at boot.
run: pytest tests/test_apply_blocks.py -v