f2d57420fb86343e43600dbbfeda1f88737c590c
3
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 flan
|
bb26edf351 |
Make nested snapshots opt-in; fix snapshot leaks found in audit
Opt-in ------ Nested-dataset snapshot support changes how backups read their source data, so it is now off by default and gated behind a marker file: install.sh --enable-nested-snapshots install.sh --disable-nested-snapshots With neither flag install.sh preserves the current setting, so a routine `git pull && bash install.sh` can never silently flip it. When disabled, apply.sh skips the patch entirely and the stock guard remains. uninstall.sh tears down staging mounts and removes the marker. Snapshot lifecycle ------------------ zfs.snapshot.delete defaults to recursive=False and stock restic_backup() calls it with no options. Stock is safe only because its validation means recursive is never True in the field. Enabling nested datasets makes recursive snapshots real: the parent then has one child snapshot per descendant dataset (160+ on an Apps pool), so stock's delete would orphan every child on EVERY successful run. The patch now owns the lifecycle end to end: - delete_snapshot_tree() sweeps the parent and all children, and is idempotent against stock's finally winning the race once our mounts are released - on a staging failure the tree is deleted here, because sync.py never completes `snapshot, local_path = await create_snapshot(...)` and so its finally deletes nothing at all - the snapshot is recorded in a sidecar file before anything is mounted, so a middlewared restart mid-backup cannot orphan it - a crashed run's snapshot tree is reclaimed on the next run instead of being overwritten and leaked Silent-omission fix ------------------- The dataset list is now enumerated AFTER the snapshot. Read beforehand it could miss a dataset created in the gap, which the recursive snapshot would capture but the staging plan would not -- silently omitting its data. Read afterwards, an unsnapshotted dataset trips the staging check and fails the run loudly. Also from the audit ------------------- - plan_staging scopes by dataset name, so skipped-dataset warnings no longer include every mountpoint-less dataset on the box, which buried the ones that matter - staging_root_for rejects "." / ".." components that would escape the staging base, and resolves STAGING_BASE at call time rather than freezing it into a default argument - uninstall.sh no longer `rm -rf`s a tree that may still contain live bind mounts, and unmounts by path depth rather than string length - apply_plan takes an injectable isdir; verify_staged drops an unused parameter - pin the shellcheck action instead of tracking @master 61 tests, ruff and shellcheck clean. |
||
|
flan
|
a572eb2164 |
Support ZFS snapshots on datasets with child datasets
TrueCloud Backup's "Take Snapshot" option is rejected on any path containing child datasets: This option is only available for datasets that have no further nesting That excludes every pool running Apps, where each app is its own dataset and often has config/pgdata children. Without the option the backup reads live files, so databases are captured mid-write and an app that continuously rewrites its files can stall a run as restic chases a moving target. The stock guard is correct and must not simply be removed. create_snapshot() already takes a recursive ZFS snapshot, but points the backup tool at the parent dataset's .zfs/snapshot/, and ZFS does not expose child datasets there: /mnt/Tap/.zfs/snapshot/<snap>/apps/ -> 0 entries /mnt/Tap/apps/lidarr/config/.zfs/snapshot/<snap>/ -> the real data Deleting the check would make restic walk a near-empty tree, report success, and upload almost nothing. Implement the missing traversal instead. After the recursive snapshot is taken, each descendant dataset's own .zfs/snapshot/<snap> is bind-mounted into a staging tree mirroring the original layout, and the backup tool is pointed at the staging root. The guard is relaxed only after that machinery is in place. Safety properties: - staging failure aborts the backup; a partial tree is never handed to restic - a post-mount pass asserts every target is a mountpoint and the root is non-empty, so this cannot regress into the empty backup it exists to prevent - apply.sh patches crud.py last, so a partial failure leaves the guard intact rather than exposing "guard removed, traversal missing" - every injected block no-ops when _truecloud_nested is absent - unmountable/locked datasets are skipped and reported, never dropped silently - scoped to cloud_backup; cloudsync has no teardown wired in, so its guard stays The staging root is stable per task, so restic can find its parent snapshot between runs; stock's timestamped .zfs path changes every run and forces a full re-scan. Add CI (shellcheck, bash -n, ruff, pytest on 3.11-3.13), including tests that compile the *_BLOCK strings, which are Python source appended to live middlewared modules and were previously unchecked. Also: sync stale version strings, untrack a committed .pyc, gitignore __pycache__. |
||
 flan
|
70e84038d6 |
Repo is the install location; all scripts self-locate
Users now clone to a persistent ZFS pool and the repo stays in place. No files are copied on install — the PREINIT hook points directly into the clone. Scripts derive PATCH_DIR from their own path at runtime. - install.sh: PATCH_DIR=$(dirname $0); register patch/apply.sh as PREINIT target; chmod only, no cp; update pipe-install error message - patch/apply.sh: PATCH_DIR=$(dirname $0)/..; substitute PATCH_DIR into sitecustomize.py via sed when writing to site-packages; reference patch_ui.py as patch/patch_ui.py - recover.sh, uninstall.sh: PATCH_DIR=$(dirname $0) - uninstall.sh: look for patch/apply.sh in PREINIT registry - patch/create_task.py: _PATCH_DIR derived from __file__; apply.log path in error message derived from _PATCH_DIR - patch/sitecustomize.py: /data/truecloud-patch remains as placeholder substituted by apply.sh on each install - .gitignore: exclude runtime files (apply.log, hook_status.json, disabled) - README: document clone-to-pool install; update all example paths |