42 Commits

Include renames

Author	SHA1	Message	Date
flan	827c4358fd	Skip the chmod-based unreadable-sidecar tests when running as root ... The Gitea runner image executes jobs as root, and chmod(0) cannot make a file unreadable for root (CAP_DAC_OVERRIDE) — the two tests failed there while passing on GitHub's non-root runner. Skipping as root keeps the scenario exercised everywhere it is constructible.	2026-08-03 20:28:37 +00:00
flan	e426045255	An unchanged fingerprint froze the report's body, not just its comments ... Two questions were sharing one answer. "Have the findings changed?" gates COMMENTS. They notify, and a daily "still broken, same as yesterday" is what teaches everyone to ignore the one that finally matters. "Is the body still true?" gates the BODY. Editing an issue body notifies nobody on either forge, so keeping it honest is free. Conflated, an unchanged fingerprint froze the body -- and the fingerprint ignores, by design, everything that moves on its own: healthy rows, the hardware-verified column, point releases, and how a row is LABELLED. So the master -> 27-dev relabel would have shipped to the README and never to the issue anybody actually opens. The report would have gone on saying "master (unreleased) BROKEN" -- the precise false alarm the relabel exists to kill -- until iX happened to break something else. The body is now rewritten whenever it is stale; comments stay strictly a changelog of real changes. Bodies are compared after normalising line endings, because a forge that round-trips \r\n would otherwise trigger a silent rewrite every run and leave the issue looking freshly touched every morning.	2026-07-14 03:05:01 +00:00
flan	862bdd3399	master is 27-dev, not the next release; and check the next maintenance release ... Two ways the matrix misled the person it exists for -- somebody deciding whether to trust this with their backups. master is not the next release. iX branches each major onto its own release/ line and master rolls straight on to the one after: every recent commit on master targets 27.0.0-BETA.1 while 26 is still in beta. So a BROKEN master row, rendered "master (unreleased)", read as "the version you are about to install is broken" when the breakage is a major release away on a line nobody can download. It is now labelled from the newest major in the matrix plus one, so it becomes 28-dev by itself once 27 branches. The break, for the record, is NAS-141498 (2026-06-24), "Convert cloud_backup plugin to the typesafe pattern": it re-signatures restic_backup and get_restic_config, splitting entry/credentials out of the cloud_backup dict. Not being chased while the 27 line churns. The next maintenance release was never checked -- and it is the one that reaches users. Shipped came from TS-* tags, unreleased from release/* branches carrying -BETA/-RC. A branched-but-untagged MAINTENANCE release is neither: release/25.10.5 has no tag, and its line has already shipped, so the "a prerelease of a shipped line is history" filter threw it out. It was invisible, and it is exactly what a 25.10.4 box gets on its next update; a break in it reaches real users before the daily check ever looks, on the only line anybody runs. A plain release/X.Y.Z branch now counts when its line HAS shipped and it sorts NEWER than that line's newest tag. Both exclusions fall out of the same rule: release/24.10-RC.2 sorts older than TS-24.10.2.4 (history), and the typo branch release/25.20.2.2 is on a line with no tag at all (not a release line). This surfaced two refs never checked before -- release/25.10.5 and release/24.10.2.5 -- both of which pass. is_unreleased() keys off where a ref came from (a branch is by construction not shipped) rather than hunting -BETA/-RC in the name. Otherwise release/25.10.5 counts as shipped and a break in it fails the build as a live outage, on a version nobody is running yet.	2026-07-14 03:03:21 +00:00
flan	52b11eada2	The bug-report bot re-filed itself every run on Gitea; nine copies ... `find_issue()` skipped pull requests by testing for the PRESENCE of the `pull_request` key. GitHub omits that key on a plain issue. Gitea sends it as `null`. So on Gitea every issue was thrown away as if it were a PR, the lookup came back empty on every run, and the bot took the "nothing filed yet" branch and opened a brand-new report instead of editing the one already open. Nine of them piled up on the canonical forge. Four were filed AFTER the commit that was supposed to stop exactly this -- the same failure it was written to prevent, moved from comments to issues. It survived because `find_issue` was the one function here with no test; the mirror deduped correctly, and the mirror is what anybody would have looked at. Test the value, not the key: it is the only form that is true on both forges. Pinned by tests that carry each forge's payload shape by hand. Also send both paging parameters. GitHub reads `per_page`, Gitea reads `limit`, and each ignores the other's; Gitea's default page is 30, so the lookup would have begun missing the report once the pile it was creating outgrew one page.	2026-07-14 02:37:32 +00:00
flan	df412eeff7	test: make three vacuous tests actually test something; drop one duplicate ... The test suite is not bloated -- 3,701 lines of test code against 3,760 lines of product code, one duplicate pair in 356 tests, 8% single-assertion tests. The waste was not volume, it was four tests that looked like coverage and provided none: - test_parents_are_mounted_before_children: the fixture was already in depth order, so the sort it exists for was never exercised. Deleting `mounts.sort(key=_depth)` passed the whole suite. It now uses datasets whose NAME order differs from their MOUNTPOINT depth, which is the only case the sort is for. - test_it_NEVER_touches_the_current_run: the "current" snapshot was a minute old, so the age floor excluded it regardless and the same-snapname guard never ran. That guard only matters for a run that OUTLIVES the floor -- which a first full upload easily does, and where collecting it would yank the snapshot out from under a backup that is still reading from it. Now tested with a 12-hour-old current run. - the fingerprint/unknown test put the unreadable file in `providers`, which is not broken -- so fingerprint() skipped the whole module via is_broken() and the filter under test never executed. The blip has to land in the module that IS broken. - test_the_real_truenas_versions was a byte-identical copy of test_async_middleware_is_detected under a name promising more. Replaced with the fact actually worth pinning: the flavour probe must read STOCK source, because apply.sh re-runs on an already-patched overlay and our own SNAPSHOT_SYNC block is a plain `def create_snapshot` -- reading it would report a 25.10 box as synchronous and inject the wrong wrapper. All four now fail when the code they name is broken.	2026-07-14 01:41:25 +00:00
flan	086b20ed23	fix: fourth audit — two regressions from the last fix, and the boot preflight had no test ... Two of these were mine, from the previous round. - mounted_snapshots still swallowed OSError. I said I had fixed it and had not: the edit never matched, and I did not read it back. With the mount table unreadable the GC loses its in-use protection entirely and can destroy the snapshots of a backup that is still uploading (a first upload easily outlives the 1h age floor). It raises now, and both behaviours are tested. - The foreign-dataset check added last round had two bugs of its own. It ignored `mounted`, so a locked/encrypted dataset from a sibling tree turned a working nightly backup into a permanent failure — it belongs in `skipped`, exactly as an in-tree one does. And it tested `mp.startswith(path + "/")`, so a foreign dataset mounted EXACTLY at the backup path slipped through — the very hole the check was added to close, one character wide, and the worse case of the two because it SHADOWS the base dataset's own directory. - _read_sidecar's new raise broke cleanup_all, which is what recover.sh and uninstall.sh call — i.e. the code that must work when the box is ALREADY stuck. One unreadable sidecar aborted it before it unmounted anything, leaving the staging tree mounted, which pins the snapshots, which is the state recover.sh exists to escape. It now reports and carries on — and does not delete a record it could not read. - compat could report a FALSE OK: `defined` was collected by walking the whole file, so any function named `delete` anywhere in it — on an unrelated class, or nested inside another method — satisfied "this namespace defines delete". The runtime is stricter (a plugin class on the service's MRO), so the two could disagree in the ok direction. compat now looks in the class that declares the namespace. Same question on both sides, which is what pick_snapshot_service's docstring has been claiming all along. - apply.sh's compat preflight — the guard that refuses to patch a middleware whose assumptions no longer hold, on every boot, on a live NAS — had no test at all. It could be turned into a no-op eight different ways with the suite still green. The SHIPPED heredoc is now extracted and driven directly against fake verdicts. Also pinned: the Tap/Tap2 prefix collisions (a sweep that treats "Tap2/data@snap" as part of Tap's tree DESTROYS another pool's snapshot), and the GC's in_use wiring. 355 tests. Verified on TrueNAS 26.0.0-BETA.1: 292-dataset backup, 0 orphans, 0 leaked mounts, byte-identical restore of a 4-deep child dataset.	2026-07-14 01:34:13 +00:00
flan	8a41d7d7ef	fix: third audit — a cross-tree dataset was omitted silently, and the block tests passed on comments ... D1, the only cardinal-rule violation left. plan_staging scopes by dataset NAME, which is right (a dataset with no mountpoint cannot be scoped by path). But ZFS lets any dataset mount anywhere, so one from a DIFFERENT tree can sit inside the backup path: Tank/photos mountpoint=/mnt/Tap/apps/photos It holds data inside the path, and `zfs snapshot -r Tap@...` does NOT cover it — recursion follows the dataset tree, not the directory tree. It fell out of the name filter and vanished: not staged, not in `skipped`, no error. The backup reported SUCCESS with that data missing. Stock has the same blind spot but refuses the nested config outright; we are the ones relaxing that guard, so the hole is ours. It now raises. The test suite was the real weakness. apply.sh's injected blocks carry the highest-consequence logic in the project — the run_in_thread hop, the flavour selection, the finally-teardown, the re-raise — and were guarded only by substring greps. Two of them passed on COMMENTS: `assert "raise" in block` was satisfied by a comment reading "a cleanup that raises...", and `assert "cleanup_task" in block` by "cleanup_task gets logger=None". Deleting the actual re-raise (restic then backs up the UN-STAGED path — the silently-empty backup this module exists to prevent) and deleting the actual cleanup call from the finally (~250 orphans per run) both left the suite green. They are asserted structurally now, against the parsed block. Eleven regressions the audit found surviving now fail the suite, including: a swallowed staging failure, a missing teardown, an inverted flavour mapping, blocking work back on the asyncio event loop, the host's deleted get_dataset_recursive, query_filesystems quietly preferring the filtered middleware query, and a re-frozen `runner`/`sleep`/ `mounts_file` default (which would silently re-arm 19 tests reading the real mount table on the NAS). Also: _read_sidecar conflated "no sidecar" with "cannot read the sidecar", so cleanup_task took the empty branch and UNLINKED the only record of a tree it could not read. mounted_snapshots returned an empty set on error, silently switching off the GC's protection for snapshots a concurrent run is using. Both raise now. Verified on TrueNAS 26.0.0-BETA.1: zvol-orphan case 0 orphans, 292-dataset backup 0 orphans / 0 leaked mounts, byte-identical restore of a 4-deep child dataset.	2026-07-14 00:38:45 +00:00
flan	ce6998a935	fix: second audit — the delete check did nothing on a real box, and five guards were untestable ... The most important finding is that the FIRST audit's fix was wrong. _can_delete() asked `callable(getattr(service, "delete"))`. But CRUDService defines `delete` on the BASE class and dispatches to self.do_delete at call time, so a bound `delete` exists on every CRUDService subclass whether or not it still implements one. The check was therefore answering "is this a CRUDService?" — precisely the weaker "is the namespace registered?" question its own docstring said must never be asked. It would still have picked a gutted pool.snapshot and failed every delete. It now walks the MRO and ignores middlewared.service.* plumbing, so only a PLUGIN class defining delete/do_delete counts. The test double was equally wrong: it modelled a gutted service as object(), a shape middlewared cannot produce, so the test passed against a fake it could never have caught in the field. It is now CRUDService-shaped. Also: - The recursive delete's fast path returned [] without confirming anything was destroyed. A delete that returns cleanly is not proof — iX has already gutted pool.snapshot.do_update on master into a no-op that returns None. cleanup_task read "no survivors" as a clean sweep, dropped the sidecar (the only record), and would have orphaned ~250 snapshots per run, silently. It confirms against ZFS now, and the by-name sweep trusts ZFS rather than the API's return value. - When ZFS cannot be read, the sweep no longer claims success. The two mistakes are not symmetric: a false survivor self-heals (sidecar kept, next run reclaims, record clears), a lost record does not. - _write_sidecar swallowed OSError. The sidecar is the only record the snapshots exist; failing to write it must never be invisible. - stage_nested now refuses UP FRONT when middleware has no usable snapshot delete, rather than discovering it after restic has already run. Tests. The autouse fixture added in the last commit did not work: `runner=_run`, `mounts_file="/proc/self/mounts"` and `sleep=time.sleep` are frozen into __defaults__ at def time, so monkeypatching the module attribute never reached them. 19 tests were still reading the real mount table — one matching name from running a real `umount` on the NAS — and the retry loop really slept. All three are late-bound now; the suite reads nothing outside tmp_path and runs in 1.1s. Every mutation the audit reported as SURVIVING now fails the suite: the naive delete check, the unconfirmed fast path, the malformed-row guard, a disconnected GC, eager service resolution, compat's method check, compat's unknown handling, a single-quoted filtered query in apply.sh, and the get-service assumption. Also: fingerprint() folded `unknown` problems into a broken module, so one transient 429 rewrote the bug report and the next clean run rewrote it back. Problems are state-tagged; only definite breakage is digested. Verified on TrueNAS 26.0.0-BETA.1: zvol-orphan case 0 orphans, 292-dataset backup 0 orphans / 0 leaked mounts / 0 stale sidecars, byte-identical restore of a 4-deep child dataset.	2026-07-14 00:10:10 +00:00
flan	413cd60ed4	fix: own the snapshot sweep unconditionally; align the runtime and the manifest ... Four audits of the TrueNAS 26 branch. The findings, in severity order. 1. TrueNAS 26 orphaned a snapshot on every run, with no backstop. Stock decides `recursive` by its own rule, and on 26 that rule is no longer ours. <= 25.10 its create_snapshot called get_dataset_recursive() — the same function this module vendors — so "stock went recursive" and "we have something to stage" were the same question. 26 uses filesystem.statfs: recursive = (path == the dataset's mountpoint). A dataset whose only descendants are ZVOLs or legacy/none-mountpoint datasets now gets a RECURSIVE snapshot while the patch sees nothing to stage. The patch then handed the snapshot back to stock, which destroys the parent only. No staging tree meant no sidecar, and the GC only ever ran from stage_nested — so nothing on the box would ever have found the children. Reproduced on the VM: one orphan per zvol, every run, forever, backup green. Ownership of the sweep is no longer conditional on staging (own_snapshot()). 2. The runtime resolved a NAMESPACE; compat.py verified a METHOD. get_service() only proves a namespace is registered. compat checks the namespace AND that it defines delete/do_delete. So if iX guts the method but keeps the service — which they have already done to pool.snapshot.do_update on master — compat falls through to zfs.snapshot and reports the box healthy, while the runtime picks pool.snapshot and fails every delete. Both sides now ask "can this namespace delete?", and a test binds the two lists together. 3. query_filesystems() silently dropped malformed rows — the one remaining silent-omission path, and a direct contradiction of the cardinal rule. It raises now. A missing `zfs` binary raised FileNotFoundError rather than ZfsError; also fixed. 4. The retry loop discarded the delete error and reported every survivor as "(still busy?)" — naming the one cause that is benign and hiding the ones that are permanent. It keeps and reports the real error. Also: the staging-failure handler could lose the original exception if its own sweep raised; get_service is now a checked assumption; normalise_dataset and two dead MiddlewareCall properties removed; stale comments corrected. Tests: five of them were shelling out to the REAL pool (`zfs list -r Tap`, 2148 snapshots) and passed here only because this box has no zfs binary — they would have gone red on the NAS, which is the one machine the release process requires them green on. An autouse fixture now makes that impossible. Mutation-tested: reverting any of the five fixes above now fails the suite; before, all 293 passed. Verified on TrueNAS 26.0.0-BETA.1 (zvol leak reproduced, then closed; 292-dataset backup, 0 orphans, byte-identical restore of a 4-deep hidden dataset) and on 25.10.4 (pool.snapshot.delete honours recursive=True).	2026-07-13 23:30:48 +00:00
flan	605231b39f	feat: TrueNAS 26 support; enumerate datasets and snapshots from ZFS, not middleware ... TrueNAS 26 deletes plugins/zfs_/ outright, taking the private zfs.dataset.query, zfs.snapshot.query and zfs.snapshot.delete with it. All three were on the nested module's critical path, so nested snapshots were BROKEN on 26. Snapshot deletion now resolves its namespace at runtime: pool.snapshot on 25.10 and 26, zfs.snapshot on 24.10 and 25.04. No single namespace spans every supported release. tools/compat.py checks the same list the runtime uses, so what CI verifies and what runs cannot drift apart. Enumeration does NOT move to pool.dataset.query / pool.snapshot.query, and that is the point of this commit. Those methods exist, are documented, and are covered by iX's deprecation policy — and they are not like-for-like replacements. They apply a visibility policy that hides ix-apps/*, .system/* and .ix-virt/*: 84 of 270 datasets on a real pool, including live application data. Staging from that view omits them silently, and plan_staging never sees them, so they do not even reach the skipped list. The snapshot query hides the same datasets' snapshots, so the sweep orphans one per hidden dataset on every run. So: read the truth from ZFS, make changes through middleware. zfs list cannot be filtered by policy and behaves identically on every release. A failing zfs list raises rather than returning an empty list — "no datasets" and "the command broke" must never look the same. No shipped release is affected: v0.6.1 and earlier use the private zfs.dataset.query, which returns all 270 datasets. The bug existed only in this port. Verified on a real TrueNAS 26.0.0-BETA.1 install: 274-snapshot recursive backup of a 292-dataset pool, zero orphaned snapshots, zero leaked mounts, and a byte-identical restore of a four-level-deep child dataset that pool.dataset.query hides.	2026-07-13 22:43:20 +00:00
flan	7cc0826c2c	The matrix bot would never have worked: wrong permissions, wrong forge ... Two bugs, both of which would have failed silently on the first scheduled run: - permissions were while the step pushes a branch and opens a PR. It would have died with a 403 and I would have had a bot that never worked. - it opened the PR on GITHUB, which is a one-way MIRROR. A PR merged there would be clobbered by the next fleet-repos mirror push from Gitea. A bot opening PRs against a mirror is a bot doing nothing, slowly. Now: contents+pull-requests write, and the PR is opened on Gitea (canonical) via its API. One long-lived PR, force-pushed in place -- a daily PR is the same mistake as a daily comment, wearing a hat.	2026-07-13 20:28:30 +00:00
flan	82084b6806	The bug-report bot was spamming; make it say something only when there is something to say ... It commented on every run that found a break. In one day it left ELEVEN identical 3,000-character comments on the same issue. That is not a warning system, it is a mute button with extra steps -- and the next real finding would have been scrolled past, which defeats the entire reason for building it. Now: the issue BODY is the current truth, edited in place. COMMENTS are a changelog of changes. A fingerprint of the findings (broken ref/module/problem triples only) is embedded in the body; a run whose findings match it says nothing at all. It closes the issue when everything is fixed. The fingerprint deliberately ignores anything that moves on its own -- healthy rows, the hardware-verified column, TrueNAS point releases -- so TS-25.10.4 becoming TS-25.10.5 is not news and does not wake anybody up. Also: - The two near-identical per-forge shell steps are gone, replaced by one tested implementation (tools/compat_publish.py). Two copies of 'find the issue, decide whether to comment' is two chances to drift, and the Gitea one duplicated an issue for real. - The README matrix refresh now opens a PULL REQUEST instead of pushing straight to main from CI. An unattended push to main is exactly what the release barrier exists to prevent: a bot that can move main can move it somewhere nobody looked.	2026-07-13 20:20:35 +00:00
flan	841e0364fd	CHANGELOG: repair a section spliced into the middle of a bullet, and guard it ... An edit matched the literal '## Unreleased' inside a backticked phrase in a prose bullet and spliced a whole new section into the middle of it, splitting the sentence in half. The release body IS this file, so that would have shipped to every user. Tests now assert: no empty version section, versions descend, no heading is indented inside a list item, and every bullet's bold phrases are balanced (ignoring code spans -- '*args, **kwargs' is a literal, not markup).	2026-07-13 19:59:35 +00:00
flan	0d04c2cd1c	Collect orphaned snapshots by name: the sidecar lives in tmpfs ... A reboot mid-backup orphaned the entire tree, permanently. The sidecar is the record of which snapshots a run pinned -- and /run is tmpfs. A reboot or crash between the recursive snapshot and its cleanup destroyed that record, leaving one snapshot per descendant dataset (250+ on a real pool) with nothing pointing at them. Nothing would ever have found them. gc_stale_snapshots() identifies leftovers by NAME, so it works when the record is gone. It runs after the sidecar reclaim -- the recorded path stays authoritative and the collector only mops up what the record lost. It deletes data on a name match, which is a weaker claim than a recorded fact, so the selection is a pure function with the harshest tests here. A snapshot is collected only if the name is exactly <dataset>@<task>-<YYYYMMDDHHMMSS>, it is not the current run's, NOTHING IS MOUNTED FROM IT (this, not the age guard, is what protects a concurrent backup), and it is over an hour old. Checked against the real pool: of 4728 snapshots including 2341 periodic ones, it selects exactly the orphans of the task being run and nothing else.	2026-07-13 19:56:32 +00:00
flan	2b8ef107f7	The sidecar must carry EVERY pending tree, not just the newest ... Found live, in the code written to prevent exactly this. The sidecar held ONE snapshot. So a run that reclaimed an older tree, failed to finish reclaiming it, and then recorded its own snapshot OVERWROTE the only record of the survivor -- orphaning it permanently. Observed: job 24 left one snapshot busy and kept the sidecar (correct). Job 46 reclaimed it, hit ZFS's 300s automount window (the runs were minutes apart), left it behind again, and then wrote its own snapshot over the record. Permanent orphan, created by the safety net. The sidecar is now a list. stage_nested carries forward whatever a reclaim could not delete; cleanup_task sweeps every pending tree and writes back only the survivors. cleanup_all reports them one per line instead of formatting a list into an f-string at the user during uninstall. Job 46 also confirms the automount fix itself: it swept all 256 of its own snapshots with no straggler.	2026-07-13 19:35:44 +00:00
flan	b50567e9a7	A few snapshots leaked on every nested run, forever ... Found on real hardware: a 256-snapshot backup of /mnt/Tap swept 253 and left 3 with 'dataset is busy'. ZFS AUTOMOUNTS <dataset>/.zfs/snapshot/<snap> when it is read, and keeps it mounted for zfs_expire_snapshot seconds (300 default) after the last access. teardown() unmounts OUR bind mounts but not the automount underneath, so zfs destroy refuses for exactly the datasets restic read most recently. cleanup_task() then removed the sidecar anyway -- destroying the only record those snapshots existed. Nothing would ever have reclaimed them. - release_snapdirs() unmounts ZFS's own automounts (deepest first) before deleting. - delete_snapshot_tree() retries the transient busy and RETURNS what it could not delete, instead of swallowing it. - The sidecar is removed only on a confirmed-clean sweep -- including on the staging-failure path, which used to remove it before the caller swept. A sidecar left behind when the tree is gone costs one no-op delete; a sidecar removed while the tree exists is unrecoverable.	2026-07-13 19:18:22 +00:00
flan	469a2e4651	Installing the patch permanently blocked updating it ... install.sh chmod +x's update.sh, and git recorded update.sh as 100644 -- so the chmod was a TRACKED modification, and update.sh refuses to run over a dirty tree. Install once and you could never update again. The error even told you to 'git checkout -- .', which just undoes the exec bit so the next install can re-dirty it. Found on the real box, which had been sitting on v0.4.1 for exactly this reason. Fixed on both sides: the scripts install.sh chmods are executable in git (so the chmod is a no-op), and update.sh's dirty check now looks at CONTENT, not mode -- git diff --numstat reports 0 0 for a mode-only change. A test asserts every script in install.sh's chmod loop is already 100755 in git.	2026-07-13 18:58:46 +00:00
flan	8c1b4c45f5	release: rc notes resolve to the base version; publish without jq ... release_notes.py 'notes v0.6.0-rc1' looked for a CHANGELOG section literally named v0.6.0-rc1. check() already used base_version(); extract_notes() did not. So the release workflow cut the tag, passed every gate, and then died extracting the body -- the candidate existed but was never published. Caught in an rc, which is the entire point of having them. Also: the Gitea publish and issue steps used jq, which is not guaranteed on a self-hosted runner. A publish step that dies on a missing tool leaves a tag with no release behind it, and a bug report that dies on one is a warning system that does not warn. Both now use python3, which setup-python guarantees.	2026-07-13 18:38:02 +00:00
flan	252696f1de	docs: split the 969-line README; put Install at the top ... Install was at line 517 of 969, under the boot sequence, the snapshot lifecycle and the release process. Someone deciding whether to trust this with their backups should not have to scroll past any of that. README is now 211 lines: what it does, the minimum version, install, the support matrix, updating, uninstall. Everything else moved to docs/ (nested snapshots, how it works, recovery, CLI, releasing). A test enforces it: every internal link resolves, Install stays near the top, and the README does not grow back. Moving Markdown breaks cross-references -- it broke eight of them here, including one in a recovery doc, where the person following the link is by definition already having a bad day.	2026-07-13 18:34:44 +00:00
flan	a54b4dd7f6	Never touch CloudSync tasks; restore the logger the async cleanup path dropped ... create_snapshot is module-global in plugins/cloud/snapshot.py, and cloud_sync.py imports it as well as cloud_backup/sync.py. So the wrapper sat in the path of every rclone/Storj CloudSync task with snapshot=true, and ran a zfs.dataset.query before concluding it had nothing to do -- a new failure mode for jobs that worked before this patch existed. Worse: a CloudSync task that ever got staged would never be torn down. The teardown is wired into cloud_backup's restic_backup finally, and CRUD_BLOCK deliberately leaves CloudSync's guard intact, so the bind mounts would pin the snapshot forever. The staging path now bails out unless the snapshot is named cloud_backup-*, before any middleware call. Separately: the async wrapper's finally dropped logger=, which the sync one passes. run_in_thread forwards **kwargs, so a cleanup that failed to unmount a bind mount or delete a snapshot tree logged nothing at all -- on the only platform anyone runs.	2026-07-13 18:30:35 +00:00
flan	ecb64878ff	compat: check the middlewared METHODS we call, not just the symbols we wrap ... This gap was hiding a catastrophe. TrueNAS 26 deletes plugins/zfs_/dataset.py and plugins/zfs_/snapshot.py outright, taking zfs.dataset.query, zfs.snapshot.query and zfs.snapshot.delete with them (26 uses filesystem.statfs and zfs.resource.*). Nothing about the five cloud_backup files reveals that, so every other check went green -- including the one I had just added. The patch would have applied cleanly and then failed on the first backup, or worse: snapshotted fine and failed to DELETE, orphaning one snapshot per descendant dataset (250 on a real pool) on every run, forever. So 26 is BROKEN and the nested module will not apply there. The async/sync wrapper work and the vendored get_dataset_recursive stay -- they are correct and necessary -- but 26 is not supported until the ZFS calls are ported, and that needs a real 26 box to verify. Shipping a port nobody has run is the failure this project exists to avoid. Also: do_delete is recognised as delete (24.10/25.04 use the CRUDService convention), which was reporting both as BROKEN -- a false verdict that would have disabled nested snapshots on boxes where they work.	2026-07-13 18:25:14 +00:00
flan	498b2690e1	TrueNAS 26 support: one sync implementation, two wrappers ... 26 rewrites cloud_backup from async to synchronous AND deletes get_dataset_recursive(), which SNAPSHOT_BLOCK called out of the host module's namespace. Either is a broken backup found at restore time. The nested module is now one synchronous implementation talking to middlewared via call_sync, behind two thin wrappers. apply.sh reads which flavour the installed middleware declares and injects the matching one: <= 25.10 reaches it through 'await middleware.run_in_thread(...)', 26 is already in a worker thread and calls it directly. The snapshot/bind-mount/failure logic exists once -- an async twin would mean every future fix had to land twice. A middleware whose three wrapped functions disagree about asyncness is refused, not guessed at. get_dataset_recursive is vendored, removing the dependency on both versions rather than asserting it. master stays BROKEN on purpose: iX are still renaming middleware->context, cloud_backup->entry and adding a required credentials param there. Chasing a branch that moves daily is how you ship a patch nobody tested.	2026-07-13 18:18:28 +00:00
flan	aa725fb198	Pin the two native probes against drift ... The split-literal squash is implemented twice: inline in apply.sh's runtime probe and as compat._squash in the static checker. That subtlety already caused one silent bug (the probe concluded iX had removed the nesting guard, which means 'retire the module'). Both are now exercised against the same inputs, including the split-across-literals form stock actually uses.	2026-07-13 17:59:27 +00:00
flan	ea090c7f72	Audit fixes: a compat verdict must never be able to brick a working box ... The audit found the new machinery could do more harm than the bugs it prevents. - apply.sh reused the 'nothing left to do' exit -- which touches the PERMANENT kill switch, cleared only by install.sh, never by update.sh -- for the incompatible case. On TrueNAS 26 (providers ok, nested opt-out) both modules go quiet, so the switch would fire and the release that fixed 26 could never re-enable itself. Retirement and incompatibility now take different exits. - A network blip, a re-export, or a conditional def all read as BROKEN. Each is now 'unknown', which changes nothing, rather than evidence strong enough to disable a module. - 'native' outranked BROKEN everywhere but apply.sh, so a TrueNAS that reworded the guard AND reshaped the functions rendered as good news. - compat.py --tree read B2_BLOCK's own 'restic = True' as native support, so the documented way to check a live box lied on every patched machine. - The signature check was a name-subset test. It passed reorders, kw-only conversions, and added required params -- and it had already passed a real bug: restic_backup takes 4 args on 24.10/25.04, and the wrapper forwarded 5. Nested backups have been raising TypeError on those releases the whole time. The wrapper now forwards *args/**kwargs. - release.sh --promote was unreachable: it died if the tag existed, the gate died if it did not. The tests hid it by always tagging first.	2026-07-13 17:58:15 +00:00
flan	cd39489c7f	compat/release: stop interpolating ${{ }} into run: bodies ... The report body is full of backticks, so 'echo "${{ steps.report.outputs.body }}"' pasted it into the shell text and bash executed create-snapshot, def and async as commands. The report is built from iX's middleware source, so that was an injection vector as well as a bug. inputs.tag on workflow_dispatch had the same shape. Data goes through files, scalars through env:. Tests enforce it across every workflow.	2026-07-13 17:30:50 +00:00
flan	5cbb7def6f	Compatibility watch: check the patch's assumptions against every TrueNAS release line ... TrueNAS 26 rewrites cloud_backup from async to sync. Every block the nested module injects is an async wrapper around an awaited original, so on 26 it hands sync.py a coroutine where it unpacks a tuple. tools/compat.py records what each module assumes and checks it two ways: CI runs it against iX's source at every release line (including master and the current BETA) and files a bug report when an unreleased line breaks; apply.sh runs it against the middlewared actually installed and refuses to apply a module whose assumptions no longer hold. Stock TrueNAS without a feature beats TrueNAS with a broken one. Workflows run on both forges; only the release/issue API calls differ.	2026-07-13 17:25:37 +00:00
flan	4ced730d65	Make Gitea canonical; derive the changelog URL from the remote instead of hard-coding GitHub	2026-07-13 17:21:30 +00:00
flan	bf6d37e621	v0.5.1: the update alert could have broken middlewared at startup ... middlewared's alert.load() imports every file in alert/source/ with NO try/except: def load(self): for module in load_modules(.../alert/source): for cls in load_classes(module, AlertSource, (ThreadedAlertSource,)): ... and it runs during setup. A module that raises on import therefore takes middlewared's startup down with it -- exactly the class of failure this project exists to avoid. apply.sh now COMPILES the substituted alert source and refuses to write it if it does not parse. An uninstalled alert is a missing convenience; a broken one is a broken box. @PATCH_DIR@ is also substituted with repr() rather than raw, so a repository path containing a quote or backslash yields a valid Python literal instead of a syntax error in the installed module. The alert source no longer mutates sys.path. It loaded tools/release_notes.py via sys.path.insert(0, ...), which shadows the stdlib for that interpreter -- and ThreadedAlertSource runs in middlewared's thread pool, so mutating sys.path is a race. It now loads by file path with importlib. New tests guard every import-time failure mode: the module compiles, apply.sh compiles before writing, awkward paths (quotes, backslashes) still produce valid modules, nothing but imports/constants/classes runs at module scope, every AlertClass name ends in "AlertClass" (AlertClassMeta raises NameError otherwise), the alert text placeholders match the args passed, and no git command that writes to .git is ever used. 152 tests, ruff and shellcheck -S style clean.	2026-07-13 16:52:23 +00:00
flan	4e0814028c	v0.5.0: TrueNAS alert when an update is available ... Raises a real alert in the TrueNAS UI bell -- not a log line nobody reads. On by default, checked once a day. install.sh --no-update-alerts turns it off. It does not nag --------------- A release whose CHANGELOG contains only a "### Docs" section changed no code and raises nothing. Anything else raises INFO; a "### Security" section raises WARNING. The CHANGELOG's own section headings are the signal, and a security fix anywhere in the range escalates the whole span -- a docs-only release sitting on top of a security fix still reports as security rather than hiding it. Why an AlertSource and not midclt ---------------------------------- TrueNAS cannot raise an alert from the CLI. midclt exposes only alert.dismiss, alert.list, alert.list_categories, alert.list_policies and alert.restore -- alert CREATION is internal to middlewared, and none of its ~60 one-shot classes is generic enough to reuse. Registering an AlertSource is the only way. It is also the least invasive thing this patch does. The providers and nested modules both APPEND CODE TO STOCK middleware files; the alert source ADDS ONE FILE and modifies none. It is the native mechanism -- the same one every built-in TrueNAS alert uses -- and TrueNAS polls it itself, so there is no cron job and no systemd timer. - Fail-safe: every error path returns None; it cannot take middlewared down. - Read-only: `git ls-remote` plus an HTTPS fetch of the CHANGELOG. It never writes to .git, so it cannot leave root-owned objects behind the way a `git fetch` from middlewared (running as root) would. - Removed by uninstall.sh (mw_patch.revert_all). - It only tells you; it never updates anything. Verified against the real repo and remote, with middlewared stubbed: on v0.4.1, only a README-only v0.4.2 available -> NO ALERT on v0.4.0, v0.4.1 fixed real bugs -> INFO on v0.3.2, v0.3.3 was the password fix -> SECURITY / WARNING 139 tests, ruff and shellcheck -S style clean.	2026-07-13 16:45:26 +00:00
flan	126756498c	v0.3.4: one implementation of apply/revert (patch/mw_patch.py) ... The "strip the TRUECLOUD_PATCH block" logic existed twice -- in apply.sh's heredoc and in an inline heredoc in uninstall.sh -- and the uninstall copy was the untested one. That is precisely how the two could have drifted apart, with apply.sh reverting one set of files and uninstall.sh another. Both now call patch/mw_patch.py. 17 new tests cover it, including that revert_nested never touches restic.py: that file carries a TRUECLOUD_PATCH block too, but it belongs to the providers module, and removing it would silently break B2 backups. apply.sh imports it fail-safe -- on ImportError the backend patch is skipped and middlewared starts stock, which is this script's whole design principle. The import uses sys.path.append, never insert(0): prepending would give patch/ precedence over the stdlib for that interpreter, so a future patch/json.py would shadow the real json module and break the boot. Also: the README's create_task.py example still taught `--password <secret>`, which is how a security fix quietly fails to land. It now shows --password-stdin. 132 tests, ruff and shellcheck clean.	2026-07-13 16:02:53 +00:00
flan	60b3ac4557	v0.3.3: keep the restic repo password out of argv and shell history ... Security -------- create_task.py shelled out to `midclt call cloud_backup.create '<json>'`, and that JSON carries the restic repository password -- so it sat in the subprocess's argv, which is world-readable via ps, for the duration of the call. That password is the encryption key for the entire cloud backup repository. It now talks to the middleware through truenas_api_client, the library that backs midclt itself, so the password never leaves this process's memory. Verified on a live box: list-tasks and list-credentials work through the new transport. --password is also no longer required, because passing a secret as a CLI argument writes it to shell history permanently. --password-stdin reads it from stdin, and with neither flag the tool prompts via getpass. --password still works but warns. Fixed ----- uninstall.sh could leave every patch installed. It reverted by unmounting the overlay -- but apply.sh only mounts one when the target directory is read-only. On a writable /usr it patches the real files in place, and uninstall would remove the boot hook, report success, and leave the patch applied. It now strips the appended blocks from the middleware files explicitly. This also covers the case where the overlay unmount fails. create_task.py's __version__ had been stuck at 0.2.0 through three releases. The version-drift check added in v0.3.1 only looked at VERSION= in shell scripts, so it missed the one file that actually shows a version to users (--version). The check now covers __version__ too -- and caught this immediately. 118 tests, ruff and shellcheck clean.	2026-07-13 15:43:24 +00:00
flan	8aa9038226	Fix: --disable-nested-snapshots did not disable anything until reboot ... apply.sh only ever ADDED patches; there was no revert path anywhere. Disabling removed the opt-in marker and then merely skipped re-applying -- but the overlay persists for the whole boot, so the previously patched cloud/{snapshot,crud}.py, cloud_backup/sync.py and _truecloud_nested.py were all still on disk, and middlewared re-imported them on the restart install.sh performs. It printed "DISABLED (stock guard restored)" while the feature kept running until the next reboot. Someone disabling it because they were worried about it would have believed it was off. apply.sh now reverts on every not-needed path (opt-out, or superseded by native support): remove the module FIRST -- every injected block is guarded by `if _tc_nested is not None`, so the stock guard comes back even if a later step fails -- then strip the appended blocks from the three patched files. restic.py also carries a TRUECLOUD_PATCH block but belongs to the providers module; reverting it would silently break B2 backups, so it is explicitly excluded. Verified: the three nested files restore byte-for-byte to stock, the module is removed, and restic.py's block survives. install.sh --disable also tears the staging tree down first, since those bind mounts pin ZFS snapshots that could otherwise never be destroyed. Updating WITHOUT the flag was always correct and is unchanged: the nested module is never installed into middleware unless explicitly enabled. 109 tests, ruff and shellcheck clean.	2026-07-13 15:23:27 +00:00
flan	eb91a337cd	Automate releases from tags ... Releases were manual and had drifted: v0.2.0 and v0.2.1 were tagged but never released, so the releases page jumped v0.1.0 -> v0.3.0 and hid the fix for the incident that took every app down. Pushing a v* tag now runs the full suite and cuts a GitHub release whose body is the matching CHANGELOG.md section -- one source of truth for release notes, so there is no second place for them to be wrong. The workflow refuses to publish when: - the tests, ruff, or bash -n fail (a tagged commit is what people install; it must be at least as good as main) - the tag does not match the VERSION= declared by every script - CHANGELOG.md has no section for the tag, or the section is empty That version check is not theoretical: VERSION= had drifted to three different values across install.sh / uninstall.sh / recover.sh / apply.sh and nothing noticed until this release. tests/test_release_notes.py now asserts the scripts agree with each other and with the newest CHANGELOG entry, so the drift cannot come back. workflow_dispatch takes an existing tag, so releases can be backfilled for tags that were pushed before this existed. 106 tests, ruff and shellcheck clean.	2026-07-13 15:05:46 +00:00
flan	51bf5326d9	Refuse to write a bundle whose parens we unbalanced ... Commit 47cdf72 shipped a pattern that matched one closing paren and emitted one, netting an extra `)` in the Angular bundle: c(2,"filterByProviders",["STORJ_IX","S3","B2"]))("required",!0) ^^ syntax error The TrueNAS web UI went blank. Worse, MARKER was now present in the file, so every later run reported "already patched" and skipped -- the patch could not heal itself, and the bundle had to be hand-restored from the .pre-truecloud-patch backup. patch_ui.py now compares the parenthesis balance before and after substitution and refuses to write if it changed. A bundle we cannot patch correctly is left exactly as it was: an unpatched UI is a missing dropdown entry, a corrupted one is a dead web UI. Tests cover the real regression (verbatim 47cdf72 pattern) end to end: it still matches, the balance still shifts, main() refuses, and the file on disk is byte-for-byte unchanged. README documents the manual recovery for anyone who already hit it. 93 tests, ruff and shellcheck clean.	2026-07-13 14:57:16 +00:00
flan	8a2028bfa7	Add test coverage for the Angular bundle patch ... patch_ui.py rewrites minified third-party JavaScript by regex and had no tests. It is the easiest place in this project to do real damage: a pattern that matches nothing silently leaves the dropdown Storj-only, and one that consumes a paren too many is a syntax error in the bundle that blanks the entire TrueNAS web UI. Tests run the real patterns against verbatim snippets from a TrueNAS 25.x chunk-*.js (the chained property(...)(...) form) and a 24.x literal array, and assert: exactly one match, all three providers present, the paren balance is UNCHANGED, surrounding code untouched, re-patching is a no-op, unrelated JS is never matched, and every pattern stays anchored to filterByProviders. The paren-balance assertion is the load-bearing one -- a plausible-but-wrong pattern that eats both parens and re-emits none shifts the delta from 1 to 2 and is caught. Also restore the comment explaining why the 25.x pattern is shaped the way it is, and correct the module docstring, which showed the binding with a single closing paren; the real bundle wraps it in a chained property call. 90 tests, ruff and shellcheck clean.	2026-07-13 14:37:43 +00:00
flan	8421a34d8d	Delete the snapshot tree atomically instead of 252 calls ... delete_snapshot_tree removed the parent and every child snapshot individually. On a real pool `zfs snapshot -r` creates one snapshot per descendant dataset -- 252 on Tap -- so cleanup was 252 sequential middleware calls. Slow, but the real problem is that it is not atomic: a job killed part-way through the sweep leaves behind exactly the orphaned snapshots this function exists to prevent. zfs.snapshot.delete accepts {"recursive": True}, which destroys the parent and all children in one call. Use that as the fast path and keep the name-by-name sweep as the fallback -- it is still needed when the parent is already gone (stock's finally can win the race once our mounts are released), which makes a recursive delete fail while the children survive. The test fake now emulates real `zfs destroy -r` semantics, so a test cannot pass while the shipped code deletes only the parent. 76 tests, ruff and shellcheck clean.	2026-07-13 14:35:04 +00:00
flan	150a241a0f	Fix native-nested probe: guard message is split across string literals ... The probe searched plugins/cloud/crud.py for the contiguous string "no further nesting". Stock does not contain it. The message is split across adjacent string literals: verrors.add(f"{name}.snapshot", "This option is only available for datasets that have no further " "nesting") Python concatenates those at runtime, so the errmsg IS contiguous and CRUD_BLOCK's runtime filter matches correctly -- but the SOURCE never contains the whole phrase. The probe therefore found nothing, concluded iX had removed the guard, and skipped the nested module as "already native" on every boot. apply.log would say "TrueNAS now handles nesting natively" and the feature would never work. It fails safe -- the stock guard stays in place, so no backup could be misconfigured and no data was at risk -- but the module was 100% dead. Verified against real middlewared on a live box: the probe returned native=yes (wrong) before this change and native=no (correct) after. The probe now strips whitespace and quote characters before matching, which is robust to any wrapping or concatenation style. Added a regression test that EXECUTES apply.sh's own probe code (extracted, not reimplemented -- a reimplementation would pass while the shipped probe stayed broken) against the real wrapped source, a single-line variant, and a three-way split. 75 tests, ruff and shellcheck clean.	2026-07-12 22:44:26 +00:00
flan	24f1f2c648	Fix post-merge audit findings; unify staging teardown ... Audit ----- - create_task.py verify failed on a DEFAULT install. hook_status.json emitted a per-file entry for the nested module with ok:false whenever the feature was switched off -- the default -- so verify printed [FAIL] and exited 1, right after the README tells users to run it. Status is now per MODULE with an `active` flag, and verify renders an inactive module as [SKIP]. - A partial apply suppressed the middlewared restart. The exit code conflated "nothing applied" with "one module applied, one failed", so a failing providers patch would prevent the restart that a freshly-applied nested patch needs, leaving it on disk and never loaded. Exit 2 now means partial and the restart still fires. - The native-nested probe could never fire. It scanned crud.py for the guard message, but our own injected block quotes that message, so once applied the probe would always conclude the guard was still present. It now reads only the stock portion of the file. - recover.sh did not unmount staging trees, so an emergency recovery left bind mounts pinning ZFS snapshots that could then never be destroyed. - uninstall.sh deleted sidecar files without reading them. A sidecar is the only record that an interrupted run's snapshot tree is still on disk; both scripts now name the snapshot before clearing it. - Removed a dead branch in the restart gate (unreachable: the kill switch exits). Refactor -------- - Staging teardown had been copy-pasted into uninstall.sh and recover.sh -- two untested shell copies of the fiddly depth-ordering and lazy-umount logic. Both now call `python3 patch/truecloud_nested.py cleanup`, so there is exactly one implementation and it is the one under test. - Dropped the in-memory ACTIVE dict. The sidecar file was already the source of truth; a second in-process record could only desync -- and it is precisely the middlewared-restart case (which empties it) that must not orphan a snapshot tree. One record, on disk, or none. Not done: the overlay-unmount loop is duplicated across apply.sh/uninstall.sh/ recover.sh. It is pre-existing, and apply.sh runs at PREINIT under a tight timeout -- giving it a source dependency would trade 10 lines of duplication for a boot-time failure mode. 74 tests, ruff and shellcheck clean.	2026-07-12 22:29:31 +00:00
flan	eca6eb3f3b	Retire the two modules independently; calm the README down ... The native-support check looked only for native B2 restic support and, on finding it, set the kill switch and disabled the whole patch. That was fine when providers were the only thing here. With nested-dataset snapshots in the patch it is wrong: TrueNAS is likely to ship one capability long before the other, and a single all-or-nothing kill switch would silently take a still-needed module down with the superseded one. apply.sh now treats the patch as two independent modules: providers b2.py + restic.py + the UI credential dropdown native when B2RcloneRemote carries a real get_restic_config() nested plugins/cloud/{snapshot,crud}.py + cloud_backup/sync.py (opt-in) native when the "no further nesting" validation is gone from plugins/cloud/crud.py Each is detected and skipped on its own. The kill switch fires only once BOTH are done (native, or nested was never enabled). The UI patch belongs to providers and is skipped with it. The deferred middlewared restart now fires when ANY still-needed module landed -- keying it off providers alone would have left a freshly-patched nested module on disk and never loaded on a native-B2 box. hook_status.json reports each module with an active flag and a reason. README: dropped the warning boxes and the disclaimer's fear-bulleting in favour of plain statements, documented the two-module design and the per-module auto-disable, and added a Development section disclosing AI assistance. The one caveat kept, as a plain sentence rather than a banner: the mount --bind staging step has not yet been exercised by a live backup run. 67 tests, ruff and shellcheck clean.	2026-07-12 22:17:37 +00:00
flan	a80de88078	Distinguish a missing snapshot from an unreadable one ... os.path.isdir() returns False both when a snapshot directory does not exist and when it cannot be stat'd. Staging aborted either way -- correct -- but reported every case as "has no snapshot", which sends you hunting for a snapshot that is sitting right there. Found while dry-running the planner against a real recursive snapshot of Tap: running as a non-root user, /mnt/Tap/apps/paperless/data is mode 0700 and the probe reported "has no snapshot" when `zfs list` showed the snapshot present. Middleware runs as root so this would not fire in production, but a backup system must not misreport why it failed. plan_staging now takes a probe() that classifies the path as ok / missing / unreadable, and the error names which. Dry-run results against Tap (250 datasets, real `zfs snapshot -r`): - 170 mounted filesystems under /mnt/Tap/, and the plan produces exactly 170 descendant mounts -- no omissions - 18 legacy-mountpoint datasets reported as skipped, never dropped silently - Tap/ix-apps mounts at /mnt/.ix-apps, correctly outside the backup path - parent snapshot exposes 0 entries under /apps; the staged sources expose 71, and lidarr/config resolves with lidarr.db present - snapshot_tree_names() identifies all 250; deleting only the parent (what stock does) leaves 249 orphans, and the sweep clears them	2026-07-12 22:04:56 +00:00
flan	bb26edf351	Make nested snapshots opt-in; fix snapshot leaks found in audit ... Opt-in ------ Nested-dataset snapshot support changes how backups read their source data, so it is now off by default and gated behind a marker file: install.sh --enable-nested-snapshots install.sh --disable-nested-snapshots With neither flag install.sh preserves the current setting, so a routine `git pull && bash install.sh` can never silently flip it. When disabled, apply.sh skips the patch entirely and the stock guard remains. uninstall.sh tears down staging mounts and removes the marker. Snapshot lifecycle ------------------ zfs.snapshot.delete defaults to recursive=False and stock restic_backup() calls it with no options. Stock is safe only because its validation means recursive is never True in the field. Enabling nested datasets makes recursive snapshots real: the parent then has one child snapshot per descendant dataset (160+ on an Apps pool), so stock's delete would orphan every child on EVERY successful run. The patch now owns the lifecycle end to end: - delete_snapshot_tree() sweeps the parent and all children, and is idempotent against stock's finally winning the race once our mounts are released - on a staging failure the tree is deleted here, because sync.py never completes `snapshot, local_path = await create_snapshot(...)` and so its finally deletes nothing at all - the snapshot is recorded in a sidecar file before anything is mounted, so a middlewared restart mid-backup cannot orphan it - a crashed run's snapshot tree is reclaimed on the next run instead of being overwritten and leaked Silent-omission fix ------------------- The dataset list is now enumerated AFTER the snapshot. Read beforehand it could miss a dataset created in the gap, which the recursive snapshot would capture but the staging plan would not -- silently omitting its data. Read afterwards, an unsnapshotted dataset trips the staging check and fails the run loudly. Also from the audit ------------------- - plan_staging scopes by dataset name, so skipped-dataset warnings no longer include every mountpoint-less dataset on the box, which buried the ones that matter - staging_root_for rejects "." / ".." components that would escape the staging base, and resolves STAGING_BASE at call time rather than freezing it into a default argument - uninstall.sh no longer `rm -rf`s a tree that may still contain live bind mounts, and unmounts by path depth rather than string length - apply_plan takes an injectable isdir; verify_staged drops an unused parameter - pin the shellcheck action instead of tracking @master 61 tests, ruff and shellcheck clean.	2026-07-12 21:52:09 +00:00
flan	a572eb2164	Support ZFS snapshots on datasets with child datasets ... TrueCloud Backup's "Take Snapshot" option is rejected on any path containing child datasets: This option is only available for datasets that have no further nesting That excludes every pool running Apps, where each app is its own dataset and often has config/pgdata children. Without the option the backup reads live files, so databases are captured mid-write and an app that continuously rewrites its files can stall a run as restic chases a moving target. The stock guard is correct and must not simply be removed. create_snapshot() already takes a recursive ZFS snapshot, but points the backup tool at the parent dataset's .zfs/snapshot/, and ZFS does not expose child datasets there: /mnt/Tap/.zfs/snapshot/<snap>/apps/ -> 0 entries /mnt/Tap/apps/lidarr/config/.zfs/snapshot/<snap>/ -> the real data Deleting the check would make restic walk a near-empty tree, report success, and upload almost nothing. Implement the missing traversal instead. After the recursive snapshot is taken, each descendant dataset's own .zfs/snapshot/<snap> is bind-mounted into a staging tree mirroring the original layout, and the backup tool is pointed at the staging root. The guard is relaxed only after that machinery is in place. Safety properties: - staging failure aborts the backup; a partial tree is never handed to restic - a post-mount pass asserts every target is a mountpoint and the root is non-empty, so this cannot regress into the empty backup it exists to prevent - apply.sh patches crud.py last, so a partial failure leaves the guard intact rather than exposing "guard removed, traversal missing" - every injected block no-ops when _truecloud_nested is absent - unmountable/locked datasets are skipped and reported, never dropped silently - scoped to cloud_backup; cloudsync has no teardown wired in, so its guard stays The staging root is stable per task, so restic can find its parent snapshot between runs; stock's timestamped .zfs path changes every run and forces a full re-scan. Add CI (shellcheck, bash -n, ruff, pytest on 3.11-3.13), including tests that compile the *_BLOCK strings, which are Python source appended to live middlewared modules and were previously unchecked. Also: sync stale version strings, untrack a committed .pyc, gitignore __pycache__.	2026-07-12 21:20:22 +00:00