ecb64878ffc68dacbaedc95391cb5633df1d8099
22
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 flan
|
ecb64878ff |
compat: check the middlewared METHODS we call, not just the symbols we wrap
This gap was hiding a catastrophe. TrueNAS 26 deletes plugins/zfs_/dataset.py and plugins/zfs_/snapshot.py outright, taking zfs.dataset.query, zfs.snapshot.query and zfs.snapshot.delete with them (26 uses filesystem.statfs and zfs.resource.*). Nothing about the five cloud_backup files reveals that, so every other check went green -- including the one I had just added. The patch would have applied cleanly and then failed on the first backup, or worse: snapshotted fine and failed to DELETE, orphaning one snapshot per descendant dataset (250 on a real pool) on every run, forever. So 26 is BROKEN and the nested module will not apply there. The async/sync wrapper work and the vendored get_dataset_recursive stay -- they are correct and necessary -- but 26 is not supported until the ZFS calls are ported, and that needs a real 26 box to verify. Shipping a port nobody has run is the failure this project exists to avoid. Also: do_delete is recognised as delete (24.10/25.04 use the CRUDService convention), which was reporting both as BROKEN -- a false verdict that would have disabled nested snapshots on boxes where they work. |
||
|
flan
|
498b2690e1 |
TrueNAS 26 support: one sync implementation, two wrappers
26 rewrites cloud_backup from async to synchronous AND deletes get_dataset_recursive(), which SNAPSHOT_BLOCK called out of the host module's namespace. Either is a broken backup found at restore time. The nested module is now one synchronous implementation talking to middlewared via call_sync, behind two thin wrappers. apply.sh reads which flavour the installed middleware declares and injects the matching one: <= 25.10 reaches it through 'await middleware.run_in_thread(...)', 26 is already in a worker thread and calls it directly. The snapshot/bind-mount/failure logic exists once -- an async twin would mean every future fix had to land twice. A middleware whose three wrapped functions disagree about asyncness is refused, not guessed at. get_dataset_recursive is vendored, removing the dependency on both versions rather than asserting it. master stays BROKEN on purpose: iX are still renaming middleware->context, cloud_backup->entry and adding a required credentials param there. Chasing a branch that moves daily is how you ship a patch nobody tested. |
||
|
flan
|
aa725fb198 |
Pin the two native probes against drift
The split-literal squash is implemented twice: inline in apply.sh's runtime probe and as compat._squash in the static checker. That subtlety already caused one silent bug (the probe concluded iX had removed the nesting guard, which means 'retire the module'). Both are now exercised against the same inputs, including the split-across-literals form stock actually uses. |
||
|
flan
|
ea090c7f72 |
Audit fixes: a compat verdict must never be able to brick a working box
The audit found the new machinery could do more harm than the bugs it prevents. - apply.sh reused the 'nothing left to do' exit -- which touches the PERMANENT kill switch, cleared only by install.sh, never by update.sh -- for the incompatible case. On TrueNAS 26 (providers ok, nested opt-out) both modules go quiet, so the switch would fire and the release that fixed 26 could never re-enable itself. Retirement and incompatibility now take different exits. - A network blip, a re-export, or a conditional def all read as BROKEN. Each is now 'unknown', which changes nothing, rather than evidence strong enough to disable a module. - 'native' outranked BROKEN everywhere but apply.sh, so a TrueNAS that reworded the guard AND reshaped the functions rendered as good news. - compat.py --tree read B2_BLOCK's own 'restic = True' as native support, so the documented way to check a live box lied on every patched machine. - The signature check was a name-subset test. It passed reorders, kw-only conversions, and added required params -- and it had already passed a real bug: restic_backup takes 4 args on 24.10/25.04, and the wrapper forwarded 5. Nested backups have been raising TypeError on those releases the whole time. The wrapper now forwards *args/**kwargs. - release.sh --promote was unreachable: it died if the tag existed, the gate died if it did not. The tests hid it by always tagging first. |
||
|
flan
|
cd39489c7f |
compat/release: stop interpolating ${{ }} into run: bodies
The report body is full of backticks, so 'echo "${{ steps.report.outputs.body }}"'
pasted it into the shell text and bash executed create-snapshot, def and async as
commands. The report is built from iX's middleware source, so that was an injection
vector as well as a bug. inputs.tag on workflow_dispatch had the same shape.
Data goes through files, scalars through env:. Tests enforce it across every
workflow.
|
||
|
flan
|
5cbb7def6f |
Compatibility watch: check the patch's assumptions against every TrueNAS release line
TrueNAS 26 rewrites cloud_backup from async to sync. Every block the nested module injects is an async wrapper around an awaited original, so on 26 it hands sync.py a coroutine where it unpacks a tuple. tools/compat.py records what each module assumes and checks it two ways: CI runs it against iX's source at every release line (including master and the current BETA) and files a bug report when an unreleased line breaks; apply.sh runs it against the middlewared actually installed and refuses to apply a module whose assumptions no longer hold. Stock TrueNAS without a feature beats TrueNAS with a broken one. Workflows run on both forges; only the release/issue API calls differ. |
||
|
flan
|
4ced730d65 | Make Gitea canonical; derive the changelog URL from the remote instead of hard-coding GitHub | ||
 flan
|
bf6d37e621 |
v0.5.1: the update alert could have broken middlewared at startup
middlewared's alert.load() imports every file in alert/source/ with NO try/except:
def load(self):
for module in load_modules(.../alert/source):
for cls in load_classes(module, AlertSource, (ThreadedAlertSource,)):
...
and it runs during setup. A module that raises on import therefore takes
middlewared's startup down with it -- exactly the class of failure this project
exists to avoid.
apply.sh now COMPILES the substituted alert source and refuses to write it if it
does not parse. An uninstalled alert is a missing convenience; a broken one is a
broken box.
@PATCH_DIR@ is also substituted with repr() rather than raw, so a repository path
containing a quote or backslash yields a valid Python literal instead of a syntax
error in the installed module.
The alert source no longer mutates sys.path. It loaded tools/release_notes.py via
sys.path.insert(0, ...), which shadows the stdlib for that interpreter -- and
ThreadedAlertSource runs in middlewared's thread pool, so mutating sys.path is a
race. It now loads by file path with importlib.
New tests guard every import-time failure mode: the module compiles, apply.sh
compiles before writing, awkward paths (quotes, backslashes) still produce valid
modules, nothing but imports/constants/classes runs at module scope, every
AlertClass name ends in "AlertClass" (AlertClassMeta raises NameError otherwise),
the alert text placeholders match the args passed, and no git command that writes
to .git is ever used.
152 tests, ruff and shellcheck -S style clean.
|
||
|
flan
|
4e0814028c |
v0.5.0: TrueNAS alert when an update is available
Raises a real alert in the TrueNAS UI bell -- not a log line nobody reads. On by default, checked once a day. install.sh --no-update-alerts turns it off. It does not nag --------------- A release whose CHANGELOG contains only a "### Docs" section changed no code and raises nothing. Anything else raises INFO; a "### Security" section raises WARNING. The CHANGELOG's own section headings are the signal, and a security fix anywhere in the range escalates the whole span -- a docs-only release sitting on top of a security fix still reports as security rather than hiding it. Why an AlertSource and not midclt ---------------------------------- TrueNAS cannot raise an alert from the CLI. midclt exposes only alert.dismiss, alert.list, alert.list_categories, alert.list_policies and alert.restore -- alert CREATION is internal to middlewared, and none of its ~60 one-shot classes is generic enough to reuse. Registering an AlertSource is the only way. It is also the least invasive thing this patch does. The providers and nested modules both APPEND CODE TO STOCK middleware files; the alert source ADDS ONE FILE and modifies none. It is the native mechanism -- the same one every built-in TrueNAS alert uses -- and TrueNAS polls it itself, so there is no cron job and no systemd timer. - Fail-safe: every error path returns None; it cannot take middlewared down. - Read-only: `git ls-remote` plus an HTTPS fetch of the CHANGELOG. It never writes to .git, so it cannot leave root-owned objects behind the way a `git fetch` from middlewared (running as root) would. - Removed by uninstall.sh (mw_patch.revert_all). - It only tells you; it never updates anything. Verified against the real repo and remote, with middlewared stubbed: on v0.4.1, only a README-only v0.4.2 available -> NO ALERT on v0.4.0, v0.4.1 fixed real bugs -> INFO on v0.3.2, v0.3.3 was the password fix -> SECURITY / WARNING 139 tests, ruff and shellcheck -S style clean. |
||
|
flan
|
126756498c |
v0.3.4: one implementation of apply/revert (patch/mw_patch.py)
The "strip the TRUECLOUD_PATCH block" logic existed twice -- in apply.sh's heredoc and in an inline heredoc in uninstall.sh -- and the uninstall copy was the untested one. That is precisely how the two could have drifted apart, with apply.sh reverting one set of files and uninstall.sh another. Both now call patch/mw_patch.py. 17 new tests cover it, including that revert_nested never touches restic.py: that file carries a TRUECLOUD_PATCH block too, but it belongs to the providers module, and removing it would silently break B2 backups. apply.sh imports it fail-safe -- on ImportError the backend patch is skipped and middlewared starts stock, which is this script's whole design principle. The import uses sys.path.append, never insert(0): prepending would give patch/ precedence over the stdlib for that interpreter, so a future patch/json.py would shadow the real json module and break the boot. Also: the README's create_task.py example still taught `--password <secret>`, which is how a security fix quietly fails to land. It now shows --password-stdin. 132 tests, ruff and shellcheck clean. |
||
|
flan
|
60b3ac4557 |
v0.3.3: keep the restic repo password out of argv and shell history
Security -------- create_task.py shelled out to `midclt call cloud_backup.create '<json>'`, and that JSON carries the restic repository password -- so it sat in the subprocess's argv, which is world-readable via ps, for the duration of the call. That password is the encryption key for the entire cloud backup repository. It now talks to the middleware through truenas_api_client, the library that backs midclt itself, so the password never leaves this process's memory. Verified on a live box: list-tasks and list-credentials work through the new transport. --password is also no longer required, because passing a secret as a CLI argument writes it to shell history permanently. --password-stdin reads it from stdin, and with neither flag the tool prompts via getpass. --password still works but warns. Fixed ----- uninstall.sh could leave every patch installed. It reverted by unmounting the overlay -- but apply.sh only mounts one when the target directory is read-only. On a writable /usr it patches the real files in place, and uninstall would remove the boot hook, report success, and leave the patch applied. It now strips the appended blocks from the middleware files explicitly. This also covers the case where the overlay unmount fails. create_task.py's __version__ had been stuck at 0.2.0 through three releases. The version-drift check added in v0.3.1 only looked at VERSION= in shell scripts, so it missed the one file that actually shows a version to users (--version). The check now covers __version__ too -- and caught this immediately. 118 tests, ruff and shellcheck clean. |
||
|
flan
|
8aa9038226 |
Fix: --disable-nested-snapshots did not disable anything until reboot
apply.sh only ever ADDED patches; there was no revert path anywhere. Disabling
removed the opt-in marker and then merely skipped re-applying -- but the overlay
persists for the whole boot, so the previously patched cloud/{snapshot,crud}.py,
cloud_backup/sync.py and _truecloud_nested.py were all still on disk, and
middlewared re-imported them on the restart install.sh performs.
It printed "DISABLED (stock guard restored)" while the feature kept running until
the next reboot. Someone disabling it because they were worried about it would
have believed it was off.
apply.sh now reverts on every not-needed path (opt-out, or superseded by native
support): remove the module FIRST -- every injected block is guarded by
`if _tc_nested is not None`, so the stock guard comes back even if a later step
fails -- then strip the appended blocks from the three patched files.
restic.py also carries a TRUECLOUD_PATCH block but belongs to the providers
module; reverting it would silently break B2 backups, so it is explicitly
excluded. Verified: the three nested files restore byte-for-byte to stock, the
module is removed, and restic.py's block survives.
install.sh --disable also tears the staging tree down first, since those bind
mounts pin ZFS snapshots that could otherwise never be destroyed.
Updating WITHOUT the flag was always correct and is unchanged: the nested module
is never installed into middleware unless explicitly enabled.
109 tests, ruff and shellcheck clean.
|
||
|
flan
|
eb91a337cd |
Automate releases from tags
Releases were manual and had drifted: v0.2.0 and v0.2.1 were tagged but never
released, so the releases page jumped v0.1.0 -> v0.3.0 and hid the fix for the
incident that took every app down.
Pushing a v* tag now runs the full suite and cuts a GitHub release whose body is
the matching CHANGELOG.md section -- one source of truth for release notes, so
there is no second place for them to be wrong.
The workflow refuses to publish when:
- the tests, ruff, or bash -n fail (a tagged commit is what people install; it
must be at least as good as main)
- the tag does not match the VERSION= declared by every script
- CHANGELOG.md has no section for the tag, or the section is empty
That version check is not theoretical: VERSION= had drifted to three different
values across install.sh / uninstall.sh / recover.sh / apply.sh and nothing
noticed until this release. tests/test_release_notes.py now asserts the scripts
agree with each other and with the newest CHANGELOG entry, so the drift cannot
come back.
workflow_dispatch takes an existing tag, so releases can be backfilled for tags
that were pushed before this existed.
106 tests, ruff and shellcheck clean.
|
||
|
flan
|
51bf5326d9 |
Refuse to write a bundle whose parens we unbalanced
Commit |
||
|
flan
|
8a2028bfa7 |
Add test coverage for the Angular bundle patch
patch_ui.py rewrites minified third-party JavaScript by regex and had no tests. It is the easiest place in this project to do real damage: a pattern that matches nothing silently leaves the dropdown Storj-only, and one that consumes a paren too many is a syntax error in the bundle that blanks the entire TrueNAS web UI. Tests run the real patterns against verbatim snippets from a TrueNAS 25.x chunk-*.js (the chained property(...)(...) form) and a 24.x literal array, and assert: exactly one match, all three providers present, the paren balance is UNCHANGED, surrounding code untouched, re-patching is a no-op, unrelated JS is never matched, and every pattern stays anchored to filterByProviders. The paren-balance assertion is the load-bearing one -- a plausible-but-wrong pattern that eats both parens and re-emits none shifts the delta from 1 to 2 and is caught. Also restore the comment explaining why the 25.x pattern is shaped the way it is, and correct the module docstring, which showed the binding with a single closing paren; the real bundle wraps it in a chained property call. 90 tests, ruff and shellcheck clean. |
||
|
flan
|
8421a34d8d |
Delete the snapshot tree atomically instead of 252 calls
delete_snapshot_tree removed the parent and every child snapshot individually.
On a real pool `zfs snapshot -r` creates one snapshot per descendant dataset --
252 on Tap -- so cleanup was 252 sequential middleware calls.
Slow, but the real problem is that it is not atomic: a job killed part-way
through the sweep leaves behind exactly the orphaned snapshots this function
exists to prevent.
zfs.snapshot.delete accepts {"recursive": True}, which destroys the parent and
all children in one call. Use that as the fast path and keep the name-by-name
sweep as the fallback -- it is still needed when the parent is already gone
(stock's finally can win the race once our mounts are released), which makes a
recursive delete fail while the children survive.
The test fake now emulates real `zfs destroy -r` semantics, so a test cannot pass
while the shipped code deletes only the parent.
76 tests, ruff and shellcheck clean.
|
||
|
flan
|
150a241a0f |
Fix native-nested probe: guard message is split across string literals
The probe searched plugins/cloud/crud.py for the contiguous string
"no further nesting". Stock does not contain it. The message is split across
adjacent string literals:
verrors.add(f"{name}.snapshot", "This option is only available for datasets that have no further "
"nesting")
Python concatenates those at runtime, so the errmsg IS contiguous and CRUD_BLOCK's
runtime filter matches correctly -- but the SOURCE never contains the whole
phrase. The probe therefore found nothing, concluded iX had removed the guard, and
skipped the nested module as "already native" on every boot. apply.log would say
"TrueNAS now handles nesting natively" and the feature would never work.
It fails safe -- the stock guard stays in place, so no backup could be
misconfigured and no data was at risk -- but the module was 100% dead.
Verified against real middlewared on a live box: the probe returned native=yes
(wrong) before this change and native=no (correct) after.
The probe now strips whitespace and quote characters before matching, which is
robust to any wrapping or concatenation style. Added a regression test that
EXECUTES apply.sh's own probe code (extracted, not reimplemented -- a
reimplementation would pass while the shipped probe stayed broken) against the
real wrapped source, a single-line variant, and a three-way split.
75 tests, ruff and shellcheck clean.
|
||
|
flan
|
24f1f2c648 |
Fix post-merge audit findings; unify staging teardown
Audit ----- - create_task.py verify failed on a DEFAULT install. hook_status.json emitted a per-file entry for the nested module with ok:false whenever the feature was switched off -- the default -- so verify printed [FAIL] and exited 1, right after the README tells users to run it. Status is now per MODULE with an `active` flag, and verify renders an inactive module as [SKIP]. - A partial apply suppressed the middlewared restart. The exit code conflated "nothing applied" with "one module applied, one failed", so a failing providers patch would prevent the restart that a freshly-applied nested patch needs, leaving it on disk and never loaded. Exit 2 now means partial and the restart still fires. - The native-nested probe could never fire. It scanned crud.py for the guard message, but our own injected block quotes that message, so once applied the probe would always conclude the guard was still present. It now reads only the stock portion of the file. - recover.sh did not unmount staging trees, so an emergency recovery left bind mounts pinning ZFS snapshots that could then never be destroyed. - uninstall.sh deleted sidecar files without reading them. A sidecar is the only record that an interrupted run's snapshot tree is still on disk; both scripts now name the snapshot before clearing it. - Removed a dead branch in the restart gate (unreachable: the kill switch exits). Refactor -------- - Staging teardown had been copy-pasted into uninstall.sh and recover.sh -- two untested shell copies of the fiddly depth-ordering and lazy-umount logic. Both now call `python3 patch/truecloud_nested.py cleanup`, so there is exactly one implementation and it is the one under test. - Dropped the in-memory ACTIVE dict. The sidecar file was already the source of truth; a second in-process record could only desync -- and it is precisely the middlewared-restart case (which empties it) that must not orphan a snapshot tree. One record, on disk, or none. Not done: the overlay-unmount loop is duplicated across apply.sh/uninstall.sh/ recover.sh. It is pre-existing, and apply.sh runs at PREINIT under a tight timeout -- giving it a source dependency would trade 10 lines of duplication for a boot-time failure mode. 74 tests, ruff and shellcheck clean. |
||
|
flan
|
eca6eb3f3b |
Retire the two modules independently; calm the README down
The native-support check looked only for native B2 restic support and, on
finding it, set the kill switch and disabled the whole patch. That was fine when
providers were the only thing here. With nested-dataset snapshots in the patch it
is wrong: TrueNAS is likely to ship one capability long before the other, and a
single all-or-nothing kill switch would silently take a still-needed module down
with the superseded one.
apply.sh now treats the patch as two independent modules:
providers b2.py + restic.py + the UI credential dropdown
native when B2RcloneRemote carries a real get_restic_config()
nested plugins/cloud/{snapshot,crud}.py + cloud_backup/sync.py (opt-in)
native when the "no further nesting" validation is gone from
plugins/cloud/crud.py
Each is detected and skipped on its own. The kill switch fires only once BOTH are
done (native, or nested was never enabled). The UI patch belongs to providers and
is skipped with it. The deferred middlewared restart now fires when ANY
still-needed module landed -- keying it off providers alone would have left a
freshly-patched nested module on disk and never loaded on a native-B2 box.
hook_status.json reports each module with an active flag and a reason.
README: dropped the warning boxes and the disclaimer's fear-bulleting in favour
of plain statements, documented the two-module design and the per-module
auto-disable, and added a Development section disclosing AI assistance. The one
caveat kept, as a plain sentence rather than a banner: the mount --bind staging
step has not yet been exercised by a live backup run.
67 tests, ruff and shellcheck clean.
|
||
|
flan
|
a80de88078 |
Distinguish a missing snapshot from an unreadable one
os.path.isdir() returns False both when a snapshot directory does not exist and when it cannot be stat'd. Staging aborted either way -- correct -- but reported every case as "has no snapshot", which sends you hunting for a snapshot that is sitting right there. Found while dry-running the planner against a real recursive snapshot of Tap: running as a non-root user, /mnt/Tap/apps/paperless/data is mode 0700 and the probe reported "has no snapshot" when `zfs list` showed the snapshot present. Middleware runs as root so this would not fire in production, but a backup system must not misreport why it failed. plan_staging now takes a probe() that classifies the path as ok / missing / unreadable, and the error names which. Dry-run results against Tap (250 datasets, real `zfs snapshot -r`): - 170 mounted filesystems under /mnt/Tap/, and the plan produces exactly 170 descendant mounts -- no omissions - 18 legacy-mountpoint datasets reported as skipped, never dropped silently - Tap/ix-apps mounts at /mnt/.ix-apps, correctly outside the backup path - parent snapshot exposes 0 entries under /apps; the staged sources expose 71, and lidarr/config resolves with lidarr.db present - snapshot_tree_names() identifies all 250; deleting only the parent (what stock does) leaves 249 orphans, and the sweep clears them |
||
|
flan
|
bb26edf351 |
Make nested snapshots opt-in; fix snapshot leaks found in audit
Opt-in ------ Nested-dataset snapshot support changes how backups read their source data, so it is now off by default and gated behind a marker file: install.sh --enable-nested-snapshots install.sh --disable-nested-snapshots With neither flag install.sh preserves the current setting, so a routine `git pull && bash install.sh` can never silently flip it. When disabled, apply.sh skips the patch entirely and the stock guard remains. uninstall.sh tears down staging mounts and removes the marker. Snapshot lifecycle ------------------ zfs.snapshot.delete defaults to recursive=False and stock restic_backup() calls it with no options. Stock is safe only because its validation means recursive is never True in the field. Enabling nested datasets makes recursive snapshots real: the parent then has one child snapshot per descendant dataset (160+ on an Apps pool), so stock's delete would orphan every child on EVERY successful run. The patch now owns the lifecycle end to end: - delete_snapshot_tree() sweeps the parent and all children, and is idempotent against stock's finally winning the race once our mounts are released - on a staging failure the tree is deleted here, because sync.py never completes `snapshot, local_path = await create_snapshot(...)` and so its finally deletes nothing at all - the snapshot is recorded in a sidecar file before anything is mounted, so a middlewared restart mid-backup cannot orphan it - a crashed run's snapshot tree is reclaimed on the next run instead of being overwritten and leaked Silent-omission fix ------------------- The dataset list is now enumerated AFTER the snapshot. Read beforehand it could miss a dataset created in the gap, which the recursive snapshot would capture but the staging plan would not -- silently omitting its data. Read afterwards, an unsnapshotted dataset trips the staging check and fails the run loudly. Also from the audit ------------------- - plan_staging scopes by dataset name, so skipped-dataset warnings no longer include every mountpoint-less dataset on the box, which buried the ones that matter - staging_root_for rejects "." / ".." components that would escape the staging base, and resolves STAGING_BASE at call time rather than freezing it into a default argument - uninstall.sh no longer `rm -rf`s a tree that may still contain live bind mounts, and unmounts by path depth rather than string length - apply_plan takes an injectable isdir; verify_staged drops an unused parameter - pin the shellcheck action instead of tracking @master 61 tests, ruff and shellcheck clean. |
||
|
flan
|
a572eb2164 |
Support ZFS snapshots on datasets with child datasets
TrueCloud Backup's "Take Snapshot" option is rejected on any path containing child datasets: This option is only available for datasets that have no further nesting That excludes every pool running Apps, where each app is its own dataset and often has config/pgdata children. Without the option the backup reads live files, so databases are captured mid-write and an app that continuously rewrites its files can stall a run as restic chases a moving target. The stock guard is correct and must not simply be removed. create_snapshot() already takes a recursive ZFS snapshot, but points the backup tool at the parent dataset's .zfs/snapshot/, and ZFS does not expose child datasets there: /mnt/Tap/.zfs/snapshot/<snap>/apps/ -> 0 entries /mnt/Tap/apps/lidarr/config/.zfs/snapshot/<snap>/ -> the real data Deleting the check would make restic walk a near-empty tree, report success, and upload almost nothing. Implement the missing traversal instead. After the recursive snapshot is taken, each descendant dataset's own .zfs/snapshot/<snap> is bind-mounted into a staging tree mirroring the original layout, and the backup tool is pointed at the staging root. The guard is relaxed only after that machinery is in place. Safety properties: - staging failure aborts the backup; a partial tree is never handed to restic - a post-mount pass asserts every target is a mountpoint and the root is non-empty, so this cannot regress into the empty backup it exists to prevent - apply.sh patches crud.py last, so a partial failure leaves the guard intact rather than exposing "guard removed, traversal missing" - every injected block no-ops when _truecloud_nested is absent - unmountable/locked datasets are skipped and reported, never dropped silently - scoped to cloud_backup; cloudsync has no teardown wired in, so its guard stays The staging root is stable per task, so restic can find its parent snapshot between runs; stock's timestamped .zfs path changes every run and forces a full re-scan. Add CI (shellcheck, bash -n, ruff, pytest on 3.11-3.13), including tests that compile the *_BLOCK strings, which are Python source appended to live middlewared modules and were previously unchecked. Also: sync stale version strings, untrack a committed .pyc, gitignore __pycache__. |