469a2e465162df55b170a3a1381e66b96f5cd68d
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 flan
|
f927773f81 |
docs: TrueNAS compatibility matrix and the two-stage release process
The matrix is regenerated daily by CI rather than typed once and forgotten — a support table that quietly goes stale is a false promise to someone deciding whether to trust this with their backups. |
||
 flan
|
4e0814028c |
v0.5.0: TrueNAS alert when an update is available
Raises a real alert in the TrueNAS UI bell -- not a log line nobody reads. On by default, checked once a day. install.sh --no-update-alerts turns it off. It does not nag --------------- A release whose CHANGELOG contains only a "### Docs" section changed no code and raises nothing. Anything else raises INFO; a "### Security" section raises WARNING. The CHANGELOG's own section headings are the signal, and a security fix anywhere in the range escalates the whole span -- a docs-only release sitting on top of a security fix still reports as security rather than hiding it. Why an AlertSource and not midclt ---------------------------------- TrueNAS cannot raise an alert from the CLI. midclt exposes only alert.dismiss, alert.list, alert.list_categories, alert.list_policies and alert.restore -- alert CREATION is internal to middlewared, and none of its ~60 one-shot classes is generic enough to reuse. Registering an AlertSource is the only way. It is also the least invasive thing this patch does. The providers and nested modules both APPEND CODE TO STOCK middleware files; the alert source ADDS ONE FILE and modifies none. It is the native mechanism -- the same one every built-in TrueNAS alert uses -- and TrueNAS polls it itself, so there is no cron job and no systemd timer. - Fail-safe: every error path returns None; it cannot take middlewared down. - Read-only: `git ls-remote` plus an HTTPS fetch of the CHANGELOG. It never writes to .git, so it cannot leave root-owned objects behind the way a `git fetch` from middlewared (running as root) would. - Removed by uninstall.sh (mw_patch.revert_all). - It only tells you; it never updates anything. Verified against the real repo and remote, with middlewared stubbed: on v0.4.1, only a README-only v0.4.2 available -> NO ALERT on v0.4.0, v0.4.1 fixed real bugs -> INFO on v0.3.2, v0.3.3 was the password fix -> SECURITY / WARNING 139 tests, ruff and shellcheck -S style clean. |
||
|
flan
|
bb26edf351 |
Make nested snapshots opt-in; fix snapshot leaks found in audit
Opt-in ------ Nested-dataset snapshot support changes how backups read their source data, so it is now off by default and gated behind a marker file: install.sh --enable-nested-snapshots install.sh --disable-nested-snapshots With neither flag install.sh preserves the current setting, so a routine `git pull && bash install.sh` can never silently flip it. When disabled, apply.sh skips the patch entirely and the stock guard remains. uninstall.sh tears down staging mounts and removes the marker. Snapshot lifecycle ------------------ zfs.snapshot.delete defaults to recursive=False and stock restic_backup() calls it with no options. Stock is safe only because its validation means recursive is never True in the field. Enabling nested datasets makes recursive snapshots real: the parent then has one child snapshot per descendant dataset (160+ on an Apps pool), so stock's delete would orphan every child on EVERY successful run. The patch now owns the lifecycle end to end: - delete_snapshot_tree() sweeps the parent and all children, and is idempotent against stock's finally winning the race once our mounts are released - on a staging failure the tree is deleted here, because sync.py never completes `snapshot, local_path = await create_snapshot(...)` and so its finally deletes nothing at all - the snapshot is recorded in a sidecar file before anything is mounted, so a middlewared restart mid-backup cannot orphan it - a crashed run's snapshot tree is reclaimed on the next run instead of being overwritten and leaked Silent-omission fix ------------------- The dataset list is now enumerated AFTER the snapshot. Read beforehand it could miss a dataset created in the gap, which the recursive snapshot would capture but the staging plan would not -- silently omitting its data. Read afterwards, an unsnapshotted dataset trips the staging check and fails the run loudly. Also from the audit ------------------- - plan_staging scopes by dataset name, so skipped-dataset warnings no longer include every mountpoint-less dataset on the box, which buried the ones that matter - staging_root_for rejects "." / ".." components that would escape the staging base, and resolves STAGING_BASE at call time rather than freezing it into a default argument - uninstall.sh no longer `rm -rf`s a tree that may still contain live bind mounts, and unmounts by path depth rather than string length - apply_plan takes an injectable isdir; verify_staged drops an unused parameter - pin the shellcheck action instead of tracking @master 61 tests, ruff and shellcheck clean. |
||
|
flan
|
a572eb2164 |
Support ZFS snapshots on datasets with child datasets
TrueCloud Backup's "Take Snapshot" option is rejected on any path containing child datasets: This option is only available for datasets that have no further nesting That excludes every pool running Apps, where each app is its own dataset and often has config/pgdata children. Without the option the backup reads live files, so databases are captured mid-write and an app that continuously rewrites its files can stall a run as restic chases a moving target. The stock guard is correct and must not simply be removed. create_snapshot() already takes a recursive ZFS snapshot, but points the backup tool at the parent dataset's .zfs/snapshot/, and ZFS does not expose child datasets there: /mnt/Tap/.zfs/snapshot/<snap>/apps/ -> 0 entries /mnt/Tap/apps/lidarr/config/.zfs/snapshot/<snap>/ -> the real data Deleting the check would make restic walk a near-empty tree, report success, and upload almost nothing. Implement the missing traversal instead. After the recursive snapshot is taken, each descendant dataset's own .zfs/snapshot/<snap> is bind-mounted into a staging tree mirroring the original layout, and the backup tool is pointed at the staging root. The guard is relaxed only after that machinery is in place. Safety properties: - staging failure aborts the backup; a partial tree is never handed to restic - a post-mount pass asserts every target is a mountpoint and the root is non-empty, so this cannot regress into the empty backup it exists to prevent - apply.sh patches crud.py last, so a partial failure leaves the guard intact rather than exposing "guard removed, traversal missing" - every injected block no-ops when _truecloud_nested is absent - unmountable/locked datasets are skipped and reported, never dropped silently - scoped to cloud_backup; cloudsync has no teardown wired in, so its guard stays The staging root is stable per task, so restic can find its parent snapshot between runs; stock's timestamped .zfs path changes every run and forces a full re-scan. Add CI (shellcheck, bash -n, ruff, pytest on 3.11-3.13), including tests that compile the *_BLOCK strings, which are Python source appended to live middlewared modules and were previously unchecked. Also: sync stale version strings, untrack a committed .pyc, gitignore __pycache__. |
||
|
flan
|
70e84038d6 |
Repo is the install location; all scripts self-locate
Users now clone to a persistent ZFS pool and the repo stays in place. No files are copied on install — the PREINIT hook points directly into the clone. Scripts derive PATCH_DIR from their own path at runtime. - install.sh: PATCH_DIR=$(dirname $0); register patch/apply.sh as PREINIT target; chmod only, no cp; update pipe-install error message - patch/apply.sh: PATCH_DIR=$(dirname $0)/..; substitute PATCH_DIR into sitecustomize.py via sed when writing to site-packages; reference patch_ui.py as patch/patch_ui.py - recover.sh, uninstall.sh: PATCH_DIR=$(dirname $0) - uninstall.sh: look for patch/apply.sh in PREINIT registry - patch/create_task.py: _PATCH_DIR derived from __file__; apply.log path in error message derived from _PATCH_DIR - patch/sitecustomize.py: /data/truecloud-patch remains as placeholder substituted by apply.sh on each install - .gitignore: exclude runtime files (apply.log, hook_status.json, disabled) - README: document clone-to-pool install; update all example paths |