Commit Graph
11 Commits
Author SHA1 Message Date
flan 9481bf55f0 Fix three findings from final clean-pass audit
- apply.sh: gate sitecustomize.py install on backup success; a failed
  backup cp (disk full, read-only mount) previously fell through and
  overwrote the vendor file with no recovery path
- create_task.py: handle unexpected 2xx response schema in cmd_create;
  bare KeyError on result['id'] is replaced with a diagnostic print
- uninstall.sh: mv inside while loop had no error handling; under
  set -euo pipefail a failed mv aborted the script before rm -rf PATCH_DIR,
  leaving the system in partial-uninstall limbo
2026-06-15 16:10:09 +00:00
flan 2a3b15b4b2 Fix two findings from full codebase audit
- apply.sh: correct comment from 'one prior generation' to 'two prior
  generations (.1 and .2)' — rotation has always kept three log files
- uninstall.sh: add sync comment on find paths to match WEBUI_CANDIDATES
  in patch/patch_ui.py, preventing silent drift if a new path is added
2026-06-15 15:58:29 +00:00
flan cbd85af2a3 Fix six quality findings from iterative code review
- patch_ui.py: find_bundle now matches MARKER so already-patched files
  return early and print 'UI already patched' instead of the misleading
  'filterByProviders pattern not found' warning
- install.sh: scope warning grep to current run only (record log offset
  before apply.sh, tail -c +N to read only new bytes)
- install.sh: fix misleading 'before continuing' wording on warning banner
- install.sh: fix grep anchor (^WARNING: missed [truecloud-patch] WARNING: lines)
- uninstall.sh: add import-middlewared verification after Python detection,
  matching apply.sh fallback logic
2026-06-15 15:53:06 +00:00
flan cc43b2831d Fix five quality findings from second re-review
sitecustomize.py: when find_spec resolves real_spec as None (module absent
after a TrueNAS update), record a FAIL status and mark the module done so
hook_status.json is still written and cmd_verify shows a diagnostic FAIL
instead of the ambiguous "no status file found".

sitecustomize.py: the AttributeError fallback in the URL-fix wrapper now
writes a WARNING to stderr before returning the unmodified result, making
the unexpected ResticConfig type visible in journalctl.

apply.sh: after falling back to bare python3, verify that python3 can also
import middlewared; if not, emit a second warning so the operator knows the
backend patch may be installed in the wrong site-packages directory.

patch_ui.py: abort (return without writing) when FIND.subn produces a count
other than 1, instead of committing a doubly-patched bundle and having
subsequent runs silently accept it via the MARKER check.

uninstall.sh: when a vendor sitecustomize.py backup exists, use mv to
atomically overwrite our file rather than rm-then-mv; eliminates the window
where a read-only /usr causes rm to fail under set -e, aborting before the
backup is restored.
2026-06-15 04:07:16 +00:00
flan 8829fbb9d4 Fix four quality findings from re-review
sitecustomize.py: exec_module now records a FAIL status when the underlying
module load raises, instead of leaving hook_status.json unwritten. cmd_verify
will now show a diagnostic FAIL rather than the misleading "no status file".

sitecustomize.py: URL-fix loop covers --repository and --repository= in
addition to --repo/--repo=/-r; these are documented restic synonyms.

sitecustomize.py: _replace fallback now catches AttributeError in addition
to TypeError so an unrecognised ResticConfig return type silently falls back
to returning the unmodified result rather than crashing the backup job.

patch_ui.py: warns when FIND.subn produces a count other than 1, making
unexpected multi-replacement visible in the apply log.

uninstall.sh: find for JS bundle restore now includes /usr/share/truenas-ui,
matching all three entries in patch_ui.py's WEBUI_CANDIDATES.
2026-06-15 03:58:06 +00:00
flan 91c607a707 Improve log rotation, early-exit find_bundle, fix uninstall orphaned backup
apply.sh: preserve two log generations (.1 and .2) on rotation so the
last two boots are always available for diagnosis.

patch_ui.py: find_bundle returns on the first matching JS file instead of
collecting all matches. The multi-match warning was dead weight — the Angular
Ivy compiler produces exactly one bundle and the WARNING path was unreachable
in practice.

uninstall.sh: restore an orphaned sitecustomize.py.pre-truecloud-patch when
sitecustomize.py itself has already been removed (e.g. manual deletion while
the backup survived). Prevents leaving ghost vendor files in site-packages.
2026-06-15 03:43:03 +00:00
flan 2c9adc1ce5 Fix five quality findings; expand recovery and restore documentation
patch_ui.py:
- Write Angular bundle atomically via tmp + os.replace, matching the pattern
  already used by _record_status. Prevents a corrupt bundle if the write is
  interrupted mid-boot.

sitecustomize.py:
- Tighten _record_status count barrier comment to name _Finder._targets as
  the canonical count, making the coupling visible to future editors.

uninstall.sh:
- Verify middlewared restarted cleanly after uninstall, with journalctl
  guidance on failure — matching recover.sh's existing pattern.

apply.sh:
- Change second line of site-packages error block from WARNING: prefix
  (misleading for an instructional message) to a plain Run: hint.

create_task.py:
- Guard __doc__ against None in epilog extraction so -OO does not crash.

README.md:
- Split Emergency recovery into three named subsections: middlewared won't
  start, web UI is blank or broken (corrupt bundle recovery), and backend
  verify shows FAIL. Each gives direct commands and escalation steps.
- Add Restoring from a TrueCloud Backup section: finding the restic binary,
  gathering credentials, provider-specific env var setup for B2 and S3,
  listing and restoring snapshots, and operational notes on restore hygiene.
- Clarify that hook_status.json is written once both target modules have
  loaded (not necessarily at the instant middlewared starts).
2026-06-15 03:23:31 +00:00
flan bee405bd52 Fix six quality findings from code review
sitecustomize.py:
- Restore _record_status count barrier: write the status file only after all
  patches have reported. middlewared.plugins.cloud_backup.restic is imported
  lazily (only when a backup task runs), so without this barrier verify would
  declare "all patches active" based solely on the B2 patch that fires at
  startup. Barrier now gates on _Finder._targets rather than the removed
  _PATCHES dict.
- Add comment in exec_module noting the if/elif must stay in sync with
  _Finder._targets, making the coupling visible.

patch_ui.py:
- Merge find_webui into find_bundle: previously find_bundle(None) would crash
  with os.walk(None) if main()'s guard were removed. Merged function returns
  a 3-tuple (webui_dir, path, content); webui_dir=None means no candidate
  directory found, path=None means directory found but pattern absent.
  main() still produces distinct messages for each failure mode.

create_task.py:
- Split triple-chained .get() in cmd_list_tasks into two lines; the or {}
  handling for None credentials was buried inside a one-liner.

uninstall.sh:
- Fix find loop: replace "for x in $(find ...)" with "while IFS= read -r"
  to handle paths containing spaces or newlines.
- Add #!/usr/bin/env shebang form to Python detection, matching apply.sh.
  Without this, uninstall on a system where middlewared uses the env form
  would silently leave sitecustomize.py in the wrong site-packages.
2026-06-15 03:15:08 +00:00
flan 78f943a71b Fix kill switch not cleared on reinstall; guard midclt delete; tighten sitecustomize imports
install.sh: running install.sh after a recover.sh left /data/truecloud-patch/disabled
in place, so apply.sh silently skipped all patching and middlewared restarted
without the patch. Clear the kill switch file before running apply.sh.

uninstall.sh: midclt initshutdownscript.delete was unguarded under set -euo pipefail,
so a delete failure (already-removed entry, transient API error) aborted the script
before sitecustomize.py was cleaned up or /data/truecloud-patch/ was removed. Now
guarded with an if/else that warns and continues.

sitecustomize.py: move importlib.machinery/.util imports inside the if block in
find_spec so they only execute when intercepting our two target modules, not on
every module import across the whole process. In _install(), import os before
importlib.util so the kill switch check (cheap) runs before the importlib import
(slightly heavier on first use). Remove the unused orig variable in _patch_restic.

patch_ui.py: add explicit encoding="utf-8" to both open() calls. errors="replace"
on read so malformed bytes in a bundle don't silently skip a candidate file.
2026-06-15 02:39:34 +00:00
flan 0a54bcba9d Harden for Python 3.12+, add disclaimer, audit for robustness
- sitecustomize.py: replace deprecated find_module/load_module with
  find_spec/exec_module (required for Python 3.12+ / TrueNAS SCALE 25.x)
- apply.sh: remove set -e (PREINIT must not fail catastrophically);
  detect middlewared's actual Python binary instead of assuming python3;
  log rotation to avoid unbounded growth; independent failure per step
- patch_ui.py: detect multiple bundle matches; include TrueNAS version
  in pattern-not-found warning; better MARKER specificity
- uninstall.sh: mirror Python detection logic from apply.sh
- README: lead with Storj $5→$50 price context; prominent unsupported
  disclaimer; Python version compatibility matrix; post-update checklist
- Add MIT LICENSE
2026-06-15 02:13:24 +00:00
flan 21b9333324 Initial implementation: extend TrueCloud Backup to S3 and B2 providers
Patches middlewared at runtime via sitecustomize.py (no file edits to /usr/)
and widens the UI credential dropdown from Storj-only to S3+B2+Storj.
Persists across TrueNAS updates via PREINIT initshutdownscript stored in DB.
2026-06-15 02:02:17 +00:00