Commit Graph
8 Commits
Author SHA1 Message Date
flan cc43b2831d Fix five quality findings from second re-review
sitecustomize.py: when find_spec resolves real_spec as None (module absent
after a TrueNAS update), record a FAIL status and mark the module done so
hook_status.json is still written and cmd_verify shows a diagnostic FAIL
instead of the ambiguous "no status file found".

sitecustomize.py: the AttributeError fallback in the URL-fix wrapper now
writes a WARNING to stderr before returning the unmodified result, making
the unexpected ResticConfig type visible in journalctl.

apply.sh: after falling back to bare python3, verify that python3 can also
import middlewared; if not, emit a second warning so the operator knows the
backend patch may be installed in the wrong site-packages directory.

patch_ui.py: abort (return without writing) when FIND.subn produces a count
other than 1, instead of committing a doubly-patched bundle and having
subsequent runs silently accept it via the MARKER check.

uninstall.sh: when a vendor sitecustomize.py backup exists, use mv to
atomically overwrite our file rather than rm-then-mv; eliminates the window
where a read-only /usr causes rm to fail under set -e, aborting before the
backup is restored.
2026-06-15 04:07:16 +00:00
flan 2c9adc1ce5 Fix five quality findings; expand recovery and restore documentation
patch_ui.py:
- Write Angular bundle atomically via tmp + os.replace, matching the pattern
  already used by _record_status. Prevents a corrupt bundle if the write is
  interrupted mid-boot.

sitecustomize.py:
- Tighten _record_status count barrier comment to name _Finder._targets as
  the canonical count, making the coupling visible to future editors.

uninstall.sh:
- Verify middlewared restarted cleanly after uninstall, with journalctl
  guidance on failure — matching recover.sh's existing pattern.

apply.sh:
- Change second line of site-packages error block from WARNING: prefix
  (misleading for an instructional message) to a plain Run: hint.

create_task.py:
- Guard __doc__ against None in epilog extraction so -OO does not crash.

README.md:
- Split Emergency recovery into three named subsections: middlewared won't
  start, web UI is blank or broken (corrupt bundle recovery), and backend
  verify shows FAIL. Each gives direct commands and escalation steps.
- Add Restoring from a TrueCloud Backup section: finding the restic binary,
  gathering credentials, provider-specific env var setup for B2 and S3,
  listing and restoring snapshots, and operational notes on restore hygiene.
- Clarify that hook_status.json is written once both target modules have
  loaded (not necessarily at the instant middlewared starts).
2026-06-15 03:23:31 +00:00
flan bee405bd52 Fix six quality findings from code review
sitecustomize.py:
- Restore _record_status count barrier: write the status file only after all
  patches have reported. middlewared.plugins.cloud_backup.restic is imported
  lazily (only when a backup task runs), so without this barrier verify would
  declare "all patches active" based solely on the B2 patch that fires at
  startup. Barrier now gates on _Finder._targets rather than the removed
  _PATCHES dict.
- Add comment in exec_module noting the if/elif must stay in sync with
  _Finder._targets, making the coupling visible.

patch_ui.py:
- Merge find_webui into find_bundle: previously find_bundle(None) would crash
  with os.walk(None) if main()'s guard were removed. Merged function returns
  a 3-tuple (webui_dir, path, content); webui_dir=None means no candidate
  directory found, path=None means directory found but pattern absent.
  main() still produces distinct messages for each failure mode.

create_task.py:
- Split triple-chained .get() in cmd_list_tasks into two lines; the or {}
  handling for None credentials was buried inside a one-liner.

uninstall.sh:
- Fix find loop: replace "for x in $(find ...)" with "while IFS= read -r"
  to handle paths containing spaces or newlines.
- Add #!/usr/bin/env shebang form to Python detection, matching apply.sh.
  Without this, uninstall on a system where middlewared uses the env form
  would silently leave sitecustomize.py in the wrong site-packages.
2026-06-15 03:15:08 +00:00
flan ebca3f99cc Reduce accidental complexity across three files
sitecustomize.py:
- Replace _PATCHES dispatch dict with if/elif in exec_module; removes
  coupling between dispatch and _record_status's count barrier
- Drop _record_status count barrier entirely; both patches fire within
  milliseconds during the same import sequence, write-on-every-call is safe
- Replace _broken_url regex with str.partition + startswith checks; same
  semantics, no regex knowledge required to read
- Replace @staticmethod decorator inside plain function with explicit
  staticmethod() assignment; decorator form creates a descriptor object,
  not a callable, which confuses readers expecting class-body usage

apply.sh:
- Inline warn/ok helpers; each was one echo with a prefix, the indirection
  cost more than the abstraction saved
- Collapse patch_ui.py if/else (whose if branch was a no-op comment) to
  a single || fallback line

create_task.py:
- Remove vestigial (_client, _args) params from cmd_verify; it was pulled
  out of dispatch, the params were never used
- Replace 3-entry dispatch dict with if/elif; dict implied a uniform calling
  convention that verify already broke
2026-06-15 03:08:52 +00:00
flan 4b07ce459a Fix six audit findings: dead code, missing status records, missing fallback, unsafe JSON read
create_task.py:
- Remove dead make_client() call before the verify branch; it was called
  unconditionally with host=None/key=None, creating a broken client that was
  immediately discarded or overwritten.
- Remove "verify" from the dispatch dict; it was never reached through dispatch
  (the if/cmd==verify branch above it handled it). Dispatch now only contains
  commands that actually use a client.
- Wrap json.load() in try/except (OSError, JSONDecodeError) so a corrupt or
  partially-written status file produces a useful message instead of a traceback.

sitecustomize.py:
- Call _record_status() on the early-return paths in both _patch_b2 and
  _patch_restic. Without this, if TrueNAS natively supports B2 or the patch
  is already applied, the status file was never written and `verify` always
  reported failure even when everything was fine.
- Add idempotency guard to _record_status(): first call wins; duplicate calls
  for the same module are ignored so the entry count stays accurate.
- Make B2 get_restic_config a @staticmethod. The method never used self; the
  noqa comment was suppressing the evidence of a design mismatch. Removing the
  unused parameter makes the intent explicit.
- Add NamedTuple._replace() fallback after dataclasses.replace() in the restic
  wrapper. If ResticConfig is ever refactored to a NamedTuple, the TypeError
  from dataclasses.replace() would have surfaced as a backup job failure rather
  than a graceful recovery.
2026-06-15 02:59:57 +00:00
flan 06de1e15c0 Wrap original get_restic_config instead of replacing it; add hook status file
sitecustomize.py: _patch_restic no longer reimplements get_restic_config.
It now wraps the original: calls _orig(cloud_backup) to get a ResticConfig,
then post-processes only the -r argument to fix "b2:/bucket" → "b2:bucket"
when the URL contains a stray leading slash (the stock bug for empty-hostname
providers). Uses dataclasses.replace() to build the corrected result so new
ResticConfig fields added in future TrueNAS versions pass through unchanged.
This eliminates the transfer_setting gap, env dict mutation, and frozen-copy
drift that would occur over time.

Also adds a status file mechanism: sitecustomize.py writes
/data/truecloud-patch/hook_status.json atomically after both patches have
reported success or failure. This gives a machine-readable signal that the
hook fired correctly — without requiring log scraping.

create_task.py: new "verify" subcommand reads the status file and prints a
human-readable summary. Does not require --host or --api-key. --host and
--api-key are now optional at the parser level and validated only for
subcommands that actually need an API connection.

README: update troubleshooting to use "create_task.py verify" instead of
the manual Python introspection one-liner.
2026-06-15 02:47:59 +00:00
flan 0a54bcba9d Harden for Python 3.12+, add disclaimer, audit for robustness
- sitecustomize.py: replace deprecated find_module/load_module with
  find_spec/exec_module (required for Python 3.12+ / TrueNAS SCALE 25.x)
- apply.sh: remove set -e (PREINIT must not fail catastrophically);
  detect middlewared's actual Python binary instead of assuming python3;
  log rotation to avoid unbounded growth; independent failure per step
- patch_ui.py: detect multiple bundle matches; include TrueNAS version
  in pattern-not-found warning; better MARKER specificity
- uninstall.sh: mirror Python detection logic from apply.sh
- README: lead with Storj $5→$50 price context; prominent unsupported
  disclaimer; Python version compatibility matrix; post-update checklist
- Add MIT LICENSE
2026-06-15 02:13:24 +00:00
flan 21b9333324 Initial implementation: extend TrueCloud Backup to S3 and B2 providers
Patches middlewared at runtime via sitecustomize.py (no file edits to /usr/)
and widens the UI credential dropdown from Storj-only to S3+B2+Storj.
Persists across TrueNAS updates via PREINIT initshutdownscript stored in DB.
2026-06-15 02:02:17 +00:00