sitecustomize.py — _install():
- Scope hook to middlewared service process only via sys.argv[0] check;
previously any tool in the same venv (midclt, debug scripts) would also
get its imports of the two target modules intercepted.
- Exec-chain a displaced sitecustomize.py: if apply.sh backed up a
pre-existing sitecustomize.py to .pre-truecloud-patch, run it in a
sandboxed namespace before installing our hook so any startup code
(path additions, codec registrations) still takes effect.
sitecustomize.py — _b2_restic_config():
- Validate expected credential fields ("account", "key") before accessing
them; raise a named KeyError listing what is missing and what is present
so a schema change produces an attributable error at backup time rather
than a bare KeyError with no indication this patch is involved.
sitecustomize.py — get_restic_config wrapper:
- Extend URL fix to cover all three flag forms restic accepts:
-r <url> (existing)
--repo <url> (long two-element form, now checked)
--repo=<url> (long single-element form, now handled)
Without this, a restic CLI change from -r to --repo would silently make
the fix a no-op while verify still reported the patch as OK.
patch_ui.py — find_bundle():
- Remove UnicodeDecodeError from except clause; errors="replace" in the
open() call means the exception can never be raised, and its presence
suggested the error parameter was not understood.
patch_ui.py:
- Write Angular bundle atomically via tmp + os.replace, matching the pattern
already used by _record_status. Prevents a corrupt bundle if the write is
interrupted mid-boot.
sitecustomize.py:
- Tighten _record_status count barrier comment to name _Finder._targets as
the canonical count, making the coupling visible to future editors.
uninstall.sh:
- Verify middlewared restarted cleanly after uninstall, with journalctl
guidance on failure — matching recover.sh's existing pattern.
apply.sh:
- Change second line of site-packages error block from WARNING: prefix
(misleading for an instructional message) to a plain Run: hint.
create_task.py:
- Guard __doc__ against None in epilog extraction so -OO does not crash.
README.md:
- Split Emergency recovery into three named subsections: middlewared won't
start, web UI is blank or broken (corrupt bundle recovery), and backend
verify shows FAIL. Each gives direct commands and escalation steps.
- Add Restoring from a TrueCloud Backup section: finding the restic binary,
gathering credentials, provider-specific env var setup for B2 and S3,
listing and restoring snapshots, and operational notes on restore hygiene.
- Clarify that hook_status.json is written once both target modules have
loaded (not necessarily at the instant middlewared starts).
sitecustomize.py:
- Restore _record_status count barrier: write the status file only after all
patches have reported. middlewared.plugins.cloud_backup.restic is imported
lazily (only when a backup task runs), so without this barrier verify would
declare "all patches active" based solely on the B2 patch that fires at
startup. Barrier now gates on _Finder._targets rather than the removed
_PATCHES dict.
- Add comment in exec_module noting the if/elif must stay in sync with
_Finder._targets, making the coupling visible.
patch_ui.py:
- Merge find_webui into find_bundle: previously find_bundle(None) would crash
with os.walk(None) if main()'s guard were removed. Merged function returns
a 3-tuple (webui_dir, path, content); webui_dir=None means no candidate
directory found, path=None means directory found but pattern absent.
main() still produces distinct messages for each failure mode.
create_task.py:
- Split triple-chained .get() in cmd_list_tasks into two lines; the or {}
handling for None credentials was buried inside a one-liner.
uninstall.sh:
- Fix find loop: replace "for x in $(find ...)" with "while IFS= read -r"
to handle paths containing spaces or newlines.
- Add #!/usr/bin/env shebang form to Python detection, matching apply.sh.
Without this, uninstall on a system where middlewared uses the env form
would silently leave sitecustomize.py in the wrong site-packages.
sitecustomize.py:
- Replace _PATCHES dispatch dict with if/elif in exec_module; removes
coupling between dispatch and _record_status's count barrier
- Drop _record_status count barrier entirely; both patches fire within
milliseconds during the same import sequence, write-on-every-call is safe
- Replace _broken_url regex with str.partition + startswith checks; same
semantics, no regex knowledge required to read
- Replace @staticmethod decorator inside plain function with explicit
staticmethod() assignment; decorator form creates a descriptor object,
not a callable, which confuses readers expecting class-body usage
apply.sh:
- Inline warn/ok helpers; each was one echo with a prefix, the indirection
cost more than the abstraction saved
- Collapse patch_ui.py if/else (whose if branch was a no-op comment) to
a single || fallback line
create_task.py:
- Remove vestigial (_client, _args) params from cmd_verify; it was pulled
out of dispatch, the params were never used
- Replace 3-entry dispatch dict with if/elif; dict implied a uniform calling
convention that verify already broke
create_task.py:
- Remove dead make_client() call before the verify branch; it was called
unconditionally with host=None/key=None, creating a broken client that was
immediately discarded or overwritten.
- Remove "verify" from the dispatch dict; it was never reached through dispatch
(the if/cmd==verify branch above it handled it). Dispatch now only contains
commands that actually use a client.
- Wrap json.load() in try/except (OSError, JSONDecodeError) so a corrupt or
partially-written status file produces a useful message instead of a traceback.
sitecustomize.py:
- Call _record_status() on the early-return paths in both _patch_b2 and
_patch_restic. Without this, if TrueNAS natively supports B2 or the patch
is already applied, the status file was never written and `verify` always
reported failure even when everything was fine.
- Add idempotency guard to _record_status(): first call wins; duplicate calls
for the same module are ignored so the entry count stays accurate.
- Make B2 get_restic_config a @staticmethod. The method never used self; the
noqa comment was suppressing the evidence of a design mismatch. Removing the
unused parameter makes the intent explicit.
- Add NamedTuple._replace() fallback after dataclasses.replace() in the restic
wrapper. If ResticConfig is ever refactored to a NamedTuple, the TypeError
from dataclasses.replace() would have surfaced as a backup job failure rather
than a graceful recovery.
sitecustomize.py: _patch_restic no longer reimplements get_restic_config.
It now wraps the original: calls _orig(cloud_backup) to get a ResticConfig,
then post-processes only the -r argument to fix "b2:/bucket" → "b2:bucket"
when the URL contains a stray leading slash (the stock bug for empty-hostname
providers). Uses dataclasses.replace() to build the corrected result so new
ResticConfig fields added in future TrueNAS versions pass through unchanged.
This eliminates the transfer_setting gap, env dict mutation, and frozen-copy
drift that would occur over time.
Also adds a status file mechanism: sitecustomize.py writes
/data/truecloud-patch/hook_status.json atomically after both patches have
reported success or failure. This gives a machine-readable signal that the
hook fired correctly — without requiring log scraping.
create_task.py: new "verify" subcommand reads the status file and prints a
human-readable summary. Does not require --host or --api-key. --host and
--api-key are now optional at the parser level and validated only for
subcommands that actually need an API connection.
README: update troubleshooting to use "create_task.py verify" instead of
the manual Python introspection one-liner.
install.sh: running install.sh after a recover.sh left /data/truecloud-patch/disabled
in place, so apply.sh silently skipped all patching and middlewared restarted
without the patch. Clear the kill switch file before running apply.sh.
uninstall.sh: midclt initshutdownscript.delete was unguarded under set -euo pipefail,
so a delete failure (already-removed entry, transient API error) aborted the script
before sitecustomize.py was cleaned up or /data/truecloud-patch/ was removed. Now
guarded with an if/else that warns and continues.
sitecustomize.py: move importlib.machinery/.util imports inside the if block in
find_spec so they only execute when intercepting our two target modules, not on
every module import across the whole process. In _install(), import os before
importlib.util so the kill switch check (cheap) runs before the importlib import
(slightly heavier on first use). Remove the unused orig variable in _patch_restic.
patch_ui.py: add explicit encoding="utf-8" to both open() calls. errors="replace"
on read so malformed bytes in a bundle don't silently skip a candidate file.
If the patch ever prevents middlewared from starting, users now have a
clear escape hatch that requires no knowledge of Python internals:
bash /data/truecloud-patch/recover.sh
Or at a bare shell prompt:
touch /data/truecloud-patch/disabled
systemctl restart middlewared
sitecustomize.py checks for /data/truecloud-patch/disabled at Python
startup and skips the import hook entirely when the file exists.
apply.sh does the same so the PREINIT script also does nothing on reboot.
recover.sh is copied to /data/truecloud-patch/ by install.sh so it is
available even without the original repo directory.
README gains an "Emergency recovery" section above Troubleshooting.
sitecustomize.py: exec_module was calling importlib.util.find_spec(fullname)
after Python had already added the module to sys.modules. find_spec
short-circuits to sys.modules[name].__spec__, which is our own wrapper spec,
so real_spec.loader.exec_module(module) immediately called exec_module again
— infinite recursion. Fix: resolve the real file spec in find_spec (before
sys.modules is populated) and store it on the Loader. exec_module now uses
the stored spec directly with no find_spec call.
Also move _mark_done into the finally block so a module that fails to load
doesn't trigger infinite retry loops on subsequent import attempts.
apply.sh: warn() writes to stdout. Inside $(find_mw_python), stdout is
captured by the command substitution, so any warn() call folded its text
into $PYTHON — causing every subsequent "$PYTHON" invocation to fail. Fix:
emit the fallback warning to stderr (>&2) so it goes to the log without
being captured.
Patches middlewared at runtime via sitecustomize.py (no file edits to /usr/)
and widens the UI credential dropdown from Storj-only to S3+B2+Storj.
Persists across TrueNAS updates via PREINIT initshutdownscript stored in DB.
flan