middlewared's alert.load() imports every file in alert/source/ with NO try/except:
def load(self):
for module in load_modules(.../alert/source):
for cls in load_classes(module, AlertSource, (ThreadedAlertSource,)):
...
and it runs during setup. A module that raises on import therefore takes
middlewared's startup down with it -- exactly the class of failure this project
exists to avoid.
apply.sh now COMPILES the substituted alert source and refuses to write it if it
does not parse. An uninstalled alert is a missing convenience; a broken one is a
broken box.
@PATCH_DIR@ is also substituted with repr() rather than raw, so a repository path
containing a quote or backslash yields a valid Python literal instead of a syntax
error in the installed module.
The alert source no longer mutates sys.path. It loaded tools/release_notes.py via
sys.path.insert(0, ...), which shadows the stdlib for that interpreter -- and
ThreadedAlertSource runs in middlewared's thread pool, so mutating sys.path is a
race. It now loads by file path with importlib.
New tests guard every import-time failure mode: the module compiles, apply.sh
compiles before writing, awkward paths (quotes, backslashes) still produce valid
modules, nothing but imports/constants/classes runs at module scope, every
AlertClass name ends in "AlertClass" (AlertClassMeta raises NameError otherwise),
the alert text placeholders match the args passed, and no git command that writes
to .git is ever used.
152 tests, ruff and shellcheck -S style clean.
86 lines
3.0 KiB
Bash
Executable File
86 lines
3.0 KiB
Bash
Executable File
#!/bin/bash
|
|
# recover.sh — emergency recovery if middlewared won't start after installing truecloud-patch.
|
|
#
|
|
# Run this from the TrueNAS shell (local console, SSH, or debug shell):
|
|
#
|
|
# bash /mnt/tank/truenas-truecloud-patch/recover.sh
|
|
#
|
|
# What it does:
|
|
# 1. Creates a "disabled" file in the repo root — apply.sh checks for this
|
|
# file at boot and skips all patching, so the next boot is always clean.
|
|
# 2. Unmounts any active truecloud overlays so the original /usr files are
|
|
# visible immediately (no reboot required).
|
|
# 3. Restarts middlewared against the unpatched files.
|
|
#
|
|
# To re-enable the patch after investigating:
|
|
# rm /mnt/tank/truenas-truecloud-patch/disabled
|
|
# bash /mnt/tank/truenas-truecloud-patch/patch/apply.sh
|
|
# systemctl restart middlewared
|
|
|
|
VERSION="0.5.1"
|
|
|
|
PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
|
|
|
|
|
|
echo "=== TrueNAS TrueCloud Provider Patch v${VERSION} — Recover ==="
|
|
echo ""
|
|
|
|
if [ "$(id -u)" -ne 0 ]; then
|
|
echo "ERROR: must be run as root." >&2
|
|
exit 1
|
|
fi
|
|
|
|
if [ ! -d "$PATCH_DIR" ]; then
|
|
echo "ERROR: $PATCH_DIR not found — truecloud-patch may not be installed." >&2
|
|
exit 1
|
|
fi
|
|
|
|
touch "$PATCH_DIR/disabled"
|
|
echo "Kill switch set: $PATCH_DIR/disabled created."
|
|
|
|
echo "Unmounting truecloud overlays ..."
|
|
_any=0
|
|
for _tag in mw ui; do
|
|
if mount | grep -qF "truecloud-${_tag} on "; then
|
|
_mnt=$(mount | grep "truecloud-${_tag} on " | awk '{print $3}' | head -1)
|
|
if umount "$_mnt" 2>/dev/null; then
|
|
echo " Unmounted: $_mnt"
|
|
_any=1
|
|
else
|
|
echo " WARNING: Could not unmount $_mnt — a reboot will restore original files."
|
|
fi
|
|
fi
|
|
done
|
|
[ "$_any" -eq 0 ] && echo " No overlays active."
|
|
|
|
# Nested-snapshot staging trees are bind mounts that PIN their ZFS snapshots, so
|
|
# leaving them mounted blocks those snapshots from ever being destroyed. The
|
|
# overlays above are volatile, but these are not self-healing without a reboot,
|
|
# and recover.sh is expected to work without one.
|
|
echo "Unmounting nested-snapshot staging trees ..."
|
|
# Best-effort: never block recovery. Same tested implementation as uninstall.sh.
|
|
python3 "$PATCH_DIR/patch/truecloud_nested.py" cleanup || true
|
|
|
|
# Cancel a deferred boot restart if one is still queued — we restart ourselves.
|
|
systemctl stop truecloud-mw-restart.service 2>/dev/null
|
|
systemctl reset-failed truecloud-mw-restart.service 2>/dev/null
|
|
|
|
echo "Restarting middlewared ..."
|
|
if systemctl restart middlewared; then
|
|
echo ""
|
|
echo "middlewared started successfully."
|
|
echo "Your system is back to normal (Storj-only TrueCloud Backup)."
|
|
else
|
|
echo ""
|
|
echo "WARNING: middlewared did not start cleanly even with the patch disabled."
|
|
echo "The problem is unrelated to truecloud-patch."
|
|
echo "Check the system log for details:"
|
|
echo " journalctl -u middlewared -n 50"
|
|
exit 1
|
|
fi
|
|
echo ""
|
|
echo "To re-enable the patch once you have investigated:"
|
|
echo " rm $PATCH_DIR/disabled"
|
|
echo " bash $PATCH_DIR/patch/apply.sh"
|
|
echo " systemctl restart middlewared"
|