Files
truenas-truecloud-patch/recover.sh
T
flan 092bdeae29 v0.4.1: fix three real bugs in update.sh found by auditing it
Release-candidate tags would have been installed as stable
-----------------------------------------------------------
git's version sort ranks v0.5.0-rc1 ABOVE v0.5.0 (verified empirically), and the
release workflow deliberately supports rc/beta/alpha tags. update.sh would have
offered an RC as "the newest release". Tag selection is now filtered to plain
vX.Y.Z.

update.sh would have died mid-update on an untracked file
----------------------------------------------------------
The dirty-tree guard uses --untracked-files=no, so an untracked file that the
TARGET tracks slips past it -- and `git checkout` then aborts. Under set -e the
script died with a raw git error, after already recording the rollback point.

Not hypothetical: a hand-copied patch/wait_restart.sh blocked a pull on a real box
in exactly this way. It is now detected up front, by name. Gitignored files are
correctly not treated as blockers, since git overwrites those silently.

Special case: if update.sh ITSELF is the blocker, it was hand-copied in to
bootstrap -- and "delete update.sh, then re-run update.sh" is impossible. It now
says so and prints the git commands that bootstrap it properly.

--rollback skipped that check entirely and would have hit the identical failure.
The check is now a shared function used by both paths, and rollback also validates
that the recorded revision still exists.

Also: install.sh's chmod aborted under set -e if a listed file was missing (the
file set changes between versions, so --rollback must not be killed by a name this
version happens to know about), and --to with no value was silently ignored.

Verified end to end in a throwaway clone: forward v0.4.1 -> v0.4.2 and rollback
back, with files appearing and disappearing correctly; both guards fire.

132 tests, ruff and shellcheck -S style clean.
2026-07-13 16:30:50 +00:00

86 lines
3.0 KiB
Bash
Executable File

#!/bin/bash
# recover.sh — emergency recovery if middlewared won't start after installing truecloud-patch.
#
# Run this from the TrueNAS shell (local console, SSH, or debug shell):
#
# bash /mnt/tank/truenas-truecloud-patch/recover.sh
#
# What it does:
# 1. Creates a "disabled" file in the repo root — apply.sh checks for this
# file at boot and skips all patching, so the next boot is always clean.
# 2. Unmounts any active truecloud overlays so the original /usr files are
# visible immediately (no reboot required).
# 3. Restarts middlewared against the unpatched files.
#
# To re-enable the patch after investigating:
# rm /mnt/tank/truenas-truecloud-patch/disabled
# bash /mnt/tank/truenas-truecloud-patch/patch/apply.sh
# systemctl restart middlewared
VERSION="0.4.1"
PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
echo "=== TrueNAS TrueCloud Provider Patch v${VERSION} — Recover ==="
echo ""
if [ "$(id -u)" -ne 0 ]; then
echo "ERROR: must be run as root." >&2
exit 1
fi
if [ ! -d "$PATCH_DIR" ]; then
echo "ERROR: $PATCH_DIR not found — truecloud-patch may not be installed." >&2
exit 1
fi
touch "$PATCH_DIR/disabled"
echo "Kill switch set: $PATCH_DIR/disabled created."
echo "Unmounting truecloud overlays ..."
_any=0
for _tag in mw ui; do
if mount | grep -qF "truecloud-${_tag} on "; then
_mnt=$(mount | grep "truecloud-${_tag} on " | awk '{print $3}' | head -1)
if umount "$_mnt" 2>/dev/null; then
echo " Unmounted: $_mnt"
_any=1
else
echo " WARNING: Could not unmount $_mnt — a reboot will restore original files."
fi
fi
done
[ "$_any" -eq 0 ] && echo " No overlays active."
# Nested-snapshot staging trees are bind mounts that PIN their ZFS snapshots, so
# leaving them mounted blocks those snapshots from ever being destroyed. The
# overlays above are volatile, but these are not self-healing without a reboot,
# and recover.sh is expected to work without one.
echo "Unmounting nested-snapshot staging trees ..."
# Best-effort: never block recovery. Same tested implementation as uninstall.sh.
python3 "$PATCH_DIR/patch/truecloud_nested.py" cleanup || true
# Cancel a deferred boot restart if one is still queued — we restart ourselves.
systemctl stop truecloud-mw-restart.service 2>/dev/null
systemctl reset-failed truecloud-mw-restart.service 2>/dev/null
echo "Restarting middlewared ..."
if systemctl restart middlewared; then
echo ""
echo "middlewared started successfully."
echo "Your system is back to normal (Storj-only TrueCloud Backup)."
else
echo ""
echo "WARNING: middlewared did not start cleanly even with the patch disabled."
echo "The problem is unrelated to truecloud-patch."
echo "Check the system log for details:"
echo " journalctl -u middlewared -n 50"
exit 1
fi
echo ""
echo "To re-enable the patch once you have investigated:"
echo " rm $PATCH_DIR/disabled"
echo " bash $PATCH_DIR/patch/apply.sh"
echo " systemctl restart middlewared"