flan/truenas-truecloud-patch
1
0
Fork 0
Code Issues 1 Pull Requests 1 Actions Packages Projects Releases 12 Wiki Activity

Repo is the install location; all scripts self-locate

Browse Source 
Users now clone to a persistent ZFS pool and the repo stays in place.
No files are copied on install — the PREINIT hook points directly into
the clone. Scripts derive PATCH_DIR from their own path at runtime.

- install.sh: PATCH_DIR=$(dirname $0); register patch/apply.sh as
  PREINIT target; chmod only, no cp; update pipe-install error message
- patch/apply.sh: PATCH_DIR=$(dirname $0)/..; substitute PATCH_DIR
  into sitecustomize.py via sed when writing to site-packages;
  reference patch_ui.py as patch/patch_ui.py
- recover.sh, uninstall.sh: PATCH_DIR=$(dirname $0)
- uninstall.sh: look for patch/apply.sh in PREINIT registry
- patch/create_task.py: _PATCH_DIR derived from __file__; apply.log
  path in error message derived from _PATCH_DIR
- patch/sitecustomize.py: /data/truecloud-patch remains as placeholder
  substituted by apply.sh on each install
- .gitignore: exclude runtime files (apply.log, hook_status.json, disabled)
- README: document clone-to-pool install; update all example paths
This commit is contained in:
flan
2026-06-15 17:07:11 +00:00
parent 4c79403491
commit 70e84038d6
7 changed files with 94 additions and 76 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+6
View File
@@ -0,0 +1,6 @@
# Runtime files written to the repo root by apply.sh and sitecustomize.py
/apply.log
/apply.log.1
/apply.log.2
/hook_status.json
/disabled
README.md
+40 -32
View File
@@ -64,7 +64,7 @@ on every update) and are therefore re-applied automatically on every boot.
Both changes are **fail-safe**: if a patch cannot be applied (e.g. TrueNAS
restructured the relevant code), middlewared starts normally with Storj-only
support and the reason is logged to `/data/truecloud-patch/apply.log`.
support and the reason is logged to `apply.log` in your repo root.
## Supported providers after patching
@@ -76,12 +76,12 @@ support and the reason is logged to `/data/truecloud-patch/apply.log`.
## How persistence works
TrueNAS SCALE updates replace `/usr/` entirely. The patch survives by storing
all scripts in `/data/truecloud-patch/` (a persistent ZFS dataset) and
registering a **PREINIT initshutdownscript** in the TrueNAS database. This
causes `apply.sh` to run on every boot before `middlewared` starts, placing
`sitecustomize.py` in the correct site-packages directory and re-patching the
UI bundle.
TrueNAS SCALE updates replace `/usr/` entirely. The patch survives by keeping
this repository on a **persistent ZFS pool** (your data pool, not `/tmp` or a
system path) and registering a **PREINIT initshutdownscript** in the TrueNAS
database. On every boot, `patch/apply.sh` runs from the repo before
`middlewared` starts, placing `sitecustomize.py` in the correct site-packages
directory and re-patching the UI bundle.
## Python version compatibility
@@ -95,14 +95,20 @@ UI bundle.
## Install
Run on your TrueNAS box as root, with the system fully booted:
Clone the repository to a **persistent ZFS pool** so it survives OS updates,
then run `install.sh` from there:
```bash
git clone https://github.com/sudolulo/truenas-truecloud-patch.git
cd truenas-truecloud-patch
# Replace /mnt/tank with your pool name
git clone https://github.com/sudolulo/truenas-truecloud-patch.git \
/mnt/tank/truenas-truecloud-patch
cd /mnt/tank/truenas-truecloud-patch
bash install.sh
```
The directory you clone into becomes the permanent install location. The PREINIT
boot hook points to it — **do not delete or move the repo after install.**
Refresh your browser. S3 and B2 credentials now appear in the
**Data Protection → TrueCloud Backup → Add** credential dropdown.
@@ -124,12 +130,14 @@ If the UI still shows only Storj after refreshing (e.g. the JS bundle pattern
changed in a new TrueNAS version), create tasks directly via the REST API:
```bash
# Replace /mnt/tank/truenas-truecloud-patch with your clone path
# List your cloud credentials to find the right ID
python3 /data/truecloud-patch/create_task.py \
python3 /mnt/tank/truenas-truecloud-patch/patch/create_task.py \
--host 192.168.1.1 --api-key <key> list-credentials
# Create a task with a B2 credential (id=3)
python3 /data/truecloud-patch/create_task.py \
python3 /mnt/tank/truenas-truecloud-patch/patch/create_task.py \
--host 192.168.1.1 --api-key <key> create \
--name "tank-to-b2" \
--path /mnt/tank/data \
@@ -147,12 +155,12 @@ Get an API key from **System → API Keys → Add**.
## Uninstall
```bash
bash /data/truecloud-patch/uninstall.sh
bash /mnt/tank/truenas-truecloud-patch/uninstall.sh
```
Removes the PREINIT hook, `sitecustomize.py`, and restores the original UI
bundle from backup. The backend changes vanish on the next `middlewared`
restart.
Replace the path with your clone location. Removes the PREINIT hook,
`sitecustomize.py`, and restores the original UI bundle from backup. The
backend changes vanish on the next `middlewared` restart.
---
@@ -252,7 +260,7 @@ restic -r "$REPO" ls latest
## After a TrueNAS update
1. Check the log: `cat /data/truecloud-patch/apply.log | tail -30`
1. Check the log: `cat /mnt/tank/truenas-truecloud-patch/apply.log | tail -30`
2. If you see "WARNING: … pattern not found", the UI patch needs updating.
[Open an issue](https://github.com/sudolulo/truenas-truecloud-patch/issues)
with your TrueNAS version number.
@@ -269,18 +277,18 @@ Run this from the TrueNAS shell (local console, SSH, or the debug shell in
the UI):
```bash
bash /data/truecloud-patch/recover.sh
bash /mnt/tank/truenas-truecloud-patch/recover.sh
```
This creates a kill-switch file (`/data/truecloud-patch/disabled`).
`sitecustomize.py` checks for it at Python startup; if present, the import
hook is skipped entirely and middlewared starts clean with Storj-only support.
Nothing else on your system is affected.
Replace the path with your clone location. This creates a kill-switch file
(`disabled`) in the repo root. `sitecustomize.py` checks for it at Python
startup; if present, the import hook is skipped entirely and middlewared starts
clean with Storj-only support. Nothing else on your system is affected.
If you cannot run a script and only have a bare shell prompt:
```bash
touch /data/truecloud-patch/disabled
touch /mnt/tank/truenas-truecloud-patch/disabled
systemctl restart middlewared
```
@@ -294,8 +302,8 @@ journalctl -u middlewared -n 50
To re-enable the patch once you have investigated:
```bash
rm /data/truecloud-patch/disabled
bash /data/truecloud-patch/apply.sh
rm /mnt/tank/truenas-truecloud-patch/disabled
bash /mnt/tank/truenas-truecloud-patch/patch/apply.sh
```
---
@@ -317,22 +325,22 @@ mv /usr/share/truenas/webui/main.XXXXXXXX.js.pre-truecloud-patch \
```
Refresh your browser. The UI will return to normal (Storj-only until the
patch re-runs at next reboot, or you run `bash /data/truecloud-patch/apply.sh`
manually).
patch re-runs at next reboot, or you run
`bash /mnt/tank/truenas-truecloud-patch/patch/apply.sh` manually).
---
### Backend verify shows FAIL
```bash
python3 /data/truecloud-patch/create_task.py verify
python3 /mnt/tank/truenas-truecloud-patch/patch/create_task.py verify
```
If one or more entries show `[FAIL]`:
1. **Check the apply log** for errors during the last boot:
```bash
cat /data/truecloud-patch/apply.log | tail -40
cat /mnt/tank/truenas-truecloud-patch/apply.log | tail -40
```
2. **Check middlewared's own log** for Python tracebacks:
```bash
@@ -351,14 +359,14 @@ If one or more entries show `[FAIL]`:
**Apply log** (check after each reboot or install):
```bash
cat /data/truecloud-patch/apply.log
cat /mnt/tank/truenas-truecloud-patch/apply.log
```
**Verify backend patch is loaded** (while middlewared is running):
```bash
python3 /data/truecloud-patch/create_task.py verify
python3 /mnt/tank/truenas-truecloud-patch/patch/create_task.py verify
```
This reads `/data/truecloud-patch/hook_status.json`, written by the import
This reads `hook_status.json` in your repo root, written by the import
hook once both target modules have been loaded by middlewared. If it reports
"No hook status file found" immediately after install, restart middlewared
and try again — the file is written when middlewared imports the relevant
install.sh
+26 -27
View File
@@ -2,24 +2,29 @@
# install.sh — run once on the TrueNAS box to set up truecloud-patch.
#
# Prerequisites: run as root on TrueNAS SCALE with middlewared running.
# Clone this repository to a persistent ZFS pool first:
#
# git clone https://github.com/sudolulo/truenas-truecloud-patch \
# /mnt/<pool>/truenas-truecloud-patch
# cd /mnt/<pool>/truenas-truecloud-patch && bash install.sh
#
# What this does:
# 1. Copies patch files to /data/truecloud-patch/ (survives OS updates).
# 2. Registers a PREINIT initshutdownscript in the TrueNAS database so
# apply.sh re-applies the patches on every boot before middlewared starts.
# 3. Applies the patches immediately (no reboot required).
# 4. Restarts middlewared so the backend change takes effect now.
# 1. Registers a PREINIT initshutdownscript so patch/apply.sh re-runs on
# every boot before middlewared starts.
# 2. Applies the patches immediately (no reboot required).
# 3. Restarts middlewared so the backend change takes effect now.
set -euo pipefail
PATCH_DIR="/data/truecloud-patch"
REPO_DIR="$(cd "$(dirname "$0")" && pwd)"
# The directory containing install.sh is the permanent install location.
PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
if [ ! -f "$REPO_DIR/patch/sitecustomize.py" ]; then
echo "ERROR: patch files not found at $REPO_DIR/patch/" >&2
echo "Run install.sh from the cloned repository, not via pipe:" >&2
echo " git clone https://github.com/sudolulo/truenas-truecloud-patch" >&2
echo " cd truenas-truecloud-patch && bash install.sh" >&2
if [ ! -f "$PATCH_DIR/patch/sitecustomize.py" ]; then
echo "ERROR: patch files not found at $PATCH_DIR/patch/" >&2
echo "Run install.sh from a clone of the repository on a persistent pool:" >&2
echo " git clone https://github.com/sudolulo/truenas-truecloud-patch \\" >&2
echo " /mnt/<pool>/truenas-truecloud-patch" >&2
echo " cd /mnt/<pool>/truenas-truecloud-patch && bash install.sh" >&2
exit 1
fi
@@ -43,17 +48,11 @@ if ! midclt call core.ping &>/dev/null; then
exit 1
fi
# ── Copy files ────────────────────────────────────────────────────────────────
# ── Set permissions ───────────────────────────────────────────────────────────
echo "Copying patch files to $PATCH_DIR ..."
mkdir -p "$PATCH_DIR"
cp "$REPO_DIR/patch/sitecustomize.py" "$PATCH_DIR/"
cp "$REPO_DIR/patch/patch_ui.py" "$PATCH_DIR/"
cp "$REPO_DIR/patch/apply.sh" "$PATCH_DIR/"
cp "$REPO_DIR/patch/create_task.py" "$PATCH_DIR/"
cp "$REPO_DIR/recover.sh" "$PATCH_DIR/"
cp "$REPO_DIR/uninstall.sh" "$PATCH_DIR/"
chmod +x "$PATCH_DIR/apply.sh" "$PATCH_DIR/create_task.py" "$PATCH_DIR/recover.sh" "$PATCH_DIR/uninstall.sh"
echo "Setting permissions ..."
chmod +x "$PATCH_DIR/patch/apply.sh" "$PATCH_DIR/patch/create_task.py" \
"$PATCH_DIR/recover.sh" "$PATCH_DIR/uninstall.sh"
echo "Done."
echo ""
@@ -65,7 +64,7 @@ EXISTING_ID=$(midclt call initshutdownscript.query '[]' | \
python3 -c "
import sys, json
for s in json.load(sys.stdin):
if s.get('script') == '/data/truecloud-patch/apply.sh':
if s.get('script') == '$PATCH_DIR/patch/apply.sh':
print(s['id'])
break
" 2>/dev/null || true)
@@ -76,7 +75,7 @@ if [ -n "$EXISTING_ID" ]; then
'{"enabled": true}' > /dev/null
else
midclt call initshutdownscript.create \
'{"type":"SCRIPT","script":"/data/truecloud-patch/apply.sh","when":"PREINIT","enabled":true,"comment":"TrueCloud provider patch (S3/B2)"}' \
"{\"type\":\"SCRIPT\",\"script\":\"$PATCH_DIR/patch/apply.sh\",\"when\":\"PREINIT\",\"enabled\":true,\"comment\":\"TrueCloud provider patch (S3/B2)\"}" \
> /dev/null
echo "Registered."
fi
@@ -94,7 +93,7 @@ fi
echo "Applying patches ..."
_log_start=$(wc -c < "$PATCH_DIR/apply.log" 2>/dev/null || echo 0)
bash "$PATCH_DIR/apply.sh"
bash "$PATCH_DIR/patch/apply.sh"
echo ""
echo "Patch log ($PATCH_DIR/apply.log):"
tail -30 "$PATCH_DIR/apply.log"
@@ -121,9 +120,9 @@ fi
echo "Done."
echo ""
echo "Verify the backend patch loaded correctly:"
echo " python3 $PATCH_DIR/create_task.py verify"
echo " python3 $PATCH_DIR/patch/create_task.py verify"
echo ""
echo "Refresh your browser to pick up the UI change."
echo ""
echo "To create a TrueCloud Backup task with S3 or B2 credentials:"
echo " python3 $PATCH_DIR/create_task.py --help"
echo " python3 $PATCH_DIR/patch/create_task.py --help"
patch/apply.sh
+10 -6
View File
@@ -1,7 +1,6 @@
#!/bin/bash
# /data/truecloud-patch/apply.sh
# patch/apply.sh — registered as a TrueNAS PREINIT initshutdownscript.
#
# Registered as a TrueNAS PREINIT initshutdownscript.
# Runs on every boot BEFORE middlewared starts, so patches land before
# the first Python process for middlewared is created.
#
@@ -18,7 +17,8 @@
# A failed patch logs a warning and continues; middlewared always starts.
# Never use `set -e` in a PREINIT script.
PATCH_DIR="/data/truecloud-patch"
# Derive PATCH_DIR from this script's location (parent of the patch/ directory).
PATCH_DIR="$(cd "$(dirname "$0")/.." && pwd)"
LOG="$PATCH_DIR/apply.log"
# Rotate log at 512 KB to avoid unbounded growth on a system volume.
@@ -32,7 +32,7 @@ exec >> "$LOG" 2>&1
echo "=== $(date -Iseconds) ==="
# Kill switch: if this file exists, skip all patching and exit cleanly.
# Recovery: touch /data/truecloud-patch/disabled (then reboot or restart middlewared).
# Recovery: touch "$PATCH_DIR/disabled" (then reboot or restart middlewared).
if [ -f "$PATCH_DIR/disabled" ]; then
echo "Kill switch active ($PATCH_DIR/disabled exists) — patch not applied."
echo "To re-enable: rm $PATCH_DIR/disabled"
@@ -98,7 +98,11 @@ else
fi
if [ "$_can_install" = true ]; then
if cp "$PATCH_DIR/sitecustomize.py" "$SITE_PKG/sitecustomize.py"; then
# Substitute PATCH_DIR into the source so sitecustomize.py knows where
# to write hook_status.json and check the kill switch at runtime.
if sed "s|/data/truecloud-patch|$PATCH_DIR|g" \
"$PATCH_DIR/patch/sitecustomize.py" \
> "$SITE_PKG/sitecustomize.py"; then
echo "OK: Installed sitecustomize.py → $SITE_PKG/sitecustomize.py"
else
echo "WARNING: Failed to write $SITE_PKG/sitecustomize.py (permission error?)"
@@ -110,7 +114,7 @@ fi
echo "--- UI patch ---"
"$PYTHON" "$PATCH_DIR/patch_ui.py" || echo "WARNING: patch_ui.py exited non-zero; UI dropdown may still show Storj only."
"$PYTHON" "$PATCH_DIR/patch/patch_ui.py" || echo "WARNING: patch_ui.py exited non-zero; UI dropdown may still show Storj only."
# ── Done ──────────────────────────────────────────────────────────────────────
patch/create_task.py
+3 -2
View File
@@ -48,7 +48,8 @@ import sys
import urllib.error
import urllib.request
_STATUS_FILE = "/data/truecloud-patch/hook_status.json"
_PATCH_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
_STATUS_FILE = os.path.join(_PATCH_DIR, "hook_status.json")
def make_client(host, api_key, insecure=False):
@@ -117,7 +118,7 @@ def cmd_verify():
print("All patches installed. Run a test backup to confirm end-to-end.")
else:
print("One or more patches failed to apply.")
print("Check /data/truecloud-patch/apply.log and journalctl -u middlewared")
print(f"Check {os.path.join(_PATCH_DIR, 'apply.log')} and journalctl -u middlewared")
sys.exit(1)
def cmd_list_credentials(client, _args):
recover.sh
+7 -7
View File
@@ -3,19 +3,19 @@
#
# Run this from the TrueNAS shell (local console, SSH, or debug shell):
#
# bash /data/truecloud-patch/recover.sh
# bash /mnt/tank/truenas-truecloud-patch/recover.sh
#
# What it does:
# 1. Creates /data/truecloud-patch/disabled — sitecustomize.py checks for this
# file at startup and skips the import hook entirely, so middlewared starts
# clean without any of our code running.
# 1. Creates a "disabled" file in the repo root — sitecustomize.py checks for
# this file at startup and skips the import hook entirely, so middlewared
# starts clean without any of our code running.
# 2. Restarts middlewared.
#
# To re-enable the patch after investigating:
# rm /data/truecloud-patch/disabled
# bash /data/truecloud-patch/apply.sh
# rm /mnt/tank/truenas-truecloud-patch/disabled
# bash /mnt/tank/truenas-truecloud-patch/patch/apply.sh
PATCH_DIR="/data/truecloud-patch"
PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
if [ "$(id -u)" -ne 0 ]; then
echo "ERROR: must be run as root." >&2
uninstall.sh
+2 -2
View File
@@ -3,7 +3,7 @@
set -euo pipefail
PATCH_DIR="/data/truecloud-patch"
PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
echo "=== TrueNAS TrueCloud Provider Patch — Uninstall ==="
echo ""
@@ -26,7 +26,7 @@ IDS=$(midclt call initshutdownscript.query '[]' | \
python3 -c "
import sys, json
for s in json.load(sys.stdin):
if s.get('script') == '/data/truecloud-patch/apply.sh':
if s.get('script') == '$PATCH_DIR/patch/apply.sh':
print(s['id'])
" 2>/dev/null || true)