flan/truenas-truecloud-patch
1
0
Fork 0
Code Issues 1 Pull Requests 1 Actions Packages Projects Releases 12 Wiki Activity

Repo is the install location; all scripts self-locate

Browse Source 
Users now clone to a persistent ZFS pool and the repo stays in place.
No files are copied on install — the PREINIT hook points directly into
the clone. Scripts derive PATCH_DIR from their own path at runtime.

- install.sh: PATCH_DIR=$(dirname $0); register patch/apply.sh as
  PREINIT target; chmod only, no cp; update pipe-install error message
- patch/apply.sh: PATCH_DIR=$(dirname $0)/..; substitute PATCH_DIR
  into sitecustomize.py via sed when writing to site-packages;
  reference patch_ui.py as patch/patch_ui.py
- recover.sh, uninstall.sh: PATCH_DIR=$(dirname $0)
- uninstall.sh: look for patch/apply.sh in PREINIT registry
- patch/create_task.py: _PATCH_DIR derived from __file__; apply.log
  path in error message derived from _PATCH_DIR
- patch/sitecustomize.py: /data/truecloud-patch remains as placeholder
  substituted by apply.sh on each install
- .gitignore: exclude runtime files (apply.log, hook_status.json, disabled)
- README: document clone-to-pool install; update all example paths
This commit is contained in:
flan
2026-06-15 17:07:11 +00:00
parent 4c79403491
commit 70e84038d6
7 changed files with 94 additions and 76 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+6
View File
@@ -0,0 +1,6 @@
# Runtime files written to the repo root by apply.sh and sitecustomize.py
/apply.log
/apply.log.1
/apply.log.2
/hook_status.json
/disabled
README.md
+40 -32
View File
@@ -64,7 +64,7 @@ on every update) and are therefore re-applied automatically on every boot.
Both changes are **fail-safe**: if a patch cannot be applied (e.g. TrueNAS Both changes are **fail-safe**: if a patch cannot be applied (e.g. TrueNAS
restructured the relevant code), middlewared starts normally with Storj-only restructured the relevant code), middlewared starts normally with Storj-only
support and the reason is logged to `/data/truecloud-patch/apply.log`. support and the reason is logged to `apply.log` in your repo root.
## Supported providers after patching ## Supported providers after patching
@@ -76,12 +76,12 @@ support and the reason is logged to `/data/truecloud-patch/apply.log`.
## How persistence works ## How persistence works
TrueNAS SCALE updates replace `/usr/` entirely. The patch survives by storing TrueNAS SCALE updates replace `/usr/` entirely. The patch survives by keeping
all scripts in `/data/truecloud-patch/` (a persistent ZFS dataset) and this repository on a **persistent ZFS pool** (your data pool, not `/tmp` or a
registering a **PREINIT initshutdownscript** in the TrueNAS database. This system path) and registering a **PREINIT initshutdownscript** in the TrueNAS
causes `apply.sh` to run on every boot before `middlewared` starts, placing database. On every boot, `patch/apply.sh` runs from the repo before
`sitecustomize.py` in the correct site-packages directory and re-patching the `middlewared` starts, placing `sitecustomize.py` in the correct site-packages
UI bundle. directory and re-patching the UI bundle.
## Python version compatibility ## Python version compatibility
@@ -95,14 +95,20 @@ UI bundle.
## Install ## Install
Run on your TrueNAS box as root, with the system fully booted: Clone the repository to a **persistent ZFS pool** so it survives OS updates,
then run `install.sh` from there:
```bash ```bash
git clone https://github.com/sudolulo/truenas-truecloud-patch.git # Replace /mnt/tank with your pool name
cd truenas-truecloud-patch git clone https://github.com/sudolulo/truenas-truecloud-patch.git \
/mnt/tank/truenas-truecloud-patch
cd /mnt/tank/truenas-truecloud-patch
bash install.sh bash install.sh
``` ```
The directory you clone into becomes the permanent install location. The PREINIT
boot hook points to it — **do not delete or move the repo after install.**
Refresh your browser. S3 and B2 credentials now appear in the Refresh your browser. S3 and B2 credentials now appear in the
**Data Protection → TrueCloud Backup → Add** credential dropdown. **Data Protection → TrueCloud Backup → Add** credential dropdown.
@@ -124,12 +130,14 @@ If the UI still shows only Storj after refreshing (e.g. the JS bundle pattern
changed in a new TrueNAS version), create tasks directly via the REST API: changed in a new TrueNAS version), create tasks directly via the REST API:
```bash ```bash
# Replace /mnt/tank/truenas-truecloud-patch with your clone path
# List your cloud credentials to find the right ID # List your cloud credentials to find the right ID
python3 /data/truecloud-patch/create_task.py \ python3 /mnt/tank/truenas-truecloud-patch/patch/create_task.py \
--host 192.168.1.1 --api-key <key> list-credentials --host 192.168.1.1 --api-key <key> list-credentials
# Create a task with a B2 credential (id=3) # Create a task with a B2 credential (id=3)
python3 /data/truecloud-patch/create_task.py \ python3 /mnt/tank/truenas-truecloud-patch/patch/create_task.py \
--host 192.168.1.1 --api-key <key> create \ --host 192.168.1.1 --api-key <key> create \
--name "tank-to-b2" \ --name "tank-to-b2" \
--path /mnt/tank/data \ --path /mnt/tank/data \
@@ -147,12 +155,12 @@ Get an API key from **System → API Keys → Add**.
## Uninstall ## Uninstall
```bash ```bash
bash /data/truecloud-patch/uninstall.sh bash /mnt/tank/truenas-truecloud-patch/uninstall.sh
``` ```
Removes the PREINIT hook, `sitecustomize.py`, and restores the original UI Replace the path with your clone location. Removes the PREINIT hook,
bundle from backup. The backend changes vanish on the next `middlewared` `sitecustomize.py`, and restores the original UI bundle from backup. The
restart. backend changes vanish on the next `middlewared` restart.
--- ---
@@ -252,7 +260,7 @@ restic -r "$REPO" ls latest
## After a TrueNAS update ## After a TrueNAS update
1. Check the log: `cat /data/truecloud-patch/apply.log | tail -30` 1. Check the log: `cat /mnt/tank/truenas-truecloud-patch/apply.log | tail -30`
2. If you see "WARNING: … pattern not found", the UI patch needs updating. 2. If you see "WARNING: … pattern not found", the UI patch needs updating.
[Open an issue](https://github.com/sudolulo/truenas-truecloud-patch/issues) [Open an issue](https://github.com/sudolulo/truenas-truecloud-patch/issues)
with your TrueNAS version number. with your TrueNAS version number.
@@ -269,18 +277,18 @@ Run this from the TrueNAS shell (local console, SSH, or the debug shell in
the UI): the UI):
```bash ```bash
bash /data/truecloud-patch/recover.sh bash /mnt/tank/truenas-truecloud-patch/recover.sh
``` ```
This creates a kill-switch file (`/data/truecloud-patch/disabled`). Replace the path with your clone location. This creates a kill-switch file
`sitecustomize.py` checks for it at Python startup; if present, the import (`disabled`) in the repo root. `sitecustomize.py` checks for it at Python
hook is skipped entirely and middlewared starts clean with Storj-only support. startup; if present, the import hook is skipped entirely and middlewared starts
Nothing else on your system is affected. clean with Storj-only support. Nothing else on your system is affected.
If you cannot run a script and only have a bare shell prompt: If you cannot run a script and only have a bare shell prompt:
```bash ```bash
touch /data/truecloud-patch/disabled touch /mnt/tank/truenas-truecloud-patch/disabled
systemctl restart middlewared systemctl restart middlewared
``` ```
@@ -294,8 +302,8 @@ journalctl -u middlewared -n 50
To re-enable the patch once you have investigated: To re-enable the patch once you have investigated:
```bash ```bash
rm /data/truecloud-patch/disabled rm /mnt/tank/truenas-truecloud-patch/disabled
bash /data/truecloud-patch/apply.sh bash /mnt/tank/truenas-truecloud-patch/patch/apply.sh
``` ```
--- ---
@@ -317,22 +325,22 @@ mv /usr/share/truenas/webui/main.XXXXXXXX.js.pre-truecloud-patch \
``` ```
Refresh your browser. The UI will return to normal (Storj-only until the Refresh your browser. The UI will return to normal (Storj-only until the
patch re-runs at next reboot, or you run `bash /data/truecloud-patch/apply.sh` patch re-runs at next reboot, or you run
manually). `bash /mnt/tank/truenas-truecloud-patch/patch/apply.sh` manually).
--- ---
### Backend verify shows FAIL ### Backend verify shows FAIL
```bash ```bash
python3 /data/truecloud-patch/create_task.py verify python3 /mnt/tank/truenas-truecloud-patch/patch/create_task.py verify
``` ```
If one or more entries show `[FAIL]`: If one or more entries show `[FAIL]`:
1. **Check the apply log** for errors during the last boot: 1. **Check the apply log** for errors during the last boot:
```bash ```bash
cat /data/truecloud-patch/apply.log | tail -40 cat /mnt/tank/truenas-truecloud-patch/apply.log | tail -40
``` ```
2. **Check middlewared's own log** for Python tracebacks: 2. **Check middlewared's own log** for Python tracebacks:
```bash ```bash
@@ -351,14 +359,14 @@ If one or more entries show `[FAIL]`:
**Apply log** (check after each reboot or install): **Apply log** (check after each reboot or install):
```bash ```bash
cat /data/truecloud-patch/apply.log cat /mnt/tank/truenas-truecloud-patch/apply.log
``` ```
**Verify backend patch is loaded** (while middlewared is running): **Verify backend patch is loaded** (while middlewared is running):
```bash ```bash
python3 /data/truecloud-patch/create_task.py verify python3 /mnt/tank/truenas-truecloud-patch/patch/create_task.py verify
``` ```
This reads `/data/truecloud-patch/hook_status.json`, written by the import This reads `hook_status.json` in your repo root, written by the import
hook once both target modules have been loaded by middlewared. If it reports hook once both target modules have been loaded by middlewared. If it reports
"No hook status file found" immediately after install, restart middlewared "No hook status file found" immediately after install, restart middlewared
and try again — the file is written when middlewared imports the relevant and try again — the file is written when middlewared imports the relevant
install.sh
+26 -27
View File
@@ -2,24 +2,29 @@
# install.sh — run once on the TrueNAS box to set up truecloud-patch. # install.sh — run once on the TrueNAS box to set up truecloud-patch.
# #
# Prerequisites: run as root on TrueNAS SCALE with middlewared running. # Prerequisites: run as root on TrueNAS SCALE with middlewared running.
# Clone this repository to a persistent ZFS pool first:
#
# git clone https://github.com/sudolulo/truenas-truecloud-patch \
# /mnt/<pool>/truenas-truecloud-patch
# cd /mnt/<pool>/truenas-truecloud-patch && bash install.sh
# #
# What this does: # What this does:
# 1. Copies patch files to /data/truecloud-patch/ (survives OS updates). # 1. Registers a PREINIT initshutdownscript so patch/apply.sh re-runs on
# 2. Registers a PREINIT initshutdownscript in the TrueNAS database so # every boot before middlewared starts.
# apply.sh re-applies the patches on every boot before middlewared starts. # 2. Applies the patches immediately (no reboot required).
# 3. Applies the patches immediately (no reboot required). # 3. Restarts middlewared so the backend change takes effect now.
# 4. Restarts middlewared so the backend change takes effect now.
set -euo pipefail set -euo pipefail
PATCH_DIR="/data/truecloud-patch" # The directory containing install.sh is the permanent install location.
REPO_DIR="$(cd "$(dirname "$0")" && pwd)" PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
if [ ! -f "$REPO_DIR/patch/sitecustomize.py" ]; then if [ ! -f "$PATCH_DIR/patch/sitecustomize.py" ]; then
echo "ERROR: patch files not found at $REPO_DIR/patch/" >&2 echo "ERROR: patch files not found at $PATCH_DIR/patch/" >&2
echo "Run install.sh from the cloned repository, not via pipe:" >&2 echo "Run install.sh from a clone of the repository on a persistent pool:" >&2
echo " git clone https://github.com/sudolulo/truenas-truecloud-patch" >&2 echo " git clone https://github.com/sudolulo/truenas-truecloud-patch \\" >&2
echo " cd truenas-truecloud-patch && bash install.sh" >&2 echo " /mnt/<pool>/truenas-truecloud-patch" >&2
echo " cd /mnt/<pool>/truenas-truecloud-patch && bash install.sh" >&2
exit 1 exit 1
fi fi
@@ -43,17 +48,11 @@ if ! midclt call core.ping &>/dev/null; then
exit 1 exit 1
fi fi
# ── Copy files ──────────────────────────────────────────────────────────────── # ── Set permissions ───────────────────────────────────────────────────────────
echo "Copying patch files to $PATCH_DIR ..." echo "Setting permissions ..."
mkdir -p "$PATCH_DIR" chmod +x "$PATCH_DIR/patch/apply.sh" "$PATCH_DIR/patch/create_task.py" \
cp "$REPO_DIR/patch/sitecustomize.py" "$PATCH_DIR/" "$PATCH_DIR/recover.sh" "$PATCH_DIR/uninstall.sh"
cp "$REPO_DIR/patch/patch_ui.py" "$PATCH_DIR/"
cp "$REPO_DIR/patch/apply.sh" "$PATCH_DIR/"
cp "$REPO_DIR/patch/create_task.py" "$PATCH_DIR/"
cp "$REPO_DIR/recover.sh" "$PATCH_DIR/"
cp "$REPO_DIR/uninstall.sh" "$PATCH_DIR/"
chmod +x "$PATCH_DIR/apply.sh" "$PATCH_DIR/create_task.py" "$PATCH_DIR/recover.sh" "$PATCH_DIR/uninstall.sh"
echo "Done." echo "Done."
echo "" echo ""
@@ -65,7 +64,7 @@ EXISTING_ID=$(midclt call initshutdownscript.query '[]' | \
python3 -c " python3 -c "
import sys, json import sys, json
for s in json.load(sys.stdin): for s in json.load(sys.stdin):
if s.get('script') == '/data/truecloud-patch/apply.sh': if s.get('script') == '$PATCH_DIR/patch/apply.sh':
print(s['id']) print(s['id'])
break break
" 2>/dev/null || true) " 2>/dev/null || true)
@@ -76,7 +75,7 @@ if [ -n "$EXISTING_ID" ]; then
'{"enabled": true}' > /dev/null '{"enabled": true}' > /dev/null
else else
midclt call initshutdownscript.create \ midclt call initshutdownscript.create \
'{"type":"SCRIPT","script":"/data/truecloud-patch/apply.sh","when":"PREINIT","enabled":true,"comment":"TrueCloud provider patch (S3/B2)"}' \ "{\"type\":\"SCRIPT\",\"script\":\"$PATCH_DIR/patch/apply.sh\",\"when\":\"PREINIT\",\"enabled\":true,\"comment\":\"TrueCloud provider patch (S3/B2)\"}" \
> /dev/null > /dev/null
echo "Registered." echo "Registered."
fi fi
@@ -94,7 +93,7 @@ fi
echo "Applying patches ..." echo "Applying patches ..."
_log_start=$(wc -c < "$PATCH_DIR/apply.log" 2>/dev/null || echo 0) _log_start=$(wc -c < "$PATCH_DIR/apply.log" 2>/dev/null || echo 0)
bash "$PATCH_DIR/apply.sh" bash "$PATCH_DIR/patch/apply.sh"
echo "" echo ""
echo "Patch log ($PATCH_DIR/apply.log):" echo "Patch log ($PATCH_DIR/apply.log):"
tail -30 "$PATCH_DIR/apply.log" tail -30 "$PATCH_DIR/apply.log"
@@ -121,9 +120,9 @@ fi
echo "Done." echo "Done."
echo "" echo ""
echo "Verify the backend patch loaded correctly:" echo "Verify the backend patch loaded correctly:"
echo " python3 $PATCH_DIR/create_task.py verify" echo " python3 $PATCH_DIR/patch/create_task.py verify"
echo "" echo ""
echo "Refresh your browser to pick up the UI change." echo "Refresh your browser to pick up the UI change."
echo "" echo ""
echo "To create a TrueCloud Backup task with S3 or B2 credentials:" echo "To create a TrueCloud Backup task with S3 or B2 credentials:"
echo " python3 $PATCH_DIR/create_task.py --help" echo " python3 $PATCH_DIR/patch/create_task.py --help"
patch/apply.sh
+10 -6
View File
@@ -1,7 +1,6 @@
#!/bin/bash #!/bin/bash
# /data/truecloud-patch/apply.sh # patch/apply.sh — registered as a TrueNAS PREINIT initshutdownscript.
# #
# Registered as a TrueNAS PREINIT initshutdownscript.
# Runs on every boot BEFORE middlewared starts, so patches land before # Runs on every boot BEFORE middlewared starts, so patches land before
# the first Python process for middlewared is created. # the first Python process for middlewared is created.
# #
@@ -18,7 +17,8 @@
# A failed patch logs a warning and continues; middlewared always starts. # A failed patch logs a warning and continues; middlewared always starts.
# Never use `set -e` in a PREINIT script. # Never use `set -e` in a PREINIT script.
PATCH_DIR="/data/truecloud-patch" # Derive PATCH_DIR from this script's location (parent of the patch/ directory).
PATCH_DIR="$(cd "$(dirname "$0")/.." && pwd)"
LOG="$PATCH_DIR/apply.log" LOG="$PATCH_DIR/apply.log"
# Rotate log at 512 KB to avoid unbounded growth on a system volume. # Rotate log at 512 KB to avoid unbounded growth on a system volume.
@@ -32,7 +32,7 @@ exec >> "$LOG" 2>&1
echo "=== $(date -Iseconds) ===" echo "=== $(date -Iseconds) ==="
# Kill switch: if this file exists, skip all patching and exit cleanly. # Kill switch: if this file exists, skip all patching and exit cleanly.
# Recovery: touch /data/truecloud-patch/disabled (then reboot or restart middlewared). # Recovery: touch "$PATCH_DIR/disabled" (then reboot or restart middlewared).
if [ -f "$PATCH_DIR/disabled" ]; then if [ -f "$PATCH_DIR/disabled" ]; then
echo "Kill switch active ($PATCH_DIR/disabled exists) — patch not applied." echo "Kill switch active ($PATCH_DIR/disabled exists) — patch not applied."
echo "To re-enable: rm $PATCH_DIR/disabled" echo "To re-enable: rm $PATCH_DIR/disabled"
@@ -98,7 +98,11 @@ else
fi fi
if [ "$_can_install" = true ]; then if [ "$_can_install" = true ]; then
if cp "$PATCH_DIR/sitecustomize.py" "$SITE_PKG/sitecustomize.py"; then # Substitute PATCH_DIR into the source so sitecustomize.py knows where
# to write hook_status.json and check the kill switch at runtime.
if sed "s|/data/truecloud-patch|$PATCH_DIR|g" \
"$PATCH_DIR/patch/sitecustomize.py" \
> "$SITE_PKG/sitecustomize.py"; then
echo "OK: Installed sitecustomize.py → $SITE_PKG/sitecustomize.py" echo "OK: Installed sitecustomize.py → $SITE_PKG/sitecustomize.py"
else else
echo "WARNING: Failed to write $SITE_PKG/sitecustomize.py (permission error?)" echo "WARNING: Failed to write $SITE_PKG/sitecustomize.py (permission error?)"
@@ -110,7 +114,7 @@ fi
echo "--- UI patch ---" echo "--- UI patch ---"
"$PYTHON" "$PATCH_DIR/patch_ui.py" || echo "WARNING: patch_ui.py exited non-zero; UI dropdown may still show Storj only." "$PYTHON" "$PATCH_DIR/patch/patch_ui.py" || echo "WARNING: patch_ui.py exited non-zero; UI dropdown may still show Storj only."
# ── Done ────────────────────────────────────────────────────────────────────── # ── Done ──────────────────────────────────────────────────────────────────────
patch/create_task.py
+3 -2
View File
@@ -48,7 +48,8 @@ import sys
import urllib.error import urllib.error
import urllib.request import urllib.request
_STATUS_FILE = "/data/truecloud-patch/hook_status.json" _PATCH_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
_STATUS_FILE = os.path.join(_PATCH_DIR, "hook_status.json")
def make_client(host, api_key, insecure=False): def make_client(host, api_key, insecure=False):
@@ -117,7 +118,7 @@ def cmd_verify():
print("All patches installed. Run a test backup to confirm end-to-end.") print("All patches installed. Run a test backup to confirm end-to-end.")
else: else:
print("One or more patches failed to apply.") print("One or more patches failed to apply.")
print("Check /data/truecloud-patch/apply.log and journalctl -u middlewared") print(f"Check {os.path.join(_PATCH_DIR, 'apply.log')} and journalctl -u middlewared")
sys.exit(1) sys.exit(1)
def cmd_list_credentials(client, _args): def cmd_list_credentials(client, _args):
recover.sh
+7 -7
View File
@@ -3,19 +3,19 @@
# #
# Run this from the TrueNAS shell (local console, SSH, or debug shell): # Run this from the TrueNAS shell (local console, SSH, or debug shell):
# #
# bash /data/truecloud-patch/recover.sh # bash /mnt/tank/truenas-truecloud-patch/recover.sh
# #
# What it does: # What it does:
# 1. Creates /data/truecloud-patch/disabled — sitecustomize.py checks for this # 1. Creates a "disabled" file in the repo root — sitecustomize.py checks for
# file at startup and skips the import hook entirely, so middlewared starts # this file at startup and skips the import hook entirely, so middlewared
# clean without any of our code running. # starts clean without any of our code running.
# 2. Restarts middlewared. # 2. Restarts middlewared.
# #
# To re-enable the patch after investigating: # To re-enable the patch after investigating:
# rm /data/truecloud-patch/disabled # rm /mnt/tank/truenas-truecloud-patch/disabled
# bash /data/truecloud-patch/apply.sh # bash /mnt/tank/truenas-truecloud-patch/patch/apply.sh
PATCH_DIR="/data/truecloud-patch" PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
if [ "$(id -u)" -ne 0 ]; then if [ "$(id -u)" -ne 0 ]; then
echo "ERROR: must be run as root." >&2 echo "ERROR: must be run as root." >&2
uninstall.sh
+2 -2
View File
@@ -3,7 +3,7 @@
set -euo pipefail set -euo pipefail
PATCH_DIR="/data/truecloud-patch" PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
echo "=== TrueNAS TrueCloud Provider Patch — Uninstall ===" echo "=== TrueNAS TrueCloud Provider Patch — Uninstall ==="
echo "" echo ""
@@ -26,7 +26,7 @@ IDS=$(midclt call initshutdownscript.query '[]' | \
python3 -c " python3 -c "
import sys, json import sys, json
for s in json.load(sys.stdin): for s in json.load(sys.stdin):
if s.get('script') == '/data/truecloud-patch/apply.sh': if s.get('script') == '$PATCH_DIR/patch/apply.sh':
print(s['id']) print(s['id'])
" 2>/dev/null || true) " 2>/dev/null || true)