flan
2c9adc1ce5
patch_ui.py: - Write Angular bundle atomically via tmp + os.replace, matching the pattern already used by _record_status. Prevents a corrupt bundle if the write is interrupted mid-boot. sitecustomize.py: - Tighten _record_status count barrier comment to name _Finder._targets as the canonical count, making the coupling visible to future editors. uninstall.sh: - Verify middlewared restarted cleanly after uninstall, with journalctl guidance on failure — matching recover.sh's existing pattern. apply.sh: - Change second line of site-packages error block from WARNING: prefix (misleading for an instructional message) to a plain Run: hint. create_task.py: - Guard __doc__ against None in epilog extraction so -OO does not crash. README.md: - Split Emergency recovery into three named subsections: middlewared won't start, web UI is blank or broken (corrupt bundle recovery), and backend verify shows FAIL. Each gives direct commands and escalation steps. - Add Restoring from a TrueCloud Backup section: finding the restic binary, gathering credentials, provider-specific env var setup for B2 and S3, listing and restoring snapshots, and operational notes on restore hygiene. - Clarify that hook_status.json is written once both target modules have loaded (not necessarily at the instant middlewared starts).
384 lines
13 KiB
Markdown
384 lines
13 KiB
Markdown
# truenas-truecloud-patch
|
||
|
||
Extends TrueNAS SCALE's **TrueCloud Backup** feature to work with S3-compatible
|
||
providers and native Backblaze B2, instead of Storj only.
|
||
|
||
---
|
||
|
||
## Why this exists
|
||
|
||
In 2026, Storj raised the price of their TrueNAS-integrated storage tier from
|
||
**$5/month to $50/month** — a 10× increase. For many home lab and small-office
|
||
users, the TrueCloud Backup feature became unaffordable overnight.
|
||
|
||
TrueCloud Backup is the only native TrueNAS mechanism that provides:
|
||
- Integrated ZFS snapshot support before each backup
|
||
- Restic-based incremental deduplication
|
||
- Scheduled tasks with progress and log tracking in the UI
|
||
- Dataset lock integration
|
||
|
||
Running restic manually is possible but loses all of the above.
|
||
This patch restores access to the TrueCloud Backup feature for users who
|
||
need a provider other than Storj, with storage they already pay for or that
|
||
costs a fraction of the new Storj price.
|
||
|
||
---
|
||
|
||
## ⚠ Disclaimer — please read before installing
|
||
|
||
**This project is unofficial, unsupported, and not affiliated with iXsystems
|
||
or the TrueNAS project in any way.**
|
||
|
||
By installing this patch you accept the following:
|
||
|
||
- **Unsupported configuration.** TrueNAS support staff are not obligated to
|
||
help with any issue on a system running this patch. If you file a bug report,
|
||
remove the patch first and reproduce the issue on an unmodified system.
|
||
|
||
- **May break on TrueNAS updates.** The patch targets internal middleware APIs
|
||
that are not part of any public contract. They can change at any time. When
|
||
they do, the patch silently degrades to Storj-only behaviour rather than
|
||
breaking TrueNAS — but you should check the log after each update.
|
||
|
||
- **Your backups are your responsibility.** Verify that your backup jobs
|
||
complete successfully and that restores work before relying on them for
|
||
disaster recovery.
|
||
|
||
- **No warranty.** This software is provided as-is. See the LICENSE file.
|
||
|
||
If TrueNAS ever adds native support for additional providers in TrueCloud
|
||
Backup, uninstall this patch immediately.
|
||
|
||
---
|
||
|
||
## What is actually patched
|
||
|
||
**Nothing in TrueNAS's persistent database or configuration is modified.**
|
||
The patch operates on two files that live in `/usr/` (which TrueNAS replaces
|
||
on every update) and are therefore re-applied automatically on every boot.
|
||
|
||
| Layer | What changes | Technique |
|
||
|---|---|---|
|
||
| **Backend** | `B2RcloneRemote` gains `get_restic_config()`. `restic.py` URL builder is fixed for providers with no hostname component (`b2:bucket/path` vs the broken `b2:/bucket/path`). | `sitecustomize.py` — Python's standard startup hook; no middleware files are modified on disk |
|
||
| **UI** | The Angular bundle's `filterByProviders` binding is widened from `["STORJ_IX"]` to `["STORJ_IX","S3","B2"]` | In-place text replacement in the compiled JS bundle; original is backed up |
|
||
|
||
Both changes are **fail-safe**: if a patch cannot be applied (e.g. TrueNAS
|
||
restructured the relevant code), middlewared starts normally with Storj-only
|
||
support and the reason is logged to `/data/truecloud-patch/apply.log`.
|
||
|
||
## Supported providers after patching
|
||
|
||
| Provider | Credential type in TrueNAS |
|
||
|---|---|
|
||
| Backblaze B2 (native B2 API) | `B2` |
|
||
| AWS S3, Wasabi, Cloudflare R2, MinIO, and any S3-compatible endpoint | `S3` |
|
||
| Storj (unchanged) | `STORJ_IX` |
|
||
|
||
## How persistence works
|
||
|
||
TrueNAS SCALE updates replace `/usr/` entirely. The patch survives by storing
|
||
all scripts in `/data/truecloud-patch/` (a persistent ZFS dataset) and
|
||
registering a **PREINIT initshutdownscript** in the TrueNAS database. This
|
||
causes `apply.sh` to run on every boot before `middlewared` starts, placing
|
||
`sitecustomize.py` in the correct site-packages directory and re-patching the
|
||
UI bundle.
|
||
|
||
## Python version compatibility
|
||
|
||
`sitecustomize.py` uses the `find_spec` / `exec_module` import hook API
|
||
(introduced in Python 3.4, required from Python 3.12 onwards). This covers:
|
||
|
||
- TrueNAS SCALE 24.x — Debian 12, Python 3.11 ✓
|
||
- TrueNAS SCALE 25.x — Debian 13, Python 3.12 ✓
|
||
|
||
---
|
||
|
||
## Install
|
||
|
||
Run on your TrueNAS box as root, with the system fully booted:
|
||
|
||
```bash
|
||
git clone https://github.com/sudolulo/truenas-truecloud-patch.git
|
||
cd truenas-truecloud-patch
|
||
bash install.sh
|
||
```
|
||
|
||
Refresh your browser. S3 and B2 credentials now appear in the
|
||
**Data Protection → TrueCloud Backup → Add** credential dropdown.
|
||
|
||
## Creating a credential
|
||
|
||
Before creating a backup task, add a credential in the TrueNAS UI:
|
||
|
||
**Data Protection → TrueCloud Backup → Add → (create new credential)**
|
||
|
||
Or use **Credentials → Backup Credentials → Cloud Credentials → Add** and
|
||
select B2 or Amazon S3.
|
||
|
||
For S3-compatible providers, select **Amazon S3**, then set a custom endpoint
|
||
in the credential's advanced settings (e.g. `https://s3.wasabisys.com`).
|
||
|
||
## Creating a task via CLI
|
||
|
||
If the UI still shows only Storj after refreshing (e.g. the JS bundle pattern
|
||
changed in a new TrueNAS version), create tasks directly via the REST API:
|
||
|
||
```bash
|
||
# List your cloud credentials to find the right ID
|
||
python3 /data/truecloud-patch/create_task.py \
|
||
--host 192.168.1.1 --api-key <key> list-credentials
|
||
|
||
# Create a task with a B2 credential (id=3)
|
||
python3 /data/truecloud-patch/create_task.py \
|
||
--host 192.168.1.1 --api-key <key> create \
|
||
--name "tank-to-b2" \
|
||
--path /mnt/tank/data \
|
||
--credential 3 \
|
||
--bucket my-bucket \
|
||
--folder backups/tank \
|
||
--password "restic-repo-password" \
|
||
--keep-last 14
|
||
```
|
||
|
||
Get an API key from **System → API Keys → Add**.
|
||
|
||
---
|
||
|
||
## Uninstall
|
||
|
||
```bash
|
||
bash /path/to/truenas-truecloud-patch/uninstall.sh
|
||
```
|
||
|
||
Removes the PREINIT hook, `sitecustomize.py`, and restores the original UI
|
||
bundle from backup. The backend changes vanish on the next `middlewared`
|
||
restart.
|
||
|
||
---
|
||
|
||
## Restoring from a TrueCloud Backup
|
||
|
||
TrueCloud Backup uses [restic](https://restic.net/) under the hood. Restores
|
||
are done with the `restic` command directly — TrueNAS does not yet expose a
|
||
restore UI for TrueCloud Backup tasks.
|
||
|
||
### 1. Find the restic binary
|
||
|
||
```bash
|
||
which restic 2>/dev/null || find /usr -name restic -type f 2>/dev/null | head -1
|
||
```
|
||
|
||
Use that path in the commands below (referred to as `restic`).
|
||
|
||
### 2. Gather your repository details
|
||
|
||
You need three things from the task you created:
|
||
|
||
| Detail | Where to find it |
|
||
|---|---|
|
||
| **Bucket** and **folder** | TrueNAS UI → Data Protection → TrueCloud Backup → edit the task → Attributes |
|
||
| **Credentials** (key ID + secret) | TrueNAS UI → Credentials → Backup Credentials → edit the credential |
|
||
| **Repository password** | The `--password` value you supplied when creating the task |
|
||
|
||
### 3. Set environment variables
|
||
|
||
**Backblaze B2:**
|
||
```bash
|
||
export B2_ACCOUNT_ID="your-key-id"
|
||
export B2_ACCOUNT_KEY="your-application-key"
|
||
export RESTIC_PASSWORD="your-repo-password"
|
||
REPO="b2:your-bucket/your-folder"
|
||
```
|
||
|
||
**S3-compatible (AWS S3, Wasabi, Cloudflare R2, MinIO, etc.):**
|
||
```bash
|
||
export AWS_ACCESS_KEY_ID="your-access-key"
|
||
export AWS_SECRET_ACCESS_KEY="your-secret-key"
|
||
export RESTIC_PASSWORD="your-repo-password"
|
||
# Use just the hostname as the endpoint — no https:// prefix:
|
||
REPO="s3:s3.wasabisys.com/your-bucket/your-folder" # Wasabi example
|
||
# REPO="s3:s3.amazonaws.com/your-bucket/your-folder" # AWS S3
|
||
# REPO="s3:<account>.r2.cloudflarestorage.com/your-bucket/your-folder" # R2
|
||
```
|
||
|
||
### 4. List snapshots
|
||
|
||
```bash
|
||
restic -r "$REPO" snapshots
|
||
```
|
||
|
||
Output example:
|
||
```
|
||
ID Time Host Tags Paths
|
||
──────────────────────────────────────────────────────────
|
||
a1b2c3d4 2026-06-01 02:00:05 truenas /mnt/tank/data
|
||
e5f6a7b8 2026-06-08 02:00:07 truenas /mnt/tank/data
|
||
```
|
||
|
||
### 5. Restore files
|
||
|
||
**Restore everything from the latest snapshot to a temporary location:**
|
||
```bash
|
||
restic -r "$REPO" restore latest --target /mnt/tank/restore-tmp
|
||
```
|
||
|
||
**Restore a specific snapshot by ID:**
|
||
```bash
|
||
restic -r "$REPO" restore a1b2c3d4 --target /mnt/tank/restore-tmp
|
||
```
|
||
|
||
**Restore only specific paths from within a snapshot:**
|
||
```bash
|
||
restic -r "$REPO" restore latest \
|
||
--include /mnt/tank/data/important-dir \
|
||
--target /mnt/tank/restore-tmp
|
||
```
|
||
|
||
**Browse a snapshot without extracting (useful for finding the right file):**
|
||
```bash
|
||
restic -r "$REPO" ls latest
|
||
```
|
||
|
||
### 6. Notes
|
||
|
||
- Restore to a **different path** first, then move files into place after
|
||
verifying. Restoring directly over a live dataset can cause data loss if
|
||
the snapshot is incomplete or from the wrong point in time.
|
||
- If you created the task with `--snapshot` (ZFS snapshot before each run),
|
||
the restic snapshot captures the dataset at a consistent point in time.
|
||
- Use `restic check -r "$REPO"` periodically to verify repository integrity.
|
||
|
||
---
|
||
|
||
## After a TrueNAS update
|
||
|
||
1. Check the log: `cat /data/truecloud-patch/apply.log | tail -30`
|
||
2. If you see "WARNING: … pattern not found", the UI patch needs updating.
|
||
[Open an issue](https://github.com/sudolulo/truenas-truecloud-patch/issues)
|
||
with your TrueNAS version number.
|
||
3. The backend patch (B2 support + URL fix) is more stable — check that a
|
||
B2 backup job still completes successfully after any update.
|
||
|
||
---
|
||
|
||
## Emergency recovery
|
||
|
||
### middlewared won't start
|
||
|
||
Run this from the TrueNAS shell (local console, SSH, or the debug shell in
|
||
the UI):
|
||
|
||
```bash
|
||
bash /data/truecloud-patch/recover.sh
|
||
```
|
||
|
||
This creates a kill-switch file (`/data/truecloud-patch/disabled`).
|
||
`sitecustomize.py` checks for it at Python startup; if present, the import
|
||
hook is skipped entirely and middlewared starts clean with Storj-only support.
|
||
Nothing else on your system is affected.
|
||
|
||
If you cannot run a script and only have a bare shell prompt:
|
||
|
||
```bash
|
||
touch /data/truecloud-patch/disabled
|
||
systemctl restart middlewared
|
||
```
|
||
|
||
If middlewared **still** won't start after the kill switch is set, the problem
|
||
is unrelated to this patch. Check:
|
||
|
||
```bash
|
||
journalctl -u middlewared -n 50
|
||
```
|
||
|
||
To re-enable the patch once you have investigated:
|
||
|
||
```bash
|
||
rm /data/truecloud-patch/disabled
|
||
bash /data/truecloud-patch/apply.sh
|
||
```
|
||
|
||
---
|
||
|
||
### Web UI is blank or broken
|
||
|
||
If the TrueNAS web interface loads blank or shows JavaScript errors, the
|
||
Angular bundle may have been interrupted mid-write (e.g. power cut during
|
||
boot). The original bundle is always backed up before patching, so recovery
|
||
is straightforward:
|
||
|
||
```bash
|
||
# Find the backup (the path varies by TrueNAS version):
|
||
find /usr/share/truenas /var/www/truenas -name "*.js.pre-truecloud-patch" 2>/dev/null
|
||
|
||
# Restore it — substitute the actual path from the find output:
|
||
mv /usr/share/truenas/webui/main.XXXXXXXX.js.pre-truecloud-patch \
|
||
/usr/share/truenas/webui/main.XXXXXXXX.js
|
||
```
|
||
|
||
Refresh your browser. The UI will return to normal (Storj-only until the
|
||
patch re-runs at next reboot, or you run `bash /data/truecloud-patch/apply.sh`
|
||
manually).
|
||
|
||
---
|
||
|
||
### Backend verify shows FAIL
|
||
|
||
```bash
|
||
python3 /data/truecloud-patch/create_task.py verify
|
||
```
|
||
|
||
If one or more entries show `[FAIL]`:
|
||
|
||
1. **Check the apply log** for errors during the last boot:
|
||
```bash
|
||
cat /data/truecloud-patch/apply.log | tail -40
|
||
```
|
||
2. **Check middlewared's own log** for Python tracebacks:
|
||
```bash
|
||
journalctl -u middlewared -n 50
|
||
```
|
||
3. **A FAIL is non-fatal.** middlewared runs normally; the affected provider
|
||
falls back to Storj-only. Your existing backups are not at risk.
|
||
4. **If the detail says the module doesn't exist**, a TrueNAS update renamed
|
||
or restructured the internal API.
|
||
[Open an issue](https://github.com/sudolulo/truenas-truecloud-patch/issues)
|
||
with your TrueNAS version number and the full verify output.
|
||
|
||
---
|
||
|
||
## Troubleshooting
|
||
|
||
**Apply log** (check after each reboot or install):
|
||
```bash
|
||
cat /data/truecloud-patch/apply.log
|
||
```
|
||
|
||
**Verify backend patch is loaded** (while middlewared is running):
|
||
```bash
|
||
python3 /data/truecloud-patch/create_task.py verify
|
||
```
|
||
This reads `/data/truecloud-patch/hook_status.json`, written by the import
|
||
hook once both target modules have been loaded by middlewared. If it reports
|
||
"No hook status file found" immediately after install, restart middlewared
|
||
and try again — the file is written when middlewared imports the relevant
|
||
modules, which happens at service start.
|
||
Does not require `--host` or `--api-key`.
|
||
|
||
**Verify the UI patch** (should print your TrueNAS version):
|
||
```bash
|
||
grep -c 'STORJ_IX.*S3.*B2' \
|
||
$(find /usr/share/truenas -name '*.js' 2>/dev/null) 2>/dev/null \
|
||
| grep -v ':0'
|
||
```
|
||
|
||
**B2 backup fails with credential error**
|
||
Confirm the credential type is exactly `B2` (not `S3` with a B2 endpoint).
|
||
B2's native restic backend uses a different auth path than S3-compatible B2.
|
||
|
||
**S3-compatible backup fails**
|
||
S3 support already existed in the backend — the credential setup is the likely
|
||
issue. Verify the endpoint URL, access key, secret key, and bucket name in
|
||
the credential settings. Use `--insecure` in create_task.py only if you are
|
||
using a self-signed certificate on your S3 endpoint.
|