The bug-report bot was spamming; make it say something only when there is something to say
CI / shell (shellcheck + syntax) (push) Successful in 10s
CI / python 3.11 (push) Successful in 14s
CI / python 3.12 (push) Successful in 17s
CI / python 3.13 (push) Successful in 17s
TrueNAS compatibility / compat (push) Successful in 11s

It commented on every run that found a break. In one day it left ELEVEN identical
3,000-character comments on the same issue. That is not a warning system, it is a mute
button with extra steps -- and the next real finding would have been scrolled past,
which defeats the entire reason for building it.

Now: the issue BODY is the current truth, edited in place. COMMENTS are a changelog of
changes. A fingerprint of the findings (broken ref/module/problem triples only) is
embedded in the body; a run whose findings match it says nothing at all. It closes the
issue when everything is fixed.

The fingerprint deliberately ignores anything that moves on its own -- healthy rows,
the hardware-verified column, TrueNAS point releases -- so TS-25.10.4 becoming
TS-25.10.5 is not news and does not wake anybody up.

Also:
- The two near-identical per-forge shell steps are gone, replaced by one tested
  implementation (tools/compat_publish.py). Two copies of 'find the issue, decide
  whether to comment' is two chances to drift, and the Gitea one duplicated an issue
  for real.
- The README matrix refresh now opens a PULL REQUEST instead of pushing straight to
  main from CI. An unattended push to main is exactly what the release barrier exists
  to prevent: a bot that can move main can move it somewhere nobody looked.
This commit is contained in:
2026-07-13 20:20:35 +00:00
parent c6b252ac6b
commit 82084b6806
4 changed files with 328 additions and 91 deletions
+85
View File
@@ -43,6 +43,7 @@ from __future__ import annotations
import argparse
import ast
import hashlib
import json
import os
import sys
@@ -892,6 +893,90 @@ def render_markdown(rows: list[dict]) -> str:
return "\n".join(out) + "\n" + _LEGEND
FINGERPRINT = "<!-- compat-fingerprint:"
def fingerprint(rows: list[dict]) -> str:
"""A stable digest of WHAT IS BROKEN, and nothing else.
The bug report must be updated when the findings change and stay silent when they
do not. Without this the workflow commented on every run -- it left **11 identical
3,000-character comments** on one issue in a single day, which is not a warning
system, it is a mute button with extra steps.
Deliberately excludes anything that moves on its own: the matrix's `ok` rows, the
hardware-verified column, and the exact TrueNAS point-release (`TS-25.10.4` ->
`TS-25.10.5` is not news). Only the broken (ref, module, problem-id) triples count.
"""
findings = sorted(
(r["ref"], mod, p["id"])
for r in rows
for mod, m in r["modules"].items()
if is_broken(m)
for p in m["problems"]
)
return hashlib.sha256(repr(findings).encode()).hexdigest()[:16]
def extract_fingerprint(body: str) -> str | None:
"""The fingerprint a previous run left in the issue body, if any."""
if not body:
return None
i = body.find(FINGERPRINT)
if i == -1:
return None
return body[i + len(FINGERPRINT):].split("-->", 1)[0].strip() or None
def render_issue(rows: list[dict]) -> str:
"""The bug report body: what is broken, why it matters, and nothing else up front.
Short by design. The full matrix and the healthy versions go in a fold -- somebody
opening this wants to know what broke and whether it can hurt them, not to re-read
a table they can see in the README.
"""
broken = [r for r in rows if any(is_broken(m) for m in r["modules"].values())]
out = [
"`tools/compat.py` checks what this patch assumes about middlewared against "
"iXsystems' actual source, every day. Those assumptions no longer hold on the "
"versions below.",
"",
"**This does not break anyone today.** `apply.sh` re-checks on every boot and "
"**declines to apply** a module whose assumptions fail, so TrueNAS is left "
"stock rather than half-patched. The cost is the module's feature, not a "
"broken backup.",
"",
]
for r in broken:
out.append(f"### `{r['ref']}`")
out.append("")
for mod, m in sorted(r["modules"].items()):
if not is_broken(m):
continue
out.append(f"**{mod}**")
out.append("")
for p in m["problems"]:
out.append(f"- {p['detail']}")
out.append(f" <br><sub>{p['why']}</sub>")
out.append("")
out += [
"<details><summary>Full support matrix</summary>",
"",
render_markdown(rows),
"</details>",
"",
"_Filed and kept up to date by "
"[`compat.yml`](.github/workflows/compat.yml). It edits this body when the "
"findings change, and stays quiet when they do not._",
"",
f"{FINGERPRINT} {fingerprint(rows)} -->",
]
return "\n".join(out)
def render_matrix(rows: list[dict]) -> str:
"""A support table.
+148
View File
@@ -0,0 +1,148 @@
#!/usr/bin/env python3
"""Keep ONE bug report in sync with what compat.py currently finds.
WHY THIS IS NOT JUST "POST A COMMENT"
-------------------------------------
The first version commented on every run that found a break. In one day it left
**11 identical 3,000-character comments** on the same issue. That is not a warning
system; it is a mute button with extra steps. The next real finding would have been
scrolled past, which defeats the entire point of building it.
So:
* **The issue body is the current truth.** It is edited in place, never appended to.
* **Comments are a changelog of CHANGES.** A run whose findings are identical to the
last one says nothing at all -- no comment, no edit, no notification.
* A fingerprint of the findings (broken ref/module/problem triples only) is embedded
in the body. It deliberately ignores things that move on their own -- healthy rows,
the hardware-verified column, TrueNAS point releases -- so `TS-25.10.4` becoming
`TS-25.10.5` is not news, and does not wake anybody up.
* When everything is fixed, the issue is **closed** with a comment saying so.
Works against GitHub and Gitea, which differ only in the auth header and the issue
list URL. One implementation, so the two cannot drift.
python3 tools/compat_publish.py --api <url> --token <tok> --matrix /tmp/matrix.json
"""
from __future__ import annotations
import argparse
import json
import sys
import urllib.error
import urllib.request
sys.path.insert(0, __file__.rsplit("/", 1)[0])
from compat import ( # noqa: E402
extract_fingerprint,
fingerprint,
is_broken,
render_issue,
)
TITLE = "TrueNAS compatibility: the patch's assumptions no longer hold"
def _call(url, token, method="GET", data=None):
req = urllib.request.Request(
url, method=method,
headers={
# Gitea wants `token <t>`; GitHub accepts `Bearer <t>`. GitHub also
# accepts `token <t>`, so one header serves both.
"Authorization": f"token {token}",
"Content-Type": "application/json",
"Accept": "application/vnd.github+json",
},
data=json.dumps(data).encode() if data else None,
)
with urllib.request.urlopen(req) as r: # noqa: S310
return json.load(r) if r.length != 0 else {}
def find_issue(api, token, title):
"""The LOWEST-numbered issue with this title, open or closed.
Lowest, not "whichever the API returns first": two issues with the same title
existed once (an earlier version put the ref list in the title, so the identity
changed whenever that set changed), and an order-dependent pick would alternate
between them -- reopening one while commenting on the other.
"""
issues = _call(f"{api}/issues?state=all&per_page=100", token)
mine = [
i for i in issues
if i.get("title") == title and "pull_request" not in i # GitHub lists PRs here
]
return min(mine, key=lambda i: i["number"]) if mine else None
def main(argv):
ap = argparse.ArgumentParser(description=__doc__.split("\n")[0])
ap.add_argument("--api", required=True, help="…/repos/<owner>/<repo>")
ap.add_argument("--token", required=True)
ap.add_argument("--matrix", required=True, help="compat.py --matrix --json output")
args = ap.parse_args(argv[1:])
with open(args.matrix, encoding="utf-8") as fh:
rows = json.load(fh)
broken = [r for r in rows if any(is_broken(m) for m in r["modules"].values())]
issue = find_issue(args.api, args.token, TITLE)
# ── everything is healthy ────────────────────────────────────────────────
if not broken:
if issue and issue["state"] == "open":
_call(f"{args.api}/issues/{issue['number']}/comments", args.token, "POST",
{"body": "All of the patch's assumptions hold again on every "
"checked TrueNAS version. Closing."})
_call(f"{args.api}/issues/{issue['number']}", args.token, "PATCH",
{"state": "closed"})
print(f"closed #{issue['number']} — nothing is broken any more")
else:
print("nothing broken; no open report to close")
return 0
body = render_issue(rows)
want = fingerprint(rows)
# ── nothing to file yet ──────────────────────────────────────────────────
if issue is None:
made = _call(f"{args.api}/issues", args.token, "POST",
{"title": TITLE, "body": body})
print(f"filed #{made['number']}")
return 0
have = extract_fingerprint(issue.get("body") or "")
n = issue["number"]
# ── the findings are UNCHANGED: say nothing ──────────────────────────────
#
# This is the whole point. A daily "still broken, same as yesterday" comment is
# what taught everyone to ignore the last one.
if have == want and issue["state"] == "open":
print(f"#{n} is already current ({want}) — staying quiet")
return 0
_call(f"{args.api}/issues/{n}", args.token, "PATCH", {"body": body, "state": "open"})
if have != want:
refs = ", ".join(f"`{r['ref']}`" for r in broken)
note = (
"The findings changed — the report above has been updated.\n\n"
f"Currently broken on: {refs}."
if have else
"This report is now kept up to date automatically: the body above always "
"reflects the current findings, and a comment is only added when they "
"change."
)
_call(f"{args.api}/issues/{n}/comments", args.token, "POST", {"body": note})
print(f"updated #{n}: {have} -> {want}")
else:
print(f"reopened #{n}")
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv))