- __version__ was stuck at 0.8.2 after the 0.8.3 release, so every ratings
and manifest artifact carried the wrong pipeline_version; bumped to match
pyproject.toml and added a regression test pinning the two together.
- fetch_article_text followed an RSS entry's <link> wherever it pointed,
including through redirects, with no check on the resolved address. A
compromised or malicious feed could aim it at an internal address (cloud
metadata, LAN service). The host and every redirect hop are now resolved
and rejected if they land on a private, loopback, link-local, or otherwise
non-public range.
- artifacts.write_json wrote the target path in place, so serve.py (a
separate process reading the same releases volume) could read a partially
written or truncated artifact. It now writes to a temp file in the same
directory and swaps it into place with os.replace.
torch's multi-threaded CPU forward pass intermittently raised SIGILL under
CPU contention on the deployment host; the collector swallowed it as
non-fatal, so that cycle silently computed no new embeddings. Pinning
inference to a single thread avoids the racy parallel kernel and removes
multi-threaded reduction nondeterminism, so recomputed vectors are
bit-reproducible.
Bump 0.8.2 -> 0.8.3: a vector recomputed after this change can differ from a
pre-0.8.3 one at ~1e-6 and move a borderline article across the cluster
threshold, which may shift a rating.
Restore batch atomicity: explicit BEGIN guard before per-entry savepoints
(python-sqlite3 opens no transaction for SAVEPOINT, so pass-2's savepoints
committed items individually on the autocommit path — regression pinned by
an empirical rollback test). validate records thin-sample raters distinctly
from missing raters with the reason, in artifact and CLI. 67 tests.
Savepoints make content+metadata writes atomic per entry; cached_embed runs
no DDL and never commits a caller's batch; cycle windows derive from UTC;
ratings pin the reference-corpus state; thin-sample raters are recorded
gate failures with automation-safe exit codes; snapshot fails loudly on
missing content; audit refuses empty stores and appends custody-heads.jsonl
history; repair adopts orphans as visible 'adopt' batches; timestamps
validated tz-aware UTC at insert; cleanup sweep (shared publish path,
README drift, dead code, encoding pins). 65 tests.
- contents table: distinct texts stored once, zlib, SHA-256 keyed; wire copy
dedups across outlets; measured 12.2MB -> 6.3MB on the real corpus
- custody_log/custody_items hash chain over every ingest/reference batch;
migration chains the backlog as genesis; heads served at /custody
- tiltmeter audit: re-hash everything + walk chain; tamper tests go through
a raw non-FK connection (the realistic attacker path)
- outlets dimension table, verbatim byline capture, HTML-stripped summaries,
URL canonicalization (tracking params removed, original kept if different)
- transactional v1->v2 auto-migration preserving fingerprints verbatim: all
published manifests stay verifiable; WAL mode
- METHODOLOGY D11 + ADR-0004; glossary + plain-language docs updated
- validate: Spearman vs AllSides + Ad Fontes over shared outlets, fixed-seed
permutation p-values, gate = rho >= 0.7 both AND reliable orientation;
refuses unverified reference values unless --allow-unverified (peek only);
emits validation-{snapshot}.json, served at /validation/{id}
- sweep: rescoring across clustering-threshold grid w/ rank-corr-vs-default
stability; emits sweep-{snapshot}.json, served at /sweeps/{id}
- day-one diagnostics on the dry-run release recorded in CHANGELOG
Retired outlets aged out of the db alarmed forever; configured outlets
with no rows were invisible. Staleness now reads the config: retired
outlets excluded, never-collected outlets stale with null hours.
json.dumps failed on date objects parsed from unquoted YAML dates
(ownership retrieved fields), killing the response mid-flight; serialize
with default=str and pin the case in the serve test fixture.
flan
flan