fix: pin embedding inference to one torch thread

torch's multi-threaded CPU forward pass intermittently raised SIGILL under
CPU contention on the deployment host; the collector swallowed it as
non-fatal, so that cycle silently computed no new embeddings. Pinning
inference to a single thread avoids the racy parallel kernel and removes
multi-threaded reduction nondeterminism, so recomputed vectors are
bit-reproducible.

Bump 0.8.2 -> 0.8.3: a vector recomputed after this change can differ from a
pre-0.8.3 one at ~1e-6 and move a borderline article across the cluster
threshold, which may shift a rating.
This commit is contained in:
flan
2026-07-19 18:57:49 +00:00
parent 444ea4fd94
commit 29afcd00b1
4 changed files with 23 additions and 2 deletions
+15
View File
@@ -8,6 +8,21 @@ requires a version bump and, if it changes methodology, a decision record in
## [Unreleased]
## [0.8.3] - 2026-07-19
### Fixed
- Embedding no longer crashes under CPU contention. torch's multi-threaded
forward-pass kernel intermittently raised `Illegal instruction` (SIGILL) on
the deployment host when the collector's 6h cycle overlapped other load,
which the loop swallowed as non-fatal — so that cycle silently computed no
new embeddings. Inference is now pinned to one thread
(`torch.set_num_threads(1)`), which also removes multi-threaded reduction
nondeterminism, so newly computed vectors are bit-reproducible (D1/D10).
Cached vectors are untouched; a vector recomputed after this change may
differ from a pre-0.8.3 one at ~1e-6, which can move a borderline article
across the cluster threshold and thus shift a rating — hence the version bump.
## [0.8.2] - 2026-07-10
Audit pass 3 (zero-confirmation over the pass-2 diff): 2 findings — one a
+1 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "tiltmeter"
version = "0.8.2"
version = "0.8.3"
description = "Auditable, reproducible political-lean ratings for news outlets"
readme = "README.md"
requires-python = ">=3.13"
+6
View File
@@ -40,8 +40,14 @@ def passage(title: str, text: str | None, summary: str | None) -> str:
def _load_model():
global _model
if _model is None:
import torch
from sentence_transformers import SentenceTransformer
# Pin CPU inference to one thread. torch's multi-threaded forward-pass
# kernel intermittently raised SIGILL under CPU contention on the
# deployment host, and multi-threaded reduction order is nondeterministic
# regardless; one thread is both crash-free and bit-reproducible (D1/D10).
torch.set_num_threads(1)
_model = SentenceTransformer(MODEL_NAME, revision=MODEL_REVISION, device="cpu")
return _model
Generated
+1 -1
View File
@@ -995,7 +995,7 @@ wheels = [
[[package]]
name = "tiltmeter"
version = "0.8.2"
version = "0.8.3"
source = { editable = "." }
dependencies = [
{ name = "feedparser" },