chore: promote development to main — CI, tests and docs only (no release) - #442
Closed
Polichinel wants to merge 15 commits into
Closed
chore: promote development to main — CI, tests and docs only (no release)#442Polichinel wants to merge 15 commits into
Polichinel wants to merge 15 commits into
Conversation
chore: sync main back into development after v1.11.0
Follows the repo convention: reports/post_mortems/YYYY-MM-DD_<slug>.md,
with the required "previous postmortem items — status" section, which is
the mechanism the convention actually runs on.
THE HEADLINE IS THE CONVENTION ITSELF. Nine releases shipped between the
last post-mortem and this one and none was written. That post-mortem's
own first "do differently" item was "write post-deployment postmortems
immediately after deployment", and it had already been carried from
v1.4.0 and v1.5.0. Eleven consecutive misses. The convention recorded
itself failing, then failed nine more times.
THE SPINE is one failure mode with six instances, each true when written
and each stopping being true without anything failing:
1. "9 sources" — a count matching no artifact, in 8 places (C-164)
2. logrotate pointing at a path the pipeline left; missingok reported
success nightly for four months (C-330)
3. ADR-026 "Public GitHub is safe" while a working password sat in
committed prose (#391)
4. ADRs citing lab_grid/, a package views-metric-lab deleted (C-336)
5. validate_docs.sh — correct, passing, wired to nothing
6. views-frames>=1.0 froze the lockfile at 1.0.0 for six weeks (C-337)
Several produced green rather than silence. Two were caught by the
operator, not by the audits. #6 was missed INSIDE the audit written about
this exact failure mode, one day after writing it — which is why C-337 is
open at Tier 2 and C-336 is a standing entry.
The refresh.log incident has a full section rather than a separate file
(operator's call): timeline, the joined-heredoc mechanism, three named
failures, and the detail that the fix for the missing rotation destroyed
what rotation would have preserved.
v1.6.0's five items are statused honestly — two failed outright, one
partial, one met. The only unambiguous success is the only one backed by
a test. Per-step duration logging is CLOSED as not-worth-it rather than
carried a sixth time.
Every figure re-checked against the repo rather than recalled: 28 commits
in v1.9.0..main, 30 merged PRs, 28 open issues, 339 IDs / 293 resolved /
43 open = 0+2+11+24+6, 22 entries registered in the window with 6 struck,
all link targets resolving, and views-metric-lab's 6e1a34d confirmed to
exist. My own check caught one imprecision — the register index quoted as
1670 (true at the split) reads as wrong today at 1764 — so the document
now states both and says why. A post-mortem about unverified claims
containing an unverified claim would have been its own next entry.
Also: nine merged local branches deleted, including release/v1.11.0,
which the stale-branch deploy gate correctly caught. The remote was
already clean — delete_branch_on_merge works — and what I had been
looking at were stale remote-tracking refs from fetching without --prune
all session. My own git view was a record describing a world that had
moved on.
Verified: full suite exit 0, ruff clean, validate_docs.sh exit 0, deploy
gates green after cleanup.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…dum) (#417) Found by using it, at the end of the v1.11.0 release, and initially written off as an anecdote. The operator asked whether it was something we should address. It was. TestF8StaleBranches read `git branch -a`, which folds in remote-tracking refs — a local cache that goes stale the moment anyone deletes a branch and that only `git fetch --prune` refreshes. So the gate failed against a repository that was genuinely clean. Demonstrated rather than argued: fabricating ONLY a refs/remotes/origin/release/v1.11.0 ref — no branch on the remote, none locally — was enough to fail it. It had already cost something. delete_branch_on_merge had removed every merged branch from the remote, the gate went red anyway, and I deleted nine local branches chasing a problem that did not exist. The actual cause was a stale view of the world. This is C-320's own failure mode occurring inside C-320's own test file: a gate that reddens for reasons unrelated to what it asserts stops being read. Registered as an addendum there rather than a new concern — same root cause, existing home. Split into two checks: local — `git branch`, authoritative for your own leftovers remote — `git ls-remote`, authoritative for the remote, and skipped when offline per this file's existing idiom (C-320: skip where the environment cannot answer, do not guess) Drilled three ways: clean repo passes; the old false positive now passes; a real local leftover still fails. The post-mortem gains this as its fourth guard-drill finding, since it was found while writing the document about exactly this failure mode. The irony is kept rather than smoothed: my view of the repository was a record describing a world that had moved on. Verified: full suite exit 0, ruff clean, validate_docs.sh exit 0, register invariants unchanged. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
/register-risk on the v1.11.0 close-out. Both were acted on during the
session and never tracked. The register is the gate between "found" and
"remembered", and memory is not a control.
C-340 (Tier 3) — AUTO-MERGE FAILS SILENTLY, TWO MECHANISMS.
1. `gh pr merge --auto --<method>` refuses to change the method on an
already-armed PR and reports nothing. During v1.10.0 that left
`squash` armed on a development -> main promotion. A squash there
rewrites the release SHAs and permanently breaks the ancestry the
back-merge exists to maintain — and it would not have surfaced
until a later release diffed against a base that never existed.
Caught only by reading auto_merge.merge_method back. The GraphQL
disable/enable pair does change it.
2. Pushing a follow-up commit to a branch whose PR has already
auto-merged orphans the work. #416 merged the instant CI went
green; the next push landed on a branch with no open PR, and two
pieces of work were simply not on development. `git push` reported
success. The only signal was `commits=1` contradicting a remembered
second push.
Tier 3, not 2: no data or model-output impact, and both recoverable.
Not Tier 4: auto-merge is armed on every PR here, so both are live on
every change, and mechanism 1's damage to main would be permanent.
C-341 (Tier 4) — DEPLOY GATES RUN ONLY WHERE SOMEONE TYPES PYTEST.
The unexamined residual of C-320's own fix. Making gates skip-with-
reason where CI cannot answer was right; the consequence — that they
now assure only whoever runs the suite at the right moment — was never
looked at. It cost four hours of undetected divergence after v1.10.0
and had been silently true after every prior release. Partially
mitigated by release-topology.yml; the rest remain local-only.
Both entries name the property they share, which is why they are
registered rather than remembered: THEY FAIL GREEN. git push succeeds;
gh pr merge exits 0. Same class as C-330's nightly no-op reporting
success and C-337's lockfile frozen without an error.
Skipped after dedup, with reasons in the report: the numpy/pyarrow/zarr
floors (already inside C-337 at :566 as its open residual), the
views-faoapi monitor observations (different repo's register), and the
four-year-old gh (resolved; the residual is the operator's machine).
Verified: 339->341 IDs, 43->45 open, 0+2+12+25+6 = 45, struck-through
109 unchanged, index 1723 < 8000, header 1723 <= 3500, all 13 register
guards plus the 7 changelog guards green, full suite exit 0,
validate_docs.sh exit 0.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…419) C-324 RESOLVED AS STALE. It described a live credential sitting in a server log. The token was revoked 2026-08-01 (GDL allows one per account, so issuing the replacement forced it), the leaking harvester was superseded when the server moved to v1.11.0, and the log itself no longer exists — destroyed by C-339. Every remediation the entry prescribed had happened. The register had gone stale in exactly the way it keeps warning about, and a reader would have acted on exposure that ended three days earlier. "FAILS GREEN" CLUSTER NAMED — eight open entries (C-317, C-331, C-336, C-337, C-338, C-339, C-340, C-341) that are symptoms of one root cause: mechanisms that report success while not doing the thing. Individually small; together they say this project's characteristic failure is silence, not error. The cluster carries a design rule rather than eight separate fixes — absence of an error is not evidence of success, so any new mechanism needs an answer to "how would I know if this silently did nothing?" before it ships. FIVE TRIGGERS REWRITTEN. C-332 was perpetual ("any change to these files") and now names the act that creates the exposure. C-339 stated a rule where a trigger belongs. C-70, C-72, C-333 were vague or passive-wait. FIVE DEMOTED to tech-debt backlog on the C-136 precedent: C-46, C-116, C-117, C-147, C-155 — mechanical, single-file, never fired, loud rather than silent if they ever do. C-70 and C-72 were on the demotion list and were KEPT: their triggers had just been made concrete, and demoting an entry immediately after making it actionable is incoherent. Blind spots reported, not registered (skill rule): bus factor — one person holds every credential, all server access, and the operational knowledge, unnamed across 341 entries; upstream source discontinuation; whether a past release's grid can still be rebuilt. Counts: 45 -> 39 open, Tier 3 12->11, Tier 4 25->20, struck 110->115. METHOD NOTE, kept because it is evidence rather than an anecdote. The cluster table first used `| C-317 |` as its leading column, which collides with the summary-row format, and the repair used `replace(..., 1)` — which hit the FIRST match and silently bolded eight real summary rows. Two edits, both plausible, both wrong, neither raising an error. The guards caught it. Writing the fails-green cluster produced two instances of failing green. Verified: full suite exit 0, validate_docs.sh exit 0, all register and changelog guards green, 39 open rows == header 39. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* test: a loose dependency floor freezes the version (C-337, #422) `uv lock` keeps an existing pin while it still satisfies the constraint, so `>=1.0` does not merely permit 1.0.0 — it freezes it, and nothing in the normal workflow moves it. views-frames sat at 1.0.0 from June until 2026-08-02 that way, across three MINOR releases that changed how MAP and HDI are computed. Two guards: no runtime dependency locked exactly at its own floor, and no ALLOWED_AT_FLOOR exemption that has stopped applying. The second exists because an exemption whose reason has expired is a guard silently switched off. Nothing is frozen today. All 13 runtime floors were checked during epic planning and views-frames is the only one at its floor — legitimately, because that floor IS the latest release. So this is a regression guard, not a remediation. Two further guards were written and then deleted: "every dependency is in the lock" and "no locked version is below its floor". Drilling them showed uv refuses to resolve, or refuses to parse, before pytest starts — they cannot fail. A test that cannot fail is the defect this file guards against, so it does not ship. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(register): C-342 — a stale uv.lock is invisible, and the entry was wrong first (#430) /code-review and /review-diff on the C-337 guard. One concern registered, two docstring defects fixed, and one correction that belongs on the record. C-342 (Tier 3): `uv sync` rewrites uv.lock in place whenever pyproject has moved, and every CI job that needs Python runs it first — so a PR whose committed lock disagrees with its committed pyproject goes green. CI repairs the lock in its own checkout, tests the repaired version, and throws it away. The stale lock stays in git. Verified by doing it, not inferred. Ninth entry in the fails-green cluster. Deliberately not fixed here. The instrument is `uv lock --check` in CI; a pytest cannot see this, because the lock is already repaired by the time pytest runs. Proposed for #424. The entry was also wrong when first written — it said "every job ... all four jobs" when ci.yml has five, `docs` needs no uv, and import-enforcement is gated to main. Inferred from four grep hits rather than read: C-336's own mistake, inside an entry about a sibling of it. Caught by /review-diff the same hour, corrected, and written up in the changelog rather than quietly amended. Test docstring: the `packaging` availability argument now rests on pytest, which cannot lapse, instead of on matplotlib, which pyproject marks for removal (C-334). And an explicit note that emptying ALLOWED_AT_FLOOR makes its rot-check vacuous *correctly* — an empty allow-list is the strongest state, not a hole — so nobody "fixes" it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…drill (#431) * docs(register): C-343 — writing the deploy tag is not deploying (Tier 2) Observed on the production host, not read off the repo. The tag file said v1.11.0, the working tree was at v1.10.0, views-frames was still the frozen 1.0.0, and `git fetch --tags` reported v1.11.0 as a NEW tag. Three sources of truth disagreeing for five days with no error anywhere — and the release in question had been cut for the sole purpose of raising that views-frames floor. Two reasons the in-script `git checkout` does not rescue this, tested in a throwaway repo at 31 KB rather than reasoned about: 1. bash buffers the script and never re-reads it, so a change to refresh_pipeline.sh itself lands one run later — a month, on a monthly cron. 2. `uv sync` is never invoked by the pipeline at all, so a dependency change never lands however often it runs. Tier 2 with the justification stated: not maintainability, but the server running code nobody believes it is running, with the status page green and the heartbeat pinging. It does not corrupt data; it silently prevents fixes from landing. The trigger fires at the next release, and #423 ships a refresh_pipeline.sh change — exactly the class that lags. Blast radius on this occasion was small and the entry says so: no src/ file and no pipeline script differs between the two tags, production imports no views_frames estimator, and the PyPI wheel carried the right floor throughout. The damage was to what we could claim. Remediated live: uv sync moved views-frames 1.0.0 -> 1.10.2 and views-datafactory 1.10.0 -> 1.11.0, and FeatureFrame was imported on the host afterwards. Entry stays OPEN — nothing prevents recurrence. The instrument (pre-flight tag-vs-HEAD check, fail-loud per ADR-011) goes to #424, not here. Also fixes the contradiction that caused it: the script header said deploying was one step. It now says it is three, and why. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(register): C-317 RESOLVED by live drill — a dangling /start does alert (#427) Epic #421 Story 6. The first entry in the fails-green cluster closed by observation rather than by argument. PR #359 added a /start ping on the theory that healthchecks.io would flag a run that began and never finished — the OOM-kill case, where SIGKILL bypasses both the ERR and EXIT traps so neither the success nor the /fail ping ever fires. Nobody had watched that happen; the entry has read "OPEN pending live grace-timeout drill" since July. Closing it because the ping is sent would have been the cluster's own mistake: the ping firing was never in doubt. Drill: throwaway check at 5 min period / 1 min grace so the timeout was observable in a minute instead of 30 days + 48 hours. Production HEARTBEAT_URL and the production check untouched. One /start, then nothing. Prediction recorded before the ping. Observed: "is DOWN (success signal did not arrive on time, grace time passed)", Last Ping Type: Started, 03:35:56 +0200. Detection latency for an OOM kill drops from ~32 days to the grace window. Throwaway deleted. Two things found that nobody was looking for. healthchecks.io's schedule dialog documents the mechanism outright — "Grace Time: when a check is late, OR has received a 'start' signal" — so it was vendor-documented all along and simply never read. And the sample check reused for the drill had sat grey, never red, for two months while permanently overdue: a check that has never been pinged does not alert, so a monitor created and never wired up is indistinguishable from a healthy one. Not closed by this: the status page is still not regenerated on SIGKILL (C-338's territory), and detection is not prevention (C-173). Three header counts were wrong across three edits — resolved total, summary-vs-entry 1:1, and struck-through. All three were caught by the register guards, not by me. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(register): address review findings — four claims that were wrong (#431) /code-review medium and /review-diff on this PR, all findings fixed here. Two stale counts, both already stale before this PR and made worse by it: the fails-green cluster prose said "three of the eight" and "cheaper than eight separate ones" while the table held nine, then ten. That is C-336 happening inside the cluster section about C-336. The tally is now qualitative on purpose — a number no guard protects will go stale again — and C-336 gains the location, rather than a new concern ID. One false claim of my own: C-343 said `git diff v1.10.0..v1.11.0` touches "only pyproject.toml, uv.lock and three GitHub workflows". It spans 31 files. I had filtered out docs/reports/tests and then reported the filtered result as the whole. The load-bearing part — no src/ file, no pipeline script — is true; the framing was not. Corrected in both the register and the changelog, in a paragraph whose subject is checking before believing. One self-invalidating anchor: the entry cited refresh_pipeline.sh at 11,592 bytes, which this same PR grows to 12,385. Now anchored to v1.11.0 explicitly, so the number cannot rot. And two from /review-diff on the shell comment. It claimed "the server always runs a specific, tested version" eight lines above explaining that this is the belief C-343 disproves — now scoped to the Python, with a pointer forward. And it restated the three deploy commands that already live in server_quickref.md, which is a second copy of the procedure whose drift IS C-343. The restatement is gone; precedence is stated instead: if the two ever disagree, the quickref wins. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: the heartbeat URL leaves the command line (C-331, #423) HEARTBEAT_URL is a capability: whoever holds it can forge a success ping and silence the dead-man alert permanently. It was an argv element at three curl sites, and /proc/<pid>/cmdline is world-readable while four accounts hold shells on the box. All three now pass it on stdin as a curl config file. C-331's own prescribed fix was the bug. The entry said `printf 'url=%s\n'` — unquoted. Measured before shipping it, with a value carrying a stray space: url=%s -> parses http://h/uuid, DROPS the /fail, sends anyway url = "%s" -> exit 3, nothing sent A trailing space or CR would have turned the failure ping into a success ping, silently. The register's own remedy for a fails-green concern carried a fails-green defect, unexamined since July because a prescribed fix reads like a settled thing. Shipped form is quoted; the reason is in the script so it survives a rewrite. The /proc claim is drilled with a negative control, since a clean scan otherwise only proves the scanner is broken. Control leaked the canary from curl's argv; the fix showed `curl -fsS --max-time 20 -K -` in flight with nothing anywhere carrying it, three times. A local listener confirmed all three paths arrive byte-exact — re-run against the three lines extracted verbatim from the committed file, not retyped. The guard was wrong first. It asserted "no line contains both curl and HEARTBEAT_URL", which fails against the FIXED script, because `printf ... "$HEARTBEAT_URL" | curl ...` puts both on one line. The property is "never AFTER the curl token". Drilling it is what found that; it now fails on the pre-change file and passes on this one. Three stale line citations fixed and the hole that hid them closed: ADR-018 had drifted ~55 lines, ADR-051 cited one number that was never right, and C-331's own Location read 93,163,290 against 112/182/309 — in an entry whose trigger was "next edit to this file". test_docs_citations could see none of them: its pattern was `.py` only. Widened to `.py|.sh`, drilled, and those three were the only offenders. Also: server_operations said "two signals" while listing three, and 24h grace where the live check is 48h; its two operator verification commands used the argv form, i.e. ran this exact exposure on the box. Not live on the server. #423 said no server change was needed; per C-343 the change lands one run after a deploy, so worst case it is in production two months from merge. Residual left open: HEARTBEAT_URL still sits in ~/.profile, and if that is not mode 600 it is readable by all four accounts permanently. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * security: C-344 — views-deploy's ~/.profile was world-readable (Tier 2) Chasing C-331's residual instead of writing it down as a caveat. The operator ran three commands on the host: stat -c '%a %n' /home/views-deploy/.profile 644 stat -c '%a %n' /home/views-deploy 751 test -r /home/views-deploy/.profile READABLE BY simmaa_prio 751 on the home lets any account traverse in; 644 on the file then lets it read. Confirmed by direct test from a second account, not inferred from permission bits. Exposed: UCDP_API_TOKEN, ACLED_USERNAME, ACLED_PASSWORD, GDL_API_TOKEN, HEARTBEAT_URL — to dylpin, simmaa_prio and sonja_prio, continuously, from deployment until today. Not a window; a standing condition. A Tier 4 story about a 10-second exposure surfaced a Tier 2 one about a permanent exposure. Fixed with chmod 600 and verified in BOTH directions: no longer readable by the other account, still readable by the owner with the pipeline path intact. A permission fix that also breaks the pipeline is not a fix. Swept the rest of the home rather than assuming. .netrc was already 600 — the data-server credentials were never exposed. .ssh 700, all history files 600. One hypothesis raised and killed: .local is 775 and refresh_pipeline.sh prepends $HOME/.local/bin to PATH, which would have been code execution as views-deploy rather than mere disclosure — but `getent group views-deploy` returns no other members. Both setup guides caused this: each showed `>> ~/.profile` with no chmod, so the mode was whatever umask gave it. Both now carry the chmod and a verification line. Left open on purpose: whether the four credentials need rotating. The readers are trusted colleagues, so this is a policy judgement rather than a breach — but C-322's GDL token was rotated on weaker evidence. The operator's call, deliberately unmade. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(register): C-344 rotation declined — record the decision and its basis The operator's call: no rotation. The three accounts belong to known colleagues and there is no indication any read the file. Recorded with the basis, because the basis is what can expire. This rests on WHO HOLDS THE ACCOUNTS, not on evidence of non-access — no audit record exists that could establish the latter and none was consulted. Reasonable on a single-team research host; it would not survive the accounts being held outside the team. Revisit trigger named: a new shell account (C-88). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test: harden the heartbeat guard — it was wrong three times (#432) /code-review medium and /review-diff on this PR. The shell change came back clean; the guard did not. The reviewer found a false negative: route the secret through an intermediate variable — FAIL_URL="$HEARTBEAT_URL/fail"; curl -fsS "$FAIL_URL" — and C-331 is fully reintroduced with the test still green. Not contrived: that is exactly what collapsing the three WET ping lines into a helper would produce, and the file's own docstring anticipates a future rewrite. The fix for that did not work either. Assignment-tracking was added; the drill returned rc=0 where it had to return rc=1, because the regex anchored at ^ with no allowance for indentation and every assignment in that script is indented. Only running it against a state it was not written for exposed that. Three versions, three claims narrower than the property: v1 "line contains both curl and HEARTBEAT_URL" -> failed on the FIXED script (printf ... | curl ... has both on one line) v2 "literal HEARTBEAT_URL after curl" -> passed a two-line reintroduction of the defect; caught by review, not the author v3 assignment-tracking to a fixpoint -> matched no indented assignment; caught by drill, not by reading v4 drills clean four ways: clean tree passes, direct argv fails, one indirection fails, two indirections fail. Also from /review-diff, both mine: the docstring still described v2's property, and bare substring matching would over-match a one-character tainted name (the two-level drill produces `A`). Word boundaries now. /register-risk: 0 new, 1 merged. This is C-336's mechanism a third time — a claim narrower than the property, failing green — so it is a second addendum there rather than a new ID. The generalisation: a guard is a claim about a property, and only running it against states it was not written for tells the two apart. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…hat was red (#433) * docs(register): C-345 — the instrument that detects this cluster is a member of it Tier 2. Twice in one session a failing suite was reported as passing. uv run pytest -q | tail -2; echo "EXIT=$?" A pipeline's exit status is its LAST element's, so tail returned 0 while pytest had exited 1. Then a backgrounded run's task notification said "exit code 0" for the same reason — the command ended in an echo. The second was caught only by reading the output file, after the user had already been told the suite was running and would be folded in. One step from reporting green on red. Registered rather than remembered because this project has two recorded false-readiness incidents that each cost a full day; because the suite is the gate on every story in this epic; and because C-343 means a defect that slips through can sit in production for two months. C-339 is the precedent for registering a workflow hazard rather than a code one. The shape is the cluster's own, which is the uncomfortable part. C-330 was a nightly no-op exiting 0. C-337 was a lockfile frozen with no error. C-343 was a deploy that deployed nothing. This is the same defect in the instrument used to find all three. Mitigation adopted and explicitly NOT a control: redirect to a file, capture $? unpiped, grep ^FAILED as a second independent reader. A habit is not machinery. The instrument — refusing to report a result not obtained unpiped — is proposed for #424. Also carries the changelog note on what was NOT registered: the unbounded poll loop (one-line fix, no consequence) and the group-writable dotfiles (group has no other members, already inside C-344 as a killed hypothesis). The guard-narrower-than-the-property pattern went to C-336 as a second addendum during #432. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(register): two false claims in C-345, both mine (#433) /code-review medium fact-checked the entry against the repository rather than against memory, and found two claims that do not survive it. 1. "up to two months in production", attributed to C-343. C-343 says ONE month and never derives the larger figure — the two-month version requires an unstated compounding worst case. Present in TWO places: C-345, and inside C-331 where it had already shipped in #432. Both corrected, in the register and the changelog, with a visible correction note rather than a silent edit. 2. "two recorded false-readiness incidents that each cost a full day." A grep of the register, the changelog and every post-mortem returns only the sentence making the claim. The incidents are real and known to the operator, but THIS REPOSITORY RECORDS NEITHER. Citing evidence a reader cannot find is how a register stops being checkable, which is C-336's subject. Removed rather than softened, and the fact that they went unrecorded is now flagged for the next post-mortem. C-345's tier now rests on the one basis that is verifiable in the repo: it fired twice in one session. That is stated as deliberately the only evidence cited. Also from the review, verified and left alone: the shell claim is exactly right (pipefail changes it, the mitigation works), the header arithmetic is correct and guarded, the cluster section's counts match its eleven rows and nine open members, and every cross-reference points at an entry in the state implied. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
#424) (#435) * ci: run the deploy gates somewhere other than one laptop (C-341, C-342, #424) The story shipped smaller than its issue described, and measuring is what showed why. #424 claimed four gates would start running once the workflow got a step and a token. Simulating a runner checkout first: - actions/checkout leaves exactly ONE local branch, so the gates' bare `git merge-base --is-ancestor main development` exits 128 and they skip themselves — even at fetch-depth: 0, which the workflow already had. Two `git branch -f` lines fix it. Without them the step would have run, passed, and asserted nothing. - The local-clone branch gate would have PASSED TRIVIALLY on a runner rather than skipped: a fresh runner has one local branch, so it reports coverage it does not have. It now skips with a reason. - TestF1VersionBumped is xfail. It cannot fail a suite anywhere, so scheduling it would add a green tick and no information. So of the four named: one redundant, one unblocked, one demoted to an honest skip, one impossible until #425. C-341 is NARROWED, not closed — closing it would claim a coverage the xfail denies. The gain the issue never mentioned: the conflict-free back-merge check, which nothing was running anywhere, now runs. C-342 resolved: `uv lock --check` in the test job, before `uv sync`. Drilled clean/dirty/clean. It is the one merge-blocking check added and it differs in kind from the deploy gates — those redden for reasons unrelated to the change (a diverged main is CORRECT between a release and its back-merge, which is C-320); this reddens only when the PR itself left the lock stale. Stated in the workflow comment so nobody re-derives it. Confirmed on its own change: this PR adds pyyaml for the new guard, and the check agreed after re-locking. Two near-misses, both recorded rather than quietly fixed: A PARSE IS NOT A VERIFICATION. The lock-check step was first inserted between `- name: Install dependencies` and its `run:`, producing a duplicate `run:` key. yaml.safe_load reported "YAML parses" — PyYAML silently keeps the last duplicate — so the check had been overwritten by `uv sync` and would never have run. Twenty minutes after registering C-345. tests/test_ci_gates.py now asserts step well-formedness. AND THE DRILL OF THAT GUARD WAS WRONG FIRST. Its criterion was rc != 0, which cannot tell "guard caught it" from "the file failed to import" — and it was failing to import, because pyyaml was not a dependency. All four mutations reported CAUGHT while nothing ran. Re-drilled with rc == 1 meaning caught and rc >= 2 meaning error: five mutations, five genuine catches. Workflow display name widened to "Release hygiene"; filename kept so run history and existing citations stay valid. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * ci: harden the output delimiter and scope the C-341 table (#435 review) /code-review medium found no defect in the workflows. It independently verified the two things most likely to be silently wrong, and both were already right: the gateresult step reads `steps.gates.outcome` and NOT `conclusion` (with continue-on-error, conclusion is forced to 'success' whatever pytest did — one field name from a gate that always reports clean); and `set -o pipefail` inside a `run:` block does take effect, which matters because Actions' default shell is `bash -e {0}` without pipefail, so `pytest | tee` would otherwise report tee's status and mask every failure. That is C-345's shape inside the step registering C-345. It also confirmed adversarial content cannot escape the issue-body heredocs — backticks and $(...) in pytest output are substituted literally — and that the legacy issue title in the close step matches byte-for-byte including the em-dash. One hardening taken: the $GITHUB_OUTPUT delimiter was a fixed string. `git log --oneline` cannot produce a bare "EOF" line, but pytest output carries no such guarantee, so the gate log now uses a random delimiter. /review-diff: C-341's residue table read as exhaustive while omitting TestF7ProductPlanCurrency and TestDF2ReleaseGuideCompleteness. They are absent deliberately — file-only, already in the PR suite — but omission does not say so. Now stated. Method note: my own check of the hardening asserted `"conclusion" not in run` and failed on its own explanatory comment about why not to use conclusion. Tightened to the expression `steps.gates.conclusion`. The guard-narrower-than-the-property shape again, at miniature scale. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
) (#436) * test: the version gate could not fail, so it is gone (C-341, C-346, #425) Closes #363. "Delete it" was not one deletion. TestF1VersionBumped asserted "the current version is not already tagged". Version here is bumped only at release time, so from the moment a release lands until the next bump the version IS a tag that exists — the entire inter-release period. Measured in both states rather than argued: version tagged (steady state) XFAIL exit 0 version untagged (just bumped) XPASS exit 0 Green either way. It asked "have you bumped yet?", which repo state cannot answer, because "about to release" is not knowable from the repo — only from the tag that triggers a release. Deleted, and replaced by the two halves that ARE answerable: TestVersionMatchesItsTag (if HEAD is on a tag, the version must equal it) and an unskippable guard in publish_package.yml comparing github.ref_name to the version before the build. The new test was drilled to a genuine rc=1 failure in an isolated worktree at v1.11.0; its predecessor could not fail in any state. C-341's last residue is gone, so C-341 resolves. Grepping for the thing being deleted found FOUR MORE copies, in three other deploy suites. They use a conditional xfail that reads as more rigorous and is worse: it runs only when the version is untagged, then asserts the version is untagged — asserting the condition that selected it. Registered as C-346 and left in place; removing four classes plus the meta-test that ENFORCES the marker is its own change, and the replacement now exists. The publish guard had a real bug the drill caught: it read the version with bare `python3 -c "import tomllib"`. tomllib needs >= 3.11, local python3 is 3.10, and the runner's is whatever the image ships. Pinned to `uv run --no-project --python 3.12`, matching the guard beside it. That one would have surfaced at a release, against an immutable tag. Two of my own drills were wrong before they were right, the same way both times. `git checkout v1.11.0` was silently refused because of uncommitted changes, so the "want FAIL" case ran off-tag, skipped, and returned rc=0. Then pytest was invoked from the main repo while the test shells out to `git describe` in the CURRENT directory — still the wrong tree. Fourth and fifth instances this week of a drill whose setup did not happen while the exit code looked fine. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test: the replacement had a fails-green of its own (#436 review) /code-review medium on this PR found a defect in the test written to close one — and in the only direction that matters. _head_tag() used `git describe --tags --exact-match HEAD`. That returns EXACTLY ONE tag, the lexicographically smallest, and a commit can carry several. Reproduced in a throwaway repo: tags on HEAD: checkpoint v1.11.0 git describe --tags --exact-match HEAD -> checkpoint _head_tag() -> None -> the test SKIPS So a stray non-v tag on a release commit hides the release tag, and the test goes quiet precisely when a real mismatch is sitting on that commit. Switched to `git tag --points-at`, which lists all of them, and drilled the multi-tag case that had no drill before: A. single tag, version matches rc=0 B. multi-tag (checkpoint + v1.11.0), match rc=0 C. multi-tag, MISMATCH rc=1, names the v* tag Where it was NOT a problem is the useful part: publish_package.yml compares github.ref_name directly and never shells out to git describe, so the unskippable half was immune. The two halves failing differently is the argument for having both — which I would have asserted without evidence if the review had not produced it. Also verified by the review and left alone: `set -euo pipefail` does abort the guard when `uv run` fails, so it never compares against an empty string; the guard runs after setup-uv; and the deploy_v130 meta-test still runs non-vacuously after the deletion — it greps and checks each surviving sibling for real, confirmed by running it in a state where its early-return does not short-circuit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…kipped (C-340, #426) (#437) Two mechanisms, two different fixes, because they fail differently. MECHANISM 2 gets a git hook. #416 merged the instant CI went green; a follow-up commit was pushed to that branch, which by then had no open PR, and two pieces of work were simply not on development. `git push` reported success. scripts/git-hooks/pre-push refuses that push, names the PR and when it merged, and says what to do instead. It is a hook rather than a checklist item deliberately. Everything else in this epic runs whether or not anyone remembers; a step someone has to recall would be the same class of defect the epic exists to remove. It ALLOWS the push whenever it cannot answer — gh absent, unauthenticated, offline — and says why. A hook that blocks work when it does not know gets uninstalled within a day and then guards nothing. Same idiom as the deploy gates (C-320): skip where the environment cannot answer. MECHANISM 1 gets a helper that reads the value back. `gh pr merge --auto --<method>` on an already-armed PR silently refuses to change the method — prints nothing, exits 0. scripts/arm_automerge.sh uses the GraphQL disable/enable pair, which does honour it, then reads auto_merge.merge_method back and exits non-zero on mismatch. Reading back is the point; arming is the easy part. Seven drills, all against the real repository: merged PR (#436) REFUSE exit 1, names the PR and merge time branch with no PR ALLOW exit 0 development ALLOW exit 0 gh absent from PATH ALLOW exit 0 with the reason gh unauthenticated ALLOW exit 0 with the reason arm on a MERGED pr exit 1, refuses to mutate arm with bad arguments exit 2 The gh-absent drill was wrong first: PATH=/usr/bin:/bin still found gh at ~/.local/bin, so it tested nothing. Redone with a sandbox PATH containing git and bash but not gh, with the absence verified before the run. tests/test_git_hooks.py guards the three things that fail silently: the executable bit (git ignores a non-executable hook with no warning), the fail-open paths, and the documented install step. Drilled by breaking each — three mutations, three catches. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…y (C-351) (#440) Ten runs, ten failures, from the day the workflow was added on 2026-08-03. Every one died the same way: failed to run git: fatal: not a git repository (or any of the parent directories): .git `gh` infers the repository from the git remote. This job has no `actions/checkout` — deliberately, and correctly: it fetches one URL and reads it, and never touches the repo. So every `gh issue` call failed, and because that is a hard error rather than an empty result, it took the whole run with it. The freshness check for served data — the half of ADR-051 §2 that Better Stack's free tier cannot do — therefore did not work for its entire existence. It was found by a code review aimed at something else, nine days in. WHY GH_REPO AND NOT actions/checkout Both silence the error. `actions/checkout` does it by giving `gh` a remote to infer from, which means cloning a repository this job has no use for to satisfy an inference step. GH_REPO supplies the input that was actually missing. NOT A REGRESSION, WHICH CHANGES WHICH CLUSTER IT BELONGS TO It shipped broken and was never verified — C-350's sibling, not C-338's. A dispatch after writing it would have caught this on day one, which is C-347's rule applied to the workflow that most needed it. GUARD tests/test_workflow_gh_context.py checks every workflow step that calls `gh` has either a checkout or GH_REPO, and carries a counterweight so it cannot pass by matching no `gh` at all. Drilled: removing GH_REPO reddens it, naming serving-freshness.yml:freshness:Open. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…340) (#441) `development` has been carrying a pre-push hook that does not work, and `publishing_to_pypi.md` told you to install it — twice. Anyone running `git config core.hooksPath scripts/git-hooks` got a hook that checks the WRONG BRANCH (it read `git rev-parse HEAD` instead of the refs git supplies on stdin) and permanently refuses branch names reused from old pull requests. That is a trap, and it shipped in #437. Four hook versions and three detector versions were written to close C-340 mechanism 2; each was defeated by a different property of git, gh or GitHub. The harm being guarded against is two orphaned commits in ~440 pull requests, both recovered by cherry-pick. The guards cost far more than the failures, so there is no guard — and the guide now says so plainly instead of pointing at one. removed scripts/git-hooks/pre-push removed tests/test_git_hooks.py (vacuous: reconstructing the broken v1 hook passed all seven of its behavioural tests) guide install instructions replaced by what to do if it happens Mechanism 1 is unaffected: scripts/arm_automerge.sh stays, works, and was drilled live. Closes the last of epic #421, which is closed as a whole. validate_docs PASSED | ruff clean | PYTEST_EXIT=0 | FAILED=0 Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…443) (#444) * chore: lower requires-python to >=3.11 — the platform floor outlier (#443) views-datafactory was the only repo on the platform above 3.11, and the views-models conda envs run 3.11.14/3.11.15, so 28 requirements files could not install it at all. ADR-030 set >=3.12 because "tifffile's current releases require it" — a fact about a vendor at a moment, not an architectural need. 3.11 was never considered there; the string does not appear in the ADR. Verified at both ends, not one: pytest exit 0 on 3.11.13 and 3.13.7, ruff and mypy clean on both, same six pre-existing xfails. THE COST IS REAL. uv.lock now forks: 3.11 resolves tifffile 2026.3.3 / imagecodecs 2026.3.6, >=3.12 keeps 2026.5.15 / 2026.5.10. Both upstreams dropped 3.11 permanently. The required `test` job pins the floor, so it decodes rasters with a different codec build than the server — registered as C-347, with `test-py313` covering the production line until it can be made a required check. No strategy.matrix: it renames the required `test` context and deadlocks every PR under enforce_admins. Three things this turned up that were not predicted: 1. test_dependency_floors._locked_versions collapsed duplicate package names via a dict comprehension. That path was flagged in review on #430, scored 25, and CORRECTLY refuted — on the premise that this project had a single requires-python and no markers. This change deletes that premise. A refutation is only as durable as its premise, and nothing was watching the premise. Drilled: on the identical forked lock the new parser fails naming both resolutions and the old one passes. 2. No test in this repository had ever written a compressed TIFF. imagecodecs decodes 100% of production GHS-POP/GHS-BUILT-S rasters, is imported by nothing, and blocking it raises KeyError "<COMPRESSION .LZW: 5> requires the 'imagecodecs' package" while every existing test keeps passing. An import-graph audit would have called it removable. Now covered. 3. Pinning mypy to the floor made it parse tifffile at 3.11 too, so it died on tifffile's own PEP 695 syntax under 3.13 while passing under 3.11. Then the fix for that was written inside [tool.mypy], where a [[overrides]] header silently captured disallow_any_generics and took mypy from clean to 98 errors. The TOML parsed and meant something else. ADR-030 amended, not superseded — the tooling decision is untouched. The Hetzner guide's "Install Python 3.10+" is rewritten: it produced an uninstallable environment for three months, and that number now selects which raster decoder the server installs. C-348 records that nothing asserts the server's interpreter. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: address the review of the 3.11 floor — eight findings (#443, #444) /code-review found seven, /review-diff one more. Three were defects in the change's own reasoning, not in the code it touched. REVERTED python_version = "3.11" from mypy. It was added so the declared floor would be checked at the keyboard, but mypy then parses third-party source at 3.11 too and dies on tifffile's PEP 695 under 3.13. The escape — follow_imports = "skip" for tifffile — silently deleted type checking of read_geotiff, the ONLY tifffile call site in this repo. Drilled both ways: with the skip, page.asarray(maxworkerz=1) type-checks clean; without it, mypy names the typo. A convenience CI already provides is not worth a real check. Reverting also removed the [[overrides]] table that, written mid-section, had captured disallow_any_generics and taken mypy from clean to 98 errors. The CI-pin guard overclaimed. Its docstring and the PR body said it would have caught the guide-vs-pyproject drift; checked against 12d5afa, it would have been GREEN the whole window, because CI and pyproject agreed and only the prose disagreed. Docstring rewritten to say what it does not cover; the residual gap is registered as C-349. The guard was also too narrow and then too broad. Widened from ci.yml to every workflow plus a --python flag check, since publish_package.yml builds the artefact consumers actually get. That version then reddened on a legitimate --python 3.13 inside test-py313 and on a version named in a comment — C-320, introduced into a change citing C-320 three times. Now parses YAML, exempts the off-floor job, strips comments; re-drilled all three ways. ADR-030's Implementation Notes still said "Bump requires-python to >=3.12" and Open Question 4 was still open, both after the amendment. An implementer reading only that section would have reversed this change. Inline superseded flags added rather than editing the original. Also: uv sync --frozen was a false guarantee in test-py313, since bare uv run re-locks by default — now uv run --frozen. The deployment guide stopped installing python3 while three later steps still invoked it — now uv run python. And the non-floor coverage guard accepted import-enforcement's single-file pytest — now requires a bare full suite. pytest exit 0, ruff clean, mypy clean at 3.11 and 3.13, validate_docs PASSED, register 349 IDs / 42 open / tier sum 42. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
Author
|
Closing in favour of the v1.12.0 release promotion, which does the same job with an accurate description. This PR is not wrong, it has drifted. It was opened to get workflow fixes onto What went stale while it sat open:
The last one is why I am closing rather than editing. A promotion PR whose description has quietly diverged from its contents is the exact artifact this week keeps finding — and correcting the prose in place would leave the CI history attached to a body that once said something else. Nothing is lost: every commit here is already on |
This was referenced Aug 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a promotion, not a release. No version bump, no tag, no GitHub Release — and therefore no PyPI publish:
publish_package.ymltriggers only onrelease: published, verified.Why now
Scheduled workflows run the default branch's copy of the file, and the default branch is
main. Several fixes ondevelopmentare workflow-shaped and are therefore inert until they reachmain:maindevelopmentserving-freshness.ymlGH_REPOserving-freshness.ymlhas failed every one of its runs since 2026-08-03 —ghcould not infer the repository in a job with no checkout. #440 fixed it, but the 07:00 cron keeps runningmain's broken copy until this lands. Same applies to the deploy gates added in #435.What is in it — 14 commits, zero
src/changesviews-framesat 1.0.0, so CI had been testing pre-amendment MAP/HDI semantics since JuneHEARTBEAT_URLoff the curl command line — capability URL,/proc/<pid>/cmdlineis world-readablepytestserving-freshness.yml— first successful run in its existenceNo product code changed, so no server redeploy is required —
refresh_pipeline.shis untouched by this promotion.How this must be merged
As a MERGE COMMIT, never a squash. A squash onto
mainrewrites the release SHAs and permanently breaks the ancestry the back-merge maintains. During v1.10.0 a promotion was armedsquash, re-armed with--merge, and stayedsquash—gh pr merge --auto --<method>silently refuses to change the method on an already-armed PR and exits 0 (C-340 mechanism 1).Arm it with the script that reads the method back:
Required follow-up — do not skip
Back-merge
mainintodevelopmentafterwards, also as a merge commit. Without itmainis not an ancestor ofdevelopmentand the topology gate reddens:gh pr create --base development --head main --title "chore: sync main back into development"🤖 Generated with Claude Code