Conversation
…pository's own trial as facts about every repository this plugin manages (#367) skills/manager/SKILL.md already carried the fix, one paragraph below a sentence in agents/developer.md itself that showed the same labelled shape. Scope the wall clock, the narration-turn count and the single-op share to this project's own history (#316), and correct the single-op figure, which was a mis-parsed 82% combined across models rather than the per-model split scripts/transcript_refusals.py actually measures. tests/test_content_invariants.py grades whether each measurement anchor carries a scoping label in its own paragraph, with a must-fire fixture on the pre-#367 wording and a must-not-fire fixture built up one label at a time. Co-Authored-By: Max <noreply>
… a copied phrase The reviewer spawn found that _unlabelled_measurements graded only the first regex match per anchor via re.search, so a document that scoped its first mention of a raw measurement and repeated the same figure bare later on would pass silently. Switched to re.finditer so every occurrence is graded, with a regression test reproducing the exact shape. It also flagged that the wall-clock paragraph copied SKILL.md's "priced evidence" wording verbatim, which fits a pricing decision but not a stopwatch measurement. Changed to "measured evidence" in that one paragraph. Co-Authored-By: Max <noreply>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
agents/developer.md is plugin content loaded verbatim by every developer lane in every repository this plugin manages, on every machine that installs the artifact. It carried three measurements from this repository's own trial -- a 27m36s wall clock, a narration-turn-count claim, and a single-op-share claim of 82% -- with no label saying they came from this one repository's history rather than being generic facts every installation should expect.
skills/manager/SKILL.md already carried the correct pattern for a similar number (the developer's default model): "a maintainer decision, not a fact about every repository this plugin manages, so the priced evidence it rests on lives in this project's own history (#316) rather than repeated here as a number a different installation would read as generic guidance." That fix landed in the same commit as the three unlabelled passages in agents/developer.md -- applied to one shared document and not the other, one paragraph below a sentence in agents/developer.md itself ("Three lanes on this repository ...") that already showed the labelled shape.
This change:
27m36sstring intact since an existing test pins it.scripts/transcript_refusals.py) is a per-model split, not one combined percentage, so the doc now points at the issue rather than repeating a number that has already drifted once and will keep drifting as more lanes run.27m36s,no tool call at all,single-op reads ran) carries a scoping label within its own paragraph, with a must-fire fixture on the pre-doctor: report the interpreter architecture and the worker count -n auto will actually ask for #367 wording and a must-not-fire fixture built up one label at a time.Self-review (Explore + oss:auditor, both run against the committed diff) found two real issues, both fixed in a second commit: the label check only graded the first regex match per anchor via
re.search, so a repeated unlabelled mention of the same figure later in the document would pass silently -- switched tore.finditerover all matches, with a regression test; and the wall-clock paragraph copied SKILL.md's "priced evidence" wording verbatim, which fits a pricing decision but not a stopwatch measurement -- changed to "measured evidence" in that one paragraph.Closes #369.
Test plan
python3 -m pytest tests/test_content_invariants.py -q --no-cov-- red before the fix (3 unmet anchors), green after (156 passed)python3 scripts/assemble_changelog.py --check --dir changelog.d --changelog CHANGELOG.md-- ok, 15 fragmentsCorrected by the maintainer before publishing: the closing reference
This body originally read
Closes #367, and #367 is somebody else's open issue — an unrelatedrequest for
doctorto report the interpreter architecture and the worker count-n autowill askfor. Merging as written would have closed it.
The mistake is mine, not the lane's. The finding came out of a release audit and I delegated it
without filing an issue first, naming the branch
fix/367from the next free number I assumedwas free. It was not. The lane inferred the closing reference from the branch name, which is the only
signal it had.
Filed properly as #369 and the reference corrected here. Recorded rather than quietly fixed,
because it is the exact failure the merge gates warn about — a body that reads correctly, a reference
GitHub would have bound to the wrong issue, and a board that reads clean afterwards. It was caught by
reading
gh-issue:367before publishing rather than by any check.Two things follow that are worth more than the fix:
fix/Ncorresponds toan existing issue N, or to this work's issue.
branch_patternisfix/{issue}, so theconvention actively invites the inference the lane made.
gh-pr-createreports a body with no closing keyword, but it cannot report one that closes thewrong issue — the reference was well-formed and would have resolved. The op would have printed
Issue: #367and exited 0.Verified by the maintainer
Independent red re-run, this branch's
tests/test_content_invariants.pyagainstorigin/mainwith the fix absent:
One assertion fires, names all three offenders individually, and 155 hold. Precise rather than broad,
inside a file whose whole job is invariants — a guard that reddened more of it would be the wrong
guard.
Dropping the 82% rather than replacing it is a better answer than the one I proposed. I told this
lane to re-derive the figure. It declined to carry any figure: the corrected measurement is a
per-model split rather than one combined percentage, and a number that has already drifted once will
drift again as lanes accumulate. So the instruction now points at the evidence instead of
transcribing a snapshot of it. That closes the defect at the class rather than at the instance —
the next wrong number cannot be introduced, because there is no number.
The
re.search→re.finditerfinding is the one that mattered in its own review. The firstversion graded only the first match per anchor, so a repeated unlabelled mention of the same figure
later in the document would have passed. A guard that checks the first occurrence of a thing it
expects to occur once is this repository's own defect class pointed at the fix for this repository's
own defect class.
On the rank, since accepting a block deserves the same scrutiny as arguing one down. The auditor
supplied the argument against itself —
tests/test_content_invariants.py:97pins27m36sdeliberately, so a reader could call this intended rather than leaked. I did not take that exit. The
deciding fact is that
agents/developer.mdis plugin content, so the tag is the mechanism by whichthese sentences reach machines that have no idea whose suite took 27 minutes; that is
ships-local-stateas the row is written, and the row blocks.
What this does not fix, stated so the next audit does not have to rediscover it. The guard added
here checks three named anchors. It does not express the general property — a number presented as
generic that is true of exactly one checkout — and the lane was explicitly permitted to conclude
that the general form cannot be written without firing on every digit. Three anchors are three
anchors; the class stays open behind them.