Skip to content

Scope developer.md's three trial measurements to this repository, not every one (#369) - #370

Merged
fdaviddpt merged 2 commits into
mainfrom
fix/367
Aug 20, 2026
Merged

Scope developer.md's three trial measurements to this repository, not every one (#369)#370
fdaviddpt merged 2 commits into
mainfrom
fix/367

Conversation

@fdaviddpt

@fdaviddpt fdaviddpt commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

agents/developer.md is plugin content loaded verbatim by every developer lane in every repository this plugin manages, on every machine that installs the artifact. It carried three measurements from this repository's own trial -- a 27m36s wall clock, a narration-turn-count claim, and a single-op-share claim of 82% -- with no label saying they came from this one repository's history rather than being generic facts every installation should expect.

skills/manager/SKILL.md already carried the correct pattern for a similar number (the developer's default model): "a maintainer decision, not a fact about every repository this plugin manages, so the priced evidence it rests on lives in this project's own history (#316) rather than repeated here as a number a different installation would read as generic guidance." That fix landed in the same commit as the three unlabelled passages in agents/developer.md -- applied to one shared document and not the other, one paragraph below a sentence in agents/developer.md itself ("Three lanes on this repository ...") that already showed the labelled shape.

This change:

Self-review (Explore + oss:auditor, both run against the committed diff) found two real issues, both fixed in a second commit: the label check only graded the first regex match per anchor via re.search, so a repeated unlabelled mention of the same figure later in the document would pass silently -- switched to re.finditer over all matches, with a regression test; and the wall-clock paragraph copied SKILL.md's "priced evidence" wording verbatim, which fits a pricing decision but not a stopwatch measurement -- changed to "measured evidence" in that one paragraph.

Closes #369.

Test plan

  • python3 -m pytest tests/test_content_invariants.py -q --no-cov -- red before the fix (3 unmet anchors), green after (156 passed)
  • python3 scripts/assemble_changelog.py --check --dir changelog.d --changelog CHANGELOG.md -- ok, 15 fragments

Corrected by the maintainer before publishing: the closing reference

This body originally read Closes #367, and #367 is somebody else's open issue — an unrelated
request for doctor to report the interpreter architecture and the worker count -n auto will ask
for. Merging as written would have closed it.

The mistake is mine, not the lane's. The finding came out of a release audit and I delegated it
without filing an issue first, naming the branch fix/367 from the next free number I assumed
was free. It was not. The lane inferred the closing reference from the branch name, which is the only
signal it had.

Filed properly as #369 and the reference corrected here. Recorded rather than quietly fixed,
because it is the exact failure the merge gates warn about — a body that reads correctly, a reference
GitHub would have bound to the wrong issue, and a board that reads clean afterwards. It was caught by
reading gh-issue:367 before publishing rather than by any check.

Two things follow that are worth more than the fix:

  • A branch name is not an issue number. Nothing in this loop verifies that fix/N corresponds to
    an existing issue N, or to this work's issue. branch_pattern is fix/{issue}, so the
    convention actively invites the inference the lane made.
  • gh-pr-create reports a body with no closing keyword, but it cannot report one that closes the
    wrong issue
    — the reference was well-formed and would have resolved. The op would have printed
    Issue: #367 and exited 0.

Verified by the maintainer

Independent red re-run, this branch's tests/test_content_invariants.py against origin/main
with the fix absent:

1 failed, 155 passed
assert not {'narration-turns-not-scoped', 'single-op-share-not-scoped', 'wall-clock-not-scoped'}

One assertion fires, names all three offenders individually, and 155 hold. Precise rather than broad,
inside a file whose whole job is invariants — a guard that reddened more of it would be the wrong
guard.

Dropping the 82% rather than replacing it is a better answer than the one I proposed. I told this
lane to re-derive the figure. It declined to carry any figure: the corrected measurement is a
per-model split rather than one combined percentage, and a number that has already drifted once will
drift again as lanes accumulate. So the instruction now points at the evidence instead of
transcribing a snapshot of it. That closes the defect at the class rather than at the instance —
the next wrong number cannot be introduced, because there is no number.

The re.searchre.finditer finding is the one that mattered in its own review. The first
version graded only the first match per anchor, so a repeated unlabelled mention of the same figure
later in the document would have passed. A guard that checks the first occurrence of a thing it
expects to occur once is this repository's own defect class pointed at the fix for this repository's
own defect class.

On the rank, since accepting a block deserves the same scrutiny as arguing one down. The auditor
supplied the argument against itself — tests/test_content_invariants.py:97 pins 27m36s
deliberately, so a reader could call this intended rather than leaked. I did not take that exit. The
deciding fact is that agents/developer.md is plugin content, so the tag is the mechanism by which
these sentences reach machines that have no idea whose suite took 27 minutes; that is ships-local-state
as the row is written, and the row blocks.

What this does not fix, stated so the next audit does not have to rediscover it. The guard added
here checks three named anchors. It does not express the general property — a number presented as
generic that is true of exactly one checkout
— and the lane was explicitly permitted to conclude
that the general form cannot be written without firing on every digit. Three anchors are three
anchors; the class stays open behind them.

…pository's own trial as facts about every repository this plugin manages (#367)

skills/manager/SKILL.md already carried the fix, one paragraph below a sentence in
agents/developer.md itself that showed the same labelled shape. Scope the wall clock, the
narration-turn count and the single-op share to this project's own history (#316), and
correct the single-op figure, which was a mis-parsed 82% combined across models rather
than the per-model split scripts/transcript_refusals.py actually measures.

tests/test_content_invariants.py grades whether each measurement anchor carries a scoping
label in its own paragraph, with a must-fire fixture on the pre-#367 wording and a
must-not-fire fixture built up one label at a time.

Co-Authored-By: Max <noreply>
… a copied phrase

The reviewer spawn found that _unlabelled_measurements graded only the first regex
match per anchor via re.search, so a document that scoped its first mention of a raw
measurement and repeated the same figure bare later on would pass silently. Switched
to re.finditer so every occurrence is graded, with a regression test reproducing the
exact shape.

It also flagged that the wall-clock paragraph copied SKILL.md's "priced evidence"
wording verbatim, which fits a pricing decision but not a stopwatch measurement.
Changed to "measured evidence" in that one paragraph.

Co-Authored-By: Max <noreply>
@fdaviddpt
fdaviddpt merged commit 34d1952 into main Aug 20, 2026
14 checks passed
@fdaviddpt
fdaviddpt deleted the fix/367 branch August 20, 2026 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant