test-arg-flag-values.sh listed its scripts by hand, so a new tool was untested and nothing said so (#188)

[fdaviddpt]

Closes #188.

What was wrong

tests/test-arg-flag-values.sh sweeps every tool for the missing-flag-value hang of #114. Its header said a thing added later was covered without anyone having to remember it. That was true of flags — each script's own argument loop is parsed for arms carrying shift 2 — and false of scripts, which were four hand-written drive_script calls at the bottom of the file.

So a new tool under scripts/ was silently untested by the suite whose job is to sweep every tool, and untested in the way this repository is named after: nothing errored, the totals went up as assertions were added elsewhere, and the coverage that was lost appeared in no number the suite printed. #183 added the jit-doctor.sh line and a comment naming the gap; the gap itself stayed open.

What changed

The script list is now git ls-files -- scripts, the same enumeration tests/test-dogfood-entries.sh already ran for rule coverage.

Enumeration turns "not in the list" into "must be driven", so a tool that legitimately takes no flag arguments needs an answer of its own. classify_script() gives every tracked file one of five verdicts, read out of that file's own bytes, and each of the five prints a line naming the script:

Verdict	Meaning	Result
drive	the canonical loop, with shift 2 arms	every flag driven
no-flags	no loop, no sign of flag parsing anywhere	named PASS
boolean-flags-only	a loop, but no arm reaches for $2	named PASS
loop-no-flags	a loop reaching for $2, no shift 2 arm to read	FAIL
flags-elsewhere	no loop, but getopts / shift 2 / a dash case arm	FAIL

Today's tree: 4 driven, 6 with no flags.

There is no skip list, deliberately. The issue named the choice between an explicit skip list and a sweep that can tell "no flags to test" from "never ran". A skip list is the hand-written list again, one indirection further out, stale in the same silence — so the second was built. What it costs, stated rather than hidden: a tool parsing flags in a shape this suite cannot read is red, not skipped, and there is no opt-out short of changing the classifier.

The header comment no longer overstates itself. Both halves are read from the repository now, and it says which one was typed until when.

Proving it catches an unlisted script

Observing that the sweep passes on today's tree says nothing — today's tree is exactly what the old hand-written list already covered. So the enumeration and the classifier are driven on every run against a throwaway git repository holding five scripts this suite has never heard of, one per verdict. The enumeration finds them, the classifier sorts them, and the unlisted tool actually goes red for want of a positive control — paired with an assertion that the same fixture's refusal path passed on the same run, so the red cannot be a broken fixture reading as a working guard.

The sweep's own coverage is guarded two ways that fail differently: two floors (at least one drive and at least one no-flag verdict in the real tree) catch a classifier stuck on one answer, and an independently written cross-check — every file mentioning shift 2 anywhere must be one the sweep drove — catches a classifier that quietly loses a single script.

When the fixture cannot be built at all, the suite exits 2 and its summary line counts the skipped section. A tally identical to a clean run would be this issue's own defect one level up, inside its own fix.

Mutation evidence

Every mutation applied to a copy outside the tree:

Mutation	Result
tracked_scripts() reduced to the old four-script hand list	60/64, 4 failed
has_arg_loop() always false	14/24, 10 failed
parses_flags_somehow() always false	69/70, 1 failed
loop_consumes_value() inverted either way	73/74, 1 failed
meta fixture git init forced to fail	exit 2, 62/62 passed, 0 failed, 1 section(s) SKIPPED

The first row is the one that matters: reverting to the arrangement this issue is about reddens the suite.

Also in the diff

• .claude/jit-context/paths/00-manual/tooling.md — the consequence for whoever adds the sixth tool: while [ $# -gt 0 ] is now a shape, not a habit. Body-only edit, no frontmatter change, so no index rebuild.
• changelog.d/188.fixed.md.

Nothing under scripts/ is touched.

Checks

• bash tests/run-all.sh — all suites passed; this suite 74/74 (was 65/65).
• shellcheck -S warning scripts/*.sh tests/*.sh — clean.
• python3 .oss/assemble_changelog.py --check — ok.
• awk, gawk and mawk all agree on the has_arg_loop idiom (observed); a CRLF line still matches the loop pattern (observed). Windows and Linux legs are reasoned, not observed.

Reviewed by an Explore reviewer (no findings) and oss:auditor (two findings, both accepted and fixed in the second commit — a skipped section that could not reach the tally, and a loop-no-flags verdict that had no fixture and was wrong for an all-boolean tool).

Verified by the maintainer

The load-bearing mutation, re-run by me in a fresh worktree at origin/main with only this suite file copied in: unmutated it is 74/74 passed, 0 failed, 0 SKIPPED, exit 0. Reverting tracked_scripts() to the four-script hand list this issue is about gives 58/63 passed, 5 failed, exit 1. That is the claim that matters — a suite whose enumeration cannot be reverted without reddening is a suite that is actually enumerating.

And one accident worth reporting, because it exercised a guard nobody wrote a fixture for. My first attempt staged the tree with git archive rather than a worktree, so there was no .git and git ls-files returned nothing. The suite did not pass over an empty sweep — it failed and said so in as many words: "and this suite would have reported a clean run over an empty sweep." That is the floor doing exactly its job against a case I produced by mistake, which is better evidence than a fixture I would have built to agree with it.

The design call is right, and it is the one I could not settle when I briefed it. A skip list is the hand-written list one indirection further out, stale in the same silence. Five verdicts read from each file's own bytes is the harder build and the correct one. The cost is stated plainly in the body rather than buried: a tool parsing flags in a shape the classifier cannot read goes red, not skipped, with no opt-out short of changing the classifier. I accept that trade — a false red is debuggable in one run, and the alternative failure is a tool with flags reading as a tool with none, which is this issue verbatim.

loop-no-flags was a real bug, not just an untested arm. The auditor found the missing fixture; acting on it turned up that the verdict would have failed any all-boolean tool with the message "the parser rotted". That is the second time today an audit finding has been the thread that pulled a defect out of a diff, and it is the argument for the two-reviewer arrangement over one.

Filed rather than carried: #193 — the classifier reads bash argument-loop shapes, so a tool written in another language under scripts/ classifies as no-flags and goes quiet rather than red. Correct today only because CLAUDE.md forbids anything but bash, awk and perl there, and nothing connects the two rules.