Skip to content

A tool: Agent rule could never fire, and the class it belongs to was silent (#182) - #184

Merged
fdaviddpt merged 2 commits into
mainfrom
fix/182
Aug 18, 2026
Merged

A tool: Agent rule could never fire, and the class it belongs to was silent (#182)#184
fdaviddpt merged 2 commits into
mainfrom
fix/182

Conversation

@fdaviddpt

@fdaviddpt fdaviddpt commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

scripts/pre-tool-hook.sh built the subject it matches tools rules against out of four tool_input keys — command, skill, file_path, pattern. An Agent dispatch carries none of them, so cmd was empty, the hook printed {} and exited 59 lines before the layer loop that would have consulted any rule at all. A rule with tool: Agent and match: ~.* — the broadest matcher there is — was written, validated by the frontmatter parser, indexed by rebuild-tsv.sh, counted by every report that counts rules, and inert. That included mode: block, which failed open in the one dimension that can refuse a call.

The subject for an Agent dispatch is subagent_type, and only that

prompt and description are excluded deliberately. They are author-written prose, and two things go wrong with prose as a subject:

  • require, forbid and ~match are matched against the whole subject, so a prompt saying "do not run git push in this repo" trips a deny-list rule written about git push — a mode: block rule refusing a call because of what somebody wrote in prose.
  • cmd is cut at the first ; & | or ", all four of which occur constantly in prose, so a substring rule would be compared against an arbitrary prefix of the prompt. That is The require column blocks a command that carries the required flag, and a line-number table has no guard #7 rebuilt on a new field.

Cost says the same thing more weakly and is not the reason to lead with. Measured on a two-rule index, 40 calls per point interleaved, one-true-awk 20200816 on darwin 24.3.0, read out of the hook's own timing line in hooks.log rather than wall clock around the process: a 7-byte subject is 91 ms median, a 4.4 KB one 97 ms, a 44 KB one 207 ms. A prompt is routinely in the second band and can reach the third; subagent_type is always in the first.

Agent was the instance, not the class — and the fix contains no list of tool names

The subject comes from a fixed set of input keys while tool: accepts any tool name, and nothing joined those two facts. TodoWrite, WebFetch, WebSearch, ExitPlanMode, BashOutput and every mcp__… rule are in the same position. That set is not enumerable by anything committed to this repository: an MCP server defines its own input schema at connect time.

So the hook reports the third state on evidence instead, once per session: a real dispatch arrived, nothing in it could be made into a subject, and rules in the tree name that tool. Reported by row position, beside the existing sibling notices for a refused row and an unread layer. A tree whose rules are all about Bash still answers a TodoWrite with {}.

Refusing such a row at index time was considered and rejected: it needs a tool-to-key map committed to disk and shipped to strangers, which would refuse rules that work and accept rules that do not — the layer-list defect of #176 in a new spelling, in the least revisable place. The trade is that an author hears about an unreachable rule at fire time rather than at authoring time, which is later; in exchange nothing can be wrong about a tool it has never met, and no existing tree is broken by a rebuild that suddenly refuses rows it accepted yesterday.

Two defects found in this diff by its own author, both fixed with a test

The census shared jit_refuse_cut with the refusal list. That flag is program-scope so that whichever refusal site overflows first adds the cut line once. Two different lists sharing it means the second to overflow drops rows silently, with no cut line, under a count that still names the whole total — a false statement produced by a defence. Section G drives both lists over the 4096-byte threshold on one call and asserts each carries its own cut line.

The census was gated on cmd rather than on the whole subject. cmd is the subject cut at the first ; & | or ", so it is empty for two unrelated reasons, and the notice asserts which one it is. Gating on it made the hook tell an author that every Bash rule in their tree was unreachable, on {"command":"\""} or {"command":"; cat x"} — a call that carried a command the whole time, naming rules that are fine. Gated on the whole subject now; a subject that was built and then cut to nothing keeps the silent exit it had before this change. Section H drives both cut shapes.

Tests

tests/test-agent-subject.sh, 41 assertions across 8 sections. Every "must not fire" assertion sits beside a "must fire" positive control in the same fixture, same tree, same hook — the ambiguity this issue is about would be embarrassing to reproduce in its own test. Every index is written by rebuild-tsv.sh from real frontmatter, because half of what hid this is that the rebuild has always indexed a tool: Agent row happily.

Red before the fix: PASS: 19 FAIL: 10, every failure an {} where a rule body was expected, every positive control green. Green after: PASS: 41 FAIL: 0. Full suite green twice, 2675 assertions, 0 skipped, exit 0.

Not in this PR

scripts/jit-dry-run.sh has no way to dry-run an Agent rule — --tool Agent answers SKIPPED: --tool needs a target. Add --command or --file. An --agent flag is the obvious companion and is left out on purpose: naming it, and deciding whether every subject key gets a flag of its own, is a design call that wants its own issue rather than a rider on this one.

Closes #182

Verified by the maintainer

The review layer on this diff did not run, and that is recorded rather than smoothed over. Both of the author's generalist reviewer spawns returned a summary saying findings existed without stating them — "Reported one confirmed, reproduced finding", then "beyond those two" — so two to three findings are lost and unrecoverable. The author's four-class audit did return complete. Per contract a second empty return is a finding, not a third attempt, so the author stopped and said so; the one defect recorded under review was found by the author re-reading their own diff afterwards, not recovered from a reviewer.

So I dispatched an independent auditor against the committed diff. It read scripts/pre-tool-hook.sh in full, scripts/common.sh:1440-1560, the whole new suite and the unified diff — reads, not a restatement of the brief. Result: four classes checked, both focus checks checked, 0 findings, 0 could-not-check. Three things it established by reading rather than accepting:

  • No third list shares either truncation flag. jit_refuse_add gates on jit_refuse_cut at all seven sites building refused; jit_unreached_add gates on its own jit_unreached_cut at the one site building unreached; jit_layers_notice is assembled in bash and touches neither. And n_unreached increments before the truncating append, so a printed total can never name more rows than were printed.
  • The census gate produces three structurally distinct states, not two: nothing could be made into a subject; a subject was built and then cut to nothing; and an ordinary no-match. Section H drives the middle one with both cut shapes.
  • Nothing payload-derived reaches the notice. It is built from jit_row_id() — a disk-scanned layer name and an integer — plus a derived kind, and the once-per-session marker key is a constant string. Test D pins that the file-name column is absent from the output.

Red re-run, mine, against main at 3553f5f with the fix absent: PASS: 25 FAIL: 16, the failures concentrated in sections B–F. Their positive controls fail there too, which is correct — they assert behaviour that does not exist on main.

The one thing this PR leaves open is now filed as #186, not carried in a comment: a subject that was built and then cut to nothing still reaches no rule and says nothing, and because a ~match rule is matched against full_command — non-empty on such a call — a mode: block regex rule fails open on any command whose first character is ;, &, | or ". Predates this diff, correctly scoped out of it. #187 carries the other one: jit-dry-run.sh cannot dry-run the rules this PR just made reachable.

Florian DAVID added 2 commits August 18, 2026 13:01
…silent (#182)

pre-tool-hook.sh built the subject it matches tools rules against out of four
tool_input keys -- command, skill, file_path, pattern. An Agent dispatch carries
none of them, so cmd was empty, the hook printed {} and exited 59 lines before
the layer loop that would have consulted any rule. A rule with tool: Agent and
match: ~.* -- the broadest matcher there is -- was written, validated, indexed,
counted by every report, and inert. That included mode: block, which failed open
in the one dimension that can refuse a call.

An Agent dispatch is now matched against subagent_type and against that alone.
prompt and description are excluded on purpose: they are prose that routinely
quotes the very commands a forbid list is written about, and the command-words
cut ends the subject at the first ; & | or quote, so a prose subject would be
compared as an arbitrary prefix of itself -- #7 rebuilt on a new field. Cost
says the same thing more weakly: measured on a two-rule index, 40 calls per
point, one-true-awk 20200816, hook-internal timing, 7 bytes is 91 ms median,
4.4 KB is 97 ms and 44 KB is 207 ms.

Agent was the instance, not the class. The subject comes from a fixed set of
input KEYS while tool: accepts any tool NAME, and nothing joined those two
facts -- so TodoWrite, WebFetch, ExitPlanMode and every mcp__* rule are in the
same position, and that set is not enumerable because an MCP server defines its
own input schema at connect time. So there is no list of tool names in this fix.
The hook reports the third state on evidence instead, once per session: a real
dispatch arrived, nothing in it could be made into a subject, and rules in the
tree name that tool. A tree whose rules are all about Bash still answers a
TodoWrite with {}.

Refusing such a row at index time was considered and rejected: it needs a
tool-to-key map committed to disk and shipped to strangers, which would refuse
rules that work and accept rules that do not -- the layer-list defect of #176 in
a new spelling, in the least revisable place.

The census gets its own truncation flag rather than sharing jit_refuse_cut.
Two different lists sharing one program-scope flag means the second to overflow
drops rows with no cut line under a count that still names the whole total --
a false statement produced by a defence. Section G drives both lists over the
threshold on one call.

tests/test-agent-subject.sh: 36 assertions, every "must not fire" beside a
"must fire" in the same fixture. Full suite green, 2670 assertions, 0 skipped.

Co-Authored-By: Max <noreply>
…ied (#182)

Found reviewing my own diff, and reproduced. The first cut gated the
unreachable-rule census on `cmd`, which is `full_command` cut at the first ; & |
or double quote. That is empty for two unrelated reasons: no tool_input key
yielded anything, or a key yielded something the cut then took. The notice makes
a factual claim about which one it is, so on {"command":"\\""} or
{"command":"; cat x"} it told the author that every Bash rule in their tree was
unreachable -- on a call that carried a command the whole time, naming rules
that are fine. A wrong answer arriving calmly, out of a report added to stop
exactly that.

Gated on full_command instead. A subject that was built and then cut to nothing
takes the old silent exit and behaves exactly as it did before #182. It is
arguably its own third state -- a ~match rule would have matched full_command
and never ran -- but that behaviour predates this fix and is reported for filing
rather than widened here.

That also makes the vocabulary-pass guard dead by construction rather than by
coincidence, and its comment now says so: the shape it was written for no longer
reaches that line.

Section H drives both cut shapes with a positive control in the same fixture.
41 assertions in the suite, full suite green at 2675, 0 skipped.

Co-Authored-By: Max <noreply>
@fdaviddpt
fdaviddpt merged commit 643fb29 into main Aug 18, 2026
5 checks passed
@fdaviddpt
fdaviddpt deleted the fix/182 branch August 18, 2026 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A tools-dimension rule with 'tool: Agent' can never fire: the hook builds no subject for Agent dispatches and exits before the layer loop

1 participant