Skip to content

Broaden the hook trigger surface: add WebFetch, decide on shell research, hook the research dispatch #174

Description

@A1igator

Corrected. This issue originally proposed adding a tenjin precheck verb on the premise that the check depended on model recall. That premise was wrong: the hook already exists, is enabled, and works. Verified by direct test, see the companion issue. What follows is the corrected ask.

The hook watches PreToolUse with matcher WebSearch. The research that missed it used curl, npm view, npm pack, gh api, git show, cast and WebFetch. The fix is trigger surface, not new machinery.

1. Add WebFetch to the matcher, today

Cheapest possible change and it covers a large share of real research. An agent fetching a spec page or a changelog is asking a research question just as much as one typing a query. Same hook, same payload shape, one more matcher entry in what tenjin install writes.

2. Decide what to do about shell-based research

This is the hard part and the one worth designing rather than patching.

A PreToolUse matcher on Bash would fire on every command, which is unacceptable noise: most shell calls are ls, git status, and test runs. But the subset that is unmistakably research is narrow and pattern-matchable in the hook itself, which already exits silently on anything it does not care about:

  • curl or wget to a non-localhost host
  • npm view, npm pack, pip download
  • gh api against a repo that is not one of the operator's own
  • cast or other chain reads against a public RPC

The hook is already built to be cheap and silent, so a Bash matcher that inspects the command string and exits 0 immediately on a non-match is consistent with its existing design. The risk is nag fatigue, which argues for a dedup window per question rather than per command.

3. Hook the research dispatch, which is the real decision point

The most valuable trigger is not any individual web call. It is the moment an agent decides to spend a lot on finding something out.

In a harness with subagents, that moment is the spawn of a research task, and it happens once, before the tokens are committed. A PreToolUse matcher on the task-dispatch tool would have fired exactly twice for those 2.07M tokens, with the full research question available in the prompt, which is a far better search input than any single query string. Two prompts against two dispatches, versus dozens of individual calls, is also the lowest-nag option available.

This is the one I would build if only one gets built.

4. Make the outcome visible

searches.json records what the hook saw. Nothing records what it never saw. Because the ledger only grows on a trigger, a day of heavy research that never touched WebSearch is indistinguishable from a quiet day. That is precisely why this went unnoticed until someone who knew the piece existed asked about it.

Anything that lets an operator ask "did we check today, and against what" would have surfaced this immediately.

The signal, restated correctly

The original filing said an installed, available, triggered skill still failed. That was wrong. The accurate version is sharper:

A well-built hook, enabled and working, watching the tool that agents used to use for research, was bypassed entirely by an agent that had a shell. The gap is not reliability, it is that the definition of "web search" moved. Any buyer whose agent has shell access has the same blind spot, and like this one, they will not know, because a miss and a never-triggered look identical from the outside.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions