Skip to content

feat(brains): teach cutoff_strength and the cut verdict in brains-read [BRNS-DQ-049] — HOLD until #1577 is in prod - #34

Open
ayaz-ssvlabs wants to merge 1 commit into
mainfrom
feat/arm-cutoff-strength-and-cut
Open

feat(brains): teach cutoff_strength and the cut verdict in brains-read [BRNS-DQ-049] — HOLD until #1577 is in prod#34
ayaz-ssvlabs wants to merge 1 commit into
mainfrom
feat/arm-cutoff-strength-and-cut

Conversation

@ayaz-ssvlabs

Copy link
Copy Markdown
Contributor

Important

Draft on purpose. Do not merge until ssvlabs/brains#1577 is deployed to production.
Merging main here publishes, and the marketplace has auto-update on, so this reaches users immediately.

Why it must wait

#1577 is merged to stage (c26d36d3) and not yet on mainmain is 8 commits behind, with no promotion PR open at time of writing.

Publishing this early would teach every agent a parameter the live server does not implement. Production would accept cutoff_strength and silently discard it, because the enforcement that rejects unknown arguments is on stage too. That is precisely the defect #1577 exists to remove — a guide teaching a parameter the server quietly ignores — recreated by our own hand, and invisible because nothing errors.

Measured against production a few minutes ago, for the record:

query_rows text="…" board_id=…  →  items: [], count: 0, candidates: 50

Fifty candidates retrieved, none returned, and no cut to say whether the gate discarded them or nothing matched. That is the state this skill still correctly describes today.

What changes

One statement is now false. The skill says: "Don't send cutoff (or entity) with [a keyword-only call] — query_pages rejects them, the other arms ignore them." After #1577 the other arms do not ignore a non-default cutoff_strength, they reject it, and none of the four emit a cut for a call with nothing to gate on. Restated with the reason rather than the behaviour: keyword hits carry no similarity, so there is no statistic for the gate to judge.

cutoff_strength added next to the knob it modifies — lenient after a scoped call comes back thin, strict for a last narrowing step, and not accepted alongside cutoff: false.

The response contract gains cut and visible_at_most, which is the half that changes agent behaviour. count: 0 alone cannot distinguish "nothing matched" from "the gate threw away everything it found", and that ambiguity is what had agents retrying the same text instead of relaxing the gate. Stated with the trap named: cut is absent when no gate ran, and an agent reading a missing field as "the gate approved this" would conclude the opposite of the truth.

The surface is stated as closed. The arms now reject an undeclared argument rather than dropping it, so leaving a reader to guess produces 400s.

Version 2.10.1 → 2.11.0 on both manifests. Published content reaches nobody without a bump; minor, per the repo's convention for feat(...).

Verified

  • scripts/generated-artifact-guard.sh origin/mainguard: OK, both bumps detected.
  • brains-read/SKILL.md is hand-authored, not a generated artifact — checked against plugins/brains/generated/capability-catalog.json, which does not name it. Worth stating because editing a build output here would be silently reverted by the next regeneration.

Merge sequence

  1. stagemain promotion in ssvlabs/brains, carrying #1577.
  2. Confirm prod serves it (a query_rows call returns cut / visible_at_most).
  3. Mark this ready and merge.

Related and still blocked on access: the brains-desktop prompt must teach the same three additions, and its CI fails until it does. Patch for the harness guide is on #1577; neither that nor the prompt can be pushed from this account.

🤖 Generated with Claude Code

…d [BRNS-DQ-049]

The four arm tools gained `cutoff_strength` and now report `cut` and
`visible_at_most` (ssvlabs/brains#1577). Two things in this skill are wrong
as a result, and one was wrong in a way that matters.

WRONG NOW: "don't send `cutoff` with a keyword-only call — `query_pages`
rejects it, the other arms ignore it." After #1577 the other arms do not
ignore a non-default `cutoff_strength`, they reject it, and none of them
emit a `cut` for a call with nothing to gate on. Keyword hits carry no
similarity, so there is no statistic to judge — which is the reason, and
worth stating rather than listing the behaviour.

MISSING: the response contract. `count: 0` alone cannot say whether nothing
matched or the gate threw away everything it found, and that ambiguity is
what made agents retry the same text instead of relaxing the gate. `cut`
answers it: `admitted: false` with a non-zero `candidates` means retrieved
and discarded, so relax rather than rewrite. It is ABSENT when no gate ran,
and absence is not a passing verdict — an agent that reads a missing field
as "the gate approved this" would draw exactly the wrong conclusion.

Also states the surface is closed. The arms now reject an undeclared
argument instead of dropping it silently, so a skill that leaves the reader
guessing is a skill that produces 400s.

Version 2.10.1 -> 2.11.0 on both manifests: content that ships to users
reaches nobody without a bump, and this is a minor feature per the repo's
convention.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@ayaz-ssvlabs
ayaz-ssvlabs marked this pull request as ready for review September 8, 2026 10:09
@ayaz-ssvlabs

Copy link
Copy Markdown
Contributor Author

Hold released — the release gate is open, verified against production, not against the git ref.

c26d36d3 reached main, but that was not sufficient and would have been the wrong trigger: for a while afterwards prod still advertised the old five parameters. What the server actually accepts is the signal, so I asked it:

query_pages:      [cutoff, cutoff_strength, legs, limit, text, type]
query_rows:       [board_id, cutoff, cutoff_strength, legs, limit, text]
query_boards:     [cutoff, cutoff_strength, legs, limit, text]
query_mini_sites: [cutoff, cutoff_strength, grain, legs, limit, text]

And the four behaviours this skill now teaches, exercised live against mcp.mybrains.ai:

1. default                    count=0 candidates=50 cut={admitted:false, dropped:2, strength:normal} visible_at_most=3102
2. cutoff_strength=lenient    count=0 candidates=50 cut={admitted:false, dropped:2, strength:lenient}
3. snippet_chars=120          ERROR  invalid arguments for query_rows: snippet_chars is not an accepted argument
4. strength + legs={vec:false} ERROR  `cutoff_strength` has no effect on a keyword-only call…

Line 1 is the reason this change exists. That exact query returned a bare count: 0 with no explanation an hour ago; it now says the gate rejected the section and how much it discarded, which is the difference between "nothing exists" and "relax the gate" — the ambiguity that had agents retrying the same text.

Line 2 is BRNS-DQ-046's lenient setting reachable for the first time. Worth noting for whoever picks that ticket up: lenient does not recover this section, so "the cutoff over-rejects at lenient" now has its first real datapoint rather than an inferred one.

Ready for review. Still not merging — merging here publishes, and that call is the maintainers'.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant