Skip to content

Harden the docs-audit grep patterns: they have under-matched on four consecutive PRs (line breaks defeat them) #235

Open

Description

@Siddardth7

Process finding from the #205 PR 3 review (F4). Not blocking #232.

Problem

The /ship docs audit rule is right and has been repeatedly vindicated: audit docs by git grep pattern, never by a hand-listed set of files. PR 1's and PR 2's blocking findings were both stale docs, and PR 2 still missed a fifth instance because the reviewer's list was fixed instead of the pattern behind it.

But the patterns themselves keep under-matching. On #232, this stale doc was missed by the P3 pattern:

apps/spc/docs/ASSUMPTIONS_LOG.md:19-23

It claimed capability "still lives there — PR 3 of #205 moves it". The phrase wrapped across a line break as:

PR 3
> of #205

so no alternation in #205 PR 3|PR 3 of 3|until #205|Drop this line when matched. It was found only because someone read the surrounding lines of a neighbouring P2 hit — i.e. by luck, not by the mechanism.

This is the fourth consecutive PR where a doc pattern under-matched. The rule is sound; the implementation is brittle.

Root cause

The patterns are single-line regexes applied to prose that is hard-wrapped (and, in blockquoted Markdown, re-prefixed with > after the break). Any multi-word phrase can be split by wrapping at any point.

Suggested fixes

  • Match on the shortest stable token, not the phrase: prefer #205 or PR 3 alone with manual triage of the hits, over a multi-word phrase that a line break can split.
  • For phrase matching, normalise first — e.g. pipe through something that joins wrapped lines and strips > prefixes before grepping, rather than grepping the raw file.
  • Add a self-check to the docs-audit step: deliberately introduce a line-wrapped instance of the phrase being hunted and confirm the pattern catches it. A doc pattern deserves a negative control exactly as much as a test does.

Where this lives

The four patterns are specified per-issue in .pipeline/spec.md §9 during research. The durable fix belongs in whatever guidance the research stage uses to generate them, so every future audit PR inherits it rather than re-deriving the same brittle patterns.

Activity

  1. added a commit that references this issue on Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions