Conversation
Alignment needs one number and one place it comes from. `similarity()` compares token sequences rather than characters -- the leaf differ's own tokens, so the ratio alignment measures is the ratio the diff will report, and two orders of magnitude cheaper besides. The backend is selected rather than detected (ADR-0032): "auto" resolves to rapidfuzz when the [fuzzy] extra is installed, an explicit name is honoured, and asking for a backend that is absent raises rather than quietly running the other one -- a silent downgrade would make #143's with-and-without report a gap of zero. `SequenceScorer` holds the test side as difflib's seq2 and applies both cheap upper bounds before either backend computes anything. Those bounds are sound for both, because a common subsequence can never be longer than the multiset intersection, so the candidate set does not depend on which library is installed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Six named passes over an anchored sibling-scoped descent (ADR-0032), with every pair carrying both ADR-0029 addresses, the pass that found it and the similarity that pass measured. Scope is what makes a label mean something: eight schedule items are near-ties across a document and trivially distinguished among their own siblings. Order is inclusion-only and fixed, because two orderings are load-bearing. `exact` before `label` is what makes renumbering fall out of the record rather than needing a search -- the sample pair's 3.3 and 3.4 are byte-identical, so the label difference is read straight off -- and the label floor is what stops the renumbered clause matching the new clause that took its number, which score 0.20 against each other. Cost is bounded three ways so ADR-0008's permitted quadratic fits N2: anchors partition each sibling group, a window caps an unanchored gap, and one run-wide budget backs both and reports itself when spent. Measured: 2,000 anchored blocks in 30 ms, 2,000 wholly unanchored ones in 0.24 s. The move pass is a marked hook for #132: the name exists, is configurable and is counted, and it matches nothing, so a moved block is a delete and an insert -- silent rather than wrong. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The move pass was a hook: a block that changed section came out as a delete and an insert. It now searches, and it is the only global work in alignment. Two stages over what the descent could not place, both restricted to move_kinds and to blocks of at least move_min_tokens tokens. Exact pairs a normalised text that is unique among the leftovers on *both* sides, which is the whole false-positive story: thirty clauses reading "Intentionally omitted." stay ambiguous and pair with nothing. Fuzzy takes the best candidate above move_min_similarity only when it beats the runner-up by move_tie_margin, so a near-tie is silence. Both spend the run-wide budget and stop generating candidates when it is gone, leaving budget_exhausted to say that "nothing found" was really "we stopped looking". Every pair either stage makes is descended into at once, so a moved subtree's children align inside it by the ordinary passes. That is what turns the sample pair's change 2 into one move at confidence 1.0 plus one fuzzy edit at 0.933 on its body, rather than one indivisible event or a second move on the child. Separately, the reordering the cross-scope search structurally cannot see: within one sibling group, anchors outside the longest increasing subsequence of (source order, test order) have genuinely crossed, and _moved now reads that off the record too. move_kinds applies there as well, because it says which kinds may be reported as moved, not only which may be searched for; move_min_tokens does not, because a crossing pair was already matched by another pass and there is no guess left to guard. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A move is the only claim alignment makes that a reader cannot check by looking at one place in one document, and ADR-0009 says it is the claim that costs most when it is wrong. So the new module asserts silence as carefully as it asserts matches: an ambiguous text, a near-tie, a block too short to identify, a container kind, an exhausted budget -- each of those has a delete and an insert as its right answer, and each has a test saying so with the measurement behind it. The three prototype failures ADR-0032 records become regressions here, since each was a confident wrong answer rather than a crash: the table must pair structurally or its rows are never compared, the pass must exclude cell/row/section kinds, and it must enforce the token minimum. Without all three the markdown side produced thirteen cell "moves". The near-tie is parametrised over both backends, because the two disagree by up to +0.333 on short sequences and a silence that depended on which extra is installed would not be a promise. The hash-seed matrix is here as well as in #135's module: the move pass buckets leftovers by text, which is precisely the shape that goes wrong quietly. The tie margin is also checked from the source end, which ADR-0032 does not state. Read only over test blocks, one source clause with two plausible destinations would pair with whichever came first while two sources and one destination went silent -- the same ambiguity answered differently depending on which document was called source. The existing expectations that were written as "until #132 lands" flip with it, and the sample pair now reports its clause 7.5 -> 9.6 as one move at confidence 1.0 plus one fuzzy edit at 0.933 on its body. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Adds tests/test_alignment_renumber.py: dedicated coverage for #133, pinning the sample pair's renumber cascade and cross-reference clause, a renumbered-and-edited clause staying one pair even with only the mandatory passes running, and a synthetic multi-clause cascade with a cross-reference span that must survive the shift underneath it. No production code changed: AlignedPair.renumbered plus the labels already sitting on both sides of the pair are what #133 asked to expose, and align() already produces both (#131). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Adds tests/test_alignment_tables.py: rows matching exact-then- positional, cells pairing strictly by sibling index including ragged rows on either side, the sample pair's real inserted row, and the near-identical-rows case table_fuzzy is off by default to avoid -- pinned under both similarity backends, and contrasted against turning the knob on so the default's actual cost (a low-confidence positional guess) is visible next to what the knob buys back. No production code changed: the table behaviour #134 asks for was already built into align()'s cell rule and table_fuzzy default (#131). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Adds tests/helpers/hash_seed.py: a reusable subprocess helper that runs a self-contained script under several PYTHONHASHSEED values and asserts byte-identical stdout. It is deliberately independent of what it serialises -- today the sample pair and a synthetic document that exercises all six alignment passes, printing Alignment.to_dict() -- so the same function is the hook #137's JSON v2 test can reuse on Comparison.to_dict() once that module exists. tests/test_determinism.py also asserts that the alignment configuration in force, including the resolved similarity backend, appears on the wire (#135), and that an explicitly requested backend is recorded as both asked and resolved. mypy needs explicit_package_bases (pyproject.toml) once tests/helpers/ is a real package sitting inside the unpackaged tests/ directory, which otherwise resolves ambiguously between 'helpers.hash_seed' and 'tests.helpers.hash_seed'. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`align` says which block is which; this says what happened to it. The result is ADR-0033's flat, document-ordered list of `Change` nodes, each carrying both addresses, both labels, the role, the span types the change touched, the alignment pass that found the pair, and -- where text was edited -- inline ops in characters into each block's own text. Three rules do most of the work and all three are the ones that keep the list readable: kind precedence `move > renumber > modify` with the losing kind's inline ops kept, so a renumbered-and-edited clause does not lose its edit; topmost-wins granularity for insert, delete and move, so an inserted table row is one node and not four; and "an address shift alone is never a change", which is what stops one insertion producing a hundred nodes. The leaf differ is reused rather than reimplemented (ADR-0010). `Redlines.changes`'s body moves out verbatim as `redlines_from_opcodes`, which the facade now calls, and the character-offset conversion shares its opcode filter -- so the two representations cannot disagree about which edits there were. They stay two functions because v1's `Redline` reports no source position for an insert and so cannot carry the insertion point a character offset needs. `_strip_sentence_markers` moves to `redlines.processor`, beside the marker it strips, because both callers need it and `changes` may not import `redlines`. v1's behaviour is unchanged: `output_json`, `output_markdown` and `changes` are byte-identical before and after the extraction. Refs #136 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Four stages composed once -- read, read, align, build -- returning a `Comparison` that carries both block trees, the alignment, the change tree and the configuration in force. `compare`, `Comparison`, `ComparisonConfig`, `ChangeTree`, `Change`, `InlineOp` and `AlignmentConfig` are exported from `redlines`, so the headline API is `from redlines import compare`. It lives in a new `redlines.comparison` rather than in `redlines.pipeline`, whose docstring claimed it: nothing in `pipeline` is re-exported, and a re-exported function inside a deliberately-not-re-exported module would make that module half-public (ADR-0033). That sentence is corrected here, pointing the reader at the module that does have it. Three decisions worth reading twice. A bare `str` is content, never a path -- a `Document` is how a file gets in, so the library never stats the filesystem behind the caller. Format detection is per side, and two sides that detect differently raise rather than being silently read as one format. And `Comparison.alignment` is public, because an unchanged matched pair produces no change node, so the correspondence set the benchmark scores is not expressible in the change tree at all; on the wire it stays optional, under `to_dict(include_alignment=True)`. The eight named assertions of #144's first phase land with it, in `tests/test_sample_pair_change_tree.py`: one test per row of the sample pair's CHANGES.md, written against the designed API before this module existed and asserting kind, addresses, labels, span types and inline ops -- with `matched_by` asserted only on the move and on the renumber run, where the pass is the point. All eight pass, so none carries a strict-xfail mark. Refs #136 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…claude/m2-142-hand-set
Clones each Common Paper standard-agreement repository at the tag pair compared and downloads two short public-domain U.S. bills at two published stages each, then applies two deterministic, recorded normalisations: strip_inline_html removes decorative markup present in every Common Paper tag, and promote_span_ids_to_labels fixes the load-bearing hazard where a newer tag encodes a clause number only in an `id` attribute instead of the literal text older tags carry -- without it the label alignment pass would have nothing to match a newer-tagged clause against. Every normalisation actually applied lands in a per-pair prepare_manifest.json, which benchmark/label.py reads next to build labels.yaml's provenance. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
init runs compare() over a prepared pair and drafts labels.yaml with every correspondence, insertion and deletion at status: proposed, plus two human-facing documents: worksheet.md (every source block, its proposed counterpart, and a ? flag for anything worth a second look) and move_worksheet.md (the two clause lists side by side, address and label only, with no engine proposal anywhere on the page). No row is ever written with kind: move. A block the engine reports as moved is written conservatively as same/renumber instead -- ADR-0034 requires the move gate to be labelled independently of the engine it checks, so seeding it from that engine is exactly the self-marking risk ADR-0021 exists to catch. The move worksheet is where a human labels moves, starting from a blank sheet. check validates schema, re-derives every digest, and checks totality. sign stamps review: with a name, a date and a content digest, and refuses while any row is still proposed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
) Eight Common Paper standard-agreement tag pairs (CSA x2, DPA, SLA, Design Partner Agreement, PSA, Partnership Agreement, Pilot Agreement), each CC BY 4.0 with its own NOTICE.md, and two U.S. bill version pairs from govinfo.gov (H.R. 7385 Introduced-to-Engrossed, H.R. 4668 Introduced-to-Reported), each public domain under 17 U.S.C. Sec 105. Every pair carries the text benchmark/prepare.py produced, its prepare_manifest.json, and the draft labels.yaml/worksheet.md/ move_worksheet.md benchmark/label.py init wrote from the current alignment engine. Every row is status: proposed; no row anywhere carries kind: move. These are drafts awaiting a maintainer's labelling pass and a separate review pass -- not ground truth yet. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The top-level README gets the licence line ADR-0034 asks for: the corpus under benchmark/corpus/hand/ carries CC BY 4.0 and public-domain material alongside redlines' own MIT licence, with each pair's NOTICE.md as the authoritative per-pair statement. benchmark/README.md documents prepare.py/label.py's layout and the init/check/sign workflow, and records that every committed pair is still at the init stage. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…st (#142) Parametrised across every directory in benchmark/corpus/hand/: labels.yaml loads and validates, every digest still matches the committed files with totality holding, NOTICE.md is present and names the right licence, and no correspondence carries kind: move while still status: proposed -- the durable invariant that keeps the move gate from ever being seeded by the engine it checks, rather than a snapshot of today's all-proposed state. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The committed worksheet.md was generated under a difflib similarity resolution; the standard dev environment has rapidfuzz installed, so similarity="auto" resolves to rapidfuzz and label.py init now produces confidence 0.65 (was 0.64) for /section[1]/list_item[1]/list_item[1]. Re-running init and committing the regenerated file keeps worksheet.md matching what init actually produces today. labels.yaml and move_worksheet.md are unaffected. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
The tooling half of #142:
benchmark/prepare.py(fetch + normalise),benchmark/label.py(
init/check/sign), and the ten prepared hand-labelled pairs themselves, committed underbenchmark/corpus/hand/<pair>/. This branch mergesclaude/m2-136-change-tree(forcompare()) intoclaude/m2-141-generator; the diff below is clean once both are merged in,in that order.
Eight pairs come from Common Paper's standard agreements (CC BY 4.0, real git-tagged
before/afters), two from U.S. bill versions on govinfo.gov (public domain, 17 U.S.C. § 105),
covering CSA, DPA, SLA, Design Partner Agreement, PSA, Partnership Agreement and Pilot
Agreement as asked. Every Common Paper tag pair checked showed a genuine edit (line-level
diff ratios between 0.14 and 0.99, never 1.0), so none needed replacing.
Decisions and ADRs honoured
Wikipedia, no customer documents, no repository-owned documents.
literal text ("1.1 Access and Use."); newer tags carry the same number only in an
idattribute, because the markdown reader strips its own ordered-list marker before label
matching (
redlines/profiles/builtin/markdown.yaml's own comment).prepare.py'spromote_span_ids_to_labelsturns theidback into a literal prefix (and a top-levelheader_2id into a nested ATX##, matching the older tags' own shape for sectionheadings);
strip_inline_htmlremoves the rest. Onlycsa-1.1-to-2.0andsla-1.0-to-2.0actually needed the promotion (verified by diffing tag pairs directly); every other pair
records only
strip_inline_html(or nothing, for the two plain-text bill pairs). Everynormalisation actually applied is recorded in
prepare_manifest.jsonand from there inlabels.yaml'sprovenance.normalisations— never silently.label.py initnever writeskind: move. A block the engine reports as moved is written conservatively assame/renumber, flagged?inworksheet.mdfor visibility, and left out oflabels.yamlunder that claim entirely — the address pair is still recorded exactly once,just not as a move.
move_worksheet.mdis the engine-independent artefact: two clause listsside by side, address and label only, no proposed correspondence anywhere on the page.
tests/test_benchmark_hand_set.pyencodes this as a durable invariant (nokind: moverowis ever
status: proposed), not a snapshot of today's state.benchmark/labels/schema.json:initbuilds directly on the label-format PR'sbenchmark/labels.py(dataclasses, digests, totality) rather than re-deriving any of it.prepare.pyshells out to the systemgitandcurlrather thanurllib.request— several sandboxed Python installs on this machine carry no CA bundle oftheir own, while system
curlalready trusts the OS keychain. Dev-only, run by hand, neverimported by a test or CI, mirroring
benchmark/fetch_neurotic.py's own rule.Reinterpretation of issue wording
None beyond what ADR-0034 and the M2 decisions record already settle.
label.py checkdoesnot refuse a fully
proposedfile (onlysigndoes) — that split is stated explicitly inthis task's own instructions and matches the fact that every pair here is deliberately left at
the
initstage.What was deliberately left out
labels.yamlisstatus: proposed; noreview:block exists anywhere. These are drafts for a maintainer towork through via each pair's
worksheet.md, correct where wrong, label moves independentlyfrom
move_worksheet.md, and sign — on a separate day from the review pass, per §1.11.5.benchmark/label.py sign's reviewer flow is implemented but unexercised here: nothingis signed, since nothing has been labelled yet.
Per-pair summary
?rowscsa-1.0-to-1.1csa-1.1-to-2.0dpa-1.0-to-1.1sla-1.0-to-2.0design-partner-agreement-1.0-to-1.1psa-1.0-to-1.1partnership-agreement-1.0-to-1.1pilot-agreement-1.0-to-1.1govinfo-hr7385-ih-to-ehgovinfo-hr4668-ih-to-rhAll ten labels are drafts awaiting the maintainer's labelling pass and a separate review pass.
Test evidence
tests/test_benchmark_hand_set.py(42 of the above, parametrised across all ten pairs) checks:every pair's
labels.yamlloads and validates; every digest is fresh against the committedfiles with totality holding;
NOTICE.mdis present and names the right licence; and nokind: moverow is everstatus: proposed. No existing test was weakened, and noxfailwasflipped.
(this PR closes no issue)
🤖 Generated with Claude Code