M1: block model, readers, profiles, semantic pass, sample pair, Pyodide CI

[houfu]

What this lands

The M1 milestone in full: the block model, both 1.0 readers, the built-in profiles, the semantic pass, the section 3a sample pair with its expected trees, and the Pyodide CI check. Nothing is exported from the top-level package yet; Redlines, Document and the CLI are unchanged apart from one import line.

Wave A: foundations

• redlines/blocks.py — the frozen block model (Block model dataclasses: kind, text, label, level, path, children, attrs #98) with role and spans (Semantic fields: role on blocks, spans in blocks #99); matched_by, confidence, dropped reporting and a derived fallback_count (dropped reporting, per-block matched_by and confidence, tree-level fallback count #106); XPath-style addresses assigned by one function, a heading breadcrumb, and to_dict/from_dict round-tripping.
• redlines/readers/ — the runtime-checkable Reader protocol and registry (Reader interface with a worked third-party example #105), a ParagraphReader as the degrade path, a shared input size cap, and detect.py for txt and md detection that reports unknown types instead of guessing (Format detection for txt and md; unknown types reported, not guessed #107).
• examples/custom_reader.py — the worked third-party reader, executed by the test suite.
• ADR-0029 (address syntax) and ADR-0030 (matched_by and confidence semantics).

Wave B1: text reader, built-in profiles, Pyodide CI

• redlines/readers/labels.py and text.py — the plain-text reader over PRD § 6b's five stages (Plain-text reader: normalise, segment, re-join wraps, detect labels, infer hierarchy #102). The § 6b hard cases live under tests/corpus/hard_cases/: four pass (alpha/roman at (i), numbering restarts inside schedules, one-line clauses that look like headings, mixed label styles) and three are strict xfails with reasons (run-on definitions, cross-references in prose, PDF page headers, which must not crash and do not).
• redlines/profiles/builtin/ — generic, contract and markdown as commented YAML (Built-in profiles: generic, contract, markdown #101). The markdown profile repeats most of contract's rules; that is the evidence ADR-0028 asked Built-in profiles: generic, contract, markdown #101 to gather before deciding on composition, and the decision is left open.
• Pyodide job in python-package.yml (Pyodide import check in CI #109). Its first run found a real bug: an undeclared typing_extensions import that broke clean installs on Python 3.10 and in the browser. Fixed here.
• A reconciliation commit after the reader met the built-in profile: single-letter (i) now reaches the style stack, hard-wrap re-joining follows the PRD's lowercase rule, and the stack prefers the smaller forward jump when two runs are open.

Wave B2: markdown reader, semantic pass

• redlines/readers/markdown.py — stdlib-regex reader over the same label and hierarchy logic (Markdown reader: headings, nested lists, clause patterns, pipe tables, fenced code #103). A markdown contract and its plain-text twin produce the same kinds, labels, levels and addresses; tests/corpus/markdown_cases/twin_contract.* proves it block by block.
• redlines/semantic.py — apply_semantics(tree, profile), pure and idempotent (Semantic pass: definitions, schedules, cross-references, parties, dates, amounts #104): roles first-match-wins with ancestor proximity as ADR-0028's one exception, spans from every extractor, cross-references carrying the referenced label as the span's value, and the PRD § 6b definitions heuristic as the one rule the profile format cannot express.

Wave C: the sample pair

• tests/corpus/sample_pair/ — the PRD § 3a demo agreement (Section 3a sample pair and its expected block trees #108): a services agreement between Northwind Systems Limited and Harbour Foods Group Limited, in markdown and as a plain-text twin, plus an amended version carrying exactly one of each detectable change. CHANGES.md lists the eight changes with their addresses. expected/ freezes the four trees; tests/test_sample_pair.py rebuilds and compares them and asserts the structural facts the pair exists to show. The change-tree golden is M2's.

Follow-ups

• redlines/pipeline.py — read_document() composes the three stages a caller otherwise spells out by hand. Format is detected when not given, and an undetectable one raises with the detection reason quoted so the "coming in 1.1" message reaches the caller. A profile can be a built-in name, a Profile, a mapping or a file, with a built-in name resolved before any path so "contract" cannot be shadowed by a file of that name. With no profile it applies the default the roadmap already recorded: contract for text, markdown for markdown. The sample-pair regenerate script now calls it and the goldens are byte-identical afterwards.
• redlines/profiles/builtin/ is now a package, holding both the loader and the three YAML files it reads. The module and the data directory previously shared a name. No import spelling changed and the wheel listing is unchanged apart from the moved file.
• ROADMAP.md gets the M1 completion entry, in the shape M0 already has.

M1 exit criteria

• The sample pair parses into the expected trees under contract and markdown: 53 tests in test_sample_pair.py.
• Every § 6b hard case has a test: four pass, three strict xfail.
• The wheel imports in Pyodide: the pyodide job is green on every push.

Checks

• 939 passed, 1 skipped, 3 xfailed; mypy strict clean on 46 files; black clean.

Decisions

Settled: the sample agreement text stands as drafted, and the clause role question is now tracked as #130 rather than blocking this PR.

Still open, neither blocking:

• recital and signature are match: heading in the contract profile, so the blocks under those headings carry no role; schedule uses ancestor_heading and reaches the whole subtree. One-line profile change if you want them to match.
• Composition. ADR-0028's revisit condition is met by the built-in profiles, with the overlap pinned in a test.

Worth knowing rather than deciding: the four expected trees total about 480 KB of pretty-printed JSON, and the plain-text twin states Schedule 1's deliverables as sentences rather than a table, because the text reader has no tables (ADR-0013). The twins therefore diverge in exactly that subtree, and the test says so.

Closes #98, closes #99, closes #101, closes #102, closes #103, closes #104, closes #105, closes #106, closes #107, closes #108, closes #109.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QosmZFLHNUw7kSDgG3ejXT