Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
- Case repair's conjunction test reads the parse's `conjunction` tag rather than re-deriving the word's class (`juan e-f smith` → 'Juan E-F Smith'); the particle test still keys on lexicon membership, by decision, so `replace()` and a parse part there — 2026-08-29 #458.
- A token carrying `UNCLASSIFIED_TAG` (stamped by `replace()` only) falls back to the vocabulary for the per-word conjunction-or-initial question and never for the per-part particle question: `replace(family='velasquez y garcia')` → 'Velasquez y Garcia' — 2026-08-29 (#458 review).
- `initials()` has no such fallback, since it is handed no lexicon and guessing the default one erases fields parsed under a custom one; `replace(family='de la vega').initials()` → 'j. d. l. v.' is accepted until #464 gives that view a crossing — 2026-08-29 (#458 review, then the review of PR #463).
- A single-case word the parse put in the suffix role from the acronym vocabulary repairs to capitals unless a mask records its casing, asked before the Mac/Mc convention (`john smith mba` → 'MBA', `mcse` → 'MCSE'); the gate is the role, so a spliced suffix repairs alike and a name word sharing the spelling does not — 2026-09-13 #459.
- A single-case word the parse put in the suffix role from the acronym vocabulary repairs to capitals unless a mask records its casing, asked before the Mc convention (`john smith mba` → 'MBA', `mcse` → 'MCSE'; the Mac/Mc convention until 2026-11-02 #1014); the gate is the role, so a spliced suffix repairs alike and a name word sharing the spelling does not — 2026-09-13 #459.
- The guard on the generation exception is the pair, the suffix role and the `vocab:suffix` tag: a connective read as the generation repairs as the generation (`Carod i` forced → 'Carod I'), while a connective the suffix field merely holds keeps its lowercase (`Smith, John, and` forced → 'John Smith and', a spliced `suffix='i'` → 'i'), and a lone middle connective stays lowercase (`Carod i Rovira`) — 2026-09-20 #461/#397.
- The exceptions map holds case masks and nothing else, a value being its key's own letters and digits recased, laid over the word as the writer punctuated it wherever the word stands (`phd`, `ph.d.`, `PH.D.` → 'PhD', 'Ph.D.', 'Ph.D.'; `phd smith` → 'PhD Smith'); `md` → 'MD' and `phd` → 'PhD' are deliberate deviations from 1.4.0 through 2.3.0 — 2026-09-23 #459, DECIDED.
- A value that does not spell its key's letters and digits is a `ValueError` at every `Lexicon` entry point, compared as alphanumerics, whose message offers a recased key that constructs; a mask the word's letters miscount against is declined and the next clause repairs — 2026-09-23 #459, THE CHECK.
Expand All @@ -29,6 +29,7 @@
- A mask outranks the particle lowercase, in the shipped map (`SEÁN Ó MURCHÚ` → 'Seán Ó Murchú', with `ní` and `ua`) and in a caller's (`van` → 'Van' gives 'Ludwig Van Beethoven'); `mac` and `binti` carry none and stay lowercase — 2026-10-04 #604.
- A listed acronym credential joined by a hyphen or slash repairs whole in capitals in the suffix role (`john smith, otr/l` → 'OTR/L', `john smith, fnp-c` → 'FNP-C', which was 'Fnp-C' before), and a recorded casing for the whole token wins in any role — 2026-10-21 #808.
- A word's full stops divide it into runs written as words of their own (`j.r.r. tolkien` → 'J.R.R. Tolkien', `st.john smith` → 'St.John Smith', forced `J.Smith` → 'J.Smith'), and a word opening with a caseless modifier letter takes its capital on the letter after it (`AHMAD IBN-ʿALĪ` → 'Ahmad ibn-ʿAlī', `ʻIOLANI` → 'ʻIolani'), a digit not being passed over (`3rd`); the cost, an unlisted abbreviation whose ending follows its full stop reading as two words (the Sardinian `DUT.RA` → 'Dut.Ra', where 1.4.0 through 2.5.0 gave 'Dut.ra'; and an abbreviated second word reads as its spaced spelling does, `dr.philol.` → 'Dr.Philol.', where they gave 'Dr.philol.'), is accepted, a listed casing being the remedy, which the Italian abbreviated salutations carry (`SIG.RA` → 'Sig.ra', `dott.ssa` → 'Dott.ssa') — 2026-10-22 #838; 2026-10-22 #840.
- Case repair title-cases a word opening Mac and gives it an inner capital only from a mask, the Mc convention standing (`JOHN MACHADO` → 'John Machado', `JAMES MACKENZIE` → 'James Mackenzie', `RONALD MACDONALD` → 'Ronald MacDonald' by its mask); the shipped masks are the Mac surnames written MacX by most of their bearers, at least three writings, in en.toml as `family` readings carrying `cased`, and again in gd.toml and ga.toml by origin, a chunk after a full stop taking its own mask as a hyphen part does — 2026-11-02 #1014 (Derek), DECIDED.

Not standing: the rewording of rules/R4's vocabulary promise and its `Juan McDonald` row, carried out — 2026-08-29, DEBT DISCHARGED; the correction of mechanisms.md#RENDER-HONORS-THE-PARSE, carried out — 2026-09-23, mechanisms; the whitespace raises the branch review left for the orchestrator to file, filed and fixed as #541 — 2026-09-24 #459/#492/#478 (branch review); the measurements, which decide nothing — 2026-09-23 #459/#478.

Expand Down Expand Up @@ -113,3 +114,4 @@ Accepted costs, deferred to the rescoped #459 rather than relitigated here: the
- 2026-10-22 #840 — THE REMEDY FOR #838's ITALIAN COST SHIPS: the abbreviated salutations and letter-heading honorifics whose ending follows their full stop are titles with `cased` masks (decisions/salutation-titles), so `SIG.RA MARIA ROSSI` repairs to 'Sig.ra Maria Rossi' again, as 1.4.0 through 2.5.0 gave it, and `dott.ssa` to 'Dott.ssa'. The mask is asked before the chunk split, the precedence the 2026-10-22 #838 entry relied on. The accepted cost stands for an unlisted ending (the Sardinian `DUT.RA` → 'Dut.Ra', CLDR's `Dut.ra Prof.ra`). A spaced `Sig. ra` is out of reach: it joins as a title run, `ra` being a listed title, and forced repair writes 'Sig. Ra'.
- 2026-10-28 #357 — THE SHIM'S HALVES OF THIS RULE'S ENTRIES GO WITH THE SHIM (decisions/3-0-reevaluations' 2026-10-28 #357, REMOVAL entry): the v1 pickle load as a second producer of `UNCLASSIFIED_TAG`, leaving `replace()` the one; the first-parse raise for a v1 `Constants` carrying a mismatched mask (2026-09-23 #459, A DECIDED EXCEPTION), the core `Lexicon` raising on it as before; and the filter of v1 `Constants` entries with its folded cross-set checks (2026-10-02 #541, 2026-10-02 #582). A punctuated mask such as 1.4.0's `M.D.` stays legal in a `Lexicon`. SUPERSEDES: the 2026-09-23 #459, A DECIDED EXCEPTION, 2026-10-02 #541 and 2026-10-02 #582 entries, all about translating a v1 `Constants`, and the 2026-08-29 (#458 review) entry's second producer, the v1 pickle load.
- 2026-10-28 #939 — A CONNECTIVE BETWEEN PARTICLES: rules/R4's all-particles Except sets aside a connective that is not also particle vocabulary, standing between the part's words, as rules/R2 does, so `FRITZ FREIHERR UND VON BAR` should repair to 'Fritz Freiherr und Von Bar' where it gives 'Fritz freiherr und von bar', a deviation #939 tracks; decided and measured in decisions/R2's 2026-10-28 #754 (Derek), A CONNECTIVE AMONG PARTICLES entry.
- 2026-11-02 #1014 (Derek), DECIDED — A NAME OPENING MAC TAKES ITS INNER CAPITAL FROM A MASK, NOT FROM ITS SPELLING. The fall-through had capitalized after any `ma?c` prefix with two or more letters behind it since 0.2.3, so a single-case `MACHADO` repaired to 'MacHado', `MACIAS` to 'MacIas', `MACIEJEWSKI` to 'MacIejewski'; #56 asked in 2015 for an exceptions list beside the regex, and only the length check shipped. Measured over the mixed-case writings of every word the regex matched, each counted as its writer typed it, in the ORCID family-name box and the names of the UK Electoral Commission donors, GOV.UK people and ClinicalTrials.gov officials local corpora (2026-11-02): 4,616 writings of 970 words, the old convention matching the writer's spelling in 16.3% and plain title case in 83.7%; over the same four sources the Mc convention matched 96.8% of 6,312 Mc writings (94.3% of ORCID's 2,764), the rest mostly names Mc opens that are not Gaelic, `Mchunu` 24 the commonest, then `Mchedlishvili` and `Mchome` 5 each. The Gaelic Mac names split by family, so no shape decides them: `MacDonald` 143 against `Macdonald` 77, `Mackenzie` 59 against `MacKenzie` 44, `Mackay` 73 against `MacKay` 26, `MacLeod` 39 against `Macleod` 18. The mask records the spelling where its bearers lean one way: a word written MacX in more than half of at least three writings carries one, 42 words, which brings the measured match to 91.2% (in-sample, since the masks were chosen from the writings they are scored on; at least two writings gave 60 masks and 92.0%, at least five 25 and 90.2%). Recipe: for each source, read the field named (`family` for ORCID's `raw.jsonl`, `name` for the other three) and take every match of `(?<![^\W\d_])(Mac[^\W\d_]{2,})(?![^\W\d_])` whose letters after `Mac` are not all capitals; key it by its lowercase and count it MacX where its fourth letter is a capital; the comparators are the old rule (every word MacX), plain title case (none), and title case with a mask on each word that is MacX in more than half of at least three writings; the Mc figures are the same with `Mc` for `Mac` and the third letter. The masks stand on `family` readings, the first `cased` a name reading carries: an entry holding only such a reading compiles, so the build does not warn it inert. They are in en.toml, the anglicized spellings being English ones (the Gaelic writes `MacDhòmhnaill`, the Irish `Mac Cárthaigh`), and again in gd.toml, 37 of them, and ga.toml, the Irish `maccarthy`, `macmahon`, `macmanus` and `macdermott`, so a strict choice of a Gaelic pack, whose bearers are likeliest to be written so, keeps them: `parser_for(locales.GD, strict=True)` repairs `RONALD MACDONALD` to 'Ronald MacDonald' (decided, Derek, 2026-11-02). `mackrill`, an English name, is in en.toml alone. Every plain choice keeps all 42, no file holding another reading of them; a strict choice holding none of the three files keeps none, and a strict GA choice keeps only the Irish four (`RONALD MACDONALD` → 'Ronald Macdonald'). A chunk of a word that a full stop divides takes its own mask, as a hyphen part does, so the masks reach `J.MACDONALD SMITH` → 'J.MacDonald Smith', which the old convention gave and #838's parity between a chunk and a hyphen part kept; credential chunks take theirs too (`Phd.Md` forced → 'PhD.Md', `Msc.Ed.` forced → 'MSc.Ed.'): of the readings of ClinicalTrials.gov's officials that move, 125 forced and 1 default are of names holding no `mac` in any case, all credential chunks, against 611 forced and 8 default of names holding one (`snapshot.py --local ctgov-officials --base` the parent commit, 2026-11-02). A caller's lower-case mask on a one-letter or particle word reaches a chunk too (`{'y': 'y'}` gives `J.Y. SMITH` → 'J.y. Smith', where it gave 'J.Y. Smith'), as it reaches the spaced spelling ('J. y. Smith'); no shipped mask does. The cost is a Gaelic name written MacX that no mask records, `SHIRLEY MACLAINE` → 'Shirley Maclaine', and a family that writes Macdonald, which a forced repair recases to the mask. A caller's own spelling goes in `Lexicon.capitalization_exceptions`.
6 changes: 3 additions & 3 deletions docs/design/decisions/R5-case-repair-gate.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@

## Standing

- Case repair acts only on a name written entirely in one case, and a name whose words outside the suffix are written in more than one case is kept as written, whether that casing is right not entering into it (`juan mcdonald` → 'Juan McDonald', `SHIRLEY MACLAINE` → 'Shirley MacLaine', `Shirley Maclaine` kept), unless repair was asked for anyway (`Shirley Maclaine` forced → 'Shirley MacLaine'); the refusal predates the git history and the override shipped in 0.4.0, and the suffix's place in the test is the 2026-09-23 #492, DECIDED line's — 2026-08-29 (#407 arc); 2026-08-29, WHY R5'S STATEMENT WAS REWORDED; 2026-09-23 #492, DECIDED.
- The boundary example of rules/R5's main fence is the forced `Shirley Maclaine` line, the refusal being R5's effect rather than its absence, and no example witnesses an already-correct mixed-case name, kept and repaired being the same string there — 2026-08-29, why the gap survived; 2026-08-29, WHY R5'S STATEMENT WAS REWORDED.
- Case repair acts only on a name written entirely in one case, and a name whose words outside the suffix are written in more than one case is kept as written, whether that casing is right not entering into it (`juan mcdonald` → 'Juan McDonald', `RONALD MACDONALD` → 'Ronald MacDonald', `Ronald Macdonald` kept), unless repair was asked for anyway (`Ronald Macdonald` forced → 'Ronald MacDonald'); the refusal predates the git history and the override shipped in 0.4.0, and the suffix's place in the test is the 2026-09-23 #492, DECIDED line's — 2026-08-29 (#407 arc); 2026-08-29, WHY R5'S STATEMENT WAS REWORDED; 2026-09-23 #492, DECIDED; the examples' name, 2026-11-02 #1014.
- The boundary example of rules/R5's main fence is the forced `Ronald Macdonald` line (the forced `Shirley Maclaine` until 2026-11-02 #1014), the refusal being R5's effect rather than its absence, and no example witnesses an already-correct mixed-case name, kept and repaired being the same string there — 2026-08-29, why the gap survived; 2026-08-29, WHY R5'S STATEMENT WAS REWORDED.
- The override has one route, `capitalized(force=True)` per call, the v1 `force_mixed_case_capitalization` attribute having gone with `Constants`; the rule names none — 2026-10-28 #357.
- `capitalized_forced` is a test-side pseudo-field the doc runner resolves, not a parser field — 2026-08-29, `capitalized_forced` is a test-side pseudo-field.
- Forcing repair does not make the output independent of how the input was cased, so rules/R5 does not state that it does; the property is pinned in tests/test_capitalization.py, where `JUAN Y GARCIA` and `juan y garcia` both give 'Juan y Garcia' since a one-case name stopped reading a bare capital connective as an initial — 2026-08-29, WHAT THE OVERRIDE DOES AND DOES NOT PROMISE; 2026-08-29, and therefore NOT stated in rules; 2026-09-13 #383/#479.
Expand Down Expand Up @@ -42,4 +42,4 @@ Open: none in this file.

- 2026-09-24 #492 — DECIDED (Derek, 2026-09-24): where repair was not forced, a SUFFIX token written in more than one case is kept as written; one written in a single case is repaired like any other token, and force repairs both, unchanged. It is the gate's own one-case test asked of each suffix on its own (`_render._in_one_case`, one predicate for both), and it is the 2026-09-23 #492 reasoning read to its end: a suffix is left out of the gate because a cased one says nothing about how the writer cased the NAME, and for the same reason a mixed-case one is the writer's spelling of the credential, which repair has no evidence to overrule. THE REGRESSION THAT SURFACED IT, measured by the branch's code reviewer and re-measured 2026-09-24: once the gate stopped counting the suffix, a one-case name with a conventionally mixed-case listed acronym was admitted and the acronym clause wrote the acronym in capitals — `john smith EdD` → `John Smith EDD`, `JANE DOE, DSc` → `Jane Doe DSC`, `juan garcia PsyD` → `Juan Garcia PSYD` — where the parent 4d0680e6 and every release 1.4.0 through 2.3.0 returned those names untouched. They now give `John Smith EdD`, `Jane Doe DSc` and `Juan Garcia PsyD`. The accepted `B.Tech.` → `B.TECH.` on the default path (decisions/R4, 2026-09-23) was the same shape, and its default-path half is superseded (decisions/R4, 2026-09-24). THE COST, accepted: "as written" keeps a garbled spelling with the deliberate one, so `juan garcia Iii` gives `Juan Garcia Iii` on the default path (`Juan Garcia III` forced) and `john smith Mba` gives `John Smith Mba`, where the branch before this decision gave `III` and `MBA`; every release returned both names untouched. A rules/R5 boundary line pins `Iii`. Why not keep only a spelling the vocabulary does not know how to repair: that would re-derive, per word, a judgment about which mixed spellings are deliberate, which is the evidence R5 says repair cannot read. The masks added the same day (decisions/R4) are the other half: they serve the one-case names this rule cannot reach (`john smith psyd` → `PsyD`). No corpus name written before this decision moves by it (measured 2026-09-24 against the branch before it, over this tree's 1375 names: the only movers are the new rules.md lines). Pinned by `tests/v2/test_render.py::test_a_suffix_written_in_more_than_one_case_is_kept_as_written`, core and facade; a mutant dropping the per-token test fails it, `test_the_gate_leaves_the_suffixes_out` and two rules/R5 lines, and one dropping its `force` guard fails it and the forced `EdD` line.
- 2026-10-28 #357 — THE OVERRIDE HAS ONE ROUTE (decisions/3-0-reevaluations' 2026-10-28 #357, REMOVAL entry). The 2026-08-29 entry recorded two, `capitalized(force=True)` and the v1 `force_mixed_case_capitalization` attribute the facade resolved when no per-call value was given; the attribute went with `Constants`, as decisions/3-0-reevaluations' 2026-10-28 #925 (Derek), WARNINGS entry foresaw. rules/R5's statement, written to cover both, needs no change to cover one. SUPERSEDES: the 2026-08-29 entry recording the override's TWO routes.

- 2026-11-02 #1014 — THE EXAMPLES' NAME: rules/R5's fence named `Shirley Maclaine`, whose forced repair to 'MacLaine' was the Mac convention's, which decisions/R4's 2026-11-02 #1014 entry retires; with no mask for `maclaine` the forced line would return the name unchanged and witness nothing, so the fence names `Ronald Macdonald`, whose forced repair writes its mask, 'MacDonald'. The gate is unchanged.
Loading
Loading