Skip to content

Commit 854087b

Browse files
derek73claude
andauthored
Record the choice of languages on the Lexicon (#754, 4i-c) (#992)
* Record the choice of languages on the Lexicon (#754, 4i-c) Lexicon gains _languages/_strict (read as languages/strict, with Parser.languages/strict reading through); a choice filters the base's readings and the caller's log applies after it. parser_for unions a base's choice with its packs', joins the packs' fragments under the caller's lexicon, and refuses a clash naming the edit. _apply_choice and its delta retire. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Document the recorded choice; mark the superseded entries in effect (#754, 4i-c) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Keep a strict base's choice through a call choosing nothing; let a pack's mask beat a shipped one (#754, 4i-c review) A chained call choosing no language now keeps base's choice, its strictness included, where the first draft raised on parser_for(locales.RU, base=strict_parser). A shipped mask the caller left unchanged yields to a pack's, as shipped readings do. The kind-set branch of _edits_toward went with the retired delta. The RFC's 3b and 3j passages are put in the past tense. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Give parser_for's strict three states, as segmenter's (#754, 4i-c) Omitted (UNSET) keeps base's recorded strictness; strict=False on a strict base raises again, as decided; strict=True on a plain base with a choice makes the union strict. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test a chained mask the default lacks and the join errors' wraps (#754, 4i-c) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Lay a pack's masks over the choice's, name the pack a join fails on, load 4i-b pickles (#754, 4i-c review) parser_for moves base's masks by the choice and lays the packs' over them, as before 4i-c; a failed join or choice compile names the first pack whose fragment fails it (rules/D2), or the languages and the packs that chose them; _languages and _strict carry _ADDED_IN, so a 4i-b pickle loads choosing nothing (fixture written on db7e8fc). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Cover the base-chosen layer and a restored mask; drop an unreachable fallback (#754, 4i-c) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pin the accepted limit: a caller's mask written as the shipped value yields to a pack's (#754, 4i-c) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Offer a word's name reading before its agnomen in a 2.x refusal; cover the translation's branches (#754, 4i-c) The translation of a 2.x agnomen offered .add(w, kind='agnomen') before .add(w, kind='family'), which add() refuses; name readings now come first. The retired choice delta was the other path through the translation's property and kind branches, so the refusal table gains rows reaching them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1 parent db7e8fc commit 854087b

13 files changed

Lines changed: 988 additions & 418 deletions

‎AGENTS.md‎

Lines changed: 4 additions & 4 deletions
Large diffs are not rendered by default.

‎docs/design/decisions/vocabulary-model.md‎

Lines changed: 15 additions & 8 deletions
Large diffs are not rendered by default.

‎docs/design/mechanisms.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -332,11 +332,11 @@ a fork for all ambiguous particles on "Dr. Van Jr.".
332332

333333
**Problem shape.** Language-specific behavior needs a home that cannot drift from the core.
334334

335-
**Contract statement.** A locale pack is pure data — a choice of languages, a Lexicon fragment of ADDITIONS and a Policy patch — applied by parser_for, a shipped pack's patch compiled from its language file's `[rules]` table where one is written, each key the `Policy` field of its name: the packs' languages union into one choice, which selects among the default vocabulary's readings by the file each sits in, the fragments union onto the result, and the patches fold in order; packs contain no code paths of their own. A vocabulary pack's deviations are derived from the reading index, the words whose compiled readings its choice changes; a policy or segmentation pack declares its own in its docstring and `DEVIATES`.
335+
**Contract statement.** A locale pack is pure data — a choice of languages, a Lexicon fragment of ADDITIONS and a Policy patch — applied by parser_for, a shipped pack's patch compiled from its language file's `[rules]` table where one is written, each key the `Policy` field of its name: the packs' languages union with the base's recorded choice into one choice, recorded on the `Lexicon`, which selects among the base's shipped readings by the file each sits in, the fragments join in pack order and the caller's lexicon after them, its own readings applying after the choice, and the patches fold in order; packs contain no code paths of their own. A vocabulary pack's deviations are derived from the reading index, the words whose compiled readings its choice changes; a policy or segmentation pack declares its own in its docstring and `DEVIATES`.
336336

337-
**How it works.** What is policy for every language (an order constant) is policy, not pack data; packs are lowercase modules exposing uppercase constants; a pack error is wrapped with the pack's code (rule D2). The fragment never removes a base entry; a choice may, and only by selection, never by a list of removals: a chosen language's `default = false` readings join; on a word a chosen language holds a vocabulary reading of, other languages' vocabulary readings leave, save `mul.toml`'s; and a chosen language's conjunction loses `may_be_initial`, its clash with the initial being a shape reading a preference outranks, which is how a `pt` choice drops `e` from `conjunctions_ambiguous` (decisions/vocabulary-model, 2026-10-27 #753, items 1, 2 and 7; built 2026-10-29, phase 3b). A strict choice, `parser_for(..., strict=True)`, selects by file alone: only the chosen files and native-script vocabulary compile, `mul.toml`'s and `und.toml`'s readings leaving with every other file's, and it too is applied as a delta (decisions/vocabulary-model, 2026-10-30 #753, file-strict; built 2026-10-31, phase 3j). So a bad default is routed around only by choosing the language whose reading is right, which is why #342-class fixes still change shipped vocabulary. The choice is applied as a delta against the default's compile: it adds to the base lexicon what the choice turns on and removes from it what the choice drops, whichever of the default and the caller put a removed entry there, and leaves every other entry, a caller's own included; a pack choosing no language leaves the base's exactly as it was. The choice is one call's: a `Parser` records none, so `parser_for(EN, base=parser_for(PT))` applies `en`'s after `pt`'s and keeps neither language's `do`, where `parser_for(PT, EN)` keeps both (decisions/vocabulary-model, 2026-10-29, chained choices). Pure data means a pack can be audited: a vocabulary pack's whole content is its tag, but what it changes reaches other files too, prefer dropping their readings of its words (a `de` choice drops `ca.toml`'s conjunction `i`), so the audit is the derived surface, `changed_words`, not the pack's file alone.
337+
**How it works.** What is policy for every language (an order constant) is policy, not pack data; packs are lowercase modules exposing uppercase constants; a pack error is wrapped with the pack's code (rule D2). A choice changes the base's readings only by selection, never by a list of removals: a chosen language's `default = false` readings join; on a word a chosen language holds a vocabulary reading of, other languages' vocabulary readings leave, save `mul.toml`'s; and a chosen language's conjunction loses `may_be_initial`, its clash with the initial being a shape reading a preference outranks, which is how a `pt` choice drops `e` from `conjunctions_ambiguous` (decisions/vocabulary-model, 2026-10-27 #753, items 1, 2 and 7; built 2026-10-29, phase 3b). A strict choice, `parser_for(..., strict=True)`, selects by file alone: only the chosen files and native-script vocabulary compile, `mul.toml`'s and `und.toml`'s readings leaving with every other file's (decisions/vocabulary-model, 2026-10-30 #753, file-strict; built 2026-10-31, phase 3j). So a bad default is routed around only by choosing the language whose reading is right, which is why #342-class fixes still change shipped vocabulary. The choice filters the base's shipped readings and the caller's own `add` and `remove` calls apply after it, so a choice never drops a caller's word, and one that clashes with a reading the choice turns on is refused at `parser_for`, naming the edit; a lexicon built from `Lexicon.empty()` holds no shipped reading, so a choice turns none on in it; a pack choosing no language leaves the base's exactly as it was. The choice is recorded on the `Lexicon`, so `parser_for(EN, base=parser_for(PT))` chooses both, as `parser_for(PT, EN)` does (decisions/vocabulary-model, 2026-11-02 #754, the `Lexicon` surface, items 3 and 4; built 2026-11-02, sub-phase 4i-c). Pure data means a pack can be audited: a vocabulary pack's whole content is its tag, but what it changes reaches other files too, prefer dropping their readings of its words (a `de` choice drops `ca.toml`'s conjunction `i`), so the audit is the derived surface, `changed_words`, not the pack's file alone.
338338

339-
**Lives in.** nameparser/locales/ (packs), nameparser/_parser.py (parser_for, the one applier), nameparser/_reading_index.py (the selection, `choose`, and the derived surface, `changed_words`), nameparser/_locale.py (`rules_patch`, a file's compiled `[rules]` as a patch) and tools/vocab/build.py (`RULE_KEYS`, the rules a file may turn on, and `PACK_RULES`, the files the shipped packs read; decisions/vocabulary-model, 2026-10-29 4f). A rule a file turns on is a `Policy` field; a new rule kind is a core change, never code in a pack: a new `Policy` field (with `ADDED_IN`), a row in build.py's `RULE_KEYS` and in `_locale._RULE_VALUES`, and a mechanisms.md entry of its own for the rule kind.
339+
**Lives in.** nameparser/locales/ (packs), nameparser/_parser.py (parser_for, the one applier), nameparser/_lexicon.py (the compile under a recorded choice, `_chosen_pool` and `_built`), nameparser/_reading_index.py (the selection, `choose`, and the derived surface, `changed_words`), nameparser/_locale.py (`rules_patch`, a file's compiled `[rules]` as a patch) and tools/vocab/build.py (`RULE_KEYS`, the rules a file may turn on, and `PACK_RULES`, the files the shipped packs read; decisions/vocabulary-model, 2026-10-29 4f). A rule a file turns on is a `Policy` field; a new rule kind is a core change, never code in a pack: a new `Policy` field (with `ADDED_IN`), a row in build.py's `RULE_KEYS` and in `_locale._RULE_VALUES`, and a mechanisms.md entry of its own for the rule kind.
340340

341341
**Reach for it when.** A language fix wants an if-statement — make it vocabulary or policy in a pack instead.
342342

0 commit comments

Comments
 (0)