Skip to content

Commit 98a19d2

Browse files
derek73claude
andcommitted
Add nameparser 2.0 design RFC
The design document for 2.0: a new immutable core API (parse() -> ParsedName, Lexicon/Policy configuration) alongside HumanName as a compatibility layer through 2.x. Covers the compatibility promise, the new API, migration mechanics, locale plans, rejected alternatives, and the feedback questions. Discussion: #284. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 8f570ce commit 98a19d2

1 file changed

Lines changed: 294 additions & 0 deletions

File tree

‎docs/design/nameparser-2.0-rfc.md‎

Lines changed: 294 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,294 @@
1+
# RFC: nameparser 2.0 — a new core API
2+
3+
Status: **open for feedback** — no deadline. Implementation may proceed
4+
in parallel; this document will be amended (as commits to this PR) as
5+
implementation teaches us things. Comment on specific lines here, or on
6+
the discussion issue (#284) for general reactions.
7+
8+
## The short version, and the promise
9+
10+
nameparser 2.0 introduces a new, immutable core API and keeps the
11+
existing one working:
12+
13+
```python
14+
from nameparser import parse
15+
16+
name = parse("Dr. Juan Q. Xavier de la Vega III (Doc Vega)")
17+
name.given # "Juan"
18+
name.family # "de la Vega"
19+
name.family_base # "Vega"
20+
name.title # "Dr."
21+
```
22+
23+
**The compatibility promise:** code that runs warning-free on nameparser
24+
1.4 keeps running on every 2.x release, with identical results, getting
25+
at most `DeprecationWarning`s. `HumanName` continues to exist as a
26+
compatibility layer over the new core. The exceptions to "identical
27+
results" are (a) parse-output changes that are deliberate bug fixes,
28+
each documented in the release log with its issue, and (b) three narrow
29+
edge cases listed in the Migration section. Removal of the old API
30+
happens in 3.0, not 2.x, and only after warnings have named every
31+
replacement.
32+
33+
If you only use `HumanName` and never customize configuration, 2.0
34+
should be a non-event for you. The rest of this document is for
35+
everyone else — and for anyone who wants to push on the new design
36+
while it is still cheap to change.
37+
38+
## Why a new core
39+
40+
Fifteen years of issues point at the same architectural limits:
41+
42+
- **`HumanName` is four things in one mutable class** — parser, state,
43+
vocabulary predicates, and formatter. Every feature grows all four at
44+
once (it is currently ~1,600 lines), and mutation semantics are
45+
inconsistent: assigning `full_name` re-parses, assigning `last` does
46+
not, `capitalize()` mutates in place.
47+
- **Parse state is `list[str]`**, so position information is lost and
48+
re-derived by value lookups. That is the structural root cause of a
49+
whole family of prefix-join bugs (#100 and relatives): when the same
50+
string appears twice, the lookup answers the wrong question.
51+
- **Configuration is a shared mutable module global** (`CONSTANTS`).
52+
Most of `nameparser.config`'s ~850 lines exist to make shared
53+
mutation survivable (custom set managers, descriptors, cache
54+
invalidation), and it still causes test pollution and cross-thread
55+
surprises.
56+
- **Equality is broken by design** (#223): case-insensitive equality,
57+
equality with plain strings, and hashability are mutually
58+
inconsistent promises. 1.3.0 deprecated `==`/`hash()` and shipped
59+
`matches()`/`comparison_key()`.
60+
- **The roadmap needs structure v1 can't cheaply provide**: family-first
61+
name order (#270), per-dataset configuration without global mutation,
62+
and honest reporting of genuinely ambiguous parses.
63+
64+
The rules-based approach is not changing. nameparser's niche is
65+
deterministic, auditable, zero-dependency parsing — the same input
66+
always parses the same way. The rewrite reorganizes *how the rules run*,
67+
not what the library is.
68+
69+
## The new API
70+
71+
### Parsing produces an immutable value
72+
73+
`parse()` returns a `ParsedName`: a frozen dataclass. There is no
74+
mutation anywhere in the new API — editing is `replace()`, which
75+
returns a new value:
76+
77+
```python
78+
name = parse("John Smith")
79+
name2 = name.replace(given="Jane") # new object; `name` unchanged
80+
```
81+
82+
`parse()` is **total over `str`**: any string — empty, emoji,
83+
garbage — returns a `ParsedName` rather than raising. Content problems
84+
are represented in the result; the only exceptions are type errors
85+
(`bytes` raises `TypeError` — decode first).
86+
87+
Equality is strict structural equality, and the semantic comparisons
88+
are explicit and separate — `comparison_key()` for dedup/sorting and
89+
`matches()` for component-wise case-insensitive comparison — resolving
90+
the #223 trilemma by not conflating the three meanings of "equal".
91+
92+
### Two access layers: strings for the 95% case, tokens for the rest
93+
94+
The string properties behave like v1's: always `str`, `""` when empty.
95+
Underneath, `ParsedName` carries the actual parse: a tuple of `Token`s,
96+
each with its **span** (position in the original string), its role, and
97+
tags recording *how it was classified*:
98+
99+
```python
100+
name = parse("Juan de la Vega")
101+
name.tokens_for(Role.FAMILY)
102+
# (Token('de' @5:7 FAMILY {particle}),
103+
# Token('la' @8:10 FAMILY {particle}),
104+
# Token('Vega' @11:15 FAMILY),)
105+
```
106+
107+
This is the structural fix for the value-lookup bug family: pipeline
108+
stages address tokens by index, so "which 'de' did you mean?" cannot
109+
arise. It is also the debuggability story — "why is 'Van' in my family
110+
name?" is answered by the token's tags, and `repr(name)` prints a
111+
compact component-per-line breakdown like v1's.
112+
113+
Derived views (`family_base`, `family_particles`, `surnames`,
114+
`given_names`) are pure filters over tokens — they can never disagree
115+
with the parse.
116+
117+
### `given` and `family`, not `first` and `last`
118+
119+
The v1 field names describe Western *position* — they date from before
120+
the parser even supported comma formats, when position was all there
121+
was. They read wrongly the moment name order varies: under family-first
122+
order, `name.last` would return the first word. 2.0 adopts the names
123+
the standards world already uses (HTML autocomplete
124+
`given-name`/`family-name`, vCard, Unicode CLDR):
125+
126+
- `first` → `given`, `last` → `family`
127+
- `last_base`/`last_prefixes` → `family_base`/`family_particles`
128+
- config: `prefixes` → `particles` (every web standard uses "prefix"
129+
for honorifics like "Mr." — v1's usage collides; the linguistics term
130+
for van/de la is *particle*)
131+
- `title` and `suffix` **stay** — they have ecosystem gravity, and the
132+
positional taxonomy ("pre-nominal"/"post-nominal") is documented
133+
rather than spelled into identifiers.
134+
135+
`HumanName` keeps every v1 spelling, including `string_format` keys, so
136+
existing code is unaffected; only new-API adopters see the new names.
137+
138+
### Configuration: `Lexicon` (vocabulary) and `Policy` (behavior)
139+
140+
v1's `Constants` mixes three kinds of thing; 2.0 splits them by what
141+
varies together:
142+
143+
- **`Lexicon`** — vocabulary data (titles, suffixes, particles,
144+
conjunctions, …). Immutable; composable by union; varies by
145+
*language*.
146+
- **`Policy`** — behavior switches (`name_order`, patronymic rules,
147+
delimiter pairs, …). Immutable; varies by *data source*.
148+
- **Rendering parameters** — `string_format`, initials style,
149+
capitalization — move to render-time arguments; they vary by *output
150+
destination* and are no longer configuration at all.
151+
152+
```python
153+
from nameparser import Parser, Lexicon, Policy, FAMILY_FIRST
154+
155+
parser = Parser(
156+
lexicon=Lexicon.default().add(titles={"dra"}),
157+
policy=Policy(name_order=FAMILY_FIRST),
158+
)
159+
parser.parse("Wang Xiuying").family # "Wang"
160+
```
161+
162+
A `Parser` is immutable, thread-safe, and does all its compilation once
163+
at construction — build it at startup, share it everywhere. There is no
164+
shared mutable configuration in the new API, which retires the entire
165+
test-pollution/config-war problem class. `name_order` is an order-spec
166+
tuple (`FAMILY_FIRST = (Role.FAMILY, Role.GIVEN, Role.MIDDLE)`) rather
167+
than a boolean, because Vietnamese order — family, middle, *then*
168+
given — needs a third arrangement (#270).
169+
170+
### Honesty about ambiguity
171+
172+
Some parses are genuinely undecidable by rules — "Kelly Michael" could
173+
be either order; in "Van Johnson", "Van" is a Dutch particle *and* a
174+
common given name. v1 silently picks a reading. 2.0 picks the same
175+
reading **and says so**:
176+
177+
```python
178+
parse("Van Johnson").ambiguities
179+
# (Ambiguity(PARTICLE_OR_GIVEN, "leading 'van' can be a particle or a
180+
# given name; chose given", tokens=(...)),)
181+
```
182+
183+
`Ambiguity.kind` values are stable API you can filter on. This is also
184+
groundwork: a future `parse_all()` returning ranked alternative
185+
readings, and `explain()` returning a stage-by-stage parse trace, are
186+
designed for (the pipeline supports both) but deliberately not in 2.0.
187+
188+
### Locales: opt-in packs, never auto-detection
189+
190+
Language-specific behavior cannot be auto-detected — "Tham Jun Hoe" is
191+
token-for-token indistinguishable from a Western three-token name, and
192+
"Ali" is Arabic or Italian. So locale support is explicit:
193+
194+
```python
195+
from nameparser import parser_for, locales
196+
197+
parser_for(locales.RU).parse("Ivanova Anna Sergeevna").given # "Anna"
198+
```
199+
200+
A locale pack is pure data: a vocabulary fragment plus a partial policy
201+
patch, folded in at parser construction. 2.0.0 ships `RU` and `TR_AZ`
202+
(formalizing the patronymic support added in 1.3.0); Chinese/Korean
203+
(bundled surname lists + segmentation of unspaced input), Vietnamese,
204+
and Japanese (via a `nameparser[ja]` extra with a pluggable segmenter)
205+
are designed and staged for 2.x minors. Vocabulary that cannot misfire
206+
on other languages' names (e.g. Cyrillic honorifics) goes in the
207+
default lexicon instead of packs.
208+
209+
## Migration: what actually happens to existing code
210+
211+
**`HumanName` in 2.0** is a wrapper over the new core with v1's exact
212+
mutation semantics, field spellings, and formatting attributes. The v1
213+
test suite runs against it as the compatibility gate. v1 pickles load
214+
throughout 2.x.
215+
216+
**`CONSTANTS` keeps working** — mutations are honored exactly (the
217+
facade rebuilds its config snapshot when the shared config changes) and
218+
emit a `DeprecationWarning` pointing at the replacements. Mutating a
219+
*private* `Constants` instance does not warn.
220+
221+
**Already-scheduled removals land as warned** (every one was deprecated
222+
in 1.3.0 or 1.4 first): `==`/`hash()`, bytes input, slice item access,
223+
`empty_attribute_default`, `constants=None`, silent config-typo
224+
fallbacks, legacy pickle blobs, `add_with_encoding()`. Python floor
225+
becomes 3.11; the last dependency is dropped.
226+
227+
**The three narrow behavior exceptions** (everything else identical):
228+
229+
1. `*_list` attributes return snapshots — in-place
230+
`name.first_list.append(...)` no longer mutates parse state
231+
(assignment still does).
232+
2. Subclasses overriding parsing hooks (`parse_pieces`, `is_title`, …)
233+
are detected and warned at construction: the facade delegates to the
234+
new core, so such overrides are no longer called (#280). If you do
235+
this, we want to hear what your override does — that is a feature
236+
request we would rather absorb than break.
237+
3. Assigning to `CONSTANTS.regexes` raises with a pointer to the named
238+
`Policy` flag that replaces it. Custom regex injection has no
239+
equivalent in the new model; nothing is silently ignored.
240+
241+
**Timeline:** 2.0.0's only *new* warning is on shared-`CONSTANTS`
242+
mutation. A warning on `HumanName` construction itself comes in a later
243+
2.x minor, once the new API has proven out — and 3.0 (which removes the
244+
facade) does not ship before that warning has been out in a release.
245+
246+
## Alternatives considered and rejected
247+
248+
- **Auto-detecting name language/order** — impossible in principle, not
249+
just hard; see the Locales section.
250+
- **A statistical/ML core** — determinism and auditability are the
251+
point of this library; the token/ambiguity model leaves room for
252+
optional escalation later without changing the core.
253+
- **Keeping `first`/`last`** — see the naming section; the facade keeps
254+
them for v1 code, which serves the familiarity constituency without
255+
wiring position-based names into the new API.
256+
- **A boolean `family_name_first`** — cannot express Vietnamese; the
257+
order-spec tuple can (#270).
258+
- **Renaming `title`/`suffix` to `pre_nominal`/`post_nominal`** — more
259+
precise, but no ecosystem uses those as field names; precision lives
260+
in the docs instead.
261+
- **Vocabulary entries as records with per-word metadata** — rejected
262+
under "no vocabulary data without a consuming rule": a flag the
263+
parser doesn't read is a promise it doesn't keep. Cross-category
264+
words are handled by dual set membership (closed classes) or the two
265+
`_ambiguous` sets (closed-vs-open), each of which has a consuming
266+
rule.
267+
- **A user-replaceable regex escape hatch** — replaced by named policy
268+
flags; anything not expressible through them is a feature request we
269+
want to see.
270+
271+
## Feedback we are specifically looking for
272+
273+
No deadline — comment whenever, on lines here or on the discussion
274+
issue. The questions where answers would most change things:
275+
276+
1. **Do you compare `HumanName` objects with `==` (especially against
277+
plain strings), or use them in sets/dicts?** 2.0 changes this to
278+
object identity (warned since 1.3.0; `matches()`/`comparison_key()`
279+
are the replacements). This is the one silent behavior change in the
280+
release.
281+
2. **Do you subclass `HumanName` and override parsing methods?** Tell
282+
us what your override accomplishes — we would rather absorb it as a
283+
feature than break it silently (#280).
284+
3. **Do you mutate `CONSTANTS` after startup** (not just at import
285+
time)? Anything beyond startup-time customization we should know
286+
about?
287+
4. **Do you assign custom regexes to `CONSTANTS.regexes`?** What for?
288+
Each real use case either becomes a named policy flag or stays
289+
broken — we would like the list to be complete before 2.0.
290+
5. **`given`/`family` instead of `first`/`last`** in the new API: does
291+
this help or annoy you?
292+
6. **Is there anything you build on top of nameparser** (wrappers,
293+
pipelines) where the compatibility promise as stated would still
294+
break you?

0 commit comments

Comments
 (0)