[derek73]

parse("John⁣Smith") now gives given John, family Smith. 1.4.0 through 2.8.0 gave a single given name, John⁣Smith (checked against the 1.4.0, 2.0.0 and 2.8.0 wheels).

What changes

• U+2063, INVISIBLE SEPARATOR, joins the unconditional separators. It is added to _tokenize._INVISIBLE_SEPARATORS, beside U+200B and U+FEFF (Should a zero-width space separate words, so ​王小明 reads as 王小明 does? #851). No policy reaches it, as none reaches those two. Unicode intends it as the invisible comma between indices in mathematical text.

• It behaves exactly as the zero-width space does in every shape Should a zero-width space separate words, so ​王小明 reads as 王小明 does? #851 settled. Beside the interpunct it reads as whitespace. It doesn't count as a space where a quote or maiden marker is recognized, so Jane (née⁣Jones) Smith reads nickname née Jones. After a hyphen it divides the name.

• Excluded, each with a test row:

  • the invisible math operators U+2061, U+2062 and U+2064, which join their operands rather than separate words;
  • U+180E, the Mongolian vowel separator, which decides how the letters beside it are shaped, as the joiners Should a zero-width space separate words, so ​王小明 reads as 王小明 does? #851 excluded do.

  name_tamer strips U+180E; doing that here would change how a Mongolian name is written.

Evidence

No real name holds any of the five characters: none of the 13 local corpora's raw.jsonl does, and in the committed corpora the only occurrence is rules/T1's new example. So no measured name moves; this is robustness against text pasted from documents. Found while comparing the tokenizer with dominicsayers/name_tamer's zero-width filter.

Docs

• rules/T1 (separators, not joiners): the When, Except and Accepted wording now names the invisible separator, plus one example.
• decisions/T1: the Standing bullet and a 2026-11-02 Log entry, with the exclusions.
• Also updated: a release-log bullet, and the two _tokenize.py comments that described the old two-member set.

Checks

• Full suite, mypy and ruff pass.
• Negative control: the 3 new separator rows fail with U+2063 taken out of the set.
• The design-docs reviewer found five small issues, all fixed:
  • the corpus-count sentence now accounts for the rule's own example, which the snapshot harvests;
  • the two stale _tokenize.py comments;
  • the hyphen bullet in rules/T1;
  • the release bullet's missing link.

🤖 Generated with Claude Code