fix(#528): HumanName.initials() reads the parse's connective tags - #529
Merged
Merged
Conversation
The v1 facade decided "is this word the connective" from the lexicon plus v1's initial shape on the part's raw text, where the core's initials() reads the tag the parse recorded. #383/#479 gave rules.md#P3 a fork the shape cannot see -- in a name written wholly in one case a marked 'e' is an initial and an unmarked 'y' joins -- so the two views of one parse disagreed about the same letter. _list_for becomes one walk with two views: _list_tokens_for yields each list element's backing tokens, and the string view is built from it, so the element boundaries the walk exists to hold (a "joined" continuation, a folded middle) cannot drift between them. _process_initial keeps its v1 signature for direct callers and gains an optional `tokens`; with tokens each word asks the tag, falling back to _render._reads_as_conjunction -- case repair's own R4 helper -- for a token carrying UNCLASSIFIED_TAG, which is text a parse never read. _is_conjunction goes with its only caller. Measured over the 1174-name corpus glob: six names moved and the two views now agree on every name but 'Ph. D., John', whose divergence is the pre-existing Ph. D. merge. 'john e smith' gives j. e. s. where 1.4.0 through 2.3.0 gave j. s.; 'JUAN Y GARCIA' gives J. G. where they gave J. Y. G. Accepted cost, the R4 shape: for a word backed by a token the answer is fixed at parse time, so a C.conjunctions edit takes effect on the next full_name assignment, as capitalize()'s already did. An unpickled name is spliced text throughout and takes the vocabulary fallback, which capitalize() has done since __setstate__ stamped the tag. And a stated private-method break: a v1-shaped subclass overriding _process_initial(self, name_part, firstname=False) now raises TypeError from initials(), since initials() passes tokens= -- such an override must accept it. Pinned by a test and named in the method. Ledgers: one new _initials rule at 1.4.0 over the four names whose roles do not move -- 'john e smith', 'john e jones', 'jones, john e', 'JUAN Y GARCIA' -- written ahead of fix(initials-per-word)'s connective rule, whose equal `fields` make file order the whole decision on the two names both reach; _CROSS_RULE_WINNERS and _RECORDED_DIFFS pin that. At 2.0.0-2.2.0 fix(#462)'s two-cause paragraph becomes single-cause, the facade half having closed; at 2.3.0 both surfaces now move together. The prose that said the facade does not follow the tags is corrected where the sweep found it, tests/v2/cases.py's two #383/#479 notes included. Five gates at 0 unexplained. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Quality-review fixes on 919644b's ledger and guard prose, comments only. The #528 rule's comment said "'j. s.' -> 'j. e. s.' on the three e-names", which is true of one of them: measured against the 1.4.0 wheel, 'john e jones' and 'jones, john e' go 'j. j.' -> 'j. e. j.'. Each of the four is now stated on its own. "Six corpus names move `HumanName.initials()`" was an undated standing count and false read literally -- 'e and e' and 'juan garcia y lopez' move here too, on the 2.0.0 per-word grouping. It becomes a dated, scoped statement in both places it is written: six one-case names move for the TAG reason, measured 2026-09-13 by driving the facade before and after the change, two of them ('jose e maria santos', 'JUAN GARCIA Y LOPEZ') moving a role as well so that `_initials` never enters their diff. The equal-`fields`/file-order argument and the role-mover exclusion were each written twice inside expected_since_1.4.0.toml. The block header keeps both copies and the rule's comment points at it; the copies in compare.py and tests/v2/test_ledger_guards.py stay, being the cross-file convention. Also: _CROSS_RULE_WINNERS now gives one property per uncontested name ('jones, john e' by the trailing-whitespace requirement, 'JUAN Y GARCIA' by the case) rather than both at once; _CORPUS_CLAIMS' pronoun pointed at three rules where the two shared names are the connective rule's alone; the 2.0.0 ledger says "the wheel's FACADE gives 'J. G.'", the wheel's core giving 'J. Y. G.'; the "a connective run initials" roster's third stacked paragraph folds into a dated correction of the second; _MUST_NOT_MATCH said "the first four are MIXED-CASE" where a ninth probe had made it five; and cases.py's run-on splits. _RECORDED_DIFFS' two new rows move below #498's block so the three cohorts read down the dict in the order the PROVENANCE note tells them, and that note goes 45 -> 47 with #528's two named and dated. All five gates re-run at 0 unexplained and no moved shape. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
decisions.md gains the R3 entry for #528: the facade's initials view reads the parse's tags, in R4's shape -- a word backed by a token asks its tags, a token carrying UNCLASSIFIED_TAG was read by no parse and the vocabulary answers through the same helper _cap_word calls. Its two accepted costs are measured and pinned: the connective answer is frozen at parse time, so a C.conjunctions edit waits for the next full_name assignment, and an unpickled name is spliced text throughout and takes the fallback, both as capitalize() has behaved all along. The stated private-method break is recorded with the remedy that actually works -- an override of _process_initial must ACCEPT `tokens` AND FORWARD it, since widening the signature alone leaves the override reading the token path's empty name_part and initials() comes back "" without raising. Measured, and now pinned by test_an_override_that_forwards_tokens_keeps_working with the widen-only spelling as its negative control. The entry also carries the one corpus name whose two views still differ ('Ph. D., John', the 2.0.0 per-word merge and not this change) and the ledger picture at all five baselines. P3's facade paragraph, which deferred this to a follow-up, is closed out in place and points here. rules.md#R3 and #R4 now distinguish the TWO initials views. The two agree on which words initial in a parsed name; what still differs is GROUPING, the facade initialing a joined run as one element. And on a spliced field the facade falls back on BOTH questions where the parsed name's own view falls back on neither -- the connective question through case repair's helper, the particle question through a live lookup, so a family spliced to "de la vega" initials "j. v." on one view and "j. d. l. v." on the other. mechanisms.md: FOLDED_TAG credits the walk that does the folded-first ordering rather than the string view built on it; VOCAB-TAGS adds the facade as a tag consumer, a module a sweep stopping at _render.py misses; RENDER-HONORS-THE-PARSE gains the second reader of the vocabulary fallback and RETRACTS, for the facade view only, its old "nothing observable" conclusion about the _INITIAL sync divergence -- with `conjunctions.add("太")`, 'Wang Chen 太. Li' initials 'W. C. L.' parsed and 'W. C. 太. L.' with the identical family text spliced in, while capitalized(force=True) shows nothing either way. Accepted rather than repaired, the remedy differing by view: revise() for the parsed name, a full_name reassignment for the facade, which has no revise. usage.rst's twin sentence is scoped to the parsed name's own views. docs/release_log.rst gains the 2.4.0 Fix bullet for #528 -- with the forwarding remedy, and with the 2.0.0-2.2.0 values stated rather than folded into "1.4.0 through 2.3.0", which is false for the two capital names -- and the #527 bullet's now-wrong facade sentence is corrected in place. tools/differential/README.md's "32 of the 45 today" becomes the dated form; the rows are 47 since #528. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #529 +/- ##
=======================================
Coverage 98.68% 98.69%
=======================================
Files 45 45
Lines 3347 3366 +19
=======================================
+ Hits 3303 3322 +19
Misses 44 44 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
- initials_separator was unguarded on the live token path: the two
existing "Van Berg" tests exercise _process_initial's direct-string
branch only. Pin it on HumanName.initials()/initials_list() over
"Ph. D., John" instead, and note on the two direct-call tests that
they cover the string path alone.
- Add "john e jones" and "jones, john e" to the one-case-fork test:
they back the 1.4.0 ledger rule, and the comma form is the shape
where the connective-run regex never reaches the trailing "e".
- test_list_tokens_for_carries_the_list_view_s_own_elements asserted
the string view against _list_for(member) and that every group is
truthy -- both true by construction (_list_for IS that join; an
empty groups list is vacuously "all true"). Assert the string view
against its own per-case literal instead, and the broader sweep
against a second call for determinism.
- Rename test_initials_of_an_unpickled_name_ask_the_vocabulary_too:
copy.copy/copy.deepcopy go through the same __getstate__/__setstate__
hooks as pickle, so a copied name takes the identical vocabulary
fallback. Measured and pinned alongside the pickle case.
- Fix two stale comments: the mixed-case-controls comment covered a
one-case control ("maria y lopez") as if it were mixed-case, and
"joined" is healed by _list_tokens_for now, not _list_for (which is
only the string view built from that walk).
- Correct the corpus count: 1173 non-empty names, out of 1174 distinct
names in the glob (one is empty), not "1173 names".
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- _facade.py/_render.py: two stale "_list_for's heal" / "text the parse never classified" comments corrected to name _list_tokens_for and the actual reader (a spliced field or a direct string call); the #232 doubled-space paragraph moved down into the tokens-is-None branch it now describes, and reworded to say so. - The accepted pickle cost is really a __getstate__/__setstate__ cost: copy.copy and copy.deepcopy take it too, measured. Widened the ACCEPTED COST paragraph in decisions.md#R3, the release_log.rst 2.4.0 Fix bullet, and the one _facade.py comment this PR added that named the pickle path alone (usage.rst and mechanisms.md's existing pickle prose predates this PR and is untouched). - decisions.md#R3's STATED PRIVATE-METHOD BREAK paragraph undersold its own scope: "the two initials values above" -> "the six names above", matching what the rest of the entry already says. - rules.md#R3/#R4: the particle question is not a fallback for the facade's initials view -- _is_particle is a live vocabulary lookup for every word, backed or spliced alike, and the facade never reads a particle tag. Reworded both paragraphs to say so, and disambiguated "in both views" (which now reads as {parsed, facade} next to the new facade paragraphs) to "in both of the parsed name's views" -- there is no HumanName.revise(). - decisions.md#P3: this PR had rewritten a dated entry that already landed in #527, instead of appending a new one. Restored the original "ACCEPTED, AND DEFERRED TO A FOLLOW-UP ISSUE" bullet verbatim and appended a new 2026-09-13 #528 paragraph carrying what the rewrite said, scoping "all along" to the window it actually covers (2.3.0 gave "John e Smith"). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The design-docs review of the previous commit found three prose slips. The restored 2026-09-13 P3 bullet had dropped the sentence its own amendment points back to; it is byte-exact against master again. The #528 entry's "six names above" pointed at the wrong names, so the six are now listed. R3's enumeration of where the facade's initials view falls back said pickle only, while the decision entry and the release note already said pickle or copy. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
_initials_lists called self._process_initial("", firstname=firstname,
tokens=group) -- the empty placeholder never mattered on the token
path, since _process_initial ignores name_part whenever tokens is
given. But a v1-style subclass that widens _process_initial's
signature to swallow `tokens` (`**kwargs`, or an unforwarded
`tokens=None`) without forwarding it takes the STRING path in the
super() call, reading that same "" for name_part -- so initials()
came back "" for every group, silently, for such an override.
Pass the group's real text instead, " ".join(tok.text for tok in
group), which _list_tokens_for already builds the tuple form of (no
extra parse-path walk: _initials_lists walks tokens directly and
never calls the cheap _list_for string view). A correct override
that forwards tokens is unaffected, since the token path still wins
whenever tokens is passed.
Before: WidensOnly("john e smith").initials() == ""
After: WidensOnly("john e smith").initials() == "j. s." (the
PRE-#528 answer -- the STRING path's vocabulary fallback
reads "e" as the connective, where the token path's
parse-backed reading, restored by #528, does not)
Forwarding still receives the fix either way:
Forwards("john e smith").initials() == "j. e. s."
HumanName("john e smith").initials() == "j. e. s."
Updates the STATED BREAK comment in _process_initial, the R3 entry
in docs/design/decisions.md (written in this PR, not yet on master,
amended in place), the 2.4.0 Fix bullet in docs/release_log.rst, and
tests/v2/test_facade.py's WidensOnly negative control to assert the
new pre-#528 answer instead of "".
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Second review round on #528 found that _initials_lists moved from reading self.first_list/middle_list/last_list -- public properties a v1 subclass may override -- to calling self._list_tokens_for(member) directly. A subclass overriding one of those properties is now silently ignored by initials(), while last_base/surnames/given_names (via _split_last, which still reads self.last_list) keep honoring it. Measured before this fix (three-property Sub, "john Xavier smith"): pre-#528 (338daf7): "J. Z." 84d9000 (the bug): "j. X. s." this commit: "J. Z." And a one-property override (SubLastOnly.last_list -> ["Zorro"], on "john e smith"), pinning that an un-overridden member keeps the (post-#528) token path: this commit: "j. e. Z." Fix: _initials_lists now checks, per member, whether the class overrides that property (getattr(type(self), f"{member}_list") is not getattr(HumanName, f"{member}_list") -- three cheap identity checks, nothing on the parse path). An overridden member takes the pre-#528 STRING path over the override's own strings (reproduced from `git show 338daf7:nameparser/_facade.py`, all-particle readmission and the `if n` filter included); an un-overridden member is unchanged. Also: zip(words, conjunctions, strict=True) in _process_initial, so a future filter on one branch raises instead of silently truncating. Verified 0 differences in HumanName(n).initials() across every name in tools/differential/corpus*.jsonl (527 distinct) between 84d9000 and this tree -- the plain, un-overridden facade path is unchanged. tests/v2/test_benchmark.py stays 17 passed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Second review round on #528/the *_list-override fix found several comments that had drifted from what the code and tests actually do: - tests/v2/test_facade.py: the "deterministic" claim on _list_tokens_for's repeat-call assert was false (the method rebuilds fresh lists every call); reworded to what the assert actually pins -- an equal grouping, which a one-shot walk behind tokens_for() would not give a second time. - tests/v2/test_facade.py: fixed the __getstate__/__setstate__ attribution -- HumanName defines its own pair in _facade.py, not nameparser/_types.py's guarded pair (a different set of classes). - tests/v2/test_facade.py: "initial-shaped" misused the repo's term (matches _render._INITIAL's shape, which "e" does not); reworded to say the parse TAGS "e" an initial via rules.md#P3's one-case fork, which is exactly why the bare-word vocabulary fallback disagrees. - tests/v2/test_facade.py: added the multi-token-group pin the reviewer measured on "Ph. D., John" (HumanName/Forwards/WidensOnly all "J. P D.", WidensOnly with initials_separator="-" giving "J. P-D."), the one shape that exercises the join's actual form. - tests/v2/test_facade.py + nameparser/_facade.py: added tests and a comment pinning that _token_is_conjunction's _resolve() call is load-bearing for a keyword-constructed HumanName, not just for unpickled/copied instances. - tests/test_initials.py: corrected which of #528's four movers has rows in _CROSS_RULE_WINNERS/_RECORDED_DIFFS ("john e jones", not "jones, john e") and why -- the connective-run ledger rule's operative half is a `(?=\s)` lookahead that a string-final letter can't satisfy -- and named the rule instead of describing it loosely. - nameparser/_render.py: _reads_as_conjunction's docstring and its _INITIAL comment claimed it is asked "only of text the parse never saw", which 84d9000 falsified -- a widen-only _process_initial override now sends parse-classified text down this path too. Reworded around the caller's contract (no token passed, whether or not the parse read the text) and added __setstate__ (pickle, copy.copy, copy.deepcopy) as a producer alongside replace(). - docs/design/decisions.md, the unlanded 2026-09-13 #528 R3 entry: recast the remedy illustration onto "john e smith" (Forwards gives "j. e. s.", WidensOnly "j. s.") since every widened override gives "John Quincy Smith" its "J. Q. S." back regardless, making that example vestigial; added a paragraph recording this PR's F1 fix (an overridden public *_list property is honored, taking the pre-#528 string path for that member -- measured "J. Z." and "j. e. Z." on the two Sub subclasses); corrected the copy.copy/copy.deepcopy measurement date (2026-09-14, not 09-13 -- the pickle half stays 09-13); and added the population figure: over the deduped tools/differential/corpus*.jsonl glob (1174 distinct names) exactly 6 give a different initials() after a pickle/copy round trip than live, the same six the STATED BREAK paragraph names. - docs/release_log.rst: the 2.4.0 Fix bullet now also states that a subclass overriding one of the public first_list/middle_list/ last_list properties keeps working, that member taking the pre-2.4 vocabulary reading. The P3 bullet near decisions.md line 330 (landed on master) is untouched; verified byte-identical to the 338daf7 copy. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Widens the unlanded #528 R3 entry's second accepted cost from "the state-restoration path" to all three no-parse paths: pickle/copy.copy/ copy.deepcopy AND the keyword constructor (HumanName(first=..., ...) never runs the tokenizer either), with measured keyword-constructor numbers and a corpus-wide recompute (1174 distinct names via the two line shapes in tools/differential/corpus*.jsonl, 1 mover before #528, 7 now). Widens the release_log.rst 2.4.0 Fix bullet's parenthetical to match. Corrects the overridden-*_list paragraph's fixture description (all three overrides, not just last_list) and adds its "Pinned at" pointer. Fixes a misattributed parenthetical (only last_base routes through _split_last; surnames and given_names never do). Fixes "all four" to "all three" for the pickle/copy.copy/copy.deepcopy count. Adds a one-sentence reading recipe for "(1174 distinct names)" at its first occurrence. Renames the wrong method in the _initials_lists docstring in nameparser/_facade.py (_list_tokens_for, not first/middle/last, is the private token walk). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The decisions entry's second cost now names three no-parse paths and cites one test; that test covered two of them. A name rebuilt from the live parse's own fields now asserts the same fallback values, so the citation covers what the paragraph claims. The override test's comment also repeats the corrected attribution: only last_base goes through _split_last; surnames and given_names read the properties directly. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #528.
HumanName.initials()now reads "is this word a connective" from the parsed token's tags wherever a token backs the word — aconjunction-tagged word contributes no initial, aninitial-tagged or untagged one contributes — and falls back to the vocabulary and shape only for spliced raw text (hn.middle = ...,replace(), the v1 pickle path), through the same_render._reads_as_conjunctionthat R4's case repair uses. Particles stay a live vocabulary lookup, as_cap_wordkeeps them.HumanName("john e smith").initials()givesj. e. s.where 1.4.0 through 2.3.0 gavej. s.;JUAN Y GARCIAgivesJ. G.where 1.4.0 and 2.3.0 gaveJ. Y. G.(2.0.0 through 2.2.0 gaveJ. G.for the #462 reason). Mixed-case names are untouched;maria y lopezstaysm. l.becauseyis outside the marked set. The facade and the core now agree on 1173 of the 1174 corpus names; the one left,Ph. D., John, is the per-word grouping of a joined run (fix(initials-per-word)), not this issue's.Three accepted consequences, recorded at decisions.md#R3: the connective answer is fixed at parse time for a token-backed word, so a
C.conjunctionsedit after parsing takes effect on the nextfull_nameassignment, ascapitalize()already behaves; a pickled facade re-loaded takes the fallback, so its initials can diverge from the live parse, again ascapitalize()already does; and one stated private break — a v1 subclass overriding_process_initialwith the two-argument signature now raisesTypeErrorfrominitials(), and the remedy is to accepttokens=and forward it tosuper()(widening the signature alone returns""for every group, measured and pinned as the negative control).Measured: six one-case corpus names' facade initials move for this reason; two of them also move a role against 1.4.0, so four carry the new 1.4.0
_initialsrule, and the equal-fieldscontest with the connective-run rule is pinned by file order and_CROSS_RULE_WINNERS. At 2.0.0 through 2.2.0 nothing new diffs (the e-names hide behind their_ambiguitiesreport;JUAN Y GARCIA's two-cause row became single-cause). Design review (design-docs-reviewer) ran on the docs/design diff; its findings landed in the docs commit, including a measured facade witness that retracts the old "nothing observable" conclusion about_INITIALversus the pipeline in a no-initials script for the facade view.Three commits: the fix with its tests, ledgers and guards; the ledger comments corrected to what was measured; the record.
Gate: 422 / 349 / 263 / 126 / 10 intentional at 1.4.0 / 2.0.0 / 2.1.0 / 2.2.0 / 2.3.0, 0 unexplained, 0 radar unclassified. Full suite 7964 passed; sphinx doctest 250 / 0;
sphinx-buildhtml clean; mypy and ruff clean.🤖 Generated with Claude Code