Skip to content

Simplify the 3.0 cycle's rule-stage changes (post_rules, group, pieces, vocab, classify) - #1084

Open
derek73 wants to merge 2 commits into
masterfrom
refactor/simplify-rule-stages
Open

derek73 wants to merge 2 commits into
masterfrom
refactor/simplify-rule-stages

Conversation

@derek73

@derek73 derek73 commented Nov 5, 2026 •

Copy link
Copy Markdown
Owner

Pass (b) of the three pre-3.0.0 simplify passes. It covers only the rule stages (_post_rules, _group, _pieces, _vocab, _classify), limited to code changed since v2.8.0. This is a quality-only change: no reading moves.

Changes

  • suffix_kind. The credential arm of the suffix-word branch could never run, because the CREDENTIAL_WORD_TAG return above it already catches that tag. Removed.
  • _DOTTED_INITIALS. It now lives in _vocab, which builds _INITIALS_SHAPE from it. Before, the two modules held hand copies of one pattern, each declared its own source. _post_rules imports the same object, and test_regex_sync's roster records it.
  • P1's fold. Under a family-first order it computes the chain's reach once and passes it to _family_first_lead_folds. The restated lead_fires and test is gone.
  • P10's guard. The no-marked-word guard moved from _given_phrase_middle to its call site in post_rules. A name with no particle-given word now pays no call, which takes one frame off every parse.
  • name_word_count. It now checks the numeral's shape in C before calling ends_in_a_generation, as group's count already does. The shape match is a necessary condition, so no answer moves.
  • New test. test_a_dotted_member_keeping_a_comma_in_its_clause_is_set_apart fills the gap the test review found (see Reviews).

The first commit also folded two hand copies into shared predicates: a _dotted_member helper and split_pair asking suffix_as_written. The frames review showed each added a frame for every dotted clause word and every Ph./D. pair, so the fix commit reverted both to their inline forms (see Reviews).

Proof that nothing moved

  • tools/differential/snapshot.jsonl is byte-identical to master (git diff --quiet origin/master -- tools/differential/snapshot.jsonl). test_snapshot.py passes and nothing was re-recorded.
  • cldr_agreement and every doc test, test_doc_citations.py included, pass unchanged.
  • The full suite passes on 3.11 and 3.15: 37,298 passed, 29 skipped, 5 xfailed on each.
  • uv run pytest -n0 -m serial passes, 54 of 54 on each interpreter. Three wall-clock scaling rows failed one run each during this work, one of them bracket_members. Each passed alone 3 of 3. The load average during those runs was 40–48, from parallel sessions.
  • The test_benchmark.py scaling and shift-count tables are unchanged.
  • uv run mypy and uv run ruff check are clean.

Frames per parse

Measured with sys.setprofile "call" events, master (git archive at 5138879, path asserted) against the branch head. Each cell shows the default order / FAMILY_FIRST.

name 3.11 master 3.11 branch 3.15 master 3.15 branch
Dr. Juan0001 de la Vega III (reference) 312 / 313 311 / 312 293 / 294 292 / 293
Smith, John 165 / 164 164 / 163 143 / 142 142 / 141
John Smith 155 / 156 154 / 155 138 / 139 137 / 138
John Smith, Jr. 212 / 213 211 / 212 189 / 190 188 / 189
Smith, John Q. Jr. 241 / 240 240 / 239 219 / 218 218 / 217
John (MA) Smith 239 / 240 238 / 239 217 / 218 216 / 217
St de Mesnil Jean 272 / 310 271 / 303 249 / 283 248 / 277
Freiherr von Richthofen 277 / 292 276 / 286 253 / 268 252 / 263
María del Carmen García 334 / 304 334 / 304 310 / 285 310 / 285
  • Every row drops by at least one frame, from P10's guard, except the P10 name María del Carmen García, which stays level.
  • Family-first chains drop further, because their reach is computed once.
  • No row rises.
  • The reference row is at 311 against _CALL_BASELINE's 307, +1.3%, inside the ±2% band. Pass (a), Simplify the output side's 3.0 changes (assemble, types, render) #1085, takes about 5 more off every parse, and these changes stack on top of that.

Reviews

  1. Behavior equivalence. It argued each change equivalent on every input, then checked with a grid run on both trees.
    • The grid: 1,733 names (130 core shapes, each in upper, lower and NFD spellings and without periods, crossed with comma parts, an empty head and a leading title) under 14 parsers. Those were the three orders, two_family_names, two_given_names, middle_as_family, and four phd lexicons (removed, ambiguous, as a word credential, with a given reading).
    • Each row compared repr, ambiguity kind, detail and alternative, token readings, initials() and capitalized().
    • Result: 0 of 24,262 rows differ, and the outputs are byte-identical. A control run with a swapped order moved 4,762 rows, which proves the harness can see a difference.
  2. Frames. It compared 160 names on both trees on 3.11 and 3.15. It found _dotted_member adding +1 per dotted clause or nickname word (John (M.B.A, M.A) Smith +3) and split_pair's call into suffix_as_written adding +1 per Ph./D. pair. Both ran on paths that grow with the input. Resolved: both reverted to the inline tests in 22dc6f1. Every other site removes frames or leaves them unchanged.
  3. Comments. Every comment and citation quote was found accurate, and no removed text carried history. It found two problems: the S1 citation block was left heading the inserted _dotted_member, and test_layering's count note named S1 for an S13 load. Resolved: both lived in code the fix commit reverted.
  4. Test coverage. It mutated each refactored site in a scratch copy, and the suite caught every mutation but two. One was a real gap: dropping .strip(",") from S1's set-apart dotted-member test failed nothing, while John Smith (M.A, PhD) reads nickname M.A, PhD without it. Resolved: a direct test in test_post_rules.py, whose control I ran: it fails with the strip removed. A cases.py row was tried first and tripped test_stage_field_ownership (see follow-ups). The other surviving mutation, _family_settled's default-reach path, is masked by the main fold and unchanged in behavior.
  5. Design docs. Not needed: no file under docs/design/ and no AGENTS.md changed.
  6. Second round on 22dc6f1. No defects found.
    • git diff origin/master over nameparser/ shows only the five surviving changes. The reverted hunks match master byte for byte, with no leftover import or comment, and test_layering.py is identical to master.
    • A 17-name grid under the three orders was identical on both trees, with a mutant control that moves 18 lines.
    • The new test's comment is accurate, and the test fails with the strip removed.
    • Two notes, neither a code defect. First, 22dc6f1's message misstates the mutant's output: it reads suffix '' and nickname 'M.A, PhD'. Second, the separate strip that rewrites the word's text (_post_rules.py:738, master code) is pinned by no test; dropping it reads John (Dr, Prof) Smith as title 'Dr, Prof'. That gap is in the follow-ups below.

Behavior-changing candidates (not applied)

Each was measured by patching a scratch copy of the branch and parsing the 6,939 snapshot names plus a 22-name grid under all three orders against the unpatched branch.

  1. One dotted-member test for both sites, stripping the kept comma in both. Raised by the simplification, reuse and altitude reviewers. The delimited-member report would strip the comma as the set-apart pass does. It moves 3 grid names and 0 snapshot names. Each move adds a suffix-or-nickname report: John (M.A,) Smith, John (M.A.,) Smith, (M.A,), PhD.
  2. set_apart_kind asking suffix_as_written instead of its own suffix test. Raised by the altitude reviewer. A joined credential in a clause becomes the suffix: John Smith (MD/PhD), John (MD/PhD) Smith and John Smith (MD-PhD) read suffix where they read nickname. It moves 3 grid names and 0 snapshot names. It would also absorb the separate dotted-member test, but loses the member flag rules/S13 reads, so it needs a design pass.
  3. generation_word_before_a_name delegating its "name word behind" test to _pieces._name_word_behind. Raised by the altitude reviewer. The two exclude differently: initials, connectives, titles, maiden markers, shape-read numerals, and capitals in a mixed-case name. It moves 6 names, 2 of them snapshot names, and all for the worse:
    • Eric H. Holder Jr. Attorney General loses title and suffix (middle H. Holder Jr. Attorney).
    • dr. Margaritta Marionna Maria de Planta jr. CF reads family CF.
    • John Smith Jr. CF, John Smith Jr. J. Jones, John Smith Jr. Dr. Jones and John Smith Jr. JONES each move a suffix run into the name.
      Not a candidate as written. Any merge needs a rules/S8 decision.

Follow-ups outside this pass's files

  • The stage-ownership map omits a post_rules write. S1's set-apart pass writes a token's text and span when it trims a kept comma. The map in ParseState's docstring and tests/v2/pipeline/test_state.py doesn't list that write, and it has passed vacuously because no case row trims one.
  • S1's set-apart text rewrite has no test. The second strip at _post_rules.py:738 rewrites a word's text. No test pins it: John (Dr, Prof) Smith reads title 'Dr, Prof' without it. It is master code, and the same write the stage map omits.
  • _comma._whole_name's C3 numeral clause. A C-level ROMAN pre-check would save two frames per suffix-comma name with two or more words before the comma. _comma.py belongs to no pass.
  • _post_rules._idx. About 10 calls per parse; grouping them would save about 20 frames on 3.11. That code predates v2.8.0.
  • ParsedName.order validation (about 5 frames per parse). Taken by pass (a) in Simplify the output side's 3.0 changes (assemble, types, render) #1085.

Considered and not taken

  • The merge_pieces comment's join_connectives history and its 2.4x–2.9x / 54,877-name measurement. decisions/ doesn't hold them, so deleting the comment would lose them.
  • A _has_letter helper for P10's and O7's letter tests, and one loop for O7's two tag tests. Both are cosmetic and on rare paths.

🤖 Generated with Claude Code

Quality-only pass over the changes since v2.8.0 to _post_rules, _group,
_pieces, _vocab and _classify; no reading moves (snapshot byte-identical).

- suffix_kind: drop the credential arm of the suffix-word branch, which
  the earlier CREDENTIAL_WORD_TAG return makes unreachable.
- split_pair: the Ph./D. test asks suffix_as_written, whose dotted and
  joiner arms bare `phd` cannot reach, instead of a hand copy of its
  middle arm.
- _post_rules: the dotted-member test of S1's set-apart pass and of the
  delimited-member report is one helper, _dotted_member, each caller
  passing the text it passed before and asking "." first in C.
- _DOTTED_INITIALS lives in _vocab, which builds _INITIALS_SHAPE from
  it; _post_rules imports the same object (test_regex_sync roster).
- P1's fold computes the chain's reach once under a family-first order
  and hands it to _family_first_lead_folds; a restated lead_fires test
  goes.
- P10's no-marked-word guard moves to post_rules' call site, so a name
  with no particle-given word pays no call: one frame off every parse.
- name_word_count asks the numeral's shape in C before
  ends_in_a_generation, as group's count does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73 derek73 added this to the 3.0 milestone Nov 5, 2026
@derek73 derek73 self-assigned this Nov 5, 2026
Review of the first commit found two folds paying a frame on a path
that grows with the input:

- `_post_rules._dotted_member` cost one frame per dotted clause or
  nickname word ('John (M.B.A, M.A) Smith' +3), where the two inline
  tests cost none. Back to the two inline copies, each asking its own
  text (the set-apart pass strips the kept comma, the report does not).
- `split_pair`'s call into `suffix_as_written` cost one frame per Ph./D.
  pair. Back to the inline test.

test_layering's 4i-a count returns to 128 with them.

The test review found S1's comma strip on a dotted member untested
(dropping `.strip(",")` failed nothing): `John Smith (M.A, PhD)` reads
suffix 'M.A, PhD', nickname 'M.A, PhD' without it. A direct test in
test_post_rules, not a case row: the row trips test_stage_field_ownership,
the set-apart pass rewriting a word's text and span, which the stage
map does not list (left for a follow-up outside this pass's files).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant