marker/extras: port pypa conformance tables + uv-pep508 carets (rstudio/package-manager#19392)

[jonyoder]

Ports pypa/packaging's marker/extras conformance tables and uv-pep508's error caret positions into three new table-driven test files. No production code changes: every mutation below was reverted before commit, and no case needed a fix.

Test-only change; no CHANGELOG entry (see "CHANGELOG" below).

---

Scope

Per the issue's own correction: test_utils.py yields one portable case (canonicalize_name), not eleven — the rest are wheelname/'s concern or have no Go counterpart at all. Scoped to that, plus test_markers.py's marker tables and uv-pep508's error-position snapshots (inputs and offsets only, never uv's message text).

Upstream pins

Source	Pin	How verified
pypa/packaging tests/test_markers.py, tests/test_utils.py	4eb0753dba8fcaaac8eb75463374e448f0931558	The repo's existing pin. Tag 26.3 dereferences to 929fd4b1410ac7ef61ef3f45b2f5d7e87711a9b5, 6 commits ahead; its only test_markers.py change is a type annotation on TestNode.test_accepts_value (not ported, TestNode has no counterpart anyway) — so the ported content is identical at 26.3.
astral-sh/uv crates/uv-pep508/src/lib.rs's mod tests	01cb90c1a4f88af09906cb60de9766d2add4a062	Release 0.12.18 (lightweight tag), confirmed by commits/01cb90c. Cargo.toml at that commit: license = "Apache-2.0 OR BSD-2-Clause"; License-BSD line 1: Copyright (c) 2023 konstin. First uv pin in this repo.

Per-file counts

File	Rows	Pass	Fix	Skip	Divergence
extras/conformance_test.go	17	17	0	0	0
marker/conformance_test.go	~2223 (incl. two 1440/600/120-case cross-products)	~2218	0	0	5
internal/pep508/error_position_test.go	45	5	0	10	30

Zero fixes: every ported case either passes as upstream states it, is an annotated divergence, or is a documented skip with a concrete reason (t.Skip, visible in go test -v's --- SKIP lines).

Correction carried from the issue

Nearly every upstream evaluate(dict) call is a partial environment: 18 of the 40 evaluate( calls in test_markers.py pass a dict, and upstream fills every key the dict omits from the live interpreter — not "~4" as the issue estimated. Every dict case here is built as baseEnvironment().With(dict), never a struct literal (see marker/conformance_test.go's header and partial_environment_test.go's TestStructLiteralZeroFillIsNotAPartialEnvironment).

Skips and divergences

extras/: the validate=True half of test_canonicalize_name_invalid is a whole-function skip — Normalize has no validate mode, documented in the file header as a half-port.

marker/: 5 divergences, all documented inline —

• '2.7.0' ~= os_name, os_name === 'posix', sys_platform === 'linux': upstream raises UndefinedComparison; Evaluate's fixed bool can't propagate an error, so it returns false (pre-existing, documented divergence, already pinned by evaluate_test.go).
• test_parses_invalid_malformed_quoted_string's two inputs: asserted as errors only, not upstream's exact wrapped message/caret (that granularity lives in internal/pep508, not here; this input overlaps internal/pep508/marker_test.go's #19401 escape table — duplication is deliberate per the issue).

internal/pep508/error_position_test.go (uv): the caret-translation rule (documented once, in the file header) — uv draws a caret as ^^^^ spanning display columns; ours draws ~~~ + one ^ at a byte offset. Every "Expected X" error in this grammar is a zero-width point at the parser's cursor; uv spans the whole offending token. Where our point equals uv's span start, that's an annotated divergence (29 rows, one shared reason) — not a bug, we just don't additionally measure a width. One more row (error_random_char) is divergent differently: our point lands at the last byte of uv's span, not the first (both anchor the same #) — 30 divergences total.

10 skips:

• error_extras_eof1/2/3, error_parenthesized_parenthesis, error_marker_incomplete4 (5): different anchor entirely — we report the cursor at end-of-input where a closing ]/) was expected; uv reports the position of the opening delimiter that never closed. Fixing this would mean widening the zero-width convention across every unclosed-delimiter call site, well past a contained fix.
• error_unnamedunnamed_url (1): uv has a bespoke "this looks like an unnamed URL" heuristic with a tailored message; pypa (our semantics anchor) has no such heuristic and neither do we.
• error_end (name_), error_extras_illegal_end's foo[bar_] case (2): the exact oracle-conflict example from the issue — pypa accepts a trailing _ in this identifier grammar, uv rejects it, we follow pypa.
• error_pep440 (numpy >=1.1.*), error_invalid_prerelease (name==1.0.org1) (2): version-string validity is checked one layer up, in package requirement via version.NewSpecifiers — this raw grammar entry point (pep508.ParseRequirement, matching grammar_conformance_test.go's own entry point) doesn't validate specifier text at all.
• error_invalid_extra_unnamed_url is excluded outright (not counted above): behind uv's non-pep508-extensions cargo feature, no counterpart.

Byte-vs-display-column note (as flagged by the issue): error_unicode_after_extra and parse_name_with_star's non-ASCII case involve a multi-byte character right at the flagged position; our zero-width Start still lands at the correct byte offset (nothing multi-byte precedes it here), so only uv's span width would differ if we ever measured one. Annotated, not "fixed".

Mutation proofs (all reverted before commit; git diff --stat on each production file is empty)

1. marker: swapped evalStringOp's "in" case to !strings.Contains(...). RED: TestConformance_Evaluates/'2.7'_in_python_version failed. Reverted, GREEN.
2. extras: dropped . from Normalize's collapsed-separator set. RED: TestConformance_CanonicalizeName/Foo.....Bar (and its sibling .-bearing rows) failed. Reverted, GREEN.
3. carets: added 1 to Start in NewSyntaxErrorAt. RED: many named error_* rows failed (e.g. error_random_char, error_whats_that). Reverted, GREEN.
4. No fix-classified rows exist in this PR, so there is nothing to revert-and-show-red for that category.

NOTICE

Append-only, at the very end of the file (after the pgregory.net/rapid paragraph) — three new paragraphs for the marker/, extras/ and internal/pep508 conformance ports. PR #60 is rewriting NOTICE elsewhere (its hunks end well above this append point), so this should merge cleanly in either order — but it still needs the same human licensing review #60 is waiting on, especially the uv-pep508 Apache-2.0 OR BSD-2-Clause / Copyright (c) 2023 konstin attribution.

CHANGELOG

No entry. Every row is test-only or an annotated/skipped divergence; no fix touched marker/*.go, extras/extras.go or internal/pep508/*.go (confirmed empty diffs on all three after each mutation revert).

Out of scope / candidate follow-ups (not filed, per the issue)

• test_python_full_version_untagged_user_provided: surfaces that Evaluate does not repair a bare trailing "+" on a caller-supplied python_full_version the way EnvironmentFromTarget does. Real gap, not fixed here.
• The unclosed-delimiter and unnamed-URL skips above, if ever worth closing the anchor-position gap with uv.

Verification

• go test ./... -count=1 and go test -race ./... -count=1: all packages pass.
• go vet ./...: clean.
• gofmt -l .: empty.
• golangci-lint run ./... at the repo's pinned v2.11.2: clean (verified live via a planted, then removed, unchecked os.Setenv).
• go list -deps ./... | grep rapid: empty.
• go.mod untouched (go 1.25.0).
• RoboRev: all commits reviewed clean, no findings.
• Independent review (a separate agent, not the author, reproducing the mutation proofs and cross-checking ~50+ rows against the pinned upstream sources directly): verdict "looks solid." One finding, fixed: the error_position_test.go header's self-reported split (5+28+7+4=44) undercounted by one row (actually 5 pass / 30 divergence / 10 skip = 45); corrected in a follow-up commit, no table or test logic changed.

🤖 Generated with Claude Code