IxVM kernel: canonical level normalization + FFT-cost pinning#447
Merged
Conversation
439dd16 to
7821fcb
Compare
Port the Rust kernel's canonical-form level machinery (level.rs normalize_level / norm_level_eq / norm_level_le, itself a line-by-line port of Lean4Lean's Level.Normalize with the covers-split soundness fix): normalize_aux with imax-path conditioning, phase-2 subsumption, and structural/dominance comparison on canonical forms. level_equal and level_leq now normalize-and-compare; the previous recursive Level.leq mirror with its two-way param-substitution split per Max/IMax — and its helpers level_subst_reduce / level_has_param / level_any_param — is deleted. level_normalize is keyed on the level alone, so each distinct level normalizes once per run. The split was exponential in the number of params and every branch materialized freshly substituted levels. On _private.…SInt.0.Int16.instRxcHasSize_eq the level family was 93% of the record at 2^31 ops (level_subst_reduce 60.8M + level_leq 53.2M + level_max 30.4M entries…), entered through Inductive's ctor-field universe constraint (level_leq), not def-eq's level_equal. Measured: Int16.instRxcHasSize_eq goes from NON-COMPLETING (killed at 178 GB RSS after 21 min, growth not converging) to 2m23s / 17.9 GB / 295.7B FFT. Int8 completes in 21s. The bench suite is unaffected (its level comparisons hit the structural fast path). 297 ixvm tests pass.
7821fcb to
95c3695
Compare
level_leq + pin per-target FFT cost## Summary Adds `expectedFftCost : Option Nat` to `AiurTestCase`. When set, `runTestCase` asserts the rounded `Aiur.computeStats.totalFftCost` matches exactly. Converts `Tests/Ix/IxVM.lean::kernelCheckNames` to `kernelCheckEntries : List (String × Nat)` and pins all 41 entries. ## Motivation Without per-circuit cost pinning, kernel changes that silently shift FFT cost can land without review. Pinning forces every cost change to be acknowledged in the test source — failures report `expected X, got Y` so the new value can be pasted back. ## Coverage Pins the 38 prior `kernelCheckNames` entries (Stdlib, `IxVMPrim`, `IxVMInd`, edge-case prelude) plus 3 newly-unlocked targets from the `level_leq` Géran-normalize fix: - `Trans.mk` — 2,732,504 - `Array.append_assoc` — 3,938,369,542 - `Vector.append` — 4,023,063,255 `Vector.extract_append` deliberately omitted (too heavy for the suite).
95c3695 to
31a35f0
Compare
johnchandlerburnham
approved these changes
Jun 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two commits on
level-normalization:IxVM: canonical level normalization for
level_equal/level_leq. Replaces the recursiveLevel.leqmirror (two-wayparam-substitution split per
Max/IMax— exponential in unresolveduniverse params) with a port of Rust's Géran canonical-form machinery
from
src/ix/kernel/level.rs::normalize_level/norm_level_eq/norm_level_le.level_equalandlevel_leqnow normalize-and-compare;the substitution helpers (
level_subst_reduce,level_has_param,level_any_param) are deleted.Tests: pin FFT cost per kernel-check target. Adds
expectedFftCost : Option NattoAiurTestCase;runTestCaseassertsthe rounded
Aiur.computeStats.totalFftCostmatches exactly. Pins all41 entries in
Tests/Ix/IxVM.lean::kernelCheckEntries.Motivation
The recursive
Level.leqsplit was exponential in the number of params andeach branch materialized freshly substituted levels. On
_private.…SInt.0.Int16.instRxcHasSize_eqthe level family was 93% of therecord at 2^31 ops (
level_subst_reduce60.8M +level_leq53.2M +level_max30.4M entries…), entered throughInductive's ctor-fielduniverse constraint (
level_leq), not def-eq'slevel_equal.The same algorithmic gap blocks several stdlib targets in the in-tree
ixvmtest suite. Beyond the fix itself, the existing suite has noper-target FFT-cost floor: a kernel change can silently shift cost on any
pinned constant without review.
What changed
Kernel (
Ix/IxVM/Kernel/Levels.lean)(path, const, vars)wherepathis the sorted list of param indices conditioning animaxchain,
constthe max constant contribution,varstheidx-sorted
(param, offset)contributions.normalize_auxwalks the level withimax-path conditioning,delegating
IMax-shape cases tonormalize_imax_dispatch.nl_subsumption_walkdrops contributions dominated by anotherentry whose path is a subset. Called once at the end of
level_normalize(with the same list as bothremandsnapshot).nl_le/nl_eqoperate on canonical forms; covers-split soundnessfix matches
level.rs.level_normalizeis keyed on the level alone, so each distinct levelnormalizes once per run; equality /
leqon canonical forms is cheap.Tests
Tests/Aiur/Common.lean: newexpectedFftCost : Option Natfield onAiurTestCase.runTestCaseconsumesqueryCounts, computesAiur.computeStats, and asserts the roundedtotalFftCostmatchesexactly. Adds
Ix.Aiur.Statisticsto the imports.Tests/Ix/IxVM.lean:kernelCheckNames : List String→kernelCheckEntries : List (String × Nat).kernelChecksnow setsexpectedFftCost := some expectedper entry.Results
Unlocked targets
Before the kernel commit these typechecks OOM'd (process killed under
14 GB ulimit). After: complete with the FFT cost reported by
lake exe check <target>.Trans.mkArray.append_assocVector.appendVector.extract_appendNo regression on previously-passing targets
8-target sample, build clean both sides:
Nat.add_commNat.addNat.decEqNat.decLeNat.sub_le_of_le_addDifferences are sub-0.1%, attributable to the inlined-
nl_subsumptionmicro-cleanup (one fewer memoized dispatch per
level_normalizecall).Pinned suite coverage
41 kernel-check constants now assert exact rounded FFT cost: the 38 prior
kernelCheckNamesentries (Stdlib,IxVMPrim,IxVMInd, edge-caseprelude) plus 3 newly-unlocked targets (
Trans.mk,Array.append_assoc,Vector.append).Vector.extract_appendisdeliberately omitted (too heavy for CI; verified manually).
How to update a pinned cost
If a future kernel change legitimately shifts FFT cost on a pinned
target, the test failure message contains the new value:
Paste the
got Nvalue back intokernelCheckEntries. No toolingbeyond
lake test -- --ignored ixvmis required.Test plan
lake build check— cleanlake build IxTests— cleanlake test -- --ignored ixvm— 41 FFT-cost pins pass, noregressions in the rest of the suite
lake exe check Trans.mk— 2.91 M FFT, passeslake exe check Vector.append— 4.02 G FFT, passeslake exe check Array.append_assoc— 3.94 G FFT, passeslake exe check Vector.extract_append— 63.76 G FFT, passeslake exe check Nat.add_comm— 56.08 M FFT, no regression