Tracks how LLVM's evolution changes the machine code generated for a pinned evmone. evmone is fixed, LLVM varies, so every recorded difference is attributable to LLVM.
Modelled on llvm-opt-benchmark, but measures disassembly rather than IR, because evmone's missed optimizations have been backend and isel issues that IR diffs cannot see.
main holds only the engine, the specs and this README. Corpora live on
data branches, one per (arch, evmone release line, format version):
asm/x86-64/evmone-0.23.0/v1 GNU objdump, closed
asm/x86-64/evmone-0.23.0/v2 llvm-objdump, current
Each commit on a data branch is one absolute snapshot: the corpus under asm/
plus a snapshot.json describing only itself, so a snapshot can be inserted
anywhere without invalidating another. Within a branch, arch and evmone are
constant by construction.
The branch is linear and reads as an annotated LLVM history, ordered by LLVM's own topology rather than by the order revisions happened to be probed:
git log --oneline asm/x86-64/evmone-0.23.0/v2
[data] b89710b49c4a [AArch64][GlobalISel] Lower vector overflow intrinsics -13
[data] d7c1e6054fb6 [SLP]Recalculate gather costs after tree trimming +12
[data] e74432084f39 [Bazel] Fix Host target's exported headers =
Full shas and the topological index are in trailers, so git log --format='%(trailers:key=LLVM-Commit,valueonly)' stays exact. Three access
paths over one dataset: git log to browse, trailers to query,
snapshot.json to measure.
Commit dates are the LLVM commit's own, so the branch reads as LLVM's timeline
and rewriting it is deterministic. Commit shas change when a snapshot is
inserted before them; the durable reference is the corpus tree hash,
git rev-parse <commit>:asm, which does not.
uv run track setup # one time, per clone
uv run track run --llvm origin/main # record a datapoint
uv run track run --llvm <sha> # record a specific revision
uv run track compare <base> <head> # evaluate without recording
uv run track compare @recorded:<sha> <head> # reuse a recorded corpus
run places the snapshot in LLVM's order whatever order you record in, so a
bisect can probe revisions freely: backfilling a revision inserts it in the
right place and re-derives the one delta that moved, rather than appending it
with a misleading diff against whatever was recorded last. It writes git
objects directly and never touches the working tree, so a run is safe while
you are working on main.
uv run track sweep <from> <to> # find every change point in a range
sweep answers "which LLVM commits changed evmone, and only those" by
recursive bisection, using the data branch as its cache. Every probe is a
permanent snapshot, so an interrupted sweep resumes where it stopped, and a
later sweep over an overlapping range pays only for what is not already
recorded. --max-runs N bounds a session. Gaps are closed narrowest first, so
stopping early leaves you with attributed change points rather than a
half-bisected partition.
Budget it: a datapoint is a clang build plus an evmone build, ~20 minutes on an idle machine. Covering a release cycle is days of compute, not minutes.
run refuses if asmtrack/ has uncommitted changes: a snapshot records
engine_commit, so the engine that produced it has to be committed for anyone
to reproduce it. It also refuses to extend a branch whose last snapshot came
from different engine source without a format-version bump, because engine
churn is the one thing that can masquerade as an LLVM change. compare
records nothing and is unrestricted.
When an engine change genuinely cannot alter corpus text (a new subcommand, a
report tweak), pass --assert-output-neutral "<reason>". The reason and the
engine hash it was asserted against are written into the snapshot, so the claim
stays checkable instead of being a decision nobody recorded. The hash covers
every module on purpose: cli.collect and cli.produce_corpus shape the
corpus too, so there is no subset that is safe to exclude from it.
Design: docs/superpowers/specs/2026-09-10-data-branches-design.md
(supersedes parts of the 2026-09-09 design).
Three snapshots recorded during the first day of use are deliberately not migrated.
Two of them (25d88db, beb41e0) predate a normalizer fix and do not
describe the corpus they claim to. normalize.py compared objdump's
section-absolute instruction addresses directly against a branch target's
symbol-relative <sym+0xNN> offset; for any symbol not starting at section
offset 0 (46% of them, measured on this corpus) that comparison was
meaningless. Branch targets either kept their raw hex address (2382 lines) or,
where the offset coincidentally matched some other instruction's absolute
address, were rewritten to a label pointing at the wrong instruction (285
lines), asserting control-flow edges that do not exist.
The 15 snapshots after that fix were produced with GNU objdump and became the
v1 branch. They are not comparable with v2: llvm-objdump suffixes
mnemonics where GNU objdump did not, so the corpus text itself differs.
Answering a question across that boundary means re-running the old revisions
under the new format, which is deliberate work rather than a diff.
Both of those episodes are why the branch layout exists. Measured on 319 symbols, the path-normalisation fix moved 15 symbols and the coordinate-system fix moved 107; a full LLVM release cycle, 8,184 commits, moved 29. Engine churn dominates the signal it is meant to measure, so the boundary between epochs has to be structural rather than a flag on a row.