Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions API-FRICTION.md
Original file line number Diff line number Diff line change
Expand Up @@ -5796,6 +5796,19 @@ Each entry records:
- Release follow-up verification: the workflow contract fixes the coordinator
ordering and disables continuous assignment for that step. The coordinator
`benchmark-check` passes against the unchanged `cd7768378` evidence.
- `0.8.0` release evidence: squash-merging the fully validated API
harmonization PR rewrote measured source commit `5a1c893` as `35832f7`
without changing any measured bytes. Both the main push and version-PR
comparison gates rejected the exact-SHA baseline.
- `0.8.0` release decision: keep the source revision as human-readable
attribution, but make the deterministic SHA-256 digest of every tracked
comparison input the authoritative equality boundary. Rewritten history can
retain evidence when its measured content is identical; any byte change
still requires a new baseline.
- `0.8.0` release verification: focused provenance tests cover history rewrite,
content drift, malformed metadata, and workspace version-only changes. The
schema-4 baseline records both commit attribution and the input digest, and
the full validation graph passes after the squash merge.

### F-198 — Union-valued axes rejected configured D3 scales

Expand Down
7 changes: 4 additions & 3 deletions benchmarks/comparison/bundle-baseline.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"schemaVersion": 3,
"generatedAt": "2026-08-09T03:44:35.418Z",
"schemaVersion": 4,
"generatedAt": "2026-08-09T03:56:56.816Z",
"packageVersions": {
"tanstack": "0.7.2",
"chartjs": "4.5.1",
Expand All @@ -11,7 +11,8 @@
"sources": {
"tanstack": {
"kind": "workspace",
"revision": "5a1c893aa5e8d5f16d05444fbaccfe29a871ca36"
"revision": "35832f753fd0e17b5215c76529dd7e4bbc222282",
"inputDigest": "sha256:938850a0ebf1876caadac6e331b12021c1caa64f6b222eadf090f698dadf3f2a"
},
"chartjs": {
"kind": "package",
Expand Down
4 changes: 2 additions & 2 deletions docs/comparison.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,14 @@ turning untested behavior into a checkmark.

| Library | Package | Measured source |
| -------------------------------------------------------------------------------------- | -------------------- | ------------------- |
| [TanStack Charts](./overview.md) | `@tanstack/charts` | workspace `5a1c893` |
| [TanStack Charts](./overview.md) | `@tanstack/charts` | workspace `35832f7` |
| [Chart.js](https://www.chartjs.org/docs/latest/) | `chart.js` | npm `4.5.1` |
| [Apache ECharts](https://echarts.apache.org/handbook/en/best-practices/canvas-vs-svg/) | `echarts` | npm `6.1.0` |
| [Recharts](https://recharts.github.io/en-US/) | `recharts` | npm `3.10.1` |
| [Observable Plot](https://observablehq.com/plot/features/plots) | `@observablehq/plot` | npm `0.6.17` |

The competitor versions are exact package pins, not latest versions inferred
at page render time. The measured TanStack workspace revision is `5a1c893`.
at page render time. The measured TanStack workspace revision is `35832f7`.

## Capability matrix

Expand Down
4 changes: 2 additions & 2 deletions packages/charts-core/docs/comparison.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,14 @@ turning untested behavior into a checkmark.

| Library | Package | Measured source |
| -------------------------------------------------------------------------------------- | -------------------- | ------------------- |
| [TanStack Charts](./overview.md) | `@tanstack/charts` | workspace `5a1c893` |
| [TanStack Charts](./overview.md) | `@tanstack/charts` | workspace `35832f7` |
| [Chart.js](https://www.chartjs.org/docs/latest/) | `chart.js` | npm `4.5.1` |
| [Apache ECharts](https://echarts.apache.org/handbook/en/best-practices/canvas-vs-svg/) | `echarts` | npm `6.1.0` |
| [Recharts](https://recharts.github.io/en-US/) | `recharts` | npm `3.10.1` |
| [Observable Plot](https://observablehq.com/plot/features/plots) | `@observablehq/plot` | npm `0.6.17` |

The competitor versions are exact package pins, not latest versions inferred
at page render time. The measured TanStack workspace revision is `5a1c893`.
at page render time. The measured TanStack workspace revision is `35832f7`.

## Capability matrix

Expand Down
11 changes: 9 additions & 2 deletions scripts/compare-chart-libraries.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import {
} from './benchmark/comparison-capabilities.mjs'
import {
comparisonInstalledVersionFailure,
tanstackComparisonInputDigest,
tanstackComparisonRevision,
tanstackComparisonSourceFailure,
} from './comparison-source-revision.mjs'
Expand Down Expand Up @@ -1111,8 +1112,9 @@ async function writeBundleBaseline(
baselineTiers,
) {
const sourceRevision = tanstackComparisonRevision(root)
const sourceInputDigest = tanstackComparisonInputDigest(root)
const baseline = {
schemaVersion: 3,
schemaVersion: 4,
generatedAt: new Date().toISOString(),
packageVersions: baselineVersions,
sources: Object.fromEntries(
Expand All @@ -1122,6 +1124,7 @@ async function writeBundleBaseline(
? {
kind: 'workspace',
revision: sourceRevision,
inputDigest: sourceInputDigest,
}
: {
kind: 'package',
Expand Down Expand Up @@ -1180,7 +1183,10 @@ async function checkBundleBaseline(
const expectedTanStackRevision = checkSourceProvenance
? tanstackComparisonRevision(root)
: undefined
if (baselineRecord.schemaVersion !== 3) {
const expectedTanStackInputDigest = checkSourceProvenance
? tanstackComparisonInputDigest(root)
: undefined
if (baselineRecord.schemaVersion !== 4) {
failures.push(
'bundle baseline schema is stale; run pnpm benchmark:update-baseline',
)
Expand Down Expand Up @@ -1213,6 +1219,7 @@ async function checkBundleBaseline(
const sourceFailure = tanstackComparisonSourceFailure(
source,
expectedTanStackRevision,
expectedTanStackInputDigest,
)
if (sourceFailure) failures.push(`${library.label}: ${sourceFailure}`)
}
Expand Down
54 changes: 51 additions & 3 deletions scripts/comparison-source-revision.mjs
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
import { execFileSync } from 'node:child_process'
import { createHash } from 'node:crypto'
import { lstatSync, readFileSync, readdirSync } from 'node:fs'
import { relative, resolve, sep } from 'node:path'

export const tanstackComparisonInputPaths = [
'benchmarks/comparison/libraries/tanstack',
Expand Down Expand Up @@ -27,15 +30,45 @@ export function tanstackComparisonRevision(repositoryRoot) {
return revision
}

export function tanstackComparisonSourceFailure(source, expectedRevision) {
export function tanstackComparisonInputDigest(
repositoryRoot,
inputPaths = tanstackComparisonInputPaths,
) {
const hash = createHash('sha256')
const files = inputPaths
.flatMap((path) => collectFiles(resolve(repositoryRoot, path)))
.sort()

for (const file of files) {
const path = relative(repositoryRoot, file).split(sep).join('/')
const contents = readFileSync(file)
hash.update(path)
hash.update('\0')
hash.update(String(contents.byteLength))
hash.update('\0')
hash.update(contents)
hash.update('\0')
}

return `sha256:${hash.digest('hex')}`
}

export function tanstackComparisonSourceFailure(
source,
expectedRevision,
expectedInputDigest,
) {
if (
source?.kind !== 'workspace' ||
!/^[0-9a-f]{40}$/u.test(source.revision)
) {
return 'bundle baseline must record its workspace revision'
}
if (source.revision !== expectedRevision) {
return `bundle baseline workspace revision ${source.revision} does not match measured inputs ${expectedRevision}`
if (!/^sha256:[0-9a-f]{64}$/u.test(source.inputDigest)) {
return 'bundle baseline must record its workspace input digest'
}
if (source.inputDigest !== expectedInputDigest) {
return `bundle baseline workspace input digest ${source.inputDigest} does not match measured inputs ${expectedInputDigest} at ${expectedRevision}`
}
}

Expand All @@ -54,3 +87,18 @@ export function comparisonInstalledVersionFailure(

return `installed version ${actualVersion} does not match baseline ${baselineVersion}`
}

function collectFiles(path) {
const stats = lstatSync(path)
if (stats.isSymbolicLink()) {
throw new Error(`Comparison input must not be a symbolic link: ${path}`)
}
if (stats.isFile()) return [path]
if (!stats.isDirectory()) {
throw new Error(`Comparison input must be a file or directory: ${path}`)
}

return readdirSync(path, { withFileTypes: true }).flatMap((entry) =>
collectFiles(resolve(path, entry.name)),
)
}
50 changes: 46 additions & 4 deletions scripts/comparison-source-revision.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { dirname, resolve } from 'node:path'
import { afterEach, describe, expect, it } from 'vitest'
import {
comparisonInstalledVersionFailure,
tanstackComparisonInputDigest,
tanstackComparisonInputPaths,
tanstackComparisonRevision,
tanstackComparisonSourceFailure,
Expand Down Expand Up @@ -44,24 +45,65 @@ describe('TanStack comparison source provenance', () => {
expect(tanstackComparisonRevision(repository)).toBe(measuredRevision)
})

it('rejects a well-formed revision from different inputs', () => {
it('uses an input digest that survives rewritten history', async () => {
const repository = await mkdtemp(
resolve(tmpdir(), 'charts-comparison-digest-'),
)
temporaryRepositories.push(repository)
const inputPath = tanstackComparisonInputPaths[1]
const coreInput = resolve(repository, inputPath, 'index.ts')
await mkdir(dirname(coreInput), { recursive: true })
await writeFile(coreInput, 'export const value = 1\n')

const inputDigest = tanstackComparisonInputDigest(repository, [inputPath])
expect(inputDigest).toMatch(/^sha256:[0-9a-f]{64}$/u)
expect(
tanstackComparisonSourceFailure(
{
kind: 'workspace',
revision: 'a'.repeat(40),
inputDigest,
},
'b'.repeat(40),
inputDigest,
),
).toBeUndefined()

await writeFile(coreInput, 'export const value = 2\n')
expect(tanstackComparisonInputDigest(repository, [inputPath])).not.toBe(
inputDigest,
)
})

it('rejects a well-formed digest from different inputs', () => {
const expectedRevision = 'a'.repeat(40)
const recordedRevision = 'b'.repeat(40)
const expectedInputDigest = `sha256:${'c'.repeat(64)}`
const recordedInputDigest = `sha256:${'d'.repeat(64)}`

expect(
tanstackComparisonSourceFailure(
{ kind: 'workspace', revision: recordedRevision },
{
kind: 'workspace',
revision: recordedRevision,
inputDigest: recordedInputDigest,
},
expectedRevision,
expectedInputDigest,
),
).toBe(
`bundle baseline workspace revision ${recordedRevision} does not match measured inputs ${expectedRevision}`,
`bundle baseline workspace input digest ${recordedInputDigest} does not match measured inputs ${expectedInputDigest} at ${expectedRevision}`,
)
})

it('uses source revisions for workspaces and versions for installed packages', () => {
expect(
comparisonInstalledVersionFailure(
{ kind: 'workspace', revision: 'a'.repeat(40) },
{
kind: 'workspace',
revision: 'a'.repeat(40),
inputDigest: `sha256:${'b'.repeat(64)}`,
},
'0.0.2',
'0.0.1',
),
Expand Down
9 changes: 7 additions & 2 deletions scripts/docs-contract.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -191,8 +191,8 @@ export function isPublicChartLibraryLinkAllowed(path, href) {

export function comparisonBaselineContractFailures(baseline, expectedVersions) {
const failures = []
if (baseline.schemaVersion !== 3) {
failures.push('comparison bundle baseline must use schema version 3')
if (baseline.schemaVersion !== 4) {
failures.push('comparison bundle baseline must use schema version 4')
}
if (
!sameStrings(baseline.matrix?.chartTypes ?? [], comparisonChartTypes) ||
Expand Down Expand Up @@ -221,6 +221,11 @@ export function comparisonBaselineContractFailures(baseline, expectedVersions) {
'comparison bundle baseline must record the TanStack workspace revision',
)
}
if (!/^sha256:[0-9a-f]{64}$/u.test(source?.inputDigest)) {
failures.push(
'comparison bundle baseline must record the TanStack workspace input digest',
)
}
} else if (
source?.kind !== 'package' ||
source.packageName !== library.packageName ||
Expand Down
5 changes: 4 additions & 1 deletion scripts/docs-contract.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -179,12 +179,13 @@ Body
'observable-plot': '0.6.17',
}
const baseline = {
schemaVersion: 3,
schemaVersion: 4,
packageVersions: versions,
sources: {
tanstack: {
kind: 'workspace',
revision: '1'.repeat(40),
inputDigest: `sha256:${'2'.repeat(64)}`,
},
chartjs: {
kind: 'package',
Expand Down Expand Up @@ -231,11 +232,13 @@ Body
const stale = structuredClone(baseline)
stale.packageVersions.chartjs = '4.5.0'
stale.sources.tanstack.revision = 'unknown'
stale.sources.tanstack.inputDigest = 'unknown'
delete stale.bundles['tanstack-line-basic']
expect(comparisonBaselineContractFailures(stale, versions)).toEqual(
expect.arrayContaining([
'comparison bundle baseline version is stale for Chart.js: expected 4.5.1',
'comparison bundle baseline must record the TanStack workspace revision',
'comparison bundle baseline must record the TanStack workspace input digest',
'comparison bundle baseline must contain the complete 60-case matrix',
]),
)
Expand Down