Support vitest 5

[GuillaumeLagrange]

TL;DR

@codspeed/vitest-plugin now supports Vitest 5 as well as 3 and 4. The plugin detects which Vitest is installed and hooks into it the right way. Users on Vitest 3/4 see no change.

What changes for users

• vitest peer range is now ^3.2 || ^4 || ^5.

• On Vitest 5, benchmarks are written with Vitest's new bench fixture, and CodSpeed picks them up as-is:

  test("fibo 15", async ({ bench }) => {
    await bench.compare(
      bench("recursive", () => recursiveFibonacci(15)),
      bench("iterative", () => iterativeFibonacci(15)),
    );
  });

  Each bench() in a bench.compare() group is reported as its own benchmark: file.bench.ts::fibo 15::recursive.

• Without CodSpeed, the plugin does nothing and Vitest runs benchmarks normally.

How it works

Vitest 5 removed the benchmark runner the plugin used to extend. Benchmarks now run through a benchmark.provider module instead.

• Vitest 3/4: same custom runner as before, moved under src/legacy/.
• Vitest 5: CodSpeed registers itself as the benchmark provider (src/v5/provider.ts). It runs each benchmark once under instrumentation in simulation mode, or drives tinybench and records walltime results in walltime mode.
• src/vitestBackend.ts picks one of the two based on the installed Vitest version; code shared by both lives in src/instrument.ts.
• globalSetup now tolerates running twice, since Vitest 5 runs it once per project and clones a separate benchmark project.

Also in this PR

• fix(core): getGitDir now finds the repository root when .git is a file (git worktrees, submodules). Before, every plugin failed with "Could not find a git repository" when run from a worktree. Inside a submodule, benchmark URIs are now relative to the submodule. Also drops the find-up dependency.
• New with-vitest-v4 and with-vitest-v5 examples, so CI covers every supported major.
• README shows the Vitest 5 fixture API.
• CLAUDE.md documents how to exercise a plugin in the three CodSpeed modes.

Testing

Ran locally:

• pnpm turbo run build lint typecheck format test test/integ passes.
• with-vitest-v3, with-vitest-v4 and with-vitest-v5 examples, plus the plugin's own benches, in fallback, simulation and walltime modes.

Known limitations

• The Vitest version is detected from the current working directory. If the Vitest that actually runs differs from the one resolved there (e.g. running a project's vitest binary with --root from another directory), the wrong integration is picked and nothing gets measured, silently. Vitest has no documented way for plugins to read its version early enough; a follow-up could resolve it from the running binary and fail loudly on a mismatch.
• On Vitest 5, the per-benchmark [CodSpeed] Checked … lines only show with --reporter=verbose.
• The Vitest 5 simulation path doesn't call mongoMeasurement.start/stop around benchmarks, unlike the Vitest 3/4 runner.

Refs COD-2931

[codspeed]

Merging this PR will regress 4 benchmarks

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 4 improved benchmarks
❌ 4 regressed benchmarks
✅ 177 untouched benchmarks
🆕 79 new benchmarks
⏩ 53 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Simulation	switch 1	204.5 µs	689.9 µs	-70.37%
❌	Memory	wait 500ms	15.5 KB	21.5 KB	-27.99%
❌	Simulation	iterative fibo 20	26.9 µs	30.7 µs	-12.36%
❌	WallTime	test_recursive_cached_fibo_10	2.3 µs	2.6 µs	-12.09%
⚡	Simulation	recursive fibo 10	1,372.1 µs	299.9 µs	×4.6
⚡	Simulation	recursive fibo 15	636.9 µs	352.1 µs	+80.88%
⚡	WallTime	switch 1	84 ns	72 ns	+16.67%
⚡	WallTime	test_iterative_fibo_10	120 ns	108 ns	+11.11%
🆕	WallTime	iterative fibo 15	N/A	672 ns	N/A
🆕	WallTime	iterative fibo 20	N/A	672 ns	N/A
🆕	WallTime	recursive fibo 15	N/A	20.4 µs	N/A
🆕	WallTime	recursive fibo 20	N/A	219.2 µs	N/A
🆕	WallTime	iterative	N/A	552 ns	N/A
🆕	WallTime	recursive	N/A	20.2 µs	N/A
🆕	WallTime	iterative	N/A	564 ns	N/A
🆕	WallTime	recursive	N/A	218.9 µs	N/A
🆕	WallTime	fibo 10	N/A	2.3 µs	N/A
🆕	WallTime	fibo 15	N/A	20.3 µs	N/A
🆕	WallTime	long body	N/A	267.2 µs	N/A
🆕	WallTime	short body	N/A	1.5 µs	N/A
...	...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing cod-2931-prepare-compatibility-with-vitest-5 (9007e6e) with main (3258048)}

Footnotes

1. 53 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[greptile-apps]

Confidence Score: 4/5

[High impact] Rewrites the vitest plugin to support version 5.

This is close, but the backend fallback should be fixed before merging.

Findings

1.  Fallback Picks Wrong Backend ▶

Fix with agent prompt

### Issue 1
packages/vitest-plugin/src/vitestBackend.ts:41
When Vitest cannot be resolved from `process.cwd()`, this still falls back to the v5 backend. A supported Vitest 3 or 4 project can hit this when the run starts outside the package root or from a workspace layout where `createRequire(join(process.cwd(), "package.json"))` cannot resolve that package's `vitest/package.json`. The plugin then wires `benchmark.provider` instead of the legacy runner, so CodSpeed does not install the Vitest 3/4 benchmark runner and the run can execute without CodSpeed instrumentation.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

This PR adds Vitest 5 support to the Vitest plugin. The main changes are:

• A new Vitest 5 benchmark provider.
• Legacy Vitest 3/4 runner code moved under legacy.
• Updated plugin config selection by Vitest major version.
• New Vitest 4 and Vitest 5 examples.
• Updated benchmark fixtures, docs, and package versions.

_{Reviews (2) · Last reviewed commit: "docs(vitest-plugin): show the vitest 5 b..." · Reviewed by Greptile}

[florian-lefebvre]

Hi @GuillaumeLagrange, what's the status of this PR? We would need it for Astro

[GuillaumeLagrange]

Hi @GuillaumeLagrange, what's the status of this PR? We would need it for Astro

Hey! The support for the benchmark provider has been merged and released upstream by vitest, we plan to merge and release this PR early this week! Don't hesitate to continue pinging me to get updates, and sorry about the delay

[JavierGOrdonnez]

Hi @GuillaumeLagrange, what's the status of this PR? I would need it for various projects. Thanks!

[greptile-apps]

Fallback Picks Wrong Backend

When Vitest cannot be resolved from process.cwd(), this still falls back to the v5 backend. A supported Vitest 3 or 4 project can hit this when the run starts outside the package root or from a workspace layout where createRequire(join(process.cwd(), "package.json")) cannot resolve that package's vitest/package.json. The plugin then wires benchmark.provider instead of the legacy runner, so CodSpeed does not install the Vitest 3/4 benchmark runner and the run can execute without CodSpeed instrumentation.

Prompt To Fix With AI

This is a comment left during a code review.
Path: packages/vitest-plugin/src/vitestBackend.ts
Line: 41

Comment:
**Fallback Picks Wrong Backend**

When Vitest cannot be resolved from `process.cwd()`, this still falls back to the v5 backend. A supported Vitest 3 or 4 project can hit this when the run starts outside the package root or from a workspace layout where `createRequire(join(process.cwd(), "package.json"))` cannot resolve that package's `vitest/package.json`. The plugin then wires `benchmark.provider` instead of the legacy runner, so CodSpeed does not install the Vitest 3/4 benchmark runner and the run can execute without CodSpeed instrumentation.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.