Skip to content

bench: add WASM init and startup timing benchmarks#52

Merged
natemoo-re merged 17 commits into
mainfrom
bench/startup-timing
Jun 1, 2026
Merged

bench: add WASM init and startup timing benchmarks#52
natemoo-re merged 17 commits into
mainfrom
bench/startup-timing

Conversation

@natemoo-re
Copy link
Copy Markdown
Member

@natemoo-re natemoo-re commented May 29, 2026

Two new benchmarks: createTerm and render-minimal, added as fixtures in bench/fixtures/. New CI job runs these on codspeed-macro with mode: walltime to measure the full E2E perf.

Updated benchmark.yml with a single build job where artifacts are shared between simulation + walltime runs. Also pinned SHAs.

@pkg-pr-new
Copy link
Copy Markdown

pkg-pr-new Bot commented May 29, 2026

Open in StackBlitz

npm i https://pkg.pr.new/clayterm@52

commit: 266259e

@codspeed-hq
Copy link
Copy Markdown
Contributor

codspeed-hq Bot commented May 29, 2026

Merging this PR will improve performance by 38.61%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 4 improved benchmarks
✅ 14 untouched benchmarks
🆕 2 new benchmarks

Performance Changes

Mode Benchmark BASE HEAD Efficiency
🆕 WallTime time to first render N/A 271.9 ms N/A
🆕 WallTime createTerm N/A 272.7 ms N/A
Simulation large list (50 items) 617.7 µs 409.8 µs +50.74%
Simulation diff render (second frame) 712.1 µs 640.6 µs +11.17%
Simulation printable ASCII (single char) 153.1 µs 88.6 µs +72.76%
Simulation long input burst (200 bytes) 1,249.6 µs 980.1 µs +27.5%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing bench/startup-timing (266259e) with main (c55a136)

Open in CodSpeed

@natemoo-re
Copy link
Copy Markdown
Member Author

natemoo-re commented May 30, 2026

benchmark seems to be a V8 de-opt due to all the benchmarks running in a shared process? digging into it

Edit: 🤞 hopefully addressed by splitting the startup benchmark to a separate process in 6eb4995

Edit v2: yeah, fixed it!

Comment thread bench/startup.bench.ts Outdated
Copy link
Copy Markdown
Collaborator

@cowboyd cowboyd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! Including the whole process startup should benefit the brotli case which has smaller source.

@cowboyd
Copy link
Copy Markdown
Collaborator

cowboyd commented May 30, 2026

On the first commit:

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com

Dunno how you feel about it, but for-profit companies whom we are paying good money to use their tools that permanently attach advertisements to our repo by injecting git metadata into commits is a practice I really find off-putting.

So much so that I added it to AGENTS.md https://github.com/bombshell-dev/clayterm/blob/main/AGENTS.md#commit-and-pr-conventions

😄

Any chance you can remove that when squashing?

@cowboyd
Copy link
Copy Markdown
Collaborator

cowboyd commented May 30, 2026

Two more "first-to-x" benchmarks we should consider before splitting out the wasm modules

  • create input

@natemoo-re
Copy link
Copy Markdown
Member Author

Dunno how you feel about it, but for-profit companies whom we are paying good money to use their tools that permanently attach advertisements to our repo by injecting git metadata into commits is a practice I really find off-putting.

@cowboyd yes, fair enough! I apologize for not reading through that doc first. I normally take care to commit everything myself manually but didn't stop the auto-commit here. Bombshell hasn't made any official contribution guidelines for agent-assisted work yet, so this is good motivation to create a policy for the whole org. I'll spin up a thread with everyone in Discord.

@natemoo-re natemoo-re force-pushed the bench/startup-timing branch from a44157b to 4dd4b4f Compare June 1, 2026 04:19
@natemoo-re
Copy link
Copy Markdown
Member Author

Co-author removed! I will add a new benchmark for createInput in the relevant branch.

@natemoo-re natemoo-re force-pushed the bench/startup-timing branch from 4dd4b4f to 5b99cba Compare June 1, 2026 04:36
@natemoo-re natemoo-re merged commit 02754a2 into main Jun 1, 2026
11 checks passed
@natemoo-re natemoo-re deleted the bench/startup-timing branch June 1, 2026 04:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants