sqlparser_bench never runs in CI, only clippy checks it. The benchmarks once measured error paths without anyone noticing (#1559), and the six parse_*_chain groups that guard the exponential parse-time fixes (#2344, #2350, #2352, #2372, #2392) would not catch a regression either.
The proposal is to run them with CodSpeed in simulation mode, which counts instructions under Valgrind on ubuntu-latest, so results stay stable on shared runners.
Two ASF prerequisites.
apache/arrow-rs#9975 and apache/datafusion#21165 plan the same setup. OpenDAL dropped CodSpeed in apache/opendal#5626 over false positives from I/O-bound benchmarks, while parsing is CPU-bound.
sqlparser_benchnever runs in CI, only clippy checks it. The benchmarks once measured error paths without anyone noticing (#1559), and the sixparse_*_chaingroups that guard the exponential parse-time fixes (#2344, #2350, #2352, #2372, #2392) would not catch a regression either.The proposal is to run them with CodSpeed in simulation mode, which counts instructions under Valgrind on
ubuntu-latest, so results stay stable on shared runners.Two ASF prerequisites.
CodSpeedHQ/actionback on the allow-list (removed as unused in Remove 14 unused wildcard actions from allowlist (part of #666) infrastructure-actions#1036).apache/arrow-rs#9975 and apache/datafusion#21165 plan the same setup. OpenDAL dropped CodSpeed in apache/opendal#5626 over false positives from I/O-bound benchmarks, while parsing is CPU-bound.
criterionforcodspeed-criterion-compatinsqlparser_bench/Cargo.toml. Bench sources and localcargo benchstay unchanged.mainplus a nightly schedule, and PR runs once the numbers prove stable.