Repository navigation
refactor(benchmarks): compare fixed workloads against Asyncband - #383
Conversation
|
Benchmark rankings · full suite at Each cell reads rank, elapsed time relative to Asyncband (median ms). #1 is fastest in that row. The Asyncband reference column is always 1.00×, regardless of its rank: 0.75× takes 25% less time; 2.00× takes twice as long. Bold marks the winner; — means the implementation did not participate. All implementations in a row performed the same fixed work. Ratios use the unrounded medians. The original ecosystem run covered 41 scenarios and 132 scenario/implementation pairs, with 10 repetitions each (1,320 measurements). The median of the 41 Asyncband reference medians was 303 ms. The separate primitives suite covered 245 cases with 3 runs each (735 measurements); the median across those case medians was 312 ms. MPSCThe winner changed on repeat for the capacity-1024 Each median covers the whole batch. Each round transfers 16,384 messages. The harness runs one additional priming batch in every timed repetition. The ratios in each row use that run’s Asyncband median as 1.00×.
In a separate Asyncband-only capacity-1024 control, the median was 184 ms with four runtime workers and 184 ms with the current-thread runtime at the same 430 rounds. In an additional 26-round inline-1 KiB diagnostic, the standard Asyncband and Tokio send paths measured 304 ms and 78 ms. Other MPSC scenarios from the full suite
MPMC
SPMC
Broadcast
Watch
WaitGroup
Reproduce the measurementscargo x bench --bench ecosystem -- --repeat 10 --raw
cargo x bench --bench primitives -- --repeat 3 --raw
cargo x bench --bench ecosystem -- mpsc/bounded/tasks-1p1c-cap1024-4w --rounds 430 --repeat 10 --raw
cargo x bench --bench ecosystem -- mpsc/bounded/tasks-4p1c-cap64-4w-inline1k --rounds 26 --repeat 10 --raw
cargo x bench --bench ecosystem -- tasks-1p1c-cap1024 --rounds 430 --baseline-only --repeat 10 --raw |
Primitive baselines and additional ecosystem comparisonsThe previous table covered 41 cross-library scenarios in six families (broadcast, MPMC, MPSC, SPMC, wait group and watch). The same benchmark revision also ran 245 Asyncband-only scenarios across 20 primitive families. Sixteen of those families had no peer comparison in the PR. This follow-up measures at least one shared operation in each of those sixteen families. Original ecosystem rankings MPSC repeat results. With one producer, one consumer, four runtime workers, capacity 1024 and Two public APIs also lacked their own timed case. The original Supplemental peer results29 scenarios, 75 scenario/implementation combinations, 10 repetitions each. Measured on PR head Read each cell as rank, elapsed time relative to Asyncband (median milliseconds for the whole batch). Asyncband is always 1.00×. A value of 0.62× means the same work took 38% less time. Rank #1 is fastest. Ranks use the displayed millisecond precision and share a place on ties. Ratios use the full-precision medians before display. A dash means there is no measurement for that combination. Locks, permits and rendezvous
The Barrier row completes a generation with four waiting parties and one final arrival. The Phaser row uses a barrier for the same fixed four-party rendezvous. Dynamic membership and cancellation are outside these rows. The upgrade row holds an upgradable read guard through the transition. The semaphore try row requests a count of one in Asyncband and Tokio and uses async-lock’s single-permit API. One-time initialization
For the passive One-shot delivery and shared completion
Each oneshot row has one receiver. The completion row publishes the same Blocking executors
All three drivers run the same futures and wake patterns. The two 249 ms ready-future results share first place at the reported precision. Events, condition variables and countdowns
Tokio uses Object pools
Both pools have capacity one and an immediately ready manager. The queued case holds the only object, registers the second borrower, then returns the object. Keyed initialization and duplicate suppression
Moka’s future cache is measured on one initialized key with no eviction during the workload. Both sides use the same hasher and blocking driver. Singleflight uses one suspended leader and three duplicate callers for the same key. All receive the successful result and only the leader runs the initializer. Shutdown
The tokio-util adapter pairs Baselines from the original
|
| Family | Cases | Example workload within family | Rounds | Median (ms) |
|---|---|---|---|---|
barrier |
4 | wait::reused_generation_fanout/4 |
1,250,000 | 251 |
blocking |
4 | block_on::self_wake |
60,000,000 | 239 |
broadcast |
34 | mpmc::bounded::try_send_and_try_recv |
9,000,000 | 293 |
completion |
9 | notify_pending_fanout/4 |
1,250,000 | 326 |
condvar |
5 | wait::notify_waiter |
3,000,000 | 416 |
event |
24 | wait::reused_waiter_handoff |
12,000,000 | 341 |
latch |
7 | wait::worker_fan_in/4 |
4,000,000 | 288 |
mpmc |
7 | bounded::try_send_then_try_recv |
30,000,000 | 351 |
mpsc |
8 | bounded::drain_backpressured_senders/4 |
1,250,000 | 346 |
mutex |
4 | lock::uncontended_reuse |
24,000,000 | 341 |
once |
9 | get_or_init::initialize_cell |
6,000,000 | 284 |
once_map |
54 | lookup::compute_hit_same_key/threads=1 |
10,000,000 | 140 |
oneshot |
3 | send::poll_before_send |
15,000,000 | 336 |
phaser |
15 | reused_phase_handoff/4 |
2,000,000 | 372 |
pool |
5 | bounded::bounded_warm_get_and_return |
6,000,000 | 384 |
rwlock |
15 | read::read_reuse |
30,000,000 | 243 |
semaphore |
17 | acquire::try_acquire_release |
30,000,000 | 285 |
shutdown |
3 | request_shutdown::signal_and_join/4 |
750,000 | 321 |
singleflight |
15 | coalescing::complete_coalesced/4 |
750,000 | 318 |
waitgroup |
3 | wait::complete_waiter_batch/4 |
1,250,000 | 312 |
SPMC and watch are measured in ecosystem. Two Asyncband reference medians there were 314 ms for 320 batches of bounded SPMC delivery (one producer and four consumers, capacity 64, four workers), and 303 ms for 1,700,000 rounds of watch notification to four waiters.
All 245 original Asyncband baselines
Each diagnostic name below omits its common primitives:: prefix. Each median is for the listed number of rounds.
| Diagnostic case | Rounds | Median (ms) |
|---|---|---|
barrier::wait::reused_generation_fanout/1 |
5,000,000 | 375 |
barrier::wait::reused_generation_fanout/4 |
1,250,000 | 251 |
barrier::wait::reused_generation_fanout/16 |
312,500 | 208 |
barrier::wait::cancel_pending |
6,000,000 | 402 |
blocking::block_on::ready |
100,000,000 | 206 |
blocking::block_on::self_wake |
60,000,000 | 239 |
blocking::block_on::cross_thread_wake |
60,000 | 268 |
blocking::wait_timeout::ready_with_timeout |
20,000,000 | 344 |
broadcast::mpmc::bounded::try_send_and_try_recv |
9,000,000 | 293 |
broadcast::mpmc::bounded::try_send_and_drain_fanout/1 |
20,000,000 | 637 |
broadcast::mpmc::bounded::try_send_and_drain_fanout/4 |
5,000,000 | 272 |
broadcast::mpmc::bounded::try_send_and_drain_fanout/16 |
1,250,000 | 183 |
broadcast::mpmc::bounded::full_channel_sender_handoff_cycle/1 |
2,000,000 | 283 |
broadcast::mpmc::bounded::full_channel_sender_handoff_cycle/4 |
500,000 | 184 |
broadcast::mpmc::bounded::full_channel_sender_handoff_cycle/16 |
125,000 | 145 |
broadcast::mpmc::bounded::deliver_to_waiting_receiver |
6,000,000 | 331 |
broadcast::mpmc::bounded::send_without_receivers |
48,000,000 | 354 |
broadcast::mpmc::bounded::try_send_when_full |
60,000,000 | 297 |
broadcast::mpmc::bounded::cancel_blocked_send |
6,000,000 | 260 |
broadcast::mpmc::bounded::drop_lagging_receiver_wakes_senders/1 |
2,000,000 | 875 |
broadcast::mpmc::bounded::drop_lagging_receiver_wakes_senders/2 |
1,000,000 | 610 |
broadcast::mpmc::bounded::drop_lagging_receiver_wakes_senders/32 |
62,500 | 260 |
broadcast::mpmc::bounded::drop_lagging_receiver_wakes_senders/256 |
7,812 | 230 |
broadcast::mpmc::unbounded::send_and_try_recv_shared |
6,000,000 | 241 |
broadcast::mpmc::unbounded::send_and_try_recv_string_shared |
3,000,000 | 177 |
broadcast::mpmc::unbounded::send_and_try_recv_string_cloned_sole |
3,000,000 | 161 |
broadcast::mpmc::unbounded::send_and_try_recv_string_cloned_fanout |
3,000,000 | 234 |
broadcast::mpmc::unbounded::deliver_to_waiting_receiver |
6,000,000 | 326 |
broadcast::mpmc::unbounded::register_and_deliver_fanout/1 |
3,000,000 | 927 |
broadcast::mpmc::unbounded::register_and_deliver_fanout/4 |
750,000 | 324 |
broadcast::mpmc::unbounded::register_and_deliver_fanout/16 |
187,500 | 248 |
broadcast::mpmc::unbounded::send_without_receivers |
12,000,000 | 325 |
broadcast::mpmc::unbounded::try_recv_empty |
60,000,000 | 325 |
broadcast::mpmc::unbounded::drain_with_receivers/peak 1 live 1 |
10,000,000 | 365 |
broadcast::mpmc::unbounded::drain_with_receivers/peak 8 live 8 |
1,250,000 | 93 |
broadcast::mpmc::unbounded::drain_with_receivers/peak 8 live 1 |
10,000,000 | 341 |
broadcast::mpmc::unbounded::drain_with_receivers/peak 32 live 32 |
312,500 | 69 |
broadcast::mpmc::unbounded::drain_with_receivers/peak 32 live 4 |
2,500,000 | 129 |
broadcast::mpmc::unbounded::drain_with_receivers/peak 32 live 1 |
10,000,000 | 341 |
broadcast::mpmc::unbounded::drain_with_receivers/peak 256 live 32 |
312,500 | 68 |
broadcast::mpmc::unbounded::drain_with_receivers/peak 256 live 1 |
10,000,000 | 342 |
broadcast::mpmc::unbounded::cancel_pending |
3,000,000 | 374 |
completion::ready_wait |
100,000,000 | 333 |
completion::complete_then_wait |
6,000,000 | 386 |
completion::notify_pending |
3,000,000 | 287 |
completion::notify_pending_fanout/2 |
2,500,000 | 529 |
completion::notify_pending_fanout/4 |
1,250,000 | 326 |
completion::notify_pending_fanout/16 |
312,500 | 228 |
completion::abandoned_wait |
96,000,000 | 299 |
completion::repoll_pending |
50,000,000 | 298 |
completion::cancel_pending |
24,000,000 | 304 |
condvar::wait::notify_waiter |
3,000,000 | 416 |
condvar::wait::notify_waiter_batch/1 |
3,000,000 | 710 |
condvar::wait::notify_waiter_batch/4 |
750,000 | 363 |
condvar::wait::notify_waiter_batch/16 |
187,500 | 282 |
condvar::wait::cancel_pending |
3,000,000 | 366 |
event::wait::is_set/false/threads=1 |
960,000,000 | 740 |
event::wait::is_set/false/threads=2 |
960,000,000 | 370 |
event::wait::is_set/false/threads=4 |
960,000,000 | 184 |
event::wait::is_set/false/threads=8 |
960,000,000 | 95 |
event::wait::is_set/true/threads=1 |
960,000,000 | 744 |
event::wait::is_set/true/threads=2 |
960,000,000 | 370 |
event::wait::is_set/true/threads=4 |
960,000,000 | 186 |
event::wait::is_set/true/threads=8 |
960,000,000 | 102 |
event::wait::wait_already_set/threads=1 |
240,000,000 | 969 |
event::wait::wait_already_set/threads=2 |
240,000,000 | 436 |
event::wait::wait_already_set/threads=4 |
240,000,000 | 218 |
event::wait::wait_already_set/threads=8 |
240,000,000 | 120 |
event::wait::set_reset_cycle |
24,000,000 | 288 |
event::wait::reused_waiter_handoff |
12,000,000 | 341 |
event::wait::waiter_fan_out/1 |
5,000,000 | 725 |
event::wait::waiter_fan_out/4 |
1,250,000 | 342 |
event::wait::waiter_fan_out/16 |
312,500 | 261 |
event::wait::cancel_pending |
6,000,000 | 394 |
event::wait::cancel_pending_reused |
24,000,000 | 337 |
event::wait::repoll_pending |
50,000,000 | 286 |
event::wait::replace_pending_waker |
48,000,000 | 358 |
event::wait::set_registered/1 |
5,000,000 | 838 |
event::wait::set_registered/16 |
312,500 | 268 |
event::wait::set_registered/256 |
19,531 | 209 |
latch::wait::worker_fan_in/1 |
16,000,000 | 1,105 |
latch::wait::worker_fan_in/4 |
4,000,000 | 288 |
latch::wait::worker_fan_in/16 |
1,000,000 | 113 |
latch::wait::waiter_fan_out/1 |
5,000,000 | 599 |
latch::wait::waiter_fan_out/4 |
1,250,000 | 277 |
latch::wait::waiter_fan_out/16 |
312,500 | 215 |
latch::wait::cancel_pending |
6,000,000 | 386 |
mpmc::bounded::try_send_then_try_recv |
30,000,000 | 351 |
mpmc::bounded::send_then_recv |
24,000,000 | 375 |
mpmc::bounded::wake_blocked_sender |
9,000,000 | 309 |
mpmc::unbounded::send_then_try_recv |
30,000,000 | 316 |
mpmc::unbounded::send_then_recv |
24,000,000 | 304 |
mpmc::unbounded::wake_pending_receiver |
9,000,000 | 289 |
mpmc::unbounded::repoll_pending_receiver |
40,000,000 | 345 |
mpsc::bounded::drain_backpressured_senders/1 |
5,000,000 | 577 |
mpsc::bounded::drain_backpressured_senders/4 |
1,250,000 | 346 |
mpsc::bounded::drain_backpressured_senders/16 |
312,500 | 278 |
mpsc::bounded::cancel_backpressured_senders/1 |
8,000,000 | 782 |
mpsc::bounded::cancel_backpressured_senders/4 |
2,000,000 | 393 |
mpsc::bounded::cancel_backpressured_senders/16 |
500,000 | 323 |
mpsc::unbounded::wake_pending_receiver |
15,000,000 | 288 |
mpsc::unbounded::reregister_pending_receiver |
40,000,000 | 269 |
mutex::lock::uncontended_reuse |
24,000,000 | 341 |
mutex::lock::queued_handoff_cycle/1 |
5,000,000 | 633 |
mutex::lock::queued_handoff_cycle/4 |
1,250,000 | 379 |
mutex::lock::queued_handoff_cycle/16 |
312,500 | 297 |
once::get_or_init::initialize_cell |
6,000,000 | 284 |
once::get_or_init::initialize_cell_waiter_batch/1 |
4,000,000 | 844 |
once::get_or_init::initialize_cell_waiter_batch/4 |
1,000,000 | 385 |
once::get_or_init::initialize_cell_waiter_batch/16 |
250,000 | 289 |
once::wait::complete_waiter |
3,000,000 | 348 |
once::wait::complete_waiter_batch/1 |
5,000,000 | 912 |
once::wait::complete_waiter_batch/4 |
1,250,000 | 346 |
once::wait::complete_waiter_batch/16 |
312,500 | 229 |
once::wait::cancel_pending_wait |
6,000,000 | 420 |
once_map::initialize::initialize_remove_cycle/0 |
3,000,000 | 281 |
once_map::initialize::initialize_remove_cycle/64 |
3,000,000 | 263 |
once_map::initialize::initialize_remove_cycle/1024 |
3,000,000 | 267 |
once_map::initialize::initialize_same_key/2 |
1,500,000 | 500 |
once_map::initialize::initialize_same_key/4 |
750,000 | 358 |
once_map::initialize::initialize_same_key/16 |
187,500 | 282 |
once_map::initialize::initialize_distinct_keys/2 |
1,000,000 | 327 |
once_map::initialize::initialize_distinct_keys/4 |
500,000 | 290 |
once_map::initialize::initialize_distinct_keys/16 |
125,000 | 251 |
once_map::initialize::initialize_error/0 |
3,000,000 | 260 |
once_map::initialize::initialize_error/64 |
3,000,000 | 245 |
once_map::initialize::initialize_error/1024 |
3,000,000 | 260 |
once_map::lookup::get_hit_same_key/threads=1 |
16,000,000 | 125 |
once_map::lookup::get_hit_same_key/threads=2 |
16,000,000 | 241 |
once_map::lookup::get_hit_same_key/threads=4 |
16,000,000 | 572 |
once_map::lookup::get_hit_same_key/threads=8 |
16,000,000 | 417 |
once_map::lookup::get_hit_distributed/64/threads=1 |
8,000,000 | 89 |
once_map::lookup::get_hit_distributed/64/threads=2 |
8,000,000 | 212 |
once_map::lookup::get_hit_distributed/64/threads=4 |
8,000,000 | 691 |
once_map::lookup::get_hit_distributed/64/threads=8 |
8,000,000 | 827 |
once_map::lookup::get_hit_distributed/1024/threads=1 |
8,000,000 | 98 |
once_map::lookup::get_hit_distributed/1024/threads=2 |
8,000,000 | 236 |
once_map::lookup::get_hit_distributed/1024/threads=4 |
8,000,000 | 577 |
once_map::lookup::get_hit_distributed/1024/threads=8 |
8,000,000 | 801 |
once_map::lookup::get_miss_distributed/0/threads=1 |
16,000,000 | 136 |
once_map::lookup::get_miss_distributed/0/threads=2 |
16,000,000 | 226 |
once_map::lookup::get_miss_distributed/0/threads=4 |
16,000,000 | 531 |
once_map::lookup::get_miss_distributed/0/threads=8 |
16,000,000 | 388 |
once_map::lookup::get_miss_distributed/64/threads=1 |
16,000,000 | 158 |
once_map::lookup::get_miss_distributed/64/threads=2 |
16,000,000 | 279 |
once_map::lookup::get_miss_distributed/64/threads=4 |
16,000,000 | 618 |
once_map::lookup::get_miss_distributed/64/threads=8 |
16,000,000 | 500 |
once_map::lookup::get_miss_distributed/1024/threads=1 |
16,000,000 | 146 |
once_map::lookup::get_miss_distributed/1024/threads=2 |
16,000,000 | 255 |
once_map::lookup::get_miss_distributed/1024/threads=4 |
16,000,000 | 566 |
once_map::lookup::get_miss_distributed/1024/threads=8 |
16,000,000 | 485 |
once_map::lookup::compute_hit_same_key/threads=1 |
10,000,000 | 140 |
once_map::lookup::compute_hit_same_key/threads=2 |
10,000,000 | 295 |
once_map::lookup::compute_hit_same_key/threads=4 |
10,000,000 | 581 |
once_map::lookup::compute_hit_same_key/threads=8 |
10,000,000 | 436 |
once_map::lookup::compute_hit_distributed/64/threads=1 |
4,000,000 | 61 |
once_map::lookup::compute_hit_distributed/64/threads=2 |
4,000,000 | 193 |
once_map::lookup::compute_hit_distributed/64/threads=4 |
4,000,000 | 442 |
once_map::lookup::compute_hit_distributed/64/threads=8 |
4,000,000 | 488 |
once_map::lookup::compute_hit_distributed/1024/threads=1 |
4,000,000 | 68 |
once_map::lookup::compute_hit_distributed/1024/threads=2 |
4,000,000 | 218 |
once_map::lookup::compute_hit_distributed/1024/threads=4 |
4,000,000 | 456 |
once_map::lookup::compute_hit_distributed/1024/threads=8 |
4,000,000 | 490 |
once_map::maintenance::remove_reinitialize_cycle/1 |
3,000,000 | 283 |
once_map::maintenance::remove_reinitialize_cycle/64 |
3,000,000 | 267 |
once_map::maintenance::remove_reinitialize_cycle/1024 |
3,000,000 | 263 |
once_map::maintenance::remove_miss/0 |
48,000,000 | 325 |
once_map::maintenance::remove_miss/64 |
48,000,000 | 326 |
once_map::maintenance::remove_miss/1024 |
48,000,000 | 345 |
oneshot::recv::repoll_pending |
140,000,000 | 309 |
oneshot::send::send_before_poll |
15,000,000 | 286 |
oneshot::send::poll_before_send |
15,000,000 | 336 |
phaser::ready_wait |
50,000,000 | 337 |
phaser::arrive_then_wait/2 |
10,000,000 | 327 |
phaser::arrive_then_wait/4 |
5,000,000 | 321 |
phaser::arrive_then_wait/16 |
1,250,000 | 315 |
phaser::reused_phase_handoff/2 |
4,000,000 | 362 |
phaser::reused_phase_handoff/4 |
2,000,000 | 372 |
phaser::reused_phase_handoff/16 |
500,000 | 351 |
phaser::repoll_pending |
50,000,000 | 299 |
phaser::cancel_pending |
24,000,000 | 306 |
phaser::register_batch/2 |
10,000,000 | 468 |
phaser::register_batch/4 |
5,000,000 | 334 |
phaser::register_batch/16 |
1,250,000 | 248 |
phaser::register_individually/2 |
8,000,000 | 411 |
phaser::register_individually/4 |
4,000,000 | 325 |
phaser::register_individually/16 |
1,000,000 | 259 |
pool::bounded::bounded_warm_get_and_return |
6,000,000 | 384 |
pool::bounded::bounded_warm_get_and_return_with_spare_capacity |
6,000,000 | 332 |
pool::bounded::bounded_contended_handoff |
3,000,000 | 432 |
pool::bounded::cancel_bounded_waiter |
3,000,000 | 246 |
pool::unbounded::unbounded_warm_try_get_and_return |
9,000,000 | 301 |
rwlock::upgradable::read_reuse |
29,000,000 | 297 |
rwlock::upgradable::write_reuse |
10,000,000 | 319 |
rwlock::upgradable::eight_reads_then_one_write |
2,600,000 | 302 |
rwlock::upgradable::upgradable_read_reuse |
15,000,000 | 315 |
rwlock::upgradable::upgradable_read_then_upgrade |
8,300,000 | 310 |
rwlock::upgradable::upgradable_read_then_try_upgrade |
10,000,000 | 306 |
rwlock::upgradable::upgradable_read_then_downgrade |
15,000,000 | 316 |
rwlock::upgradable::readers_then_queued_upgrade/1 |
1,000,000 | 92 |
rwlock::upgradable::readers_then_queued_upgrade/4 |
1,000,000 | 153 |
rwlock::upgradable::readers_then_queued_upgrade/16 |
1,000,000 | 383 |
rwlock::read::eight_reads_then_one_write |
3,000,000 | 230 |
rwlock::read::read_reuse |
30,000,000 | 243 |
rwlock::write::readers_then_queued_writer/1 |
5,000,000 | 850 |
rwlock::write::readers_then_queued_writer/4 |
1,250,000 | 292 |
rwlock::write::readers_then_queued_writer/16 |
312,500 | 140 |
semaphore::acquire::handoff_permit |
3,000,000 | 254 |
semaphore::acquire::try_acquire_release |
30,000,000 | 285 |
semaphore::acquire::try_acquire_release_with_spare_permits |
60,000,000 | 318 |
semaphore::acquire::owned_try_acquire_release |
48,000,000 | 298 |
semaphore::acquire::queued_owned_burst/1 |
4,000,000 | 780 |
semaphore::acquire::queued_owned_burst/4 |
1,000,000 | 344 |
semaphore::acquire::queued_owned_burst/16 |
250,000 | 279 |
semaphore::acquire::cancel_pending_acquire |
3,000,000 | 216 |
semaphore::acquire::owned_try_acquire_rejected |
96,000,000 | 385 |
semaphore::acquire::cancel_pending_owned_batch/1 |
8,000,000 | 533 |
semaphore::acquire::cancel_pending_owned_batch/4 |
2,000,000 | 343 |
semaphore::acquire::cancel_pending_owned_batch/16 |
500,000 | 307 |
semaphore::release::fulfill_debt_repeatedly |
300,000 | 361 |
semaphore::release::release_to_waiters/1 |
4,000,000 | 737 |
semaphore::release::release_to_waiters/2 |
2,000,000 | 463 |
semaphore::release::release_to_waiters/32 |
125,000 | 232 |
semaphore::release::release_to_waiters/256 |
15,625 | 195 |
shutdown::request_shutdown::signal_and_join/1 |
3,000,000 | 886 |
shutdown::request_shutdown::signal_and_join/4 |
750,000 | 321 |
shutdown::request_shutdown::signal_and_join/16 |
187,500 | 179 |
singleflight::coalescing::complete_coalesced/2 |
1,500,000 | 400 |
singleflight::coalescing::complete_coalesced/4 |
750,000 | 318 |
singleflight::coalescing::complete_coalesced/16 |
187,500 | 259 |
singleflight::coalescing::sequential_retries_after_leader_error/2 |
1,500,000 | 410 |
singleflight::coalescing::sequential_retries_after_leader_error/4 |
750,000 | 300 |
singleflight::coalescing::sequential_retries_after_leader_error/16 |
187,500 | 262 |
singleflight::leaders::leader_success |
3,000,000 | 263 |
singleflight::leaders::distinct_keys/2 |
1,000,000 | 315 |
singleflight::leaders::distinct_keys/4 |
500,000 | 294 |
singleflight::leaders::distinct_keys/16 |
125,000 | 241 |
singleflight::leaders::distributed_leaders/threads=1 |
1,000,000 | 96 |
singleflight::leaders::distributed_leaders/threads=2 |
1,000,000 | 180 |
singleflight::leaders::distributed_leaders/threads=4 |
1,000,000 | 369 |
singleflight::leaders::distributed_leaders/threads=8 |
1,000,000 | 313 |
singleflight::leaders::leader_error |
3,000,000 | 258 |
waitgroup::wait::complete_waiter_batch/1 |
5,000,000 | 736 |
waitgroup::wait::complete_waiter_batch/4 |
1,250,000 | 312 |
waitgroup::wait::complete_waiter_batch/16 |
312,500 | 204 |
Reproduce the supplemental peer measurements at 6102089
In a disposable checkout at 6102089, save the generator below as peer-checks.rb at the repository root. It registers a temporary Cargo bench target and generates its adapters from the PR’s workload bodies. Tokio is 1.53.1 from the PR lockfile. The other direct peer dependencies are pinned in the generator.
ruby peer-checks.rb
cargo x bench --bench peer_checks -- --test
cargo x bench --bench peer_checks -- --repeat 10 --raw# Run in a disposable checkout at 6102089. The PR's Rust workload bodies are the templates.
mf = 'benchmarks/Cargo.toml'
manifest = File.read(mf)
unless manifest.include?('name = "peer_checks"')
manifest.sub!(' "latch",', " \"latch\",\n \"lazy-cell\",")
deps = <<~TOML
async-lock = "=3.4.2"
async-std = { version = "=1.13.2", features = ["unstable"] }
async-once-cell = { version = "=0.5.4", features = ["std"] }
async_singleflight = "=0.6.2"
latches = { version = "=0.2.0", default-features = false, features = ["task", "std"] }
futures-channel = "=0.3.33"
futures-util = "=0.3.33"
futures-executor = "=0.3.33"
futures-lite = "=2.6.1"
deadpool = { version = "=0.12.3", default-features = false, features = ["managed"] }
moka = { version = "=0.12.16", features = ["future"] }
tokio-util = { version = "=0.7.20", features = ["rt"] }
TOML
manifest.sub!("[dev-dependencies]\n", "[dev-dependencies]\n" + deps)
File.write(mf, manifest + "\n[[bench]]\nname = \"peer_checks\"\npath = \"benches/peer_checks.rs\"\nharness = false\n")
end
@code = <<~RUST
#![allow(unused_imports)]
use benchmarks::harness::{Bencher, Runner};
use benchmarks::support::{bench_context, poll_ready, poll_pending, poll_pinned_ready, wait_until_open};
use std::cell::Cell;
use std::convert::Infallible;
use std::future::{IntoFuture, poll_fn};
use std::hash::{BuildHasherDefault, DefaultHasher};
use std::hint::black_box;
use std::pin::{pin, Pin};
use std::sync::{Arc, mpsc};
use std::sync::atomic::{AtomicBool, Ordering};
use std::task::{Poll, Waker};
use std::thread;
use asyncband::blocking::FutureExt;
RUST
@calls = []
def src(file, op)
text = File.read("benchmarks/benches/primitives/#{file}.rs")
text[/^fn #{op}\(bencher: Bencher[\s\S]*?^}/] or raise "missing #{file} #{op}"
end
def change(text, replacements)
replacements.each do |from,to|
raise "missing #{from}" unless text.include?(from)
text = text.gsub(from){to}
end
text
end
def row(label, n, variants, arg=nil)
id = @calls.size
calls = variants.each_with_index.map do |(impl, code),idx|
code = code.sub(/^fn \w+\(bencher: Bencher/, 'pub(super) fn run(bencher: Bencher')
raise label unless code.include?('pub(super) fn run(bencher: Bencher')
mod = "v#{id}_#{idx}"
@code << "\nmod #{mod} { use super::*;\n#{code}\n}\n"
"(\"#{impl}\", &|b| #{mod}::run(b#{arg ? ", #{arg}" : ""}))"
end
@calls << "r.compare(\"#{label}\", #{n}, &[#{calls.join(', ')}]);"
end
# Borrowed locks and owned handoffs, with the same payload, boxes and polls.
['uncontended_reuse','queued_handoff_cycle'].each do |op|
s = src('mutex/lock',op); q = op.start_with?('queued')
row("mutex/#{op}" + (q ? '/4' : ''), q ? 1_250_000 : 24_000_000,{
'Asyncband'=>"use asyncband::mutex::Mutex;\n"+s,
'Tokio'=>"use tokio::sync::Mutex;\n"+s,
'async-lock'=>"use async_lock::Mutex;\n"+s.gsub('mutex.clone().lock_owned()', 'mutex.lock_arc()')
}, q ? 4 : nil)
end
[['read_reuse',30_000_000],['eight_reads_then_one_write',3_000_000]].each do |op,n|
s=src('rwlock/read',op)
row("rwlock/#{op}",n,{'Asyncband'=>"use asyncband::rwlock::RwLock;\n"+s,'Tokio'=>"use tokio::sync::RwLock;\n"+s,'async-lock'=>"use async_lock::RwLock;\n"+s})
end
s=src('rwlock/upgradable','upgradable_read_then_upgrade')
row('rwlock/upgrade',8_300_000,{
'Asyncband'=>"use asyncband::rwlock::{UpgradableRwLock, RwLockUpgradableReadGuard};\n"+s,
'async-lock'=>"use async_lock::{RwLock, RwLockUpgradableReadGuard};\n"+change(s,{'UpgradableRwLock::new'=>'RwLock::new'})
})
s=src('semaphore/acquire','try_acquire_release')
row('semaphore/try-one-and-return',30_000_000,{
'Asyncband'=>"use asyncband::semaphore::Semaphore;\n"+s,
'Tokio'=>"use tokio::sync::Semaphore;\n"+change(s,{'try_acquire(black_box(1))'=>'try_acquire_many(black_box(1))'}),
'async-lock'=>"use async_lock::Semaphore;\n"+change(s,{'try_acquire(black_box(1))'=>'try_acquire()'})
})
s=src('semaphore/acquire','handoff_permit')
row('semaphore/pending-handoff',3_000_000,{
'Asyncband'=>"use asyncband::semaphore::Semaphore;\n"+s,
'Tokio'=>"use tokio::sync::Semaphore;\n"+change(s,{'semaphore.acquire(black_box(1))'=>'semaphore.acquire_many(black_box(1))','semaphore.release('=>'semaphore.add_permits(','let permit = poll_pinned_ready(acquire.as_mut(), &mut context);'=>'let permit = poll_pinned_ready(acquire.as_mut(), &mut context).unwrap();'})
})
s=src('barrier/wait','reused_generation_fanout')
row('barrier/complete-generation-5-parties',1_250_000,{
'Asyncband'=>"use asyncband::barrier::Barrier;\n"+s,
'Tokio'=>"use tokio::sync::Barrier;\n"+change(s,{'(participant_count + 1) as u32'=>'participant_count + 1'}),
'async-lock'=>"use async_lock::Barrier;\n"+change(s,{'(participant_count + 1) as u32'=>'participant_count + 1'})
},4)
[['send','send_before_poll',15_000_000],['send','poll_before_send',15_000_000],['recv','repoll_pending',140_000_000]].each do |file,op,n|
s=src("oneshot/#{file}",op)
row("oneshot/#{op}",n,{'Asyncband'=>"use asyncband::oneshot::channel;\n"+s,'Tokio'=>"use tokio::sync::oneshot::channel;\n"+s,'futures-channel'=>"use futures_channel::oneshot::channel;\n"+s})
end
[['initialize_cell',6_000_000,nil],['initialize_cell_waiter_batch',1_000_000,4]].each do |op,n,arg|
s=src('once/get_or_init',op)
row('once-cell/'+op+(arg ? '/4' : ''),n,{
'Asyncband'=>"use asyncband::once::OnceCell;\n"+s,
'Tokio'=>"use tokio::sync::OnceCell;\n"+s,
'async-lock'=>"use async_lock::OnceCell;\n"+s,
'async-once-cell'=>"use async_once_cell::OnceCell;\n"+change(s,{'get_or_init(|| async'=>'get_or_init(async'})
},arg)
end
s=src('once/wait','complete_waiter')
row('once/complete-registered-waiter',3_000_000,{
'Asyncband'=>"use asyncband::once::Once;\n"+s,
'async-lock'=>"use async_lock::OnceCell;\n"+change(s,{'Once::new()'=>'OnceCell::<()>::new()','once.call_once(async || {})'=>'once.get_or_init(|| async {})','once.is_completed()'=>'once.get().is_some()'})
})
s=src('condvar/wait','notify_waiter')
row('condvar/release-notify-relock',3_000_000,{
'Asyncband'=>"use asyncband::{condvar::Condvar,mutex::Mutex};\n"+s,
'async-std'=>"use async_std::sync::{Condvar,Mutex};\n"+s
})
# Poll the same futures under all three blocking drivers.
[['ready',100_000_000],['self_wake',60_000_000],['cross_thread_wake',60_000]].each do |op,n|
s=src('blocking/block_on',op)
alt=s.gsub(/async \{ black_box\(42usize\) \}\.block_on\(\)/, 'DRIVE(async { black_box(42usize) })')
if op!='ready'
alt=change(alt,{'poll_fn(|context|'=>'DRIVE(poll_fn(|context|', "})\n .block_on()" => '}))'})
end
row("blocking/#{op}",n,{'Asyncband'=>s,'futures-lite'=>change(alt,{'DRIVE'=>'futures_lite::future::block_on'}),'futures-executor'=>change(alt,{'DRIVE'=>'futures_executor::block_on'})})
end
s=src('event/wait','waiter_fan_out')
row('event/already-registered-fanout-4',1_250_000,{
'Asyncband'=>"use asyncband::event::ManualResetEvent;\n"+s,
'Tokio Notify'=>"use tokio::sync::Notify;\n"+change(s,{'ManualResetEvent::new()'=>'Notify::new()', 'event.wait()'=>'event.notified()','event.set()'=>'event.notify_waiters()'})
},4)
s=<<~RUST
fn probe(bencher: Bencher) {
let event = TYPE::new(); let mut cx=bench_context();
bencher.bench_local(|| {
let mut wait=pin!(event.WAIT()); poll_pending(wait.as_mut(), &mut cx);
event.SIGNAL(); poll_pinned_ready(wait.as_mut(), &mut cx);
});
}
RUST
row('extra/auto-reset-single-pending-waiter',16_000_000,{
'Asyncband'=>change(s,{'TYPE'=>'asyncband::event::AutoResetEvent','WAIT'=>'wait','SIGNAL'=>'set'}),
'Tokio Notify'=>change(s,{'TYPE'=>'tokio::sync::Notify','WAIT'=>'notified','SIGNAL'=>'notify_one'})
})
s=src('latch/wait','worker_fan_in')
row('latch/four-countdowns-one-waiter',4_000_000,{
'Asyncband'=>"use asyncband::latch::Latch;\n"+s,
'latches'=>"use latches::task::Latch;\n"+change(s,{'worker_count as u32'=>'worker_count'})
},4)
s=src('phaser/mod','reused_phase_handoff')
b=src('barrier/wait','reused_generation_fanout')
row('phaser/fixed-four-party-rendezvous',2_000_000,{
'Asyncband'=>"use asyncband::phaser::Phaser;\n"+s,
'Tokio Barrier'=>"use tokio::sync::Barrier;\n"+change(b,{'(participant_count + 1) as u32'=>'participant_count','0..participant_count'=>'0..participant_count - 1'}),
'async-lock Barrier'=>"use async_lock::Barrier;\n"+change(b,{'(participant_count + 1) as u32'=>'participant_count','0..participant_count'=>'0..participant_count - 1'})
},4)
s=<<~RUST
fn probe(bencher: Bencher) {
let mut cx=bench_context();
bencher.bench_local(|| {
let lazy=pin!(CONSTRUCT);
black_box(*poll_ready(ACCESS, &mut cx))
});
}
RUST
row('extra/lazy-cell-first-force',7_000_000,{
'Asyncband'=>change(s,{'CONSTRUCT'=>'asyncband::once::LazyCell::new(|| async { black_box(1usize) })','ACCESS'=>'asyncband::once::LazyCell::force_pin(lazy.as_ref())'}),
'async-once-cell'=>change(s,{'CONSTRUCT'=>'async_once_cell::Lazy::new(async { black_box(1usize) })','ACCESS'=>'lazy.as_ref().get()'})
})
s=src('completion/mod','notify_pending_fanout')
shared=<<~RUST
fn probe(bencher: Bencher, observers: usize) {
let mut context=bench_context();
bencher.bench_local(|| {
let (tx,rx)=black_box(futures_channel::oneshot::channel::<usize>());
let completion=futures_util::FutureExt::shared(rx);
let mut waiters=(0..observers).map(|_| Box::pin(completion.clone())).collect::<Vec<_>>();
for w in &mut waiters {poll_pending(w.as_mut(), &mut context);}
tx.send(black_box(1usize)).unwrap();
for mut w in waiters {black_box(poll_pinned_ready(w.as_mut(), &mut context).unwrap());}
});
}
RUST
row('completion/publish-to-four-pending-observers',1_250_000,{
'Asyncband'=>"use asyncband::completion;\n"+s, 'futures Shared+oneshot'=>shared
},4)
manager=File.read('benchmarks/benches/primitives/pool/bounded.rs')[/^struct Manager;[\s\S]*?(?=^fn bounded_warm_get_and_return\()/]
manager="use asyncband::pool::{bounded,ManageObject,ObjectStatus};\n"+manager
dead=<<~RUST
struct Manager;
impl deadpool::managed::Manager for Manager {
type Type=usize; type Error=Infallible;
async fn create(&self) -> Result<usize,Infallible> { Ok(0) }
async fn recycle(&self, _: &mut usize, _: &deadpool::managed::Metrics) -> deadpool::managed::RecycleResult<Infallible> {Ok(())}
}
RUST
[['bounded_warm_get_and_return',6_000_000],['bounded_contended_handoff',3_000_000]].each do |op,n|
s=src('pool/bounded',op)
row("pool/#{op}",n,{'Asyncband'=>manager+s,'deadpool'=>dead+change(s,{'bounded::Pool::new(bounded::PoolConfig::new(1), Manager)'=>'deadpool::managed::Pool::<Manager>::builder(Manager).max_size(1).build().unwrap()'})})
end
s=src('shutdown/request_shutdown','signal_and_join')
track=<<~RUST
fn probe(bencher: Bencher, observer_count: usize) {
let mut cx=bench_context();
bencher.bench_local(|| {
let token=tokio_util::sync::CancellationToken::new();
let tracker=tokio_util::task::TaskTracker::new();
let first=(token.clone(),tracker.token());
let guards=(0..observer_count).map(|_| first.clone()).collect::<Vec<_>>();
drop(first);
let mut waits=guards.iter().map(|g| Box::pin(g.0.cancelled())).collect::<Vec<_>>();
for w in &mut waits {poll_pending(w.as_mut(),&mut cx);}
token.cancel();
tracker.close();
for mut w in waits {poll_pinned_ready(w.as_mut(),&mut cx);}
drop(guards);
poll_ready(tracker.wait(), &mut cx);
});
}
RUST
row('shutdown/signal-and-join-4',750_000,{'Asyncband'=>"use asyncband::shutdown;\n"+s, 'tokio-util token+tracker'=>track},4)
s=<<~RUST
fn probe(bencher: Bencher) {
let map=MAP;
futures_lite::future::block_on(PRIME);
bencher.bench_local(|| black_box(futures_lite::future::block_on(HIT)));
}
RUST
row('once-map/already-initialized-compute-one-key',10_000_000,{
'Asyncband'=>change(s,{'MAP'=>'asyncband::once::OnceMap::<usize,usize,BuildHasherDefault<DefaultHasher>>::default()','PRIME'=>'map.compute(0, || async { 1usize })','HIT'=>'map.compute(black_box(0), || async { unreachable!() })'}),
'moka future cache'=>change(s,{'MAP'=>'moka::future::Cache::<usize,usize>::builder().build_with_hasher(BuildHasherDefault::<DefaultHasher>::default())','PRIME'=>'map.get_with(0, async { 1usize })','HIT'=>'map.get_with(black_box(0), async { unreachable!() })'})
})
s=src('singleflight/coalescing','complete_coalesced')
s=change(s,{'Cell::new(false)'=>'AtomicBool::new(false)','wait_until_open(&gate).await'=>'poll_fn(|_| if gate.load(Ordering::Relaxed) {Poll::Ready(())} else {Poll::Pending}).await','gate.set(true)'=>'gate.store(true, Ordering::Relaxed)'})
row('singleflight/four-callers-success',750_000,{
'Asyncband'=>"type BenchGroup=asyncband::singleflight::Group<usize,usize,BuildHasherDefault<DefaultHasher>>;\n"+s,
'async_singleflight'=>"type BenchGroup=async_singleflight::UnaryGroup<usize,usize,BuildHasherDefault<DefaultHasher>>;\n"+change(s,{'group.work(0, || async'=>'group.work(&0usize, async'})
},4)
File.write('benchmarks/benches/peer_checks.rs', @code+"\nfn main() { let mut r=Runner::from_args();\n"+@calls.join("\n")+"\nr.finish(); }\n")Port this branch's benchmarks to the fixed-work harness from apache#383. The bounded MPMC scenarios add `AsyncbandMpmcReserve` and crossbeam-channel peers, a 1P/8C thread batch, a capacity-1 thread batch, and a spinning batch. The primitives add `try_reserve_then_send` and `reserve_then_send`. New round counts put the reference, Asyncband at `c37b1a4`, near 300 ms on macOS.
Summary
Per-operation timings make it difficult to see how long different implementations take to finish the same job. Following Benchmark In Milliseconds,
cargo x bench --bench ecosystemcompares 41 fixed workloads across channels, broadcast, watch and wait groups. Each table shows median milliseconds, the full observed range and elapsed time relative to its Asyncband reference.Design Notes
--baseline-onlyand--roundsallow explicit sizing without adaptive calibration.--rawshows every run.--bench primitives, including upgradable locks, shared and cloned broadcast delivery, and pending oneshot repolls. See the benchmark guide for commands and timing boundaries.Validated with
cargo x lint,cargo x test, and smoke execution of 132 scenario/implementation pairs in the ecosystem suite and 245 primitive cases. A ten-repetition ecosystem run on an Apple M4 Max produced 1,320 measurements; the 41 Asyncband reference medians ranged from 241 to 366 ms, with a median of 303 ms across scenarios.