Repository navigation
Add confidence intervals for benchmarks #158
Description
Currently, our benchmarks are a bit messy. They show some points here and there, and there is a huge amount of noise. Criterion gives us nice confidence intervals that we can use. See how to implement them, you might find inspiration here: chartjs/Chart.js#6899
In the process, we might want to clean-up a bit that page, by stop using big "dots" in the graphs, and maybe having up to 2 graphs per row, so that the page is not so long. Maybe also some explanations on what each benchmark is checking.
Activity
Zeesky-code commentedon Mar 21, 2023
Hello, I would love to work on this issue. Please can I be assigned? :)
HalidOdat commentedon Mar 21, 2023
@Zeesky-code sure go ahead :)
Zeesky-code commentedon Mar 27, 2023
I'm sorry for the delay in working on this issue, as I haven't had time to work on it yet. To prevent delays, I would unassign myself and give someone else the opportunity to take over. Sorry for any inconvenience caused :(
gavincrawford commentedon May 28, 2024
Can I take a shot at it?
HalidOdat commentedon May 28, 2024
@gavincrawford Sure, go ahead! :)
EDIT: The benchmark visualization page code can be found in this repo https://github.com/boa-dev/boa-dev.github.io/
jedel1043 commentedon May 28, 2024
I'll transfer the issue to the other repo, since it doesn't make sense to have it here anymore.
gavincrawford commentedon May 28, 2024
Looking for some clarification on the "confidence intervals". If I'm not missing something, those aren't present in the data hosted on the data repository, meaning displaying them isn't possible without a revision of the code that updates those JSON files.
Other than that, I've gotten the filler plugin up and running, with the absolute difference from the average used as the fill parameter, just for demonstration purposes. Thoughts?
gavincrawford commentedon Jun 12, 2024
Stale. I don't think that this issue is possible to complete unless someone clarifies where you want this data pulled from, as it isn't present in the source that we're using right now. I'm unassigning myself until further clarification.
jedel1043 commentedon Jun 13, 2024
Will mark it as blocked because I realized we previously had confidence intervals, but after moving to another benchmarking test suite we don't have that data anymore.