Repository navigation
Java running time is not accurate #437
Description
Java seems to do something with threads even though the code run is completely single threaded. It's not enough to just force the garbage collector to be single threaded.
For most people this is probably not a significant timing issue, but seems easy to fix.
On a 128-core (256-virtual) machine it does cause massive issues.
My timing went down from 57s to 0.3s by adding the flag -XX:ActiveProcessorCount=1 to languages.yaml
On a 6-core (6-virtual) machine it seems to be able to cause the reported running time to be approximately double the actual running time.
Java is however often I/O bound in my experience in competitive programming setting so this may not be extremely common.
Activity
Tagl commentedon Jul 10, 2026
There might be good reason to set the number of ActiveProcessorCount to 2 instead of 1 to ensure the garbage collection can run.
A quick test on my machine does not show any significant impact to timing going from 1 to 2, but going to 4 roughly doubled the times.
Tagl commentedon Jul 13, 2026
@gkreitz Any thoughts on this before I make a PR?
Do you think it affects the measurements in Kattis too?
gkreitz commentedon Jul 13, 2026
Java's GC will definitely be counted against the submission's time, as it should. Java's argument list (problemtools, as well as Kattis) includes
-XX:+UseSerialGCwhich I'm assuming is there to make the GC less aggressive.I'm struggling to find good documentation on these flags. From a quick google, ActiveProcessorCount seems to affect other things (in particular, it sounds to me like it'll kill off multithreading for java, at least in some scenarios), so I'd be hesitant to just throw that in in order to calm down the GC.
I suspect the behavior you're reporting depends on the JVM version. Your report does not mention what JVM you're observing this on.
Tagl commentedon Jul 13, 2026
Garbage collection was my first thought too, and I do have the serial GC on during these tests.
The 128-core machine:
The 6-core machine:
gkreitz commentedon Jul 14, 2026
I'm by no means a JVM expert (if we have some reading, please chime in!). I also do not have a 128-core machine to reproduce this on, so testing is a bit tricky. :)
Based on what I can read myself to, the serial GC is single threaded, so it would seem very weird if GC is the culprit here. The other main suspect would be JIT. There are flags to control JIT too, I think. Stuff like
-XX:TieredStopAtLevel=1(for some value of 1 between 0 and 5), and-XX:CICompilerCount=1. Can you play around a bit with those and see if they affect things? I'm not sure we want to fiddle with-XX:TieredStopAtLeveltoo much, but limiting the number of threads for doing JIT is probably relatively safe. And if completely disabling the JIT still shows a large difference between 128 and 6 cores, then we know JIT isn't the culprit either, and we can try to figure out what other things java may do that scales with processor count.meisterT commentedon Jul 14, 2026
My suggestion is to compare the output of
-XX:+PrintFlagsFinalon the 256 thread machine vs a normal say 6 core machine vs withtaskset -c 0and see what the JVM decides to change.Does Kattis (not verifyproblem) not pin submissions to a single core anyway?
gkreitz commentedon Jul 15, 2026
We do not currently pin to a core, no. We also are not running on a 128 core machine. :)
Tagl commentedon Jul 15, 2026
Tagl commentedon Jul 16, 2026
Setting
-XX:TieredStopAtLevel=1does mostly fix it.It both makes the code example I have run a bit faster, but I still get a 2 second measurement for something that's taking closer to 0.1s.
Adding the
-XX:ActiveProcessorCount=2as well does actually give a timing of ~0.1s, but was ~0.3s without the-XX:TieredStopAtLevel=1flag.I did not see any effect from setting
-XX:CICompilerCount=2which is the lowest value allowed.gkreitz commentedon Jul 16, 2026
Thanks for testing, looks like we at least have a strong indication that the JIT is the problem.
After vacations, I'll see if I can gather some data to see how the flags would affect Kattis submissions, so we have more data points. From reading, I would guess
-XX:TieredStopAtLevel=1likely makes sense in our setting (but I do want to see the performance impact of that for more submissions). I guess-XX:ActiveProcessorCount=2also makes some sense.And, again, if someone actually has deeper knowledge in JVM tuning and could opine, that's very helpful. Changing these flags does take some testing and usually cause some submissions to be faster and some to be slower, so I don't want to fiddle with them too much.
Tagl commentedon Oct 2, 2026
Have you had a chance gather data on this?
gkreitz commentedon Oct 2, 2026
Not yet. There is a bit of work involved, and impact is relatively small (even if it's unfortunately affecting you), so this typically doesn't end up at the top of my list. I will get around to it at some point though.
gkreitz commentedon Oct 15, 2026
Yesterday was the day this ended up at the top of my list. :)
Thanks for raising the issue! The change had a significantly larger positive effect on Java runtime than I would have guessed when I tested on Kattis hardware.
Add JVM flags for more consistent (and typically faster) timing #437
Merge pull request #505 from gkreitz/437_jvm_options
mzuenni commentedon Oct 17, 2026
Should the same flags be set for Kotlin (maybe even Scala)?
gkreitz commentedon Oct 17, 2026
Possibly. They're even further down my priority list to test, and I would not want to make the change without proper testing (as their toolchains may generate bytecode which is affected differently by JVM optimizations.
Again, if someone has actual expertise here and wants to chime in, that's very welcome. Otherwise, we'll most likely leave it at Java.