Reuse sigv4 chunk buffer

[RanVaknin]

Benchmark

The new Sigv4ChunkedSigningBenchmark signs and fully consumes an in memory S3 PUT payload (excluding network so that SDK signing and encoding costs remain narrow and accurate). It covers 64 KiB, 1 MiB, and 16 MiB payloads across signed payload, signed checksum trailer, and unsigned checksum trailer modes; these cases exercise the important variations while showing how costs scale with payload size.

Finding and fix

JFR showed that ChunkedEncodedInputStream.getChunk accounted for 51%-74% of self CPU and 88%-100% of sampled allocation weight in the useful profiles. The implementation allocated a new 128 KiB array for every chunk and another array when probing for EOF, so a 16 MiB request allocated roughly 16 MiB of temporary chunk arrays even though only one chunk is consumed at a time.

This optimization target changescore/http-auth-aws at software.amazon.awssdk.http.auth.aws.internal.signer.chunkedencoding.ChunkedEncodedInputStream to lazily allocates one chunk array per stream, refills it only after the previous chunk is exhausted, and releases it at EOF, read failure, or close.

From a customer perspective there is no new API or behavior. Synchronous S3 customers using aws-chunked uploads benefit through lower allocation pressure and less CPU spent allocating and zeroing arrays, with the largest effect on large payloads and unsigned checksum trailer uploads where cryptographic work does not dominate.

Results

Before (64KiB unsigned trailer case)

CPU:

Memory:

After (64KiB unsigned trailer case)

CPU:

Memory:

The final comparison used Amazon Corretto 17 on the same dedicated host for baseline and candidate, with JMH -wi 5 -i 5 -f 2 -prof gc.

Payload	Mode	Baseline ops/s	Candidate ops/s	Delta	Baseline B/op	Candidate B/op	Allocation delta
64 KiB	Signed payload	12790.760	15086.183	+17.9%	286540	155460	-45.7%
64 KiB	Signed checksum trailer	12062.976	14085.332	+16.8%	295328	164192	-44.4%
64 KiB	Unsigned checksum trailer	29173.288	44726.624	+53.3%	279056	147936	-47.0%
1 MiB	Signed payload	1159.483	1322.558	+14.1%	1258366	209843	-83.3%
1 MiB	Signed checksum trailer	1127.180	1278.246	+13.4%	1267259	218548	-82.8%
1 MiB	Unsigned checksum trailer	5370.273	11216.463	+108.9%	1202080	153368	-87.2%
16 MiB	Signed payload	73.385	82.749	+12.8%	17916296	1141989	-93.6%
16 MiB	Signed checksum trailer	71.826	80.489	+12.1%	17928455	1145979	-93.6%
16 MiB	Unsigned checksum trailer	368.025	773.295	+110.1%	17024794	245609	-98.6%

[github-actions]

This pull request has been closed and the conversation has been locked. Comments on closed PRs are hard for our team to see. If you need more assistance, please open a new issue that references this one.