Skip to content

AES fixes - #526

Open
Frauschi wants to merge 2 commits into
wolfSSL:mainfrom
Frauschi:fixes
Open

Frauschi wants to merge 2 commits into
wolfSSL:mainfrom
Frauschi:fixes

Conversation

@Frauschi

@Frauschi Frauschi commented Sep 22, 2026

Copy link
Copy Markdown
Member

Summary

Two independent fixes to the AES client paths, both found on an AURIX TC4Dx port running TLS through wolfHSM.

  1. client: report WH_ERROR_REQUEST_SIZE when AES will not fit the comm buffer - an AES request larger than WOLFHSM_CFG_COMM_DATA_LEN returned WH_ERROR_BADARGS, although the arguments are valid and only the transport is too small. With COMM_DATA_LEN 8192 a TLS record above ~8.1 KiB surfaced as a decrypt failure and bad_record_mac, with nothing telling the caller why. The four AES request builders now return a distinct WH_ERROR_REQUEST_SIZE (new, -2012).
  2. Carry a small AES-GCM AAD in the request instead of over DMA - the DMA AES-GCM request already sends the IV, tag and key inline but passed the AAD as a DmaBuffer. wolfSSL builds the TLS 1.3 additional data in a stack local, and where task stacks are outside the range the server can address, that disqualified the entire request, payload buffers included. With the AAD inline, full-size TLS records go over DMA on the TC4Dx, where the comm buffer size does not apply.

Design notes

WH_ERROR_REQUEST_SIZE is a hard error, not a fallback. The cryptocb passes it straight through rather than mapping it to CRYPTOCB_UNAVAILABLE: the caller asked for the HSM through its devId, and a supported algorithm must not silently run in software because of a transport limit. Large payloads belong on the DMA path or need a larger comm buffer. The check sits ahead of every memcpy into the comm buffer and ahead of the send, so the error has no side effects.

Inline AAD is a client-side policy. WOLFHSM_CFG_DMA_INLINE_AAD_MAX_SIZE (default 128, 0 disables) bounds what the client will inline. The server reads whatever the request carries and never consults the value, so client and server may be built with different ones. Inline AAD does require a server new enough to understand the encoding; an older one rejects such a request with WH_ERROR_BADARGS rather than misreading it.

The server bounds the declared AAD length before using it. aad.sz arrives as a client-supplied uint64_t and the expected-size check is an exact equality, so an unbounded AAD term in that sum could be chosen to wrap it back onto the received size, passing validation with an arbitrary ivSz. It is bounded before the sum is formed, and one validated length then drives the size check, the DMA translation and the wc_AesGcm call alike.

Testing

Full suite passes. New and extended coverage:

  • whTest_CryptoAesCommBuffer (new) drives an oversized request through each mode's wolfCrypt entry point (CBC, CTR, ECB, GCM) and requires exactly WH_ERROR_REQUEST_SIZE. Reverting any one builder to WH_ERROR_BADARGS fails the test for that mode, and mapping the CBC result to CRYPTOCB_UNAVAILABLE fails it with a silent software success.
  • whTest_CryptoReqSize (new file) drives _HandleAesGcmDma directly with five hand-built frames, since the client always builds a self-consistent one. The case that pins the new bound declares a length whose low 32 bits match the frame exactly - truncated to uint32_t it matches, so only a check made before that cast rejects it.
  • whTest_CryptoAesDmaAsync runs the GCM round-trip twice, at the inline cap and one byte past it, so both wire forms of the AAD are exercised; setting the knob to 0 leaves only the DMA form and the test says so in its output. Each leg compares its ciphertext and tag against a software reference rather than against itself: encrypt and decrypt carry the same AAD, so an AAD read at the wrong offset or length still round-trips cleanly. An off-by-one in the DMA AAD length fails the above-cap leg's tag comparison.

Each new guard was verified by breaking it and re-running, not only by passing.

Follow-up

The same req_len > WOLFHSM_CFG_COMM_DATA_LEN -> WH_ERROR_BADARGS pattern appears at ~70 other client-side sites. Only the four AES non-DMA builders use the new code here, to keep this PR focused; converting the rest is a separate PR.

@Frauschi Frauschi self-assigned this Sep 22, 2026

@wolfSSL-Fenrir-bot wolfSSL-Fenrir-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fenrir Automated Review — PR #526

Scan targets checked: wolfhsm-core-bugs, wolfhsm-crypto-bugs, wolfhsm-src

Findings: 3
3 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Reported findings require changes before merge.

Comment thread test/wh_test_crypto.c
Comment thread test/wh_test_crypto.c
Comment thread test/wh_test_crypto.c Outdated

@wolfSSL-Fenrir-bot wolfSSL-Fenrir-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fenrir Automated Review — PR #526

Scan targets checked: wolfhsm-core-bugs, wolfhsm-crypto-bugs, wolfhsm-src

Findings: 1
1 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Reported findings require changes before merge.

Comment thread test/wh_test_crypto.c Outdated

@wolfSSL-Fenrir-bot wolfSSL-Fenrir-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fenrir Automated Review — PR #526

Scan targets checked: wolfhsm-core-bugs, wolfhsm-crypto-bugs, wolfhsm-src

Fenrir result: Approved ✅

No new issues found in the changed files.

Advisory only — this automated result does not count as a GitHub approval.

@wolfSSL-Fenrir-bot
wolfSSL-Fenrir-bot dismissed stale reviews from themself September 22, 2026 12:14

Fenrir's latest completed scan found no issues; clearing the prior automated change request.

@Frauschi Frauschi assigned wolfSSL-Bot and unassigned Frauschi Sep 22, 2026

@bigbrett bigbrett left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love:

  • The new "data too large for transport" error code

Reservations, but could go either way on:

  • Making the AES GCM DMA API variant allow for small AAD to be passed inline

I actually do like this feature, however is size always the correct metric here for determining inline vs DMA AAD? I wonder if we should make this explicit for the user. When invoked under the hood of wolfCrypt then yes, we will need to make an opinionated decision since there is no good way to pass supplementary info other than through the Client context DMA mode state. Perhaps there are multiple DMA modes instead of just on/off? I'm fine with it as-is but just brainstorming

Problems:

  • Silent SW fallback on the client side.

This behavior is incorrect based on the usage model of wolfHSM.

We should never silently fall back to software for a supported algorithm just because the transport is not large enough to handle the data. HSM offload is explicit based on a caller-supplied devId in the context. If the caller says "I want this operation offloaded to the HSM" and we can't offload to the HSM for some reason, then the operation must fail. I think the case you introduced where the key is local to the client in RAM anyway makes this less damaging, however I don't want users to have to try and reverse engineer where the crypto is happening.

Currently the water is a bit muddy on this for compound operations (e.g. algo A isnt supported for offload but it uses algo B internally so in this case we let algo A run in software with algo B offloaded inside) but I hate that too and the real solution is simply to support all those algorithms. I'm sure we missed a few edge cases in the current library but I want to flag it here so we don't add additional cases of this happening.

I know you are doing this to support offload of client-side TLS. Could wolfSSL instead just chose to do this crypto locally in this case, instead of trying to have wolfHSM silently fallback to local software crypto if arbitrary conditions are met? I think the burden needs to be on the caller here to explicitly route their crypto calls properly.

@bigbrett

Copy link
Copy Markdown
Contributor

@Frauschi spent a little more time diving into this after the review and it seems there are more places than I thought where wolfHSM through the wolfCrypt API can silently fall back to software. I think we need to fix this, however it is admittedly a bit beyond the scope of this PR. Do you think we could still prevent the silent fallback here, and I can simultaneously see if there is a uniform solution across the codebase to prevent silent fallbacks going forward?

…uffer

wh_Client_AesGcmRequest and the CBC, CTR and ECB request builders return
WH_ERROR_BADARGS when the request exceeds WOLFHSM_CFG_COMM_DATA_LEN. The
arguments are valid; the transport is just too small for them, and a caller
cannot tell the two apart. On a TC4Dx demo with COMM_DATA_LEN 8192, a TLS
record above about 8.1 KiB surfaced as a decrypt failure and bad_record_mac.

Return a distinct WH_ERROR_REQUEST_SIZE instead. The check sits ahead of
every memcpy into the comm buffer and ahead of the send, so the error has no
side effects. It stays a hard error rather than CRYPTOCB_UNAVAILABLE: the
caller asked for the HSM through its devId, and a supported algorithm must
not silently run in software because of a transport limit. Large payloads
belong on the DMA path or need a larger comm buffer.

whTest_CryptoAesCommBuffer drives an oversized request through each mode's
wolfCrypt entry point and requires exactly WH_ERROR_REQUEST_SIZE. Reverting
any one builder to WH_ERROR_BADARGS fails the test for that mode, as does
mapping the CBC result to CRYPTOCB_UNAVAILABLE.
The DMA AES-GCM request already sends the IV, the auth tag and the key as
trailing data inside the packet, but passed the AAD as a DmaBuffer - an
address the server has to translate, map and release for what is usually a
handful of bytes. Send it inline when it is small enough, and keep the DMA
path for anything larger.

This is not only about saving a round of address handling. The AAD is
frequently built somewhere the server cannot reach at all: wolfSSL assembles
the TLS 1.3 additional data in a stack local, and on targets where task
stacks are outside the address range the server can address, that
disqualified the entire request - payload buffers included - and dropped the
whole operation back to software. Measured on an AURIX TC4Dx with the CSS
engine behind wolfHSM, that was three quarters of all crypto: the port's own
self-test dispatch goes from 172 DMA / 523 comm-buffer to 692 / 3, and TLS
record decryption stops falling back to software entirely.

A request with aad.addr == 0 and aad.sz > 0 carries the AAD immediately after
the key. The server accounts for it in the expected request size, reads it
from the packet, and skips both the address translation and the release.

WOLFHSM_CFG_DMA_INLINE_AAD_MAX_SIZE bounds how much the request will carry,
defaulting to 128 bytes - ample for a TLS 1.2 or 1.3 record header and for
the AUTOSAR and CAN headers that motivate this, while leaving the rest of
WOLFHSM_CFG_COMM_DATA_LEN alone. Anything larger goes over DMA exactly as
before, as does everything when the knob is set to 0. It is a client-side
policy only: the server reads whatever the request carries and never consults
the value, so a client and a server built with different settings still
interoperate.

The server cannot lean on that knob, and must not lean on aad.sz either. It
arrives as a client-supplied uint64_t, and the expected-size check is an
exact equality, so an unbounded AAD term in that sum could be chosen to wrap
it back onto the received size - passing validation with an arbitrary ivSz
and handing GHASH a length that walks gigabytes past the message. Bound it
before the sum is formed: an inline AAD cannot exceed the request carrying
it, and no AAD may exceed what wc_AesGcmEncrypt can hash. The size check,
the DMA translation and the wc_AesGcm call then all use that one validated
length rather than validating one and truncating another.

The client always builds a self-consistent frame, so that bound can only be
reached by a hand-built packet. whTest_CryptoReqSize drives the handler
directly with five: an inline AAD the message is too short to hold, one
declared larger than the whole message, one whose low 32 bits match the frame
exactly, one sized so the 64-bit sum wraps back onto the received size, and a
correctly framed control that must not be rejected. The third is what pins
the bound rather than the pre-existing equality check: truncated to uint32_t
it matches the frame, so only a test made before that cast rejects it.

Both sides locate that AAD from the key size the CLIENT put on the wire, not
from the resolved key length. They differ for an HSM-side key: the wire
carries keySz 0 while the server replaces keyLen with the keystore key's own
length, and computing the offset from that reads the AAD past where the
client wrote it - encrypt then tags over the wrong bytes and only the decrypt
fails, as AES_GCM_AUTH_E.

Mixed versions fail closed rather than silently. An old server computes the
expected request size without the inline AAD, so the length check rejects the
request with WH_ERROR_BADARGS; an old client always sends a real address and
is unaffected. Only the inline case is new on the wire, and only in the
client-to-new-server direction.

Inlining also takes the AAD out of the DMA callbacks and the client-side
allowlist, so wh_Client_AesGcmDmaRequest and wh_Client_AesGcmDmaResponse now
document that the AAD is address-translated and POST-cleaned only when it is
large enough to travel over DMA.

The DMA form of the AAD had no coverage before - every AAD in the suite fits
the comm buffer, so all of them would now take the inline path and the DMA
branch would never run again. The AES-GCM DMA async round-trip therefore runs
twice, at the inline cap and one byte past it. Setting the knob to 0 leaves
only the DMA form, and the test says so in its output rather than reporting a
pass that covered half of what it claims.

A round-trip alone would not have been enough. Encrypt and decrypt carry the
same AAD, so an AAD the server reads at the wrong offset or length still
verifies against itself and the test passes. Each leg therefore compares its
ciphertext and tag against a software reference computed on INVALID_DEVID.
An off-by-one in the DMA AAD length fails the tag comparison on the above-cap
leg; without it the suite stays green.
@Frauschi

Copy link
Copy Markdown
Member Author

Thanks for the explanation on the fallback, I agree. I dropped it completely. The four AES builders now only return WH_ERROR_REQUEST_SIZE and the cryptocb passes it straight through, so an oversized request fails hard no matter where the key lives. Based on the inline-ADD support, the TLS motivation is also covered, as the large records go through the DMA path without a size restriction.

The remaining client-side req_len > WOLFHSM_CFG_COMM_DATA_LEN checks (~70) still return WH_ERROR_BADARGS. I'll move them to the new code in a follow-up PR to keep this one small.

On inline-AAD: I think the current approach is a reasonable first step, but more granular DMA modes could be a great future addition. However, that will surely take more brainstorming work.

The broader silent fallback issues should definitely be looked into in more detail. Even when all algorithms are supported by wolfHSM, a user may still disable some on purpose to have them executed on the client (for whatever reason, maybe performance). I think the bigger issue here is the current wolfssl handling of devId offloading:

  1. When only the wolfCrypt API is used for specific crypto operations, each crypto object (Aes, Sha, etc.) has its own devId and can decide whether to offload to wolfHSM or not. In this case, a silent fallback is very bad.
  2. However, when using the higher layer APIs, eg. for TLS, then we can only pass one devId for the whole wolfssl_ctx object, and all operations are tried to be offloaded to wolfHSM. Here, the user does not have a granular way to specify which should be offloaded. In this case, a silent fallback would at least keep the desired functionality running in many cases (eg. the AES case with the key in client memory).

The second case could also be handled by a custom cryptoCb the user provides in which he can manually do the fallbacks for the desired algorithms himself. Biggest pain-point here is that it is currently kinda hard for a user to set his own callback instead of the library provided one. Maybe we can improve that and properly document it?

@Frauschi
Frauschi requested a review from bigbrett September 23, 2026 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants