Skip to content

Refactor hash/dgst from one-shot to streaming#225

Merged
lealem47 merged 3 commits into
wolfSSL:mainfrom
rlm2002:hashStream
May 28, 2026
Merged

Refactor hash/dgst from one-shot to streaming#225
lealem47 merged 3 commits into
wolfSSL:mainfrom
rlm2002:hashStream

Conversation

@rlm2002
Copy link
Copy Markdown
Contributor

@rlm2002 rlm2002 commented May 12, 2026

Addresses issue where hash and digest function encounter hash collisions with large files. Changes from one shot file read to streaming method.

Adds unit tests for hash and digest.

python3 tests/hash/hash-test.py   /* regression test will run with sha256 */
python3 tests/dgst/dgst-test.py.   /* regression test will run with RSA */

/* test other algorithms
 * Valid hash values: md5, sha, sha256, sha384, sha512. Valid key kinds: rsa, ecc
 */
WOLFCLU_LARGE_HASH_ALG=sha512 python3 tests/hash/hash-test.py
WOLFCLU_LARGE_DGST_KEY=ecc   python3 tests/dgst/dgst-test.py

*Note: each algorithm hashes a 4.5 GiB sparse file, so `all` is slow —
budget several minutes per algorithm. The env var only accepts a single algorithm name or `all`.

Fixes zd21765

@rlm2002 rlm2002 self-assigned this May 12, 2026
Copy link
Copy Markdown

@wolfSSL-Fenrir-bot wolfSSL-Fenrir-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fenrir Automated Review — PR #225

Scan targets checked: wolfclu-bugs, wolfclu-src

No new issues found in the changed files. ✅

@rlm2002 rlm2002 assigned wolfSSL-Bot and unassigned rlm2002 May 13, 2026
@rlm2002 rlm2002 assigned lealem47 and unassigned wolfSSL-Bot May 26, 2026
@lealem47
Copy link
Copy Markdown
Contributor

@rlm2002 Could you extract the hash streaming loop in clu_dgst_setup.c and clu_hash.c into one shared function? You could put it in src/tools/clu_funcs.c

@rlm2002 rlm2002 force-pushed the hashStream branch 2 times, most recently from 16ae498 to d0359cb Compare May 28, 2026 15:06
@lealem47 lealem47 assigned rlm2002 and unassigned lealem47 May 28, 2026
rlm2002 added 3 commits May 28, 2026 10:56
refactor to use streaming instead of one shot file read in dgst setup

extract hash streaming loop logic into shared function
add macro for MAX_IO_CHUNK_SZ 4096
Copy link
Copy Markdown
Contributor

@lealem47 lealem47 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @rlm2002 !

@lealem47 lealem47 merged commit 3db446e into wolfSSL:main May 28, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants