[yosuke-wolfssl]

Problem

In builds with OPENSSL_EXTRA and WOLF_CRYPTO_CB_ONLY_ECC, wolfSSL_ECDSA_do_verify() re-encodes r/s into byte signature[ECC_MAX_SIG_SIZE] with i2d_ECDSA_SIG(). That call takes no capacity and passes its own computed length to StoreECC_DSA_Sig() as the limit. d2i_ECDSA_SIG() and ECDSA_SIG_set0() do not bound r/s to the curve order, so an oversized INTEGER overflows the stack buffer with caller-controlled bytes. Caller-supplied DER reaches it through EVP_PKEY_verify(), EVP_VerifyFinal() and EVP_DigestVerify[Final]().

The same branch assigned the wc_ecc_verify_hash() result to ret, so a valid signature returned 0 instead of 1. ECDSA_do_verify() and the EVP ECDSA verify APIs never succeeded in this configuration.

Default builds compile neither path.

Fix (src/pk_ec.c)

• Length query first: wolfSSL_i2d_ECDSA_SIG(sig, NULL) runs before encoding. A length <= 0 or > sizeof(signature) returns WOLFSSL_FATAL_ERROR before anything is written, and the encode must return the same length.
• Success result: the wc_ecc_verify_hash() return is compared directly, so ret stays 1 for a valid signature.

Closes f-11725.

Tests

• test_ECDSA_do_verify_oversized_sig: an r that encodes larger than ECC_MAX_SIG_SIZE must return -1.
• test_wolfSSL_EC_POINT and the ECDH derive part of test_wolfSSL_EVP_PKEY_encoded_public_key are compiled out under WOLF_CRYPTO_CB_ONLY_ECC, matching the library guards, so unit.test links and runs in this configuration.

Verification

No CI job builds WOLF_CRYPTO_CB_ONLY_ECC with --enable-opensslextra (cryptocb-only.yml keeps the compat layer off), so this was verified locally:

• CB-only + opensslextra, ASan: the new test aborts with stack-buffer-overflow before the fix and passes after it.
• cryptocb-only.yml ecc flags + --enable-opensslextra, GCC -Werror, Ubuntu 24.04: 0 failed, 1807 passed of 2970. Master does not link; with only the EC_POINT test guard applied, 5 fail.
• --enable-asn=original, ASan, boundary input (133-byte r, s = 0): returns -1, no ASan errors.
• --enable-all, ASan: the ossl_ec group and the guarded EVP test pass.

Not in this PR: a CI job for this rare combination.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

[yosuke-wolfssl]

@claude review

[github-actions]

MemBrowse Memory Report · Explore what changed →

zephyr-frdm_rw612

• FLASH: log_const_area -8 B, rodata -8 B, text +160 B (+0.1%, 282,608 B / 67,108,864 B, total: 0% used)

No memory changes detected for:

• gcc-arm-cortex-m0plus
• gcc-arm-cortex-m3
• gcc-arm-cortex-m4
• gcc-arm-cortex-m4-baremetal
• gcc-arm-cortex-m4-crypto-only
• gcc-arm-cortex-m4-dtls13
• gcc-arm-cortex-m4-min-ecc
• gcc-arm-cortex-m4-openssl-compat
• gcc-arm-cortex-m4-pkcs7
• gcc-arm-cortex-m4-pq
• gcc-arm-cortex-m4-rsa-only
• gcc-arm-cortex-m4-sp-math
• gcc-arm-cortex-m4-tls12
• gcc-arm-cortex-m4-tls13
• gcc-arm-cortex-m7
• gcc-arm-cortex-m7-pq
• gcc-arm-cortex-m7-tls13
• stm32-sim-stm32h753

---

_{Catch memory regressions in CI with MemBrowse. Free for open source.}

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #11926

Scan targets checked: wolfssl-src, wolfssl-bugs
Not reviewed (no scan target covers them): tests/api/test_ossl_ec.h
Coverage: 2 of 3 in-scope changed file(s) opened by the reviewer; not opened: tests/api/test_ossl_ec.c

Fenrir result: Approved ✅

No new issues found in the changed files.

Advisory only — this automated result does not count as a GitHub approval.

Review tier: Lite

[claude]

I reviewed this PR and didn't find any bugs. It fixes a stack buffer overflow in ECDSA verify, so a maintainer familiar with the WOLF_CRYPTO_CB_ONLY_ECC configuration should still take a look.

What was reviewed:

• The CB-only branch of wolfSSL_ECDSA_do_verify: it queries the length with i2d(sig, NULL), rejects lengths <= 0 or > sizeof(signature) before writing anything, and checks that the actual encode returns the same length. Valid signatures now return 1, because the wc_ecc_verify_hash result is no longer stored in ret.
• The new test in default builds: there the oversized r is caught by wc_ecc_check_r_s_range (r must be below the curve order), so -1 is the expected result on that path too. The NO_RSA guard covers builds whose bignums are too small to hold a 133-byte r.
• The new WOLF_CRYPTO_CB_ONLY_ECC exclusions for test_wolfSSL_EC_POINT and the ECDH set1/derive block only remove tests in that configuration.

Extended reasoning...

The change hardens the WOLF_CRYPTO_CB_ONLY_ECC branch of wolfSSL_ECDSA_do_verify in src/pk_ec.c, where caller-controlled r/s were DER-encoded into a fixed ECC_MAX_SIG_SIZE stack buffer with no bounds check. It also fixes that branch returning 0 instead of 1 for valid signatures. It adds a regression test and compiles out two tests in the CB-only configuration. This is memory-safety and signature-verification code in a crypto library, so it falls under the rule not to auto-approve security-sensitive paths, even though the diff is small and looks correct. The PR says no CI job builds CB-only together with opensslextra, so the fixed path is only covered by the author's local runs.