Summary
WolfCryptSignature.engineVerify() for RSASSA-PSS calls wc_RsaPSS_VerifyCheck with a fixed salt length derived from PSSParameterSpec.getSaltLength(). This fails when:
- The verifier was initialized without explicit PSS parameters (salt length unknown)
- The signer used a salt length that differs from the verifier's stored params
wolfSSL provides RSA_PSS_SALT_LEN_DISCOVER (-2) via wc_RsaPSS_Verify_ex + wc_RsaPSS_CheckPadding_ex to auto-detect the actual salt length from the signature structure. Invalid signatures are still correctly rejected.
JNI layer change needed
jni_rsa.c Java_com_wolfssl_wolfcrypt_Rsa_wc_1RsaPSS_1VerifyCheck currently calls wc_RsaPSS_VerifyCheck which does not accept RSA_PSS_SALT_LEN_DISCOVER. It needs to be split into wc_RsaPSS_Verify_ex + wc_RsaPSS_CheckPadding_ex so the salt-length discover mode can be passed through.
Java layer change
WolfCryptSignature.engineVerify() should pass Rsa.RSA_PSS_SALT_LEN_DISCOVER instead of the explicit salt length.
Discovered by
Wycheproof RsaPssTest.testSignVerifyWithParameters.
Summary
WolfCryptSignature.engineVerify()for RSASSA-PSS callswc_RsaPSS_VerifyCheckwith a fixed salt length derived fromPSSParameterSpec.getSaltLength(). This fails when:wolfSSL provides
RSA_PSS_SALT_LEN_DISCOVER(-2) viawc_RsaPSS_Verify_ex+wc_RsaPSS_CheckPadding_exto auto-detect the actual salt length from the signature structure. Invalid signatures are still correctly rejected.JNI layer change needed
jni_rsa.cJava_com_wolfssl_wolfcrypt_Rsa_wc_1RsaPSS_1VerifyCheckcurrently callswc_RsaPSS_VerifyCheckwhich does not acceptRSA_PSS_SALT_LEN_DISCOVER. It needs to be split intowc_RsaPSS_Verify_ex+wc_RsaPSS_CheckPadding_exso the salt-length discover mode can be passed through.Java layer change
WolfCryptSignature.engineVerify()should passRsa.RSA_PSS_SALT_LEN_DISCOVERinstead of the explicit salt length.Discovered by
Wycheproof
RsaPssTest.testSignVerifyWithParameters.