Skip to content

AES-GCM tests failing with wycherproof in Fedora #844

@Jakuje

Description

@Jakuje

After trying to update rust-cryptoki in Fedora that got softhsm 2.7.0 I started getting some C++ assertions. They fail in one of the wycherproof test rust-cryptoki introduced in recent release:

test aes_gcm_message_wycheproof ... ok
...
✓ Test 310: Valid - Key: 256-bit, Nonce: 10, Tag: 16, AAD: 0, PT: 16
/usr/include/c++/16/bits/stl_vector.h:1253: std::vector<_Tp, _Alloc>::reference std::vector<_Tp, _Alloc>::operator[](size_type) [with _Tp = unsigned char; _Alloc = SecureAllocator<unsigned char>; reference = unsigned char&; size_type = long unsigned int]: Assertion '__n < this->size()' failed.
error: test failed, to rerun pass `--test wycheproof`

The same is happening in Fedora 43 with softhsm 2.6.9 (but looks like it does not in upstream CI with Ubuntu):

✓ Test 310: Valid - Key: 256-bit, Nonce: 10, Tag: 16, AAD: 0, PT: 16
/usr/include/c++/15/bits/stl_vector.h:1263: std::vector<_Tp, _Alloc>::reference std::vector<_Tp, _Alloc>::operator[](size_type) [with _Tp = unsigned char; _Alloc = SecureAllocator<unsigned char>; reference = unsigned char&; size_type = long unsigned int]: Assertion '__n < this->size()' failed.

I assume the failing input is the next test we are not getting output for (output from testsuite running against kryoptic):

✓ Test 311: Invalid (expected failure) - Key: 128-bit, Nonce: 0, Tag: 16, AAD: 0, PT: 0

I did not dive deeper into the softhsm code, but it sounds like some buffer overrun that should not be there ...

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions