Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix sporadic CI failure in crypto test for nrf platform #4582

Merged
merged 4 commits into from
Jan 30, 2021

Conversation

pan-apple
Copy link
Contributor

@pan-apple pan-apple commented Jan 30, 2021

Problem

CI is intermittently failing in crypto tests while running nrf platform tests.

Failure logs are

1: [ Test-CHIP-SecureChannel ]
1: mbedTLS error: BIGNUM - Memory allocation failed
1: 
1: ../../../../../../src/transport/tests/TestSecureSession.cpp:53: assertion failed: "channel.Init(keypair, keypair2.Pubkey(), nullptr, 0, (const uint8_t *) info, sizeof(info)) == CHIP_NO_ERROR"
1: mbedTLS error: BIGNUM - Memory allocation failed
1: 
1: ../../../../../../src/transport/tests/TestSecureSession.cpp:57: assertion failed: "channel.Init(keypair, keypair2.Pubkey(), nullptr, 0, (const uint8_t *) info, sizeof(info)) == CHIP_ERROR_INCORRECT_STATE"
1: mbedTLS error: BIGNUM - Memory allocation failed
1: 
1: ../../../../../../src/transport/tests/TestSecureSession.cpp:64: assertion failed: "channel2.Init(keypair, keypair2.Pubkey(), (const uint8_t *) salt, sizeof(salt), (const uint8_t *) info, sizeof(info)) == CHIP_NO_ERROR"
1: [ Test-CHIP-SecureChannel : Init    ] : FAILED
1: mbedTLS error: BIGNUM - Memory allocation failed
1: 
1: ../../../../../../src/transport/tests/TestSecureSession.cpp:90: assertion failed: "channel.Init(keypair, keypair2.Pubkey(), (const uint8_t *) salt, sizeof(salt), (const uint8_t *) info, sizeof(info)) == CHIP_NO_ERROR"
1: ../../../../../../src/transport/tests/TestSecureSession.cpp:95: assertion failed: "channel.Encrypt(nullptr, 0, nullptr, packetHeader, mac) == CHIP_ERROR_INVALID_ARGUMENT"
1: ../../../../../../src/transport/tests/TestSecureSession.cpp:96: assertion failed: "channel.Encrypt(plain_text, 0, nullptr, packetHeader, mac) == CHIP_ERROR_INVALID_ARGUMENT"
1: ../../../../../../src/transport/tests/TestSecureSession.cpp:97: assertion failed: "channel.Encrypt(plain_text, sizeof(plain_text), nullptr, packetHeader, mac) == CHIP_ERROR_INVALID_ARGUMENT"
1: ../../../../../../src/transport/tests/TestSecureSession.cpp:101: assertion failed: "channel.Encrypt(plain_text, sizeof(plain_text), output, packetHeader, mac) == CHIP_NO_ERROR"
1: [ Test-CHIP-SecureChannel : Encrypt ] : FAILED
1: mbedTLS error: BIGNUM - Memory allocation failed
1: 
1: ../../../../../../src/transport/tests/TestSecureSession.cpp:121: assertion failed: "channel.Init(keypair, keypair2.Pubkey(), (const uint8_t *) salt, sizeof(salt), (const uint8_t *) info, sizeof(info)) == CHIP_NO_ERROR"
1: ../../../../../../src/transport/tests/TestSecureSession.cpp:124: assertion failed: "channel.Encrypt(plain_text, sizeof(plain_text), encrypted, packetHeader, mac) == CHIP_NO_ERROR"
1: mbedTLS error: BIGNUM - Memory allocation failed
1: 
1: ../../../../../../src/transport/tests/TestSecureSession.cpp:132: assertion failed: "channel2.Init(keypair2, keypair.Pubkey(), (const uint8_t *) salt, sizeof(salt), (const uint8_t *) info, sizeof(info)) == CHIP_NO_ERROR"
1: ../../../../../../src/transport/tests/TestSecureSession.cpp:137: assertion failed: "channel2.Decrypt(nullptr, 0, nullptr, packetHeader, mac) == CHIP_ERROR_INVALID_ARGUMENT"
1: ../../../../../../src/transport/tests/TestSecureSession.cpp:138: assertion failed: "channel2.Decrypt(encrypted, 0, nullptr, packetHeader, mac) == CHIP_ERROR_INVALID_ARGUMENT"
1: ../../../../../../src/transport/tests/TestSecureSession.cpp:139: assertion failed: "channel2.Decrypt(encrypted, sizeof(encrypted), nullptr, packetHeader, mac) == CHIP_ERROR_INVALID_ARGUMENT"
1: ../../../../../../src/transport/tests/TestSecureSession.cpp:143: assertion failed: "channel2.Decrypt(encrypted, sizeof(plain_text), output, packetHeader, mac) == CHIP_NO_ERROR"
1: ../../../../../../src/transport/tests/TestSecureSession.cpp:145: assertion failed: "memcmp(plain_text, output, sizeof(plain_text)) == 0"
1: [ Test-CHIP-SecureChannel : Decrypt ] : FAILED
1: Failed Tests:   3 / 3
1: Failed Asserts: 16 / 26

Summary of Changes

PASESession.Clear() was calling mSpake2p.Init(nullptr);, but it doesn't clear the state of the Spake2p object. It initialized the SHA256 context, which was eventually leaked. The test runs were calling this repeatedly that drained the buffer pool from mbedTLS library.

@@ -65,7 +65,6 @@ void PASESession::Clear()
memset(&mWS[0][0], 0, sizeof(mWS));
memset(&mKe[0], 0, sizeof(mKe));
mNextExpectedMsg = Protocols::SecureChannel::MsgType::PASE_Spake2pError;
mSpake2p.Init(nullptr);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, and the point is since we are initing mSpake2p with mCommissioningHash and we are clearing that we don't need to mess with mSpake2p? This could actually use a comment explaining why we don't need to touch that member here. Looks good to me with the comment added.

@boring-cyborg boring-cyborg bot added the crypto label Jan 30, 2021
@github-actions
Copy link

Size increase report for "esp32-example-build" from 4c56ede

File Section File VM
chip-all-clusters-app.elf .flash.text -8 -8
Full report output
BLOAT REPORT

Files found only in the build output:
    report.csv

Comparing ./master_artifact/chip-all-clusters-app.elf and ./pull_artifact/chip-all-clusters-app.elf:

sections,vmsize,filesize
.debug_loc,0,6
.debug_line,0,-4
.flash.text,-8,-8
.debug_info,0,-22


@@ -65,8 +65,11 @@ void PASESession::Clear()
memset(&mWS[0][0], 0, sizeof(mWS));
memset(&mKe[0], 0, sizeof(mKe));
mNextExpectedMsg = Protocols::SecureChannel::MsgType::PASE_Spake2pError;
mSpake2p.Init(nullptr);

// Note: we don't need to explicitly clear the state of mSpake2p object.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It says 'we don't need to' which sounds a as if we could but it does not matter.

However from the change, it seems that calling 'init' actually caused unit test failues, so we likely we 'must not init spake2p'. Should we update the comment? I think the comment may make sense within this PR, however looking in time the comment could use a bit more context. Like:

// clearing mCommisioningHash will automatically clear mSpake2p. 
// NOTE: do NOT call mSpake2p.Init(nullptr) here since that will leak the spake hash.

... assuming the above comment is really what is going on.
Could we fix the possiblity of a memory leak when init is called?

kpschoedel pushed a commit to kpschoedel/connectedhomeip that referenced this pull request Feb 1, 2021
…#4582)

* Fix sporadic CI failure in crypto test for nrf platform
austinh0 pushed a commit to austinh0/connectedhomeip that referenced this pull request Feb 3, 2021
…#4582)

* Fix sporadic CI failure in crypto test for nrf platform
austinh0 pushed a commit to austinh0/connectedhomeip that referenced this pull request Feb 3, 2021
…#4582)

* Fix sporadic CI failure in crypto test for nrf platform
austinh0 pushed a commit to austinh0/connectedhomeip that referenced this pull request Feb 4, 2021
…#4582)

* Fix sporadic CI failure in crypto test for nrf platform
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
crypto hotfix urgent fix needed, can bypass review transport
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants