tests: psa: tests/subsys/secure_storage/psa/its: test fails on multiply NXP platforms

[hakehuang]

Describe the bug

tests suite	platform
tests/subsys/secure_storage/psa/its/secure_storage.psa.its.secure_storage	frdm_k64f
tests/subsys/secure_storage/psa/its/secure_storage.psa.its.secure_storage	frdm_kw41z
tests/subsys/secure_storage/psa/its/secure_storage.psa.its.secure_storage	frdm_mcxn947/mcxn947/cpu0
tests/subsys/secure_storage/psa/its/secure_storage.psa.its.secure_storage	lpcxpresso55s28
tests/subsys/secure_storage/psa/its/secure_storage.psa.its.secure_storage	lpcxpresso55s36
tests/subsys/secure_storage/psa/its/secure_storage.psa.its.secure_storage	lpcxpresso55s69/lpc55s69/cpu0
tests/subsys/secure_storage/psa/its/secure_storage.psa.its.secure_storage	mimxrt1050_evk
tests/subsys/secure_storage/psa/its/secure_storage.psa.its.secure_storage	rd_rw612_bga
tests/subsys/secure_storage/psa/its/secure_storage.psa.its.secure_storage	rd_rw612_bga/rw612/ethernet
tests/subsys/secure_storage/psa/its/secure_storage.psa.its.secure_storage	twr_ke18f
tests/subsys/secure_storage/psa/its/secure_storage.psa.its.secure_storage.custom.transform	frdm_k64f
tests/subsys/secure_storage/psa/its/secure_storage.psa.its.secure_storage.custom.transform	lpcxpresso55s28
tests/subsys/secure_storage/psa/its/secure_storage.psa.its.secure_storage.custom.transform	lpcxpresso55s36
tests/subsys/secure_storage/psa/its/secure_storage.psa.its.secure_storage.custom.transform	lpcxpresso55s69/lpc55s69/cpu0
tests/subsys/secure_storage/psa/its/secure_storage.psa.its.secure_storage.custom.transform	mimxrt1050_evk
tests/subsys/secure_storage/psa/its/secure_storage.psa.its.secure_storage.custom.transform	twr_ke18f
tests/subsys/secure_storage/psa/its/secure_storage.psa.its.secure_storage.custom.transform	frdm_mcxn947/mcxn947/cpu0
tests/subsys/secure_storage/psa/its/secure_storage.psa.its.secure_storage.custom.transform	rd_rw612_bga
tests/subsys/secure_storage/psa/its/secure_storage.psa.its.secure_storage.custom.transform	rd_rw612_bga/rw612/ethernet
tests/subsys/secure_storage/psa/crypto/secure_storage.psa.crypto.secure_storage	frdm_k64f
tests/subsys/secure_storage/psa/crypto/secure_storage.psa.crypto.secure_storage	lpcxpresso55s28
tests/subsys/secure_storage/psa/crypto/secure_storage.psa.crypto.secure_storage	lpcxpresso55s36
tests/subsys/secure_storage/psa/crypto/secure_storage.psa.crypto.secure_storage	lpcxpresso55s69/lpc55s69/cpu0
tests/subsys/secure_storage/psa/crypto/secure_storage.psa.crypto.secure_storage	mimxrt1050_evk
tests/subsys/secure_storage/psa/crypto/secure_storage.psa.crypto.secure_storage	twr_ke18f

To Reproduce

scripts/twister -p rd_rw612_bga -T tests/subsys/secure_storage/psa/its/

Expected behavior

Test PASS

Impact

secure storage feature support

Logs and console output

*** Booting Zephyr OS build v3.7.0-4339-g1ec5ce05f9f8 ***
WARNING: Using a potentially insecure PSA ITS encryption key provider.
W: Using a potentially insecure PSA ITS encryption key provider.
I: 2 Sectors of 4096 bytes
I: alloc wra: 1, 720
I: data wra: 1, 714
Running TESTSUITE secure_storage_psa_its
===================================================================
START - test_all_offsets
E: Failed to transform data from storage. (-146)
E: Failed to write data to storage. (-142)

    Assertion failed at WEST_TOPDIR/zephyr/tests/subsys/secure_storage/psa/its/src/main.c:67: secure_storage_psa_its_test_all_offsets: ret not equal to PSA_SUCCESS

 FAIL - test_all_offsets in 0.271 seconds
===================================================================
START - test_all_sizes
E: Failed to transform data from storage. (-146)
E: Failed to write data to storage. (-142)

    Assertion failed at WEST_TOPDIR/zephyr/tests/subsys/secure_storage/psa/its/src/main.c:38: secure_storage_psa_its_test_all_sizes: ret not equal to PSA_SUCCESS

 FAIL - test_all_sizes in 0.269 seconds
===================================================================
START - test_max_num_entries
E: Failed to transform data from storage. (-146)
E: Failed to write data to storage. (-142)

    Assertion failed at WEST_TOPDIR/zephyr/tests/subsys/secure_storage/psa/its/src/main.c:90: secure_storage_psa_its_test_max_num_entries: max_num_entries > 1 is false

 FAIL - test_max_num_entries in 0.272 seconds
===================================================================
START - test_write_once_flag
E: Failed to write data to storage. (-142)

    Assertion failed at WEST_TOPDIR/zephyr/tests/subsys/secure_storage/psa/its/src/main.c:122: secure_storage_psa_its_test_write_once_flag: (ret not equal to PSA_SUCCESS)
-142
 FAIL - test_write_once_flag in 0.268 seconds
===================================================================
TESTSUITE secure_storage_psa_its failed.

------ TESTSUITE SUMMARY START ------

SUITE FAIL -   0.00% [secure_storage_psa_its]: pass = 0, fail = 4, skip = 0, total = 4 duration = 1.080 seconds
 - FAIL - [secure_storage_psa_its.test_all_offsets] duration = 0.271 seconds
 - FAIL - [secure_storage_psa_its.test_all_sizes] duration = 0.269 seconds
 - FAIL - [secure_storage_psa_its.test_max_num_entries] duration = 0.272 seconds
 - FAIL - [secure_storage_psa_its.test_write_once_flag] duration = 0.268 seconds

------ TESTSUITE SUMMARY END ------

===================================================================
RunID: 91e049c93bbd0eafd0406251b258c8eb
PROJECT EXECUTION FAILED

Environment (please complete the following information):

• OS: (e.g. Linux, MacOS, Windows)
• Toolchain (e.g Zephyr SDK, ...)
• Commit SHA or Version used: v3.7.0-4339-g1ec5ce05f9f8

Additional context

this case is failed since 85a56b5

[hakehuang]

frdm_mcxw71 has the same problem

[dkalowsk]

@tomi-font assigned to you as I think you added the secure storage feature. If I have it wrong please let me know.

[tomi-font]

There's something up with the storage. Tests that use a RAM-based test storage implementation (secure_storage.psa.its.secure_storage.custom.store, secure_storage.psa.its.secure_storage.custom.both) pass, and the logs clearly indicate something with the storage.
-142 is PSA_ERROR_INSUFFICIENT_STORAGE, returned here.

You could enable CONFIG_SECURE_STORAGE_LOG_LEVEL_DBG for some more logs, and make sure to erase the flash before running the tests.
Or maybe those platforms just need to be filtered out if they don't support settings and NVS (or NVS could be changed to another settings backend)? If that's the case, you could modify the filter here to catch those platforms as well if you have an idea how.