[BUG] vmh_test_single test fails

[kv2019i]

Describe the bug
vmh_test_single() test in work/sof/zephyr/test/vmh.c fails.

To Reproduce

• uncomment SOF_BOOT_TEST and ZTEST in sof/app/debug_overlay.conf
• build SOF with debug enabled "-d"
• boot FW, test is run on boot

Reproduction Rate
100%

Expected behavior
Test should pass

Impact
The failing test caused DSP panics in #8621 . The relation is not fully understood yet, but potentially vmh error can have serious side-impacts that we need to understand

Environment

1. Branch name and commit hash of the 2 repositories: sof (firmware/topology) and linux (kernel driver).
   • Kernel: 20231213 daily
   • SOF: af3fa41 (SOF main)
2. Name of the topology file
   • Topology: n/a
3. Name of the platform(s) on which the bug is observed.
   • Platform: Intel mtl

Screenshots or console output

Running TESTSUITE sof_boot
===================================================================
START - virtual_memory_heap
[    0.505716] <wrn> sof_boot_test: vmh_test_single: Ignoring failure to allocate 1616 in non-contiguous mode
[    0.505723] <err> sof_boot_test: vmh_test_single: Failed to allocate 26
[    0.505845] <err> sof_boot_test: vmh_test: Non-contiguous test error -12
[    0.505850] <err> sof_boot_test: sof_boot_virtual_memory_heap: virtual_memory_heap failed: -12
 FAIL - virtual_memory_heap in 0.001 seconds
===================================================================

#8621 (comment)

[kv2019i]

@dabekjakub Can you check whether the calling sequence (i.e. the test) is correct? This seems to fail 100% and is causing "interesting" side-effects when failing (DSP panics as seen in #8621 ).

FYI @marc-hb @lyakh

[dabekjakub]

Did this test pass when @lyakh submited it ? I am currently trying to setup unit testing for vmh. I will look at this one after I wrap up my tests.

[marc-hb]

Did this test pass when @lyakh submited it ?

Yes on CAVS but not on MTL; never passed on the latter. See details in #8621

[lyakh]

Did this test pass when @lyakh submited it ?

Yes on CAVS but not on MTL; never passed on the latter. See details in #8621

@dabekjakub @marc-hb I'd say it depends on what "this test" is referring to. If it's referring to the complete boot-test, then yes, Marc's answer is correct. If however it's referring to the VMH test, then no, it never passed. The boot-test passed on cAVS because VMH is disabled there (IIRC, or at least the vmh test was never enabled there)

[lgirdwood]

Ok, this is actually quite good (for ztest and we've caught an issue early) - it means the test results are being correctly captured. @kv2019i was this manual testing or automatic via CI ?

[marc-hb]

Catching errors in firmware logs is still blocked because there are a few errors left in every run:

• Catch firmware errors sof-test#1075

[marc-hb]

Screenshots or console output

@kv2019i mtrace-reader.py is the only way to see this error log, correct?

https://github.com/thesofproject/sof/tree/main/tools/mtrace