tests/kernel/lifo/lifo_usage fails on m2gl025_miv

[andyross]

tests/kernel/lifo/lifo_usage fails on m2gl025_miv

This seems to be a reproducible failure in CI right now. This board uses the renode simulator, not qemu, and it is not a default platform for sanitycheck. That simulator isn't part of the SDK (and frankly I don't know anything about it), but it's installed in CI so our integration tests try to run it.

Basically, changes to test code cause all tests to be built and run, and this one doesn't. So right now we can't get a passing CI run on changes to tests/kernel/lifo/lifo_usage

Log seen in #13704:

ninja: Entering directory `/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/sanity-out/m2gl025_miv/tests/kernel/lifo/lifo_usage/kernel.lifo.usage'
[1/1] cd /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/sanity-out/m2gl025_miv/tests/kernel/lifo/lifo_usage/kernel.lifo.usage && /usr/bin/renode --disable-xwt --pid-file renode.pid -e '$bin=@/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/sanity-out/m2gl025_miv/tests/kernel/lifo/lifo_usage/kernel.lifo.usage/zephyr/zephyr.elf; include @/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/boards/riscv32/m2gl025_miv/support/m2gl025_miv.resc; s'
00:28:43.1960 [INFO] Including script: /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/boards/riscv32/m2gl025_miv/support/m2gl025_miv.resc
00:28:43.2316 [INFO] System bus created.
00:28:44.6194 [DEBUG] Segment size automatically calculated to value 64KiB
00:28:44.6218 [DEBUG] Segment size automatically calculated to value 64KiB
00:28:44.6231 [DEBUG] Segment size automatically calculated to value 64KiB
00:28:44.6244 [DEBUG] Segment size automatically calculated to value 4MiB
00:28:45.0323 [WARNING] Writing to an output GPIO pin #0
00:28:45.0331 [WARNING] Writing to an output GPIO pin #1
00:28:45.0387 [WARNING] Writing to an output GPIO pin #2
00:28:45.7738 [DEBUG] sysbus: Loading ELF /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/sanity-out/m2gl025_miv/tests/kernel/lifo/lifo_usage/kernel.lifo.usage/zephyr/zephyr.elf.
00:28:45.7950 [INFO] sysbus: Loading segment of 29760 bytes length at 0x80000000.
00:28:45.8423 [DEBUG] sysbus: Segment loaded.
00:28:45.8427 [INFO] sysbus: Loading segment of 348 bytes length at 0x80007440.
00:28:45.8427 [DEBUG] sysbus: Segment loaded.
00:28:45.8430 [INFO] sysbus: Loading segment of 17408 bytes length at 0x80040160.
00:28:45.8430 [DEBUG] sysbus: Segment loaded.
00:28:45.9945 [INFO] cpu: Setting PC value to 0x80000000.
00:28:46.1199 [INFO] Mi-V: Machine started.
00:28:46.2226 [WARNING] plic: Unhandled write to offset 0x200000, value 0x0.
00:28:46.2569 [WARNING] uart: Unhandled write to offset 0xC. Unhandled bits: [0] when writing value 0x1. Tags: BIT8 (0x1).
00:28:46.2916 [DEBUG] uart: [+1.11s host +0s virt 0s virt from start] ***** Booting Zephyr OS v1.14.0-rc1-624-gb6c6aaa65b73 *****
00:28:46.3162 [DEBUG] uart: [+24.9ms host +0.1ms virt 0.1ms virt from start]   Running test suite test_lifo_usage
00:28:46.3173 [DEBUG] uart: [+5.54ms host +0.1ms virt 0.2ms virt from start]   ===================================================================
00:28:46.3178 [DEBUG] uart: [+0.53ms host +0s virt 0.2ms virt from start]   starting test - test_lifo_nowait
00:28:46.3252 [DEBUG] uart: [+7.34ms host +0.1ms virt 0.3ms virt from start]   PASS - test_lifo_nowait
00:28:46.3260 [DEBUG] uart: [+0.84ms host +0s virt 0.3ms virt from start]   ===================================================================
00:28:46.3265 [DEBUG] uart: [+0.44ms host +0s virt 0.3ms virt from start]   starting test - test_lifo_wait
00:28:46.3285 [DEBUG] uart: [+2.05ms host +0.1ms virt 0.4ms virt from start]   PASS - test_lifo_wait
00:28:46.3299 [DEBUG] uart: [+1.3ms host +0.1ms virt 0.5ms virt from start]   ===================================================================
00:28:46.3302 [DEBUG] uart: [+0.46ms host +0s virt 0.5ms virt from start]   starting test - test_timeout_empty_lifo
00:28:46.3430 [DEBUG] uart: [+12.53ms host +0.2ms virt 0.7ms virt from start]   PASS - test_timeout_empty_lifo
00:28:46.3436 [DEBUG] uart: [+0.76ms host +0s virt 0.7ms virt from start]   ===================================================================
00:28:46.3444 [DEBUG] uart: [+0.82ms host +0.1ms virt 0.8ms virt from start]   starting test - test_timeout_non_empty_lifo
00:28:46.3564 [DEBUG] uart: [+1.35ms host +0s virt 0.8ms virt from start]   PASS - test_timeout_non_empty_lifo
00:28:46.3565 [DEBUG] uart: [+1.41ms host +0.1ms virt 0.9ms virt from start]   ===================================================================
00:28:46.3565 [DEBUG] uart: [+0.6ms host +0s virt 0.9ms virt from start]   starting test - test_timeout_lifo_thread
00:28:46.3565 [DEBUG] uart: [+6.18ms host +0.3ms virt 1.2ms virt from start]   PASS - test_timeout_lifo_thread
00:28:46.3565 [DEBUG] uart: [+0.96ms host +0s virt 1.2ms virt from start]   ===================================================================
00:28:46.3692 [DEBUG] uart: [+14.27ms host +0.1ms virt 1.3ms virt from start]   starting test - test_timeout_threads_pend_on_lifo
00:28:46.3773 [DEBUG] uart: [+8.1ms host +0.1ms virt 1.4ms virt from start]    thread (q order: 2, t/o: 0, lifo 0x800401d4)
00:28:46.3838 [DEBUG] uart: [+6.56ms host +0.2ms virt 1.6ms virt from start]   
00:28:46.3919 [DEBUG] uart: [+8.11ms host +0.1ms virt 1.7ms virt from start]       Assertion failed at /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/tests/kernel/lifo/lifo_usage/src/main.c:141: test_multiple_threads_pending: (data->timeout_order not equal to ii)
00:28:46.3924 [DEBUG] uart: [+0.5ms host +0s virt 1.7ms virt from start]    *** thread 2 woke up, expected 1
00:28:46.3925 [DEBUG] uart: [+76s host +0s virt 1.7ms virt from start]   
00:28:46.3941 [DEBUG] uart: [+1.59ms host +0.1ms virt 1.8ms virt from start]   FAIL - test_timeout_threads_pend_on_lifo
00:28:46.3952 [DEBUG] uart: [+1.09ms host +0.1ms virt 1.9ms virt from start]   ===================================================================
00:28:46.3958 [DEBUG] uart: [+0.48ms host +0s virt 1.9ms virt from start]   Test suite test_lifo_usage failed.
00:28:46.3999 [DEBUG] uart: [+0.96ms host +0s virt 1.9ms virt from start]   ===================================================================
00:28:46.4000 [DEBUG] uart: [+0.43ms host +0s virt 1.9ms virt from start]   PROJECT EXECUTION FAILED

[PiotrZierhoffer]

The main problem is that this test also fails on real hardware, it is not Renode-specific.

As discussed with @nashif , we are about to create some docs describing Renode and how to use it, but due to Embedded World this had to be postponed.

[andyross]

OK, but it's failing right now reliably in the tree, which means that CI runs that try to build it (in this case, anything that modifies the tests/kernel/lifo/lifo_usage directory) can't pass.

Please file a bug for the hardware failure for sure, it's worth tracking. But we need to correct the booby trap in the source tree. Would you be OK blacklisting this test on that platform for now?

[PiotrZierhoffer]

(got offline for the weekend, sorry for the delay).

Sure, that makes sense. I'll file a bug, but I am not sure how to blacklist the test in the 'Zephyr' way. Only one test case fails from the whole suite, but I think the only "correct" way to handle it is via "platform_exclude". Am I right?