tests/kernel/mem_protect/protection/kernel.memory_protection.protection fails on qemu_cortex_r5

[fabiobaltieri]

Describe the bug
Hi, seems like tests/kernel/mem_protect/protection/kernel.memory_protection.protection fails started timing out on qemu_cortex_r5 since 33a2537.

@tnmysh: any chance you could look into it?

To Reproduce
./scripts/twister -v -n -M -p qemu_cortex_r5 -T tests/kernel/mem_protect/protection on main

Also https://github.com/zephyrproject-rtos/zephyr/runs/15955541825, fails with Timeout.

Bisected down to 33a2537, reverting om main fixes the issue.

Expected behavior
Test should pass.

Impact
Broken in CI, pretty bad.

Environment (please complete the following information):
Linux, main (77698e8), Zephyr SDK 0.16.1

[fabiobaltieri]

cc @tnmysh @carlocaione @uLipe

[tnmysh]

@fabiobaltieri Sure I will look into it.

[tnmysh]

I built that test and run on qemu_cortex_r5 platform with commit https://github.com/zephyrproject-rtos/zephyr/commit/33a25371b6dcc3cfeb7df31f7ae9ba014830545f and it is passing.

===================================================================                                             
TESTSUITE protection succeeded                                                                                  
                                                                                                                
------ TESTSUITE SUMMARY START ------                                                                           
                                                                                                                
SUITE PASS - 100.00% [protection]: pass = 5, fail = 0, skip = 0, total = 5 duration = 3.943 seconds             
 - PASS - [protection.test_exec_data] duration = 0.582 seconds                                                  
 - PASS - [protection.test_exec_heap] duration = 2.694 seconds                                                  
 - PASS - [protection.test_exec_stack] duration = 0.363 seconds                                                 
 - PASS - [protection.test_write_ro] duration = 0.164 seconds                                                   
 - PASS - [protection.test_write_text] duration = 0.140 seconds                                                 
                                                                                                                
------ TESTSUITE SUMMARY END ------                                                                             
                                                                                                                
===================================================================                                             

@fabiobaltieri I am not sure why it is failing with CI. Any workaround to that? Can we skip it for this platform?

[tnmysh]

I tried this in my local repo: west build -d /tmp/build -p -b qemu_cortex_r5 tests/kernel/mem_protect/protection/ -t run

and hit this issue:

[142/143] To exit from QEMU enter: 'CTRL+a, x'[QEMU] CPU: cortex-r5
/bin/sh: QEMU-NOTFOUND: command not found
FAILED: zephyr/CMakeFiles/run_qemu /tmp/build/zephyr/CMakeFiles/run_qemu 
cd /tmp/build && QEMU-NOTFOUND -nographic -machine arm-generic-fdt -dtb /scratch/openamp/src/zephyr_3_4/zephyrproject/zephyr/boards/arm/qemu_cortex_r5/fdt-single_arch-zcu102-arm.dtb -net none -pidfile qemu.pid -chardev stdio,id=con,mux=on -serial chardev:con -mon chardev=con,mode=readline -icount shift=3,align=off,sleep=off -rtc clock=vm -device loader,file=/tmp/build/zephyr/zephyr.elf,cpu-num=4 -device loader,addr=0xff5e023c,data=0x80008fde,data-len=4 -device loader,addr=0xff9a0000,data=0x80000218,data-len=4
ninja: build stopped: subcommand failed.

Just want to check If CI is running this with QEMU or not.

[fabiobaltieri]

Yeah CI is running qemu, but I realize now that @dcpleung landed on a different commit than I in #61571 and his one looks more plausible. @dcpleung should we close this and follow up on yours?

[fabiobaltieri]

So on my system 36997de works, 33a2537, fails, but reverting either on top of mains works. welp.

@ibirnbaum @microbuilder @povergoing

[dcpleung]

@dcpleung should we close this and follow up on yours?

We can use this one since this has more information.

[tnmysh]

So on my system 36997de works, 33a2537, fails, but reverting either on top of mains works. welp.

@fabiobaltieri What is the error message you get when it fails ? Also, How do you run test with QEMU? Does zephyr provide some qemu binary I should test with ? For now, I am running zephyr binary on QEMU provided with Petalinux SDK. I would like to run with QEMU provided with Zephyr SDK.

[fabiobaltieri]

@tnmysh it fails with a timeout, the test runs with ./scripts/twister -v -n -M -p qemu_cortex_r5 -T tests/kernel/mem_protect/protection (it runs on qemu) and it's using a qemu version that ships with the Zephyr SDK. That may explain why it's not failing on your side.

[fabiobaltieri]

You could also use west build -d /tmp/build -p -b qemu_cortex_r5 tests/kernel/mem_protect/protection/ -t run, that should finish with a PROJECT EXECUTION SUCCESSFUL, instead it hangs after

START - test_exec_heap
trying to call code written to 0x9858