[BUG][Multi-core] MTL RVP hangs when core1 & core2 are used simultaneously

[RanderWang]

Describe the bug
MTL RVP hangs when core1 & core2 are used simultaneously. We need to rescue the MTL RVP by cold boot.

To Reproduce
Do two streams playback on core1 & core2

Reproduction Rate
100%

Environment

1. Branch name and commit hash of the 2 repositories: sof (firmware/topology) and linux (kernel driver).
   • Kernel: main branch : 0035b02b10f1ef6417d76f469c04572a3af5b9a3
   • SOF: 005-drop-stable
2. Name of the topology file
   • Topology: development/sof-mtl-nocodec-multicore.tplg
3. Name of the platform(s) on which the bug is observed.
   • Platform: MTL RVP B0

We created internal bug 15013565662 and ask for help from SOC team

[abonislawski]

Same on main branch?

[abonislawski]

@RanderWang checked this today with our validation and we have good results when core1 & core2 are used simultaneously.
Looks like driver is not programming something correctly for this scenario?

[RanderWang]

@RanderWang checked this today with our validation and we have good results when core1 & core2 are used simultaneously. Looks like driver is not programming something correctly for this scenario?

Thanks! As you know, driver doesn't touch core1 & core2, which are maintained by FW. SOC team is checking the scan dump. I will share you the finding

[RanderWang]

@abonislawski We got the reply from SOC team and you are in the loop. Do you have any idea ? Thanks!

[RanderWang]

the scandump show DSP0 and DSP2 are issuing exclusive read to L2SRAM address 0x400ED940. however the transaction is not seen at fabric-to-L2SRAM interface.

question: are these valid program counter: DSP0 = 0x4006 600C, DSP2 = 0x400E CE82

[abonislawski]

@RanderWang it looks fine. To see exactly what's under it we need to check files from this fw build.
On my local build (releases/mtl/v5.0.1) 0x400ED940 = log_buffer so for quick check you can try if there is any difference on build without logs (CONFIG_LOG=n) but this may not be the case with a build that has actually been tested.

[RanderWang]

@abonislawski yes, the address is for log_buffer for this scan dump. I disable log but make no effect on 005 branch. I tested main branch last week, but to my surprise this bug doesn't happen on latest main branch !

[RanderWang]

@abonislawski Can you give education about "program counter: DSP0 = 0x4006 600C, DSP2 = 0x400E CE82". As I know dsp address is based on 0xa0002000, why it is 0x400xxxxx? Thanks!

[marcinszkudlinski]

0xAxxxxx is cached alias
0x4xxxxx is uncached alias
Same SRAM, just accessed through L1 or directly

[RanderWang]

@abonislawski no issue on latest main branch. The issue was fixed or workaround by e2d0d1

 kernel: allow for arch specific processing within busy loops

Give architectures that need it the ability to perform special checks
while e.g. waiting for a spinlock to become available.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>

This is proved that our SOC team reported that DSP0 and DSP2 are issuing exclusive read to L2SRAM address 0x400ED940 (log_buffer).

[mengdonglin]

@abonislawski no issue on latest main branch. The issue was fixed or workaround by e2d0d1

 kernel: allow for arch specific processing within busy loops

Give architectures that need it the ability to perform special checks
while e.g. waiting for a spinlock to become available.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>

This is proved that our SOC team reported that DSP0 and DSP2 are issuing exclusive read to L2SRAM address 0x400ED940 (log_buffer).

[wszypelt]

@nashif the problem is not on the FW side, please assign the right person to issue

[wszypelt]

Fixed with latest zephyr version, @RanderWang can you check?

[RanderWang]

fixed