Thermal issues when compiling big things

[jglathe]

Not the first time I see this, but it's a stability issue: wdk thermal profile appears to be not attuned to the wdk. Of course, it's taken from the x13s and not amended yet. Usually you won't notice, but with compiling bigger packages like the linux kernel or llvm (for gunyah docker image), you get crashes with read-only filesystem and other ones. Usually no hard crashes, but that's not far from it.

To avoid these issues my recommendation for now is to reduce the number of cores/threads used for build work to 4. This has the additional effect of using only the high performance cores of the SoC. Of ourse it increaeses build times, but the box stays stable that way.

[xlazom00]

@jglathe
I don't have this problem with 6.6.0 kernel. In my case whole soc just slow down cpu freq and I hear noise from internal fan.

But thermal sensors don't work. Any progress with this?

[xlazom00]

I found that best cooling option is to have it upside down As heated air can leave case with ventilation holes from bottom side

[jglathe]

Appears to be intermittent, no real handle on it yet. It seems to be a timing issue connected to the userspace services, and the coprocessors. Sometimes it wil cool diligently as soon as the load goes up starting silent, gradually increasing over time, sometimes it won't and does only the emergency blasts, which may not be enough. Documentation is still not really there.

[xlazom00]

@jglathe you wrote that temp sensors works fine in 6.5 kernel version
maybe it is related to this change steev@41c1855

And maybe we can also enable USB MP
steev@e9bb9fe

[jglathe]

Both these commits are in the 6.6 tree, too. Without the latter no USB-A ports on the wdk.
Digging a little, I found that my 6.5.6 build is the last one doing fine with thermals. It's also the last one where I see CPU core temps. Similar behaviour on the x13s, except that I also see memory temp there.
Digging a little into the source code, there appears to be an oddity: The thermal sensors of the SnapDragon SoC are referenced in drivers/thermal/qcom/tsens.c. The Kconfig definition for it is CONFIG_QCOM_TSENS . If you go and use make menuconfig, QCOM_TSENS will not be shown as an option, but you can search for it (and this path is orphaned). And after saving your changed configuration (for example, enabling FW zstd compression) this option will be gone from your config.
I just made the check, and yes, if CONFIG_QCOM_TSENS is actually present in your .config (and you didn't hack it there manually) the darn thing builds it and I have CPU core temps back 😀. Next will be to destillate a useful defconfig file and revised build instructions (and maybe a bug report). Oh well.

jglathe@snapdragix:~$ sensors
pm8280_2_thermal-virtual-0
Adapter: Virtual device
temp1:        +37.0°C  

cpu7_thermal-virtual-0
Adapter: Virtual device
temp1:        +47.6°C  

cpu2_thermal-virtual-0
Adapter: Virtual device
temp1:        +48.0°C  

cpu5_thermal-virtual-0
Adapter: Virtual device
temp1:        +48.3°C  

cpu0_thermal-virtual-0
Adapter: Virtual device
temp1:        +47.6°C  

cpu3_thermal-virtual-0
Adapter: Virtual device
temp1:        +47.3°C  

ath11k_hwmon-pci-60100
Adapter: PCI adapter
temp1:            N/A  

qcom_battmgr_bat-virtual-0
Adapter: Virtual device
in0:           8.80 V  
temp:         +30.1°C  

skin_temp_thermal-virtual-0
Adapter: Virtual device
temp1:        -41.0°C  

nvme-pci-20100
Adapter: PCI adapter
Composite:    +46.9°C  (low  = -40.1°C, high = +83.8°C)
                       (crit = +87.8°C)
Sensor 1:     +55.9°C  (low  = -273.1°C, high = +65261.8°C)
Sensor 2:     +46.9°C  (low  = -273.1°C, high = +65261.8°C)

cluster0_thermal-virtual-0
Adapter: Virtual device
temp1:        +47.6°C  

pm8280_1_thermal-virtual-0
Adapter: Virtual device
temp1:        +37.0°C  

cpu6_thermal-virtual-0
Adapter: Virtual device
temp1:        +47.3°C  

cpu1_thermal-virtual-0
Adapter: Virtual device
temp1:        +47.0°C  

cpu4_thermal-virtual-0
Adapter: Virtual device
temp1:        +47.6°C  

mem_thermal-virtual-0
Adapter: Virtual device
temp1:        +46.6°C  

qcom_battmgr_bat-virtual-0
Adapter: Virtual device
ERROR: Can't get value of subfeature in0_input: Kernel interface error
in0:              N/A  
temp1:        +30.1°C  

[xlazom00]

I know that both commits are in tree but channel renaming isn't in sc8280xp-microsoft-dev-kit-2023.dts

this one
steev@41c1855

I did check 2.

[xlazom00]

So it looks like some bug in Makefile dep for CONFIG_QCOM_TSENS

[jglathe]

And it is reallly stubborn in getting not set on 6.6

jglathe@snapdragix:~/src/linux_ms_dev_kit$ make -j8 devkit_defconfig
#
# configuration written to .config
#
jglathe@snapdragix:~/src/linux_ms_dev_kit$ grep TSENS .config
jglathe@snapdragix:~/src/linux_ms_dev_kit$ make -j8 defconfig
*** Default configuration is based on 'defconfig'
#
# configuration written to .config
#
jglathe@snapdragix:~/src/linux_ms_dev_kit$ grep TSENS .config
CONFIG_QCOM_TSENS=y

[jglathe]

I know that both commits are in tree but channel renaming isn't in sc8280xp-microsoft-dev-kit-2023.dts

this one steev@41c1855

I did check 2.

Okay, I'll amend this.

[xlazom00]

@jglathe
I found that
CONFIG_QCOM_TSENS depends on NVMEM_QCOM_QFPROM
https://github.com/jglathe/linux_ms_dev_kit/blob/jg/wdk2023-6.6.1/drivers/thermal/qcom/Kconfig#L4
and NVMEM_QCOM_QFPROM is missing in your config

[jglathe]

Yeah I've seen that, and adding it gives just "unexpected data" but no change. Removing the dependency on NVMEM_QCOM_QFPROM from KConfig actually helps. And, this QFPROM option appears to be independent / have vanished from the menuconfig, too.

[jglathe]

@xlazom00 thank you for the debug. Looks like its working now.

jglathe@snapdragix:~$ uname -a
Linux snapdragix 6.6.2+ #7 SMP PREEMPT Fri Nov 24 14:11:31 CET 2023 aarch64 aarch64 aarch64 GNU/Linux
jglathe@snapdragix:~$ sensors
pm8280_2_thermal-virtual-0
Adapter: Virtual device
temp1:        +37.0°C  

mem_thermal-virtual-0
Adapter: Virtual device
temp1:        +42.6°C  

cpu3_thermal-virtual-0
Adapter: Virtual device
temp1:        +43.3°C  

cpu7_thermal-virtual-0
Adapter: Virtual device
temp1:        +42.6°C  

cpu1_thermal-virtual-0
Adapter: Virtual device
temp1:        +42.9°C  

cpu5_thermal-virtual-0
Adapter: Virtual device
temp1:        +43.3°C  

skin_temp_thermal-virtual-0
Adapter: Virtual device
temp1:        -41.0°C  

nvme-pci-20100
Adapter: PCI adapter
Composite:    +39.9°C  (low  = -40.1°C, high = +83.8°C)
                       (crit = +87.8°C)
Sensor 1:     +52.9°C  (low  = -273.1°C, high = +65261.8°C)
Sensor 2:     +39.9°C  (low  = -273.1°C, high = +65261.8°C)

pm8280_1_thermal-virtual-0
Adapter: Virtual device
temp1:        +37.0°C  

cpu4_thermal-virtual-0
Adapter: Virtual device
temp1:        +43.6°C  

cluster0_thermal-virtual-0
Adapter: Virtual device
temp1:        +43.9°C  

cpu2_thermal-virtual-0
Adapter: Virtual device
temp1:        +42.3°C  

cpu6_thermal-virtual-0
Adapter: Virtual device
temp1:        +42.9°C  

cpu0_thermal-virtual-0
Adapter: Virtual device
temp1:        +43.6°C  

ath11k_hwmon-pci-60100
Adapter: PCI adapter
temp1:            N/A  

qcom_battmgr_bat-virtual-0
Adapter: Virtual device
in0:           8.80 V  
temp:         +30.1°C  

Will test a bit, then probably close it. We could try to get ath11k_hwmon to do something useful, too, but I guess soundwire over dp would be of greater benefit. Or the VM support with gunyah (if possible at all).