[CPU backend] "Threads" defaults to core count minus one despite the tooltip saying "Recommended value is your CPU core count"

[NintendoManiac64]

Describe the Issue
In the initial dialog window, if you go to the Hardware tab and have the "Backend" drop-down set to CPU, the Threads text box will list a value that is your CPU's core count minus one (e.g. 3 threads on quad-core CPU, or 7 threads on an octa-core CPU)

But the tooltip if you hover your mouse cursor over the Threads text label says "Recommended value is your CPU core count".

Additional Information:
The only situation I can think of where you'd want to default to your CPU's core count minus one is if its a traditional P-core only CPU with no SMT (e.g. i5-9600K and i7-9700K) as the presence of E-cores and/or SMT allows the GUI and OS to remain responsive (at least it does on my own 4core/8thread Xeon E3-1285L v4 and 6core/12thread Ryzen 5600).

This was tested on both a locally installed copy of Linux Mint 21.3 Cinnamon as well as the live ISO of openSUSE Tumbleweed XFCE snapshot build 2026-01-21

[LostRuins]

Core count minus one is more of a pragmatic option. Since your other stuff on your PC uses CPU compute too, e.g. your browser, your video player, if you hog all the cores it can bottleneck there and cause lags and stutters.

[NintendoManiac64]

Core count minus one is more of a pragmatic option. Since your other stuff on your PC uses CPU compute too, e.g. your browser, your video player, if you hog all the cores it can bottleneck there and cause lags and stutters.

I already addressed this though in that it only makes sense if the CPU lacks SMT (aka hyperthreading in Intel terminology) or E-cores does it not? (unless Kobold actually uses both P & E cores in which case all cores minus one still makes sense, but again only makes sense if SMT/hyperthreading isn't present)

Isn't kobold able to detect at least the presence of SMT/hyperthreading in the way that lots of other software can? (I'm reminded of how prime95 beginning with version 30.7 now has a "hyperthreading" checkbox)

[LostRuins]

the presence of E-cores and/or SMT allows the GUI and OS to remain responsive

the issue is that the OS scheduler cannot be trusted to ALWAYS give kobold the P-cores. So what can happen is say you have 6 P-Cores and 6 E-cores, and another process (say background VLC player) is using 1 P-core, then kobo requests for 6 threads, 5 will run on P-Cores and 1 runs on an E-core and that ends up overall slower than just using 5 threads.

This behavior used to be the case though its been years and the scheduler could have changed in the meantime. Do you observe a speedup if you increase the thread count? @henk717 thoughts?

[NintendoManiac64]

(Un?)fortunately all my DDR4-and-newer CPUs are AMD so I'm more in the SMT camp than the E-core camp and can't directly test this with my own hardware, and in such a situation I didn't seem to have any issues UI responsiveness isues with Kobold using all of my cores on SMT-enabled systems.

Nevertheless, since it seems like AMD and Intel are diverging on this front (I don't think AMD even makes any CPUs without SMT anymore), a "one-size-fits-all" approach may not be possible and perhaps the default thread detection really should give a different value between the (modern AMD) CPU with SMT on all cores and the (modern Intel) CPU with zero SMT but different types of cores.

EDIT: As a sanity check on my 4c/8t Broadwell everything feels fine as I'm currently playing music in VLC and typing this sentence right now while I have kobold printing "as many digits of pi as you can" across all 4 cores and purely relying on SMT threads for typing this (using a different browser though, but I was able to also browse a semi-simple webpage like linuxmint.com in the same browser as the one running kobold without seemingly any issue?). And to be clear, I have an SSD-only system (and the SSDs have DRAM caches too), so I'm not just a case of "not being able to tell the difference" like if I were used to using mechanical hard drives (in fact I was an SSD early adapter ~15 years ago due to their silence)

If you want me to double-check with lower-end CPUs from the DDR3 era that have fewer cores and therefore also fewer SMT threads, I can do that too...or just simulate it the lazy way by underclocking and down-core'ing the Broadwell I'm currently typing from.

[henk717]

The problem is that psutil doesn't give us the info we needed to program this better. So the formula if I remember correctly is threads /2 - 1. The -1 was done to ensure the system also has a core for itself. Improvements here are unlikely unless there is a python library that lets us just grab the number of P cores.

Its a best effort default after all, if people have a system that can't handle this then they need to manually specify it.

So while better methods have been discussed in the paat we lack the libraries to make them.

The only thing I can still think off and suggested in the past is doing two checks if it lets you. One for physical cores, one for logical cores devided by 2. If there is a difference between the two end results then that difference indicates the e-cores are present.

Unfortunately I no longer have the system that clearly got worse when using more cores than there were physical cores so I can't test if this still applies.

[NintendoManiac64]

Unfortunately I no longer have the system that clearly got worse when using more cores than there were physical cores

Do you remember what the CPU in that system was? I have... a lot of rather old PC hardware.

[henk717]

Yes, it was a first gen ryzen 1700X. I do still have the system itself but the CPU has been replaced which solved that issue.

[NintendoManiac64]

Ryzen 1700X?! Pfft, I was expecting something more like Sandy Bridge - I have an i5-2520M laptop (2core/4thread) that's still regularly used (and no, it does not have discrete graphics despite its 1600x900 screen, though it is running Linux which definitely has better GPU driver support for Intel stuff of that era...not that kobold can use that iGPU anyway).

[henk717]

Its possibly because first gen ryzens were nutoriously bound to memory speed. Its just not a memory optimized cpu. Chances are equally old intels perform better. Especially when we still had openblas in KoboldCpp it was a huge impact. Between that cpu and my new one the new one was 40 times faster.
But these days we don't even have openblas anymore and a much more efficient cpu method made by llamafile which is named after it. So chances are threat count increasing is less of an issue now.

[NintendoManiac64]

I would like to point out that my 4core/8thread Broadwell Xeon is from 2015 and uses DDR3-1600 in my case - this is why I said "I was expecting something more like Sandy Bridge" - because a 1700X should run circles around what I'm using, and I've had zero issues!

But other than having Intel's fastest iGPU on desktop for quite a few years (14nm++++++++++ didn't help...) and decent power efficiency gains, it was barely an upgrade over Haswell from 2013 if not even a couple percentage regression in at least one application I know of (Dolphin emulator...though Haswell is so much faster than Ivy Bridge in emulation that it wasn't until Zen 2 that AMD surpassed Haswell in that type of workload)

By the way, since you said the CPU has been replaced rather than the PC has been replaced, does that mean you're now on presumably a Zen2 or Zen3 CPU?...couldn't you just go into the BIOS and also try and see for yourself by simply underclocking and/or down-core'ing your CPU to a lower clockrate and/or lesser core count and test this out yourself? Heck you can even slow down your RAM to DDR3 speeds if you want to really make sure.