"OpenBLAS : Program will terminate because you tried to start too many threads." even though OPENBLAS_NUM_THREADS=1 (version 0.3.2)

[untom]

Hi!

I am using OpenBLAS as the underlying library for numpy and sklearn. In my program, I start a Python process which starts several Python Threads (= still 1 process), and each of them starts an instance of sklearn.svc.LinearSVC, (which uses OpenBLAS). The environment variable OPENBLAS_NUM_THREADS is set to 1.

Using Openblas 0.2.20, I got an error on one of our machines with Program is Terminated. Because you tried to allocate too many memory regions. . In #539 it was recommended to rebuild the library with a larger NUM_THREADS setting to avoid this, so I did. I upgraded to 0.3.2 when doing this. My machine has 20 threads, so I used NUM_THREADS=128 to be on the safe side. However, now I'm getting crashes with "OpenBLAS : Program will terminate because you tried to start too many threads.", when I try to start my python program using 10 threads. Please advise what I could do to fix this?

[TadasBaltrusaitis]

What OS are you using? Is this on Mac OSX?

One option is to export an environment variable (https://github.com/TadasBaltrusaitis/OpenFace/wiki/Mac-installation#openblas-performance) before calling OpenFace.

I'm trying to find a way to set the number of threads used by OpenBLAS to 1 on all operating systems (this seems to work on Windows and Ubuntu, but still has issues on Macs)

Thanks,
Tadas

[untom]

Hi! No, this was on CentOS 7. I've since recompiled OpenBLAS without multi-threaded support, and now everything works. It still kind of a pitty, since On occasion I'd like to use the MT ability of OpenBLAS, so I'd still be interestd in a solution.

[TadasBaltrusaitis]

Have you tried OpenBLAS 0.3.3, they seem to have fixed some threading issues, but not sure if they were exactly related to the ones you encountered.