macOS conda installation has missing random generators

[nicolossus]

NEST 3.6 installed via conda only has the Mersenne Twister random generator:

>> nest.rng_types
('mt19937', 'mt19937_64')

I would expect all random generators to be available, that is:

>> nest.rng_types
('Philox_32', 'Philox_64', 'Threefry_32', 'Threefry_64', 'mt19937', 'mt19937_64')

[steffengraber]

Hi @nicolossus Thank you for your notification. On which system are you working - Mac or Linux? Which version of python are you using?

[nicolossus]

@steffengraber I am using a conda environment with Python 3.11 on macOS. A minimal reproducer:

conda create --name nest_conda -c conda-forge python=3.11 nest-simulator=3.6

My guess would be that the conda build is compiled without the Random123 generators for some reason.

[steffengraber]

Ok, I tested this under ubuntu 23.04. With this I get

>>> nest.rng_types
('Philox_32', 'Philox_64', 'Threefry_32', 'Threefry_64', 'mt19937', 'mt19937_64')

It seems that there is only a problem in the MacOS packages.

[steffengraber]

The test for nrg.types is now integrated in the CI. These tests run for linux_64 and osx_64, but not for osx-arm64. This allowed me to narrow down the problem to macOS with M1 chip. I do not have the possibilities to test this. Could you possibly verify this? @nicolossus

[nicolossus]

@steffengraber I should probably have mentioned that I encountered the conda installation problem on osx-arm64 (M2 chip), so I can confirm.

However, if I build and install NEST from source on the same osx-arm64 machine, all the RNG types are included. It therefore doesn't seem like there should be a problem with Apple silicon and the Random123 generators library (which contains the Philox and Threefry generators).

[steffengraber]

if I build and install NEST from source on the same osx-arm64 machine, all the RNG types are included

Could you send me a build log of it? This could help to recognize the differences in the build process.

[nicolossus]

@steffengraber Of course, I have sent you the build log directly.

[steffengraber]

The problem might have something to do with cross-compiling the osx_arm64 version.
The report here on random123 :

...
-- Info: Check if Random123 generators work. 
-- Info: Bad CPU type in executable 
-- Info: Check if Random123 generators work. OFF
...

[steffengraber]

@nicolossus As a test, I have now deactivated the random123 check and used this to build the package for macOS ARM.

You can download one here:
https://artprodeus21.artifacts.visualstudio.com/A910fa339-c7c2-46e8-a579-7ea247548706/84710dde-1620-425b-80d0-4cf5baca359d/_apis/artifact/cGlwZWxpbmVhcnRpZmFjdDovL2NvbmRhLWZvcmdlL3Byb2plY3RJZC84NDcxMGRkZS0xNjIwLTQyNWItODBkMC00Y2Y1YmFjYTM1OWQvYnVpbGRJZC84MzE4MjEvYXJ0aWZhY3ROYW1lL2NvbmRhX2FydGlmYWN0c18yMDIzMTEyOC4xLjFfb3N4X2FybTY0X251bXB5MS4yM3B5dGhvbjMuMTEuX19fX2NweXRob241/content?format=zip

Can you test this? Unfortunately, I have no possibility to do so and the automatic tests are not executable due to cross compiling.

To test, unzip the package and change into the directory. Then:

conda create -n my-test-env -c "file://${PWD}" -c conda-forge 
conda activate my-test-env
conda install -c "file://${PWD}" nest-simulator

Instead of conda you can also use mamba, which offers some advantages when creating environments.