[Bug]: Incompatibility with numpy 2.0 on windows

[nhuet]

🐛 Bug

When trying to run the first example of the doc in an environment with numpy 2.0 on windows, an error occurs.

To Reproduce

On a github runner windows-latest

• install gymnasium + sb3 + numpy 2.0:

python -m pip install gymnasium stable-baselines3 "numpy>=2.0" 

• run the basic example (just the start is sufficient):

import gymnasium as gym

from stable_baselines3 import PPO

env = gym.make("CartPole-v1", render_mode="human")

model = PPO("MlpPolicy", env, verbose=1)
model.learn(total_timesteps=10_000)

Relevant log output / Error message

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last):  File "D:\a\workflow_testing\workflow_testing\ex.py", line 7, in <module>
    model = PPO("MlpPolicy", env, verbose=1)
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\stable_baselines3\ppo\ppo.py", line 109, in __init__
    super().__init__(
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\stable_baselines3\common\on_policy_algorithm.py", line 85, in __init__
    super().__init__(
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\stable_baselines3\common\base_class.py", line 127, in __init__
    self.device = get_device(device)
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\stable_baselines3\common\utils.py", line 153, in get_device
    device = th.device(device)
C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\stable_baselines3\common\utils.py:153: UserWarning: Failed to initialize NumPy: _ARRAY_API not found (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\utils\tensor_numpy.cpp:84.)
  device = th.device(device)
Traceback (most recent call last):
  File "D:\a\workflow_testing\workflow_testing\ex.py", line 8, in <module>
Using cpu device
Wrapping the env with a `Monitor` wrapper
    model.learn(total_timesteps=100)
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\stable_baselines3\ppo\ppo.py", line 315, in learn
Wrapping the env in a DummyVecEnv.
    return super().learn(
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\stable_baselines3\common\on_policy_algorithm.py", line 300, in learn
    continue_training = self.collect_rollouts(self.env, callback, self.rollout_buffer, n_rollout_steps=self.n_steps)
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\stable_baselines3\common\on_policy_algorithm.py", line 178, in collect_rollouts
    obs_tensor = obs_as_tensor(self._last_obs, self.device)
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\stable_baselines3\common\utils.py", line 485, in obs_as_tensor
    return th.as_tensor(obs, device=device)
RuntimeError: Could not infer dtype of numpy.float32

System Info

No response

Checklist

• My issue does not relate to a custom gym environment. (Use the custom gym env template instead)
• I have checked that there is no similar issue in the repo
• I have read the documentation
• I have provided a minimal and working example to reproduce the bug
• I've used the markdown code blocks for both code and stack traces.

[araffin]

24ebf1a#diff-60f61ab7a8d1910d86d9fda2261620314edcae5894d5aaa236b821c7256badd7

for short: sb3 is not compatible with numpy 2 because of pytorch.

[nhuet]

Thank your for the swift answer

[nhuet]

Hi, do you have still issues with numpy 2 ? Pytorch claims to be be compatible with it since torch 2.3 (see pytorch/pytorch#107302)
I tried torch 2.4.1 + python 3.12 + numpy 2.1.3, and it seemed to work fine. (I did not thoroughly test though)

[nhuet]

nice! thks

[xmatthias]

@araffin any chance to get a release cut that includes this?
The current release (2.4.0) is still pinning numpy<2.0.

[araffin]

there is already a pre-release on pypi normally: https://pypi.org/project/stable-baselines3/2.5.0a0/
I will update it soon with the recent fix for the video recorder.

Otherwise, you can always install from the master branch (see doc).

[debnath-d]

@araffin When will 2.5.0 be released?