[Feature Request] TorchRL for Python 3.13t

[patchmeifyoucan]

Hello, PyTorch recently added support for Python 3.13t, aka "no-gil" Python (pytorch/pytorch#130249).
Are there any plans to make TorchRL usable from this version? It would be nice if this could be supported soon, as the ability to run CPU code for interactions with the environment from multiple no-gil threads could lead to major performance improvements and higher GPU utilization.

[vmoens]

That's definitely on our radar!
Beyond multithreaded envs another big benefit would be reducing the cpu overheads across the lib (sampling in rbs, iterating over TensorDict etc).

Is there anything we should prioritise in your opinion?

[patchmeifyoucan]

Thank you, good to know.

For me, simply general support would be sufficient to get a better GPU utilization, that's what I'm after. I mostly use torchrl's replay buffers but their performance is far from being a bottleneck in my application.

[patchmeifyoucan]

Is it somehow possible to install torchrl with a 3.13t python right now from nightly builds? TorchRL remains my last dependency which I was not able to install onto a 3.13t python.
Or is there work in progress that needs to be finished before that release is possible? If yes, is there any expected time schedule for a new version?

[ljfp]

@vmoens

Beyond multithreaded envs another big benefit would be reducing the cpu overheads across the lib (sampling in rbs, iterating over TensorDict etc).
Is there anything we should prioritise in your opinion?

I'm a different user, but at least in my case this would be a huge win for me -> iterating over TensorDict

Thanks for your work! I'll subscribe to this Feature Request. TorchRL (wth TensorDicts) for Python 3.13 (no GIL) would be amazing to have.

[patchmeifyoucan]

Any updates here?

[vmoens]

I didn't manage to get much speed-ups for TD with python 3.13t -- but i'm not losing hope!
We just need to port it for expensive stuff like consolidate etc.
At least the packages are now shipped for this python version, it's a start!