Expose env var to disallow cuda

[IanButterworth]

While it's possible to set Flux.use_cuda[] = false, on some machines with underpowered but cuda-functional gpus, it would be nice to have a more system-wide permanent way to opt out of cuda.

So this adds env var FLUX_DISALLOW_CUDA which can be set to "true" etc. to disallow cuda

[DhairyaLGandhi]

Thanks Ian! Should we instruct users to set CUDA_VISIBLE_DEVICES instead? That seems more future proof.

[IanButterworth]

Yeah, that sounds good. I didn't know about that (for anyone else that didn't https://developer.nvidia.com/blog/cuda-pro-tip-control-gpu-visibility-cuda_visible_devices/)

As long as it appears in a google for "Flux disable CUDA" then I'd have caught it.

[mcabbott]

Can this disable the loading completely? @time using CUDA is about 4 seconds for me, the second time, and much more if things have been updated since the last time.

[DhairyaLGandhi]

Since CUDA the symbol is already in the flux namespace that might lead to dirtier code in the repo.

[mcabbott]

What does "might lead to dirtier code" even mean?

Loading CUDA unconditionally is literally half the loading time of Flux. If it can be avoided, that's a huge win. If it has to be in breaking v0.13, that's fine.

But I haven't tried, there may be some actual problem with this idea.

If possible, it would be nicest if the solution could be shared with other packages. Which means CUDA_VISIBLE_DEVICES sounds like it's probably the right thing. But does it have a standard way to encode "none"?

[darsnack]

It's possible that we can set use_cuda[] to false early only in the case that CUDA_VISIBLE_DEVICES is empty (and drop out before calling CUDA.functional() otherwise). The downside (which is what I think Dhairya meant) is that any reference to CUDA.jl stuff will need to be guarded by a use_cuda[] check.

[mcabbott]

Yes, on searching, there is more random CuArray code scattered about than I remembered. Although most of it should really move to NNlibCUDA. Maybe that's a bigger project than Ian has volunteered for... no downside to small steps first.

[CarloLucibello]

Instead of adding a new env variable, we can just document

export CUDA_VISIBLE_DEVICES=""

or more generally the use of CUDA_VISIBLE_DEVICES

[DhairyaLGandhi]

@IanButterworth do we need this still?

[IanButterworth]

I don't think so. Thanks!