Check for CUDA availability at run time.

[maleadt]

WIP: necessary changes over at CuArrays/CUDAdrv/CUDAnative have not been pushed/released yet.

Continuing the package loading / conditional dependency saga, here is another attempt to cover all requirements. With the recent CUDAapi.jl-based scheme, we made it possible to add regular Pkg dependencies on GPU packages, but that had one important flaw: once the application (e.g. Flux) was precompiled without GPU support, there was no easy way for the user to "fix" GPU support and reload Flux. So we added some hacks to detect that and remove the compile cache during __init__.

Furthermore, several users want the GPU packages and applications like Flux to be precompilable on a system without a GPU, e.g. the login node of a cluster, or during the build step of a container. This is similarly incompatible with conditionally loading GPU packages, which would bake the GPU-less state into the precompilation image.

There's also certain parts of our infrastructure, like Documenter (being tightly linked to Travis) and the new automatic package registry automerge bot, that expect packages to be loadable at all times.

Bottom line, the GPU stack should be loadable and precompilable regardless of wether it can and will be used. I'm working on exactly that right now, where CUDAdrv/CUDAnative/CuArrays would always be loadable, print a (silencable) warning if something goes, and have a $module.functional() method to query the state of the package. In this PR, I adapt Flux to use those APIs.

Advantages:

• Flux can be precompiled, including optional GPU support
• Changes to the system that affect GPU support are reflected by just reloading Flux

Disadvantages:

• functions like gpu are now type unstable (returning either an Array or CuArray)
• CUDNN is mandatory, as there's currently no easy way to conditionally enable that functionality based on a run time flag

Both disadvantages could be worked around by evaluating code at run time, but that would negate some of the precompilation advantages.

Thoughts? @MikeInnes

[MikeInnes]

Seems sensible to me, given the options we have right now. gpu not being inferable is a very minor downside. Requiring CUDNN is bigger, but if we need to eval some code later on that seems ok (there is very little code on Flux's side, so the eval doesn't take too long).

[maleadt]

Updated to include newly released GPU packages that contain the necessary functionality. That also means only supporting Julia 1.2+ though.

Requiring CUDNN is bigger, but if we need to eval some code later on that seems ok (there is very little code on Flux's side, so the eval doesn't take too long).

OK, done.

[MikeInnes]

Dropping support for 1.0 is not ideal; is there no way the loading changes can be backported to previous Cu* releases?

[DhairyaLGandhi]

Yeah, maintaining backwards compatibility would be nice.

[maleadt]

Yes, JuliaLang/julia#31403 as used in CUDAnative. I'll have a look at providing llvmcall alternatives for certain functionality.

[maleadt]

JuliaGPU/CUDAnative.jl#489

[findmyway]

Hi, I think I encounter an error after this PR. Using the latest master branch with the following code:

module TestFlux
using Flux
end

julia> using TestFlux
[ Info: Precompiling TestFlux [158674fa-8238-5cab-b5ba-03dfc80d1318]
ERROR: LoadError: InitError: UndefVarError: functional not defined
Stacktrace:
 [1] getproperty at ./Base.jl:13 [inlined]
...

[maleadt]

Yes, thanks for the report, looking into it.