[Breaking] Adds AMP<F> dtype

[coreylowman]

Resolves #424

• [Breaking] Moves Unit/Dtype to new dtypes module
• Adds dfdx::dtypes::AMP<F>
• impl Device<AMP<f16>> for Cpu/Cuda`

Ops that cast up to f32 internally:

• sum_to
• adam
• sgd
• rmsprop

Other ops may be auto cast in the future.

[nkoppel]

I'm curious: why struct Amp<F>(F) rather than struct Amp(f16)? Are there plans to support Amp<f32> as well? If not, the generic should be removed in my opinion.

[coreylowman]

I'm curious: why struct Amp<F>(F) rather than struct Amp(f16)? Are there plans to support Amp<f32> as well? If not, the generic should be removed in my opinion.

@nkoppel I was thinking we could have AMP<bf16> as well, but I don't actually know if that is done in practice. If not, we can move to what you suggest. Can you check if anyone does that?

[nkoppel]

PyTorch supports it, as seen here, so it makes sense to leave room to do that. Might it make sense to use a type alias to refer to AMP<f16> named something like AMP16 to make this easier to use?

[coreylowman]

👍 to type aliases