add simple finetuning check with fp8 emulation

[vkuzo]

Summary:

Creates a simple image classification network and finetunes it on MNIST. Baseline is fp32, and fp8 training can be enabled with a flag. Verified that fp8 training converges on this simple example. Note that fp8 compute is emulated for now as we don't have a hookup to the real fp8 matmul kernel yet.

Test Plan:

with-proxy python finetune/mnist.py --batch-size 4096
// https://gist.github.com/vkuzo/0e8cbb3df1f0610e528ac3ad15da3ace
with-proxy python finetune/mnist.py --batch-size 4096 --use-pt-fp
// https://gist.github.com/vkuzo/99b0cf2c1492a5f605c9f028f12340c3

Reviewers:

Subscribers:

Tasks:

Tags: