Recreation (overkill!) of Fizzbuzz in Pytorch
I wanted a small self contained project to help my understanding of Pytorch. Joel Grus's excellent blog post was the source of this idea and is highly worth a read. I've even borrowed some of his functions.
The classic Fizzbuzz problem can be solved imperically in a few lines of code so creating a neural network is a total overkill but its still a very interesting small project when learning the details of a designing a network, deploying to a GPU and a chance to dig into the basics.
It's also still powerful to show a pattern being learnt with data alone.
I demo'd this to my AI study group - more as a pytorch demo - it kicked off a conversation about whether a simple DNN could ever perform this task perfectly. Was there enough sophistication in feed forward neural network (with non linear activations) to learn this task? Would adding an RNN layer make the netowrk Turing Complete or does the Universal Approximation Theorem (UAT) mean that a non-recurrent network should be able to learn the task.
UAT is well discussed here and two important caveats are given; approximation and a continuous function - both of which would come into play with this example.
And the difference between the two is discussed here.
To test I added an LSTM layer to provide this memory / loop and improvements are definitely seen - more accurate results showing a greater approximation of Fizzbuzz are seen, getting up to only 1 wrong answer in the entire population vs 7 in the original model.
(Additionally the experiment was made considerably more robust by using an unseen test set after an oversight on my behalf!)