Recurrent benchmarks

[mkschleg]

Given the conversation in #1855, I think we should have benchmarks in the main repo we can use to iterate on for rnn performance. This touches benchmark_utils to add some of the Flux.Recur specific machinery we need. I only added tests for RNN using both the vector and 3d interfaces, but we can add the other cells once we are happy w/ how these operate.

PR Checklist

• Tests are added
• Entry in NEWS.md
• Documentation, if applicable

[ToucheSir]

One thing I was experimenting with locally was a more structured benchmarking system like https://github.com/JuliaGPU/CUDA.jl/blob/master/perf/runbenchmarks.jl. It would be nice to have a way to visualize results from these smaller-scale performance tests. https://speed.fluxml.ai/ currently has just E2E examples.

[mkschleg]

Having good tools for testing performance and for testing is up there with things I wish we had in Flux (#1700). With the performance measurements I think it would be good to also capture how much memory we use (to more easily catch things like #1872).

[mkschleg]

@CarloLucibello and @ToucheSir I'm cool with waiting to add RNN tests until we figure out a better framework if that is preferable! Otherwise, what do we want done in this PR to get this merged?

[mkschleg]

I made some modifications. Primarily from @ToucheSir's suggestions. I also modified the bench_utils slightly to make overloading functionality easier. Primarily by adding a pullback function pb(m, ps, x) and changing some of the other generic versions to use fw instead of a generic call.