Do gradient via mutating and unmutating cell

[oxinabox]

We can avoid copying every element every step if we just mutate the element we want to mutate on the original matrix, then mutate it back again after so we still have the original for the next step.

My micro benchmarks show this helps a bit.

[oxinabox]

I think similar could be done for the Dict case but I don't really understand how that works.
https://github.com/JuliaDiff/FiniteDifferences.jl/pull/59/files#diff-94e14cf43e653c4b17f49cc18895018cR33-R44

[longemen3000]

with the dict:

function grad(fdm, f, d::Dict{K, V}) where {K, V}
    ∇d = Dict{K, V}()
    for (k, v) in d
        temp = d[k]
        function f′(x)
            d[k] = x
            return f(d)
        end
        ∇d[k] = grad(fdm, f′, v)[1]
        d[k] = temp
    end
    return (dd, )
end

[oxinabox]

Benchmarks

using BenchmarkTools
using FiniteDifferences

const _fdm = central_fdm(2,1);
const xs = collect(1:0.1:200);
f(x) = sum(sin, x);
@btime grad($_fdm, $f, $xs);

Before:

222.774 ms (154790 allocations: 4.28 MiB)

With this PR;

212.711 ms (150807 allocations: 4.20 MiB)

So only a very modest improvement.
But still a step towards not allocating at all (beyond what the primal does).

[codecov-io]

Codecov Report

Merging #59 into master will decrease coverage by 0.55%.
The diff coverage is 92.85%.

@@            Coverage Diff             @@
##           master      #59      +/-   ##
==========================================
- Coverage    98.8%   98.24%   -0.56%     
==========================================
  Files           4        4              
  Lines         167      171       +4     
==========================================
+ Hits          165      168       +3     
- Misses          2        3       +1

Impacted Files	Coverage Δ
src/grad.jl	96.66% <92.85%> (-1.01%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 70847c8...f677183. Read the comment docs.

[willtebbutt]

Looks reasonable to me.

[willtebbutt]

Doesn't technically matter, but maybe change / remove the type constraint on x, since we know it's going to be an Array by virtue of the fact it's called from grad

[oxinabox]

Could be a a few things actually.
But yes, the constraints are not needed

[oxinabox]

I am going to keep it because the T is useful