Set jacobian of Fun to ZeroOperator

[LucasAschenbach]

The jacobian of a plain Functional should be the ZeroOperator. This is what was causing the independent variable test for the newton iteration to take considerably more iterations than if each dimension was solved in isolation.

[codecov]

Codecov Report

Patch coverage: 100.00% and project coverage change: +0.30% 🎉

Comparison is base (0e78219) 28.06% compared to head (f984484) 28.36%.
Report is 9 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #909      +/-   ##
==========================================
+ Coverage   28.06%   28.36%   +0.30%     
==========================================
  Files          11       11              
  Lines         563      564       +1     
==========================================
+ Hits          158      160       +2     
+ Misses        405      404       -1     

Files Changed	Coverage Δ
ext/ApproxFunDualNumbersExt.jl	21.27% <100.00%> (+3.88%)	⬆️
src/Extras/autodifferentiation.jl	62.50% <100.00%> (ø)

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[jishnub]

I'm not very familiar with this code. Could you comment on why this should be zero?

[LucasAschenbach]

The DualFun structure propagates the derivatives through the function such that for an operator F, the expression F(DualFun(x,T)) is expected to evaluate to DualFun(F(x),F'(x)∘T), i.e. jacobian(F(DualFun(x,T))) = F'(x)∘T (In this implementation T is assumed to be linear). Think of the DualFun structure as holding the value x and an infinitesimally small perturbation T∘𝓔(x) which get propagated through F. In this framework, a plain x::Fun has no perturbation since operators evaluate x to F(x); hence the dual part, i.e. the perturbation of the identity on x should be zero.

Let's consider the example where F is constant. Then, said expression evaluates to F(DualFun(x,T)) = F(x). Under the current implementation, jacobian(F(DualFun(x,T))) would return the identity operator which is incorrect as the derivative is zero. This is also what happened in the independent Newton test case. The first and second row are independent of the second and first variable, respectively. Therefore, the derivatives w.r.t. these variables gave the identity operator instead of the zero operator. As a consequence, the method behaved like an Inexact Newton Method converging much slower than expected.

[jishnub]

Thanks, that makes sense.