The Hessian of x-> dot(x,H,x) should be H+H', not zero

[wmkoolen]

Hi all,

ForwardDiff incorrectly computes the Hessian of three-argument-dot.

Cook up some (any) square matrix
julia> H = [1 2 3; 4 5 6; 7 8 9];

Compute Hessian of three-argument dot. Yields all zeros. Should yield H+H'.
julia> ForwardDiff.hessian(x->dot(x,H,x), zeros(3))
3×3 Array{Float64,2}:
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0

Manually expand that three argument dot call. Now result is the expected H+H':
julia> ForwardDiff.hessian(x->x'H*x, zeros(3))
3×3 Array{Float64,2}:
2.0 6.0 10.0
6.0 10.0 14.0
10.0 14.0 18.0

Curious!

Cheers,

Wouter

[mcabbott]

Notice that if you give slightly nonzero arguments, you get different results:

julia> H = [1 2 3; 4 5 6; 7 8 9];

julia> ForwardDiff.hessian(x->dot(x,H,x), zeros(3))
3×3 Matrix{Float64}:
 0.0  0.0  0.0
 0.0  0.0  0.0
 0.0  0.0  0.0

julia> ForwardDiff.hessian(x->dot(x,H,x), [0,0,0.0001])
3×3 Matrix{Float64}:
 0.0  0.0   3.0
 0.0  0.0   6.0
 3.0  6.0  18.0

julia> ForwardDiff.hessian(x->dot(x,H,x), fill(0.00001, 3))
3×3 Matrix{Float64}:
  2.0   6.0  10.0
  6.0  10.0  14.0
 10.0  14.0  18.0

So dot must be taking shortcuts when it knows x is zero. This is fixed by #481.