Third derivative of x^2 at x=0 is NaN

[tkf]

If I nest ForwardDiff.derivative three times to compute the third derivative, I found that it doesn't work with some inputs. Taking derivative of x^2 at x = 0 is the minimal example I found.

julia> deriv3(f, x::Number = 0.0) =
           ForwardDiff.derivative(
               a -> ForwardDiff.derivative(
                   b -> ForwardDiff.derivative(f, b),
                   a),
               x)
deriv3 (generic function with 2 methods)

julia> deriv3(x -> x^2)
NaN

julia> deriv3(x -> x^3)  # OK
6.0

julia> deriv3(x -> x)  # OK
0.0

julia> versioninfo()
Julia Version 0.6.3
Commit d55cadc (2018-05-28 20:20 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM)2 Quad CPU    Q9400  @ 2.66GHz
  WORD_SIZE: 64
  BLAS: libopenblas (DYNAMIC_ARCH NO_AFFINITY Penryn)
  LAPACK: libopenblas
  LIBM: libm
  LLVM: libLLVM-3.9.1 (ORCJIT, penryn)

julia> Pkg.installed("ForwardDiff")
v"0.7.5"

[tkf]

There may be some issue in how Dual treats x^2? I find that x * x works fine:

julia> using ForwardDiff: Dual

julia> x0 = 0.0
0.0

julia> x1 = Dual{:t1}(x0, 1.0)
Dual{:t1}(0.0,1.0)

julia> x2 = Dual{:t2}(x1, 1.0)
Dual{:t2}(Dual{:t1}(0.0,1.0),Dual{:t1}(1.0,0.0))

julia> x3 = Dual{:t3}(x2, 1.0)
Dual{:t3}(Dual{:t2}(Dual{:t1}(0.0,1.0),Dual{:t1}(1.0,0.0)),
          Dual{:t2}(Dual{:t1}(1.0,0.0),Dual{:t1}(0.0,0.0)))

julia> x3^2
Dual{:t3}(Dual{:t2}(Dual{:t1}(0.0,0.0),Dual{:t1}(0.0,2.0)),
          Dual{:t2}(Dual{:t1}(0.0,2.0),Dual{:t1}(2.0,NaN)))

julia> x3 * x3
Dual{:t3}(Dual{:t2}(Dual{:t1}(0.0,0.0),Dual{:t1}(0.0,2.0)),
          Dual{:t2}(Dual{:t1}(0.0,2.0),Dual{:t1}(2.0,0.0)))

julia> x3^3
Dual{:t3}(Dual{:t2}(Dual{:t1}(0.0,0.0),Dual{:t1}(0.0,0.0)),
          Dual{:t2}(Dual{:t1}(0.0,0.0),Dual{:t1}(0.0,6.0)))

(BTW, IIUC, this is the simplified version of what happens in the nested ForwardDiff.derivative call; is it correct?)

[KristofferC]

http://www.juliadiff.org/ForwardDiff.jl/stable/user/advanced.html#Fixing-NaN/Inf-Issues-1

[tkf]

Oops, I forgot to mention that I tried NANSAFE_MODE_ENABLED = true and it didn't change the result.

julia> using ForwardDiff

julia> ForwardDiff.NANSAFE_MODE_ENABLED
true

julia> deriv3(f, x::Number = 0.0) =
           ForwardDiff.derivative(
               a -> ForwardDiff.derivative(
                   b -> ForwardDiff.derivative(f, b),
                   a),
               x)
deriv3 (generic function with 2 methods)

julia> deriv3(x -> x^2)
NaN

julia> deriv3(x -> x^3)  # OK
6.0

julia> deriv3(x -> x)  # OK
0.0

julia> using ForwardDiff: Dual

julia> x0 = 0.0
0.0

julia> x1 = Dual{:t1}(x0, 1.0)
Dual{:t1}(0.0,1.0)

julia> x2 = Dual{:t2}(x1, 1.0)
Dual{:t2}(Dual{:t1}(0.0,1.0),Dual{:t1}(1.0,0.0))

julia> x3 = Dual{:t3}(x2, 1.0)
Dual{:t3}(Dual{:t2}(Dual{:t1}(0.0,1.0),Dual{:t1}(1.0,0.0)),Dual{:t2}(Dual{:t1}(1.0,0.0),Dual{:t1}(0.0,0.0)))

julia> x3^2
Dual{:t3}(Dual{:t2}(Dual{:t1}(0.0,0.0),Dual{:t1}(0.0,2.0)),Dual{:t2}(Dual{:t1}(0.0,2.0),Dual{:t1}(2.0,NaN)))

julia> x3 * x3
Dual{:t3}(Dual{:t2}(Dual{:t1}(0.0,0.0),Dual{:t1}(0.0,2.0)),Dual{:t2}(Dual{:t1}(0.0,2.0),Dual{:t1}(2.0,0.0)))