Computation of Hessian fails #1312
Comments
|
It is to be expected. |
|
Thanks for the hint. I made all the element types different and the error message changes. Here is the updated code snippet: using Zygote
using SparseArrays
using Random
using CUDA
function masked_mean(x::AbstractArray{T, N}, mask::AbstractArray{S, N}) where {T, S, N}
n = sum(mask)
if n > zero(T)
return sum(x .* mask) ./ n
else
return zero(T)
end
end
function test(x::AbstractArray{T, N}, mask::AbstractArray{S, N}, goal::R) where {T, S, R, N}
return (masked_mean(x, mask) - goal)^2
end
A = cu(sprand(200, 100, 0.8))
v = cu(rand(100))
mask = cu(round.(rand(200)))
goal = 1.f0
# test(A*v, mask, goal)
# Zygote.gradient(v -> test(A*v, mask, goal), v)
Zygote.hessian(v -> test(A*v, mask, goal), v)Then there is a warning about bad performance introduced in the Zygote code: And finally the error message: |
|
Setting |
So I am working with kernels of the type hessian(k, [1.])
ERROR: MethodError: no method matching (::IncrementalGRF.Kernels.SquaredExponential{Float64, 1})(::Vector{ForwardDiff.Dual{Nothing, Float64, 1}})
Closest candidates are:
(::IsotropicKernel{T, N})(::AbstractVector{T}) where {T, N} at ~/code/incrementalGP/src/kernels.jl:15 |
|
Barring that, adding a dispatch for |
|
Thanks for the answers. My takeaway from this thread is that Hessian computation in Julia is not stable yet. For my purposes it is not really needed, so I leave it at that. |
|
Kind of. If you must use Zygote, then everything said previously applies. If your code is amenable to working with other ADs, then libraries like ForwardDiff, ReverseDiff and https://github.com/JuliaDiff/SparseDiffTools.jl/ all have more battle-tested ways of calculating hessians. |
Hi,
I'm trying to calculate a masked squared loss as part of my loss function. The loss function and its gradient can be computed without problems, but the Hessian fails:
Error message:
If I interpret the message correctly, Zygote changes the datatype of the first parameter of
test(). Is this to be expected or a bug?The text was updated successfully, but these errors were encountered: