Error calculating gradient for basic function

[samuela]

Why doesn't this gradient work?

import Zygote

function f(x)
    # Why does fail?
    sum([x' * x, x...])

    # This also fails
    # sum([sum(x .^ 2), x...])

    # Also fails, but different error
    # sum([(x' * x)..., x...])

    # This works
    # sum([x' * x])

    # This works
    # sum(x)

    # This works
    # sum([0.0, x...])
end
Zygote.gradient(f, ones(3))

I get an error:

julia> include("bug.jl")
ERROR: LoadError: MethodError: no method matching +(::Tuple{Float64,Float64,Float64}, ::Array{Float64,1})
Closest candidates are:
  +(::Any, ::Any, ::Any, ::Any...) at operators.jl:529
  +(::ChainRulesCore.DoesNotExist, ::Any) at /Users/skainswo/.julia/packages/ChainRulesCore/Q5Nrj/src/differential_arithmetic.jl:20
  +(::ChainRulesCore.One, ::Any) at /Users/skainswo/.julia/packages/ChainRulesCore/Q5Nrj/src/differential_arithmetic.jl:64
  ...
Stacktrace:
 [1] accum(::Tuple{Float64,Float64,Float64}, ::Array{Float64,1}) at /Users/skainswo/.julia/packages/Zygote/1GXzF/src/lib/lib.jl:8
 [2] accum(::Tuple{Float64,Float64,Float64}, ::Array{Float64,1}, ::Array{Float64,1}) at /Users/skainswo/.julia/packages/Zygote/1GXzF/src/lib/lib.jl:13
 [3] #708 at /Users/skainswo/.julia/packages/Zygote/1GXzF/src/lib/array.jl:320 [inlined]
 [4] #1530#back at /Users/skainswo/.julia/packages/ZygoteRules/6nssF/src/adjoint.jl:49 [inlined]
 [5] f at /Users/skainswo/dev/research/julia/odecontrol/bug.jl:5 [inlined]
 [6] (::typeof(∂(f)))(::Float64) at /Users/skainswo/.julia/packages/Zygote/1GXzF/src/compiler/interface2.jl:0
 [7] (::Zygote.var"#37#38"{typeof(∂(f))})(::Float64) at /Users/skainswo/.julia/packages/Zygote/1GXzF/src/compiler/interface.jl:45
 [8] gradient(::Function, ::Array{Float64,1}) at /Users/skainswo/.julia/packages/Zygote/1GXzF/src/compiler/interface.jl:54
 [9] top-level scope at /Users/skainswo/dev/research/julia/odecontrol/bug.jl:22
 [10] include(::String) at ./client.jl:439
 [11] top-level scope at REPL[11]:1
in expression starting at /Users/skainswo/dev/research/julia/odecontrol/bug.jl:22

[mcabbott]

Duplicate of #599, I think.

However, splatting an array is usually a terrible idea anyway, this operation is vcat:

julia> Zygote.gradient(x -> sum([x' * x, x...]), rand(3))           
ERROR: MethodError: no method matching +(::Tuple{Float64,Float64,Float64}, ::Array{Float64,1})

julia> Zygote.gradient(x -> sum(vcat(x' * x, x)), rand(3))          
([2.3322036215131745, 2.459023092242586, 2.6029553627355084],)

[samuela]

However, splatting an array is usually a terrible idea anyway, this operation is vcat:

Could you elaborate a bit on this point? Do you mean that it's generally a terrible idea in idiomatic Julia code or that it's a terrible idea specifically in the context of functions being ADed with Zygote? And in either case why is it a bad idea? What's the difference relative to vcat?

[AzamatB]

It's a terrible idea because the cost of splatting grows linearly with array size, so while it's ok for splatting smallish tuples into the function arguments, for large arrays cost becomes really noticeable.

[samuela]

@AzamatB Ah, thanks for the explanation! I'll switch to vcat from now on.