Convolutions error using Measurements.jl

[adrhill]

Passing an Array of Measurements to a Conv layer throws the error ERROR: UndefRefError: access to undefined reference:

julia> using Flux, Measurements

(@v1.7) pkg> status Flux
      Status `~/.julia/environments/v1.7/Project.toml`
  [587475ba] Flux v0.12.10

(@v1.7) pkg> status Measurements
      Status `~/.julia/environments/v1.7/Project.toml`
  [eff96d63] Measurements v2.7.1

julia> x = rand(Float32, 5, 5, 3, 1);

julia> m = measurement.(x, 0.02f0)
5×5×3×1 Array{Measurement{Float32}, 4}:
[:, :, 1, 1] =
 0.843±0.02   0.78±0.02  0.164±0.02  0.313±0.02   0.578±0.02
 0.188±0.02  0.119±0.02  0.573±0.02  0.342±0.02  0.0012±0.02
 0.434±0.02   0.99±0.02  0.901±0.02  0.881±0.02   0.569±0.02
 0.149±0.02  0.184±0.02  0.553±0.02  0.827±0.02   0.328±0.02
 0.544±0.02  0.659±0.02  0.159±0.02  0.426±0.02   0.789±0.02

[:, :, 2, 1] =
 0.903±0.02  0.801±0.02  0.904±0.02  0.101±0.02   0.11±0.02
 0.879±0.02  0.764±0.02   0.11±0.02  0.426±0.02  0.461±0.02
  0.03±0.02  0.911±0.02  0.819±0.02   0.15±0.02  0.922±0.02
  0.91±0.02  0.392±0.02  0.838±0.02  0.356±0.02  0.277±0.02
 0.632±0.02  0.391±0.02   0.94±0.02  0.969±0.02  0.154±0.02

[:, :, 3, 1] =
 0.084±0.02  0.947±0.02  0.891±0.02  0.997±0.02  0.172±0.02
 0.815±0.02  0.285±0.02  0.243±0.02  0.427±0.02  0.639±0.02
 0.663±0.02  0.055±0.02  0.807±0.02  0.782±0.02  0.439±0.02
 0.882±0.02  0.958±0.02  0.869±0.02  0.471±0.02  0.204±0.02
 0.235±0.02  0.544±0.02  0.946±0.02  0.735±0.02  0.302±0.02

julia> c = Conv((3, 3), 3 => 1)
Conv((3, 3), 3 => 1)  # 28 parameters

julia> c(x)
3×3×1×1 Array{Float32, 4}:
[:, :, 1, 1] =
 0.127281    0.635378   -0.00389463
 0.0390705   0.0806748   0.285093
 0.0103112  -0.182715    0.127719

julia> c(m)
ERROR: UndefRefError: access to undefined reference
Stacktrace:
  [1] getindex
    @ ./array.jl:862 [inlined]
  [2] conv_direct!(y::Array{Measurement{Float32}, 5}, x::Array{Measurement{Float32}, 5}, w::Array{Float32, 5}, cdims::DenseConvDims{3, 3, 3, 6, 3}, ::Val{(3, 3, 1)}, ::Val{1}, ::Val{(0, 0, 0, 0, 0, 0)}, ::Val{(1, 1, 1)}, ::Val{(1, 1, 1)}, fk::Val{false}; alpha::Measurement{Float32}, beta::Bool)
    @ NNlib ~/.julia/packages/NNlib/TAcqa/src/impl/conv_direct.jl:111
  [3] conv_direct!(y::Array{Measurement{Float32}, 5}, x::Array{Measurement{Float32}, 5}, w::Array{Float32, 5}, cdims::DenseConvDims{3, 3, 3, 6, 3}; alpha::Measurement{Float32}, beta::Bool)
    @ NNlib ~/.julia/packages/NNlib/TAcqa/src/impl/conv_direct.jl:50
  [4] conv_direct!
    @ ~/.julia/packages/NNlib/TAcqa/src/impl/conv_direct.jl:50 [inlined]
  [5] #conv!#288
    @ ~/.julia/packages/NNlib/TAcqa/src/conv.jl:288 [inlined]
  [6] conv!
    @ ~/.julia/packages/NNlib/TAcqa/src/conv.jl:284 [inlined]
  [7] conv!(y::Array{Measurement{Float32}, 4}, x::Array{Measurement{Float32}, 4}, w::Array{Float32, 4}, cdims::DenseConvDims{2, 2, 2, 4, 2}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ NNlib ~/.julia/packages/NNlib/TAcqa/src/conv.jl:145
  [8] conv!
    @ ~/.julia/packages/NNlib/TAcqa/src/conv.jl:145 [inlined]
  [9] conv(x::Array{Measurement{Float32}, 4}, w::Array{Float32, 4}, cdims::DenseConvDims{2, 2, 2, 4, 2}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ NNlib ~/.julia/packages/NNlib/TAcqa/src/conv.jl:88
 [10] conv
    @ ~/.julia/packages/NNlib/TAcqa/src/conv.jl:86 [inlined]
 [11] (::Conv{2, 4, typeof(identity), Array{Float32, 4}, Vector{Float32}})(x::Array{Measurement{Float32}, 4})
    @ Flux ~/.julia/packages/Flux/7nTyc/src/layers/conv.jl:166
 [12] top-level scope
    @ REPL[8]:1
 [13] top-level scope
    @ ~/.julia/packages/CUDA/5jdFl/src/initialization.jl:52

@DhairyaLGandhi's suggestion from Slack's #machine-learning to set the weights as Measurements doesn't fix the issue:

julia> c2 = Conv(measurement.(c.weight, 0.0f0), measurement.(c.bias, 0.0f0))
Conv((3, 3), 3 => 1)  # 28 parameters

julia> typeof(c2)
Conv{2, 4, typeof(identity), Array{Measurement{Float32}, 4}, Vector{Measurement{Float32}}}

julia> c2(m)
ERROR: UndefRefError: access to undefined reference
Stacktrace:
  [1] getindex
    @ ./array.jl:862 [inlined]
  [2] conv_direct!(y::Array{Measurement{Float32}, 5}, x::Array{Measurement{Float32}, 5}, w::Array{Measurement{Float32}, 5}, cdims::DenseConvDims{3, 3, 3, 6, 3}, ::Val{(3, 3, 1)}, ::Val{1}, ::Val{(0, 0, 0, 0, 0, 0)}, ::Val{(1, 1, 1)}, ::Val{(1, 1, 1)}, fk::Val{false}; alpha::Measurement{Float32}, beta::Bool)
    @ NNlib ~/.julia/packages/NNlib/TAcqa/src/impl/conv_direct.jl:111
  [3] conv_direct!(y::Array{Measurement{Float32}, 5}, x::Array{Measurement{Float32}, 5}, w::Array{Measurement{Float32}, 5}, cdims::DenseConvDims{3, 3, 3, 6, 3}; alpha::Measurement{Float32}, beta::Bool)
    @ NNlib ~/.julia/packages/NNlib/TAcqa/src/impl/conv_direct.jl:50
  [4] conv_direct!
    @ ~/.julia/packages/NNlib/TAcqa/src/impl/conv_direct.jl:50 [inlined]
  [5] #conv!#288
    @ ~/.julia/packages/NNlib/TAcqa/src/conv.jl:288 [inlined]
  [6] conv!
    @ ~/.julia/packages/NNlib/TAcqa/src/conv.jl:284 [inlined]
  [7] conv!(y::Array{Measurement{Float32}, 4}, x::Array{Measurement{Float32}, 4}, w::Array{Measurement{Float32}, 4}, cdims::DenseConvDims{2, 2, 2, 4, 2}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ NNlib ~/.julia/packages/NNlib/TAcqa/src/conv.jl:145
  [8] conv!
    @ ~/.julia/packages/NNlib/TAcqa/src/conv.jl:145 [inlined]
  [9] conv(x::Array{Measurement{Float32}, 4}, w::Array{Measurement{Float32}, 4}, cdims::DenseConvDims{2, 2, 2, 4, 2}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ NNlib ~/.julia/packages/NNlib/TAcqa/src/conv.jl:88
 [10] conv
    @ ~/.julia/packages/NNlib/TAcqa/src/conv.jl:86 [inlined]
 [11] (::Conv{2, 4, typeof(identity), Array{Measurement{Float32}, 4}, Vector{Measurement{Float32}}})(x::Array{Measurement{Float32}, 4})
    @ Flux ~/.julia/packages/Flux/7nTyc/src/layers/conv.jl:166
 [12] top-level scope
    @ REPL[11]:1
 [13] top-level scope
    @ ~/.julia/packages/CUDA/5jdFl/src/initialization.jl:52

[darsnack]

Yeah changing the weights won't help, because the undefined reference it is complaining about is an index to y in conv_direct! (i.e. the output). This is created by NNlib using similar. NNlib doesn't care about the initial content in y, but it does assume that the elements are defined. That's not the case for Measurements.jl:

julia> using Measurements

julia> x = rand(Float32, 2, 2)
2×2 Matrix{Float32}:
 0.737843  0.811024
 0.440666  0.4111

julia> m = measurement.(x, 0.02)
2×2 Matrix{Measurement{Float64}}:
 0.738±0.02  0.811±0.02
 0.441±0.02  0.411±0.02

julia> similar(m)
2×2 Matrix{Measurement{Float64}}:
 #undef  #undef
 #undef  #undef

julia> similar(x)
2×2 Matrix{Float32}:
 6.071f-40  1.10454f-16
 0.0        1.0f-45

[adrhill]

So fixing this would require Measurements.jl to implement a similar method?

I'm guessing the issue would be the unique identifier tag and dictionary of derivatives der that are part of a Measurement:
https://github.com/JuliaPhysics/Measurements.jl/blob/master/src/Measurements.jl#L40-L50

[darsnack]

One fix would be to change L111 and L154 of impl/conv_direct.jl to

# old
y[w_idx, h_idx, d_idx, c_out, batch] = alpha*dotprod + beta*y[w_idx, h_idx, d_idx, c_out, batch]

# new
# somewhere outside loops
yidxs = LinearIndices(y)
# L111 and later L154
if !isassigned(y, yidxs[w_idx, h_idx, d_idx, c_out, batch])
  y[w_idx, h_idx, d_idx, c_out, batch] = yT(0)
end
y[w_idx, h_idx, d_idx, c_out, batch] = alpha*dotprod + beta*y[w_idx, h_idx, d_idx, c_out, batch]

Having Measurements.jl define a similar method would fix this issue, but I think that NNlib's assumption that similar returns defined arrays is wrong. For custom types, the default similar will return #undefs.

[ToucheSir]

I believe this is a dupe of #490. Using an initialization loop or zeros isn't ideal, but I guess that's not a goal of conv_direct! anyhow.