Dispatch error in v1.12

[JonasIsensee]

Hi,

this popped up in JLD2 JuliaIO/JLD2.jl#645.

using JLD2 # v0.5.13
struct Foo end
which(JLD2.h5fieldtype, Tuple{JLD2.JLDFile{JLD2.MmapIO}, Type{Type{Foo}}, Type{Type{Foo}}, Type{Val{false}}})

On v1.11 and earlier this gives:

h5fieldtype(f::JLD2.JLDFile, ::Type{T}, readas::Type, ::Union{Type{Val{true}}, Type{Val{false}}}) where T<:DataType
     @ JLD2 /data.lmp/isensee/.julia/dev/JLD2/src/data/writing_datatypes.jl:382

On v1.12:

h5fieldtype(f::JLD2.JLDFile, writeas, readas::Type, initialized::Union{Type{Val{true}}, Type{Val{false}}})
     @ JLD2 /data.lmp/isensee/.julia/dev/JLD2/src/data/writing_datatypes.jl:91

The method in line 382 still exists but cannot be dispatched to
When removing the definition in line 91, I get an error: (pasted as image for colors)

Oddly enough: When tested in isolation, this works just fine.

julia> testfun(::JLD2.JLDFile, writeas::Type{T}, readas::Type, initialized::JLD2.Initialized) where T<:DataType = "works"
testfun (generic function with 1 method)

julia> which(testfun, Tuple{JLD2.JLDFile{JLD2.MmapIO}, Type{Type{Foo}}, Type{Type{Foo}}, Type{Val{false}}})
testfun(::JLD2.JLDFile, writeas::Type{T}, readas::Type, initialized::Union{Type{Val{true}}, Type{Val{false}}}) where T<:DataType
     @ Main REPL[22]:1

[vtjnash]

smaller MWE

julia> for i = 1:40; @eval f(::Val{$i}) = $i; end

julia> f(::Type{<:DataType}) = 0;

julia> f(::Type) = -1;

julia> f(Type{Int}) # should == 0
-1

Note that relying on kind matching in type parameters is likely to be very unwise and cause you future problems like this, so you should consider rewriting the code not to do that (though we also plan to fix this particular regression for the v1.12 release)

[vtjnash]

Some examples of the mistakes you'll encounter include in your code today, with correct dispatch:

julia> f(Type{Vector{T}} where T) # dispatches to Type{<:DataType}, despite being !concrete
0

julia> f(Type{AbstractString}) # dispatches to Type{<:DataType}, despite being abstract
0

julia> f(DataType) # dispatches to Type{<:DataType}, despite not even being Type
0

and also type normalization.

[j-fu]

We stumbled upon this when storing instances of our grid type to disk with JLD2, see JuliaIO/JLD2.jl#645 referenced at the start of this issue.

This is its definition:
https://github.com/WIAS-PDELib/ExtendableGrids.jl/blob/783fdd36af555d93f4d8ff2b4bfe52c434477a9a/src/extendablegrid.jl#L16

The core component is a Dict{Type{<:AbstractGridComponent}, Any} with types as keys which allows to nicely dispatch between different getindex etc methods like
instantiate(grid, ::Type{NumCellRegions}) = maximum(grid[CellRegions])

So we indeed seem to do dispatch using what you call kind matching. Is there any reference which explains why you say that this is unwise ? If this is really a serious issue, we could probably rewrite the code by using singleton instances instead of the types. @pjaap

[vtjnash]

Just that JLD2 seems to be mistakenly using this as a way of computing isconcretetype, which is not what it actually computes (see counter-examples above)

[JonasIsensee]

Thank you for the examples and for getting this case fixed.
I am not sure, I fully understand in which cases this will soon begin to fail.
Type kind matching appears to be a central design principle in JLD2 for handling the serialization of arbitrary julia structs.
A quick full-text search revealed 241 ::Type statements in function definitions within JLD2.

[vtjnash]

Dispatching on Type is generally fine, just not this specific use where it dispatches on the particular representation of Type kinds (the <:DataType)