Mega-issue for packages failing PkgEval for 1.1 release

[KristofferC]

This contains a small analysis of all packages that fail PkgEval.
Gist with the results of packages failing on 1.1 vs 1.0: https://gist.github.com/ararslan/533cf231d32d87a6f80bf47d38de3182

Feel free to look at any of the packages without a cross and investigate why they fail and update this post (or post a reply).
Also, feel free to open dedicated issues for specific packages to discuss a certain bug / breaking change.

Failing packages

Packages that fail but are deemed to do so for acceptable reasons are checked.

• Zygote
• WignerSymbols
• VT100
• SymbolServer
• SVR (fixed by Mega-issue for packages failing PkgEval for 1.1 release #30374 (comment)?)
• StaticNumbers
• SplitApplyCombine
• SPH
• ResumableFunctions Mega-issue for packages failing PkgEval for 1.1 release #30374 (comment)
• QDates (explanation: Mega-issue for packages failing PkgEval for 1.1 release #30374 (comment))
• OffsetArrays (likely fixed by Support IdentityUnitRange JuliaArrays/OffsetArrays.jl#62)
• Nullables (should be fixed by codegen: disable Bool optimization for maybe-undef fields #30350)
• MultivariatePolynomials (fixed by fix #30346, specificity issue with DynamicPolynomials #30360)
• MathematicalSystems (maybe fixed Mega-issue for packages failing PkgEval for 1.1 release #30374 (comment))
• Libtask
• LegacyStrings (also fixed by Fix import from Base warnings on Julia 1.0 JuliaStrings/LegacyStrings.jl#36)
• JSONWebTokens
• IterableTables
• Distances
• DecFP: (explanation: Mega-issue for packages failing PkgEval for 1.1 release #30374 (comment))
• CSTParser
• BlockArrays
• BandedMatrices
• AxisArrays (likely fixed by Support IdentityUnitRange JuliaArrays/OffsetArrays.jl#62)
• AstroLib (PR with fix at Fix #556, use IdentityUnitRange in place of Base.Slice JuliaArrays/StaticArrays.jl#568)
• Arrow (passes locally Mega-issue for packages failing PkgEval for 1.1 release #30374 (comment))
• AbstractOperators (explanation: Mega-issue for packages failing PkgEval for 1.1 release #30374 (comment))

Analysis:

Zygote

LoadError: type StmtRange has no field first.

Uses a lot of internal features. Can likely ignore.

WignerSymbols

Testing WignerSymbols tests passed

wut?

VT100

ERROR: LoadError: LoadError: MethodError: no method matching Base.TTY(::RawFD; readable=true)

Base.TTY is internal and undocumented. Can likely ignore.

SymbolServer

LoadError: MethodError: no method matching getindex(::Pkg.Types.Project, ::String)

Uses Pkg internals that have been refactored. Should help with fixing SymbolServer to work on 1.1 but no need
for change in code,

SVR

signal (11): Segmentation fault
in expression starting at /root/.julia/packages/SVR/OpJky/test/runtests.jl:8
_ZN6KernelD0Ev at /root/.julia/packages/SVR/OpJky/src/../deps/libsvm-3.22/libsvm.so.2 (unknown line)
Allocations: 4829533 (Pool: 4828507; Big: 1026); GC: 10
ERROR: Package SVR errored during testing

Should investigate

StaticNumbers

Error During Test at /root/.julia/packages/StaticNumbers/BglY7/test/runtests.jl:104
  Test threw exception
  Expression: trystatic(2, 1:3) == static(2)
  MethodError: convert(::Type{StaticInteger{0}}, ::StaticInteger{0}) is ambiguous. Candidates:
    convert(::Type{T}, ::Union{StaticInteger{X}, StaticNumber{X}, StaticReal{X}}) where {T<:Number, X} in StaticNumbers at /root/.julia/packages/StaticNumbers/BglY7/src/StaticNumbers.jl:108
    convert(T::Type{#s14} where #s14<:Union{StaticInteger{X}, StaticNumber{X}, StaticReal{X}}, y::Number) where X in StaticNumbers at /root/.julia/packages/StaticNumbers/BglY7/src/StaticNumbers.jl:104
  Possible fix, define
    convert(::Union{Type{T<:(Union{StaticInteger{X}, StaticNumber{X}, StaticReal{X}} where X)}, Type{T<:(Union{StaticInteger{X}, StaticNumber{X}, StaticReal{X}} where X)}, Type{T<:(Union{StaticInteger{X}, StaticNumber{X}, StaticReal{X}} where X)}}, ::Union{StaticInteger{X}, StaticNumber{X}, StaticReal{X}})
  Stacktrace:
   [1] LengthUnitRange{Int64,StaticInteger{0},StaticInteger{3}}(::StaticInteger{0}, ::StaticInteger{3}) at /root/.julia/packages/StaticNumbers/BglY7/src/LengthRanges.jl:53
   [2] LengthUnitRange(::StaticInteger{0}, ::StaticInteger{3}) at /root/.julia/packages/StaticNumbers/BglY7/src/LengthRanges.jl:59
   [3] trystatic(::Int64, ::UnitRange{Int64}) at /root/.julia/packages/StaticNumbers/BglY7/src/trystatic.jl:53

SplitApplyCombine

 Unexpected Pass
 Expression: isequal(#= /root/.julia/packages/SplitApplyCombine/nrESG/test/product.jl:5 =# @inferred(product(*, [1.0, 2.0], [1, 2, missing]))::Matrix{Union{Missing, Float64}}, [1.0 2.0 missing; 2.0 4.0 missing])
 Got correct result, please change to @test if no longer broken.

Maybe we just fixed something?

SPH

ERROR: Unsatisfiable requirements detected for package SPH [9ea8cd13]:
 SPH [9ea8cd13] log:
 ├─possible versions are: [0.0.1, 0.2.1-0.2.2, 1.0.0] or uninstalled
 ├─restricted to versions * by an explicit requirement, leaving only versions [0.0.1, 0.2.1-0.2.2, 1.0.0]
 └─restricted by julia compatibility requirements to versions: uninstalled — no versions left
Stacktrace:

Package is upper bounded, seems like this is working as intended

ResumableFunctions

test_for: Error During Test at /root/.julia/packages/ResumableFunctions/YK92O/test/runtests.jl:12
  Test threw exception
  Expression: collect(test_for(4)) == [4, 5, 9, 14, 23, 37, 60, 97, 157, 254]
  MethodError: Cannot `convert` an object of type Nothing to an object of type Tuple{Int64,Int64}
  Closest candidates are:
    convert(::Type{T<:Tuple{Any,Vararg{Any,N} where N}}, !Matched::T<:Tuple{Any,Vararg{Any,N} where N}) where T<:Tuple{Any,Vararg{Any,N} where N} at essentials.jl:274
    convert(::Type{T<:Tuple{Any,Vararg{Any,N} where N}}, !Matched::Tuple{Any,Vararg{Any,N} where N}) where T<:Tuple{Any,Vararg{Any,N} where N} at essentials.jl:275
    convert(::Type{T<:Tuple}, !Matched::CartesianIndex) where T<:Tuple at multidimensional.jl:130
    ...
  Stacktrace:
   [1] setproperty!(::getfield(Main, Symbol("##364")), ::Symbol, ::Nothing) at ./sysimg.jl:19
   [2] (::getfield(Main, Symbol("##364")))(::Nothing) at /root/.julia/packages/ResumableFunctions/YK92O/src/transforms.jl:16
   [3] iterate at /root/.julia/packages/MacroTools/4AjBS/src/utils.jl:84 [inlined]
   [4] _collect(::UnitRange{Int64}, ::getfield(Main, Symbol("##364")), ::Base.HasEltype, ::Base.SizeUnknown) at ./array.jl:553
   [5] collect(::getfield(Main, Symbol("##364"))) at ./array.jl:541

should investigate,

QDates

ERROR: LoadError: LoadError: There was an error during testing
in expression starting at /root/.julia/packages/QDates/qPuFu/test/ranges.jl:3
in expression starting at /root/.julia/packages/QDates/qPuFu/test/runtests.jl:12
start testing: ranges.jl
Test Failed at /root/.julia/packages/QDates/qPuFu/test/ranges.jl:51
  Expression: iterate(dr) == (first(dr), 1)
   Evaluated: (旧2014年01月01日, (1, 1)) == (旧2014年01月01日, 1)

Seems like some state from an iterator has changed. Should investigate.

OffsetArrays

Error During Test at /root/.julia/packages/OffsetArrays/NIuVg/test/runtests.jl:81
  Test threw exception
  Expression: A[:, 3] == S[:, 3] == OffsetArray([1, 2], (A.offsets[1],))
  MethodError: no method matching similar(::OffsetArray{Int64,2,Array{Int64,2}}, ::Type{Int64}, ::Tuple{Base.IdentityUnitRange{UnitRange{Int64}}})
  Closest candidates are:
    similar(::OffsetArray, ::Type{T}, !Matched::Tuple{Vararg{Int64,N}} where N) where T at /root/.julia/packages/OffsetArrays/NIuVg/src/OffsetArrays.jl:85
    similar(::AbstractArray, ::Type{T}) where T at abstractarray.jl:572
    similar(::AbstractArray, ::Type{T}, !Matched::Union{Integer, AbstractUnitRange}...) where T at abstractarray.jl:575
    ...
  Stacktrace:
   [1] similar(::OffsetArray{Int64,2,Array{Int64,2}}, ::Tuple{Base.IdentityUnitRange{UnitRange{Int64}}}) at ./abstractarray.jl:573
   [2] _unsafe_getindex(::IndexLinear, ::OffsetArray{Int64,2,Array{Int64,2}}, ::Base.Slice{UnitRange{Int64}}, ::Int64) at ./multidimensional.jl:654
   [3] _getindex at ./multidimensional.jl:642 [inlined]

Nullables

signal (11): Segmentation fault
in expression starting at /root/.julia/packages/Nullables/Oyo3b/test/runtests.jl:342
jl_intrinsic_cvt at /buildworker/worker/package_linux64/build/src/runtime_intrinsics.c:392
toInt8 at ./boot.jl:627 [inlined]
Type at ./boot.jl:730 [inlined]

Fixed by #30350

MultivariatePolynomials

Unreachable reached at 0x7f51518adc2a

signal (4): Illegal instruction
in expression starting at /root/.julia/packages/MultivariatePolynomials/N5uDO/test/zip.jl:3
Type at ./pair.jl:12 [inlined]
Type at ./pair.jl:15

Fixed by #30360??

MathematicalSystems

Polynomial system in continuous time: Error During Test at /root/.julia/packages/MathematicalSystems/coRVN/test/continuous.jl:121
  Got exception outside of a @test
  TypeError: in typeassert, expected Variable{:y}, got Variable{:y}
  Stacktrace:
   [1] setindex!(::Array{Variable{:y},1}, ::Variable{:y}, ::Int64) at ./array.jl:764
   [2] getindex(::Tuple{Variable{:y}}, ::UnitRange{Int64}) at ./range.jl:293
   [3] merge(::Tuple{Variable{:y}}, ::Variable{:x}) at /root/.julia/packages/TypedPolynomials/aFic2/src/promotion.jl:46

Looks weird. Should investigate.

Libtask

ERROR: LoadError: LoadError: type Task has no field parent
Stacktrace:
 [1] getproperty(::Any, ::Symbol) at ./sysimg.jl:18
 [2] copy(::Task) at /root/.julia/packages/Libtask/L5wq8/src/taskcopy.jl:28
 [3] top-level scope at none:0

Using internal structure of Task

LegacyStrings

in expression starting at /root/.julia/packages/LegacyStrings/tkVHE/test/runtests.jl:210
Error During Test at /root/.julia/packages/LegacyStrings/tkVHE/test/runtests.jl:211
  Test threw exception
  Expression: next(str, 15) == ('�', 16)
  UndefVarError: next not defined

Should investigate

JSONWebTokens

ERROR: Unsatisfiable requirements detected for package JSONWebTokens [9b8beb19]:
 JSONWebTokens [9b8beb19] log:
 ├─possible versions are: [0.0.2, 0.1.0] or uninstalled
 ├─restricted to versions * by an explicit requirement, leaving only versions [0.0.2, 0.1.0]
 └─restricted by julia compatibility requirements to versions: uninstalled — no versions left

Package is upper bounded, ok.

IterableTables

Testing IterableTables tests passed

Hmm.

Distances

zero allocation colwise!: Error During Test at /root/.julia/packages/Distances/nLAdT/test/test_dists.jl:548
 Unexpected Pass
 Expression: #= /root/.julia/packages/Distances/nLAdT/test/test_dists.jl:548 =# @allocated(colwise!(z, d, a, b)) == 0
 Got correct result, please change to @test if no longer broken.

Improvement, can ignore.

DecFP

Test Failed at /root/.julia/packages/DecFP/Mxubh/test/runtests.jl:74
  Expression: #= /root/.julia/packages/DecFP/Mxubh/test/runtests.jl:74 =# @sprintf("%7.2f", T("1.2345")) == "   1.23"
   Evaluated: "   @.0\x8f" == "   1.23"

Should investigate.

CSTParser

  Expression: Meta.parse("1,") == Expr(:tuple, 1)
   Evaluated: $(Expr(:incomplete, "incomplete: premature end of input")) == (1,)

Is quite closely tied to the internals of julia.

BlockArrays

lmul!/rmul!: Test Failed at /root/.julia/packages/BlockArrays/DOXjA/test/test_blockarrays.jl:320
  Expression: Array(A) == zeros(6)
   Evaluated: [NaN, NaN, NaN, NaN, NaN, NaN] == [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]

Changed 0 * NaN = NaN in BLAS1. Desired change even though breaking.

BandedMatrices

lmul!/rmul!: Test Failed at /root/.julia/packages/BandedMatrices/kvwBE/test/test_broadcasting.jl:144
  Expression: norm(A) == 0.0
   Evaluated: NaN == 0.0

Same as package above.

AxisArrays

Indexing: Error During Test at /root/.julia/packages/AxisArrays/G6pZY/test/indexing.jl:252
  Test threw exception
  Expression: #= /root/.julia/packages/AxisArrays/G6pZY/test/indexing.jl:252 =# @inferred(A[:, atvalue(2.0)]) == OffsetArrays.OffsetArray([1, 3], 0:1)
  MethodError: no method matching similar(::OffsetArrays.OffsetArray{Int64,2,Array{Int64,2}}, ::Type{Int64}, ::Tuple{Base.IdentityUnitRange{UnitRange{Int64}}})
  Closest candidates are:
    similar(::OffsetArrays.OffsetArray, ::Type{T}, !Matched::Tuple{Vararg{Int64,N}} where N) where T at /root/.julia/packages/OffsetArrays/NIuVg/src/OffsetArrays.jl:85
    similar(::AbstractArray, ::Type{T}) where T at abstractarray.jl:572
    similar(::AbstractArray, ::Type{T}, !Matched::Union{Integer, AbstractUnitRange}...) where T at abstractarray.jl:575
    ...

AstroLib

  MethodError: no method matching similar(::SubArray{Float64,1,StaticArrays.SArray{Tuple{3},Float64,1,3},Tuple{UnitRange{Int64}},true}, ::Type{Float64}, ::Tuple{Base.IdentityUnitRange{StaticArrays.SOneTo{4}}})
  Closest candidates are:
    similar(::SubArray, ::Type, !Matched::Tuple{Vararg{Int64,N}} where N) at subarray.jl:67
    similar(::AbstractArray, ::Type{T}) where T at abstractarray.jl:572
    similar(::AbstractArray, ::Type{T}, !Matched::Union{Integer, AbstractUnitRange}...) where T at abstractarray.jl:575
    ...

Arrow

arrowformat_construct: Test Failed at /root/.julia/packages/Arrow/b4oSO/test/runtests.jl:129
  Expression: typeof(p) == BitPrimitive
   Evaluated: NullableBitPrimitive == BitPrimitive

Should investigate

AbstractOperators

  LoadError: syntax: invalid syntax (incomplete #<julia: "incomplete: premature end of input">)
  Stacktrace:
   [1] mul!(::Array{Float64,2}, ::NonLinearCompose{2,HCAT{1,3,Tuple{MatrixOp{Float64,Float64,Array{Float64,2}},Zeros{Float64,2,Float64,2},Zeros{Float64,2,Float64,2}},Tuple{Int64,Int64,Int64},Array{Float64,2}},HCAT{1,2,Tuple{Zeros{Float64,2,Float64,2},NonLinearCompose{2,HCAT{1,2,Tuple{MatrixOp{Float64,Float64,Array{Float64,2}},Zeros{Float64,2,Float64,2}},Tuple{Int64,Int64},Array{Float64,2}},HCAT{1,2,Tuple{Zeros{Float64,2,Float64,2},MatrixOp{Float64,Float64,Array{Float64,2}}},Tuple{Int64,Int64},Array{Float64,2}},Tuple{Array{Float64,2},Array{Float64,2}},Tuple{Array{Float64,2},Array{Float64,2}}}},Tuple{Int64,Tuple{Int64,Int64}},Array{Float64,2}},Tuple{Array{Float64,2},Array{Float64,2}},Tuple{Array{Float64,2},Array{Float64,2}}}, ::Tuple{Array{Float64,2},Array{Float64,2},Array{Float64,2}}) at /root/.julia/packages/AbstractOperators/7oLxM/src/calculus/NonLinearCompose.jl:90
   [2] *(::NonLinearCompose{2,HCAT{1,3,Tuple{MatrixOp{Float64,Float64,Array{Float64,2}},Zeros{Float64,2,Float64,2},Zeros{Float64,2,Float64,2}},Tuple{Int64,Int64,Int64},Array{Float64,2}},HCAT{1,2,Tuple{Zeros{Float64,2,Float64,2},NonLinearCompose{2,HCAT{1,2,Tuple{MatrixOp{Float64,Float64,Array{Float64,2}},Zeros{Float64,2,Float64,2}},Tuple{Int64,Int64},Array{Float64,2}},HCAT{1,2,Tuple{Zeros{Float64,2,Float64,2},MatrixOp{Float64,Float64,Array{Float64,2}}},Tuple{Int64,Int64},Array{Float64,2}},Tuple{Array{Float64,2},Array{Float64,2}},Tuple{Array{Float64,2},Array{Float64,2}}}},Tuple{Int64,Tuple{Int64,Int64}},Array{Float64,2}},Tuple{Array{Float64,2},Array{Float64,2}},Tuple{Array{Float64,2},Array{Float64,2}}}, ::Tuple{Array{Float64,2},Array{Float64,2},Array{Float64,2}}) at /root/.julia/packages/AbstractOperators/7oLxM/src/syntax.jl:16
   [3] test_NLop(::NonLinearCompose{2,HCAT{1,3,Tuple{MatrixOp{Float64,Float64,Array{Float64,2}},Zeros{Float64,2,Float64,2},Zeros{Float64,2,Float64,2}},Tuple{Int64,Int64,Int64},Array{Float64,2}},HCAT{1,2,Tuple{Zeros{Float64,2,Float64,2},NonLinearCompose{2,HCAT{1,2,Tuple{MatrixOp{Float64,Float64,Array{Float64,2}},Zeros{Float64,2,Float64,2}},Tuple{Int64,Int64},Array{Float64,2}},HCAT{1,2,Tuple{Zeros{Float64,2,Float64,2},MatrixOp{Float64,Float64,Array{Float64,2}}},Tuple{Int64,Int64},Array{Float64,2}},Tuple{Array{Float64,2},Array{Float64,2}},Tuple{Array{Float64,2},Array{Float64,2}}}},Tuple{Int64,Tuple{Int64,Int64}},Array{Float64,2}},Tuple{Array{Float64,2},Array{Float64,2}},Tuple{Array{Float64,2},Array{Float64,2}}}, ::Tuple{Array{Float64,2},Array{Float64,2},Array{Float64,2}}, ::Array{Float64,2}, ::Bool) at /root/.julia/packages/AbstractOperators/7oLxM/test/utils.jl:35
   [4] top-level scope at none:0

Some parser change?

[fredrikekre]

OffsetArrays fixed by JuliaArrays/OffsetArrays.jl#62, and proably AxisArrays too

[JeffBezanson]

DecFP hooks into Printf internals. It will need to be updated after #30373 is merged. Currently it's assuming digits are left in DIGITS, which is no longer true, so it's getting junk.

[fredrikekre]

#28941 seems to have caused the failure of OffsetArrays, AxisArrays and AstroLib. OffsetArrays and AxisArrays are probably fixed by JuliaArrays/OffsetArrays.jl#62 but maybe you can comment on AstroLib@mbauman?

[ararslan]

As a sanity check I reran WignerSymbols and IterableTables, which both reported failure after passing tests, and both now report success.

[JeffBezanson]

AbstractOperators has generated functions that call parse (not recommended) on inputs ending with commas, which are now considered incomplete expressions.

[ararslan]

LegacyStrings was fixed by JuliaStrings/LegacyStrings.jl#36, which is in v0.4.1, but PkgEval ran on v0.4.0. So we're good there. (Not sure how it passed on 1.0, but hey...)

[tlienart]

For ResumableFunctions, a bit of investigation based on their runtests.jl

@resumable function test_for(a::Int=0; b::Int=a+1) :: Int
  for i in 1:10
    @yield a
    a, b = b, a+b
  end
end
collect(test_for(4)) # fails

however if you do this

@resumable function test_for(a::Int=0; b::Int=a+1, n::Int=10) :: Int
  for i in 1:n
    @yield a
    a, b = b, a+b
  end
end
collect(test_for(4)) # works

(note: both work on 1.0.x, first one fails on 1.1)

Edit: the problem is in the transform_for function in src/transforms.jl with this line

$next = iterate($iterator_value, $state)

where it looks like iterate($iterator_value, $state) becomes nothing at the end and it should break there instead of trying to feed that to $next (which expects a tuple of Int). Not too sure why the assignment causes an issue here in one case but not the other but I figure someone more Julia savvy will know.

Edit2: the following fixes the problem (though I'm not certain why so it should probably be looked at)

function transform_for(expr, ui8::BoxedUInt8)
  @capture(expr, for element_ in iterator_ body__ end) || return expr
  ui8.n += one(UInt8)
  next = Symbol("_iteratornext_", ui8.n)
  state = Symbol("_iterstate_", ui8.n)
  iterator_value = Symbol("_iterator_", ui8.n)
  quote
    $iterator_value = $iterator
    $next = iterate($iterator_value)
    while $next != nothing
      ($element, $state) = $next
      $(body...)
      tmp = iterate($iterator_value, $state) # <---
      tmp === nothing && break               # <---
      $next = tmp                            # <---
    end
  end
end

Edit3 there's another (different) problem with the function test_try (in runtests.jl) ... syntax: Attempt to jump into catch block, that's beyond my julia.

FWIW I've opened an issue for both over there. (JuliaDynamics/ResumableFunctions.jl#30)

[tlienart]

Just a note that for QDates, it's also an issue with iterate. Did anything noteworthy change there?

[saschatimme]

MultivariatePolynomials got fixed by #30360.