Circular dependency on 1.10

[j-fu]

Describe the bug 🐞

I am getting circular dependency errors with LinearSolve v3 on Julua 1.10 (but not on 1.11 and nightly).
In particular, this makes the Aqua test for persistent tasks fail.

Expected behavior

Aqua test for persistent tasks works.

Minimal Reproducible Example 👇

See https://github.com/j-fu/LSolveMWE

Pkg.test the package under 1.10 vs 1.11.

Error & Stacktrace ⚠️
This is the test output on 1.10:

┌ Warning: Circular dependency detected.
│ Precompilation will be skipped for dependencies in this cycle:
│  ┌ LinearSolve → LinearSolveSparseArraysExt
│  └─ LinearSolve → LinearSolveRecursiveArrayToolsExt
│ Precompilation will also be skipped for the following, which depend on the above cycle:
│   jl_CjwJIzUtOD
│   LSolveMWE
└ @ Pkg.API.Precompilation ~/.julia/juliaup/julia-1.10.8+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/precompilation.jl:584
┌ Error: Unexpected error: /tmp/jl_CjwJIzUtOD/done.log was not created, but precompilation exited
└ @ Aqua ~/.julia/packages/Aqua/epbUr/src/persistent_tasks.jl:118
Test Failed at /home/fuhrmann/.julia/packages/Aqua/epbUr/src/persistent_tasks.jl:38
Expression: !(has_persistent_tasks(package; kwargs...))

ERROR: LoadError: There was an error during testing
in expression starting at /home/fuhrmann/Wias/work/julia/test/LSolveMWE/test/runtests.jl:5
ERROR: Package LSolveMWE errored during testing

Environment (please complete the following information):

See the CI results in https://github.com/j-fu/LSolveMWE

Additional context

@jpthiele

[j-fu]

Ok, this seems to be related to something like this:
JuliaLang/julia#52511 (comment)

I updated the MWE to remove the SparseArrays dependency of the project, and
it is still in the manifest:

(LSolveMWE) pkg> why SparseArrays
LinearSolve → Krylov → SparseArrays
LinearSolve → LazyArrays → SparseArrays
LinearSolve → SciMLBase → RecursiveArrayTools → Statistics → SparseArrays
LinearSolve → SciMLBase → Statistics → SparseArrays
LinearSolve → SparseArrays

JuliaSmoothOptimizers/Krylov.jl#955 seems to take care of the first case...

[albertomercurio]

Same issue here.

[ufechner7]

Same issue here, in KiteModels.jl.

[oameye]

Also happens for us.

[oscardssmith]

This problem is very annoying. What happens is that on 1.11, SparseArrays isn't loaded by default, so it's possible to have a weakdep on it, but on 1.10, SparseArrays is in the sysimage so there's circularity. I think this is a Julia level issue.

[j-fu]

It seems that the problem goes away if all packages LinearSolve depends on avoid having SparseArrays as strong dependency. I tested this locally.

All packages involved (Krylov, Statistics, LazyArrays) have now PRs/Issues addressing this point.

[oscardssmith]

@ChrisRackauckas the simplest solution here would be to make RecursiveArrayTools a regular dep rather than a weakdep (which it effectively already is since SciMLBase has a strong dep on it)

[j-fu]

However, the problem pops up again if we add e.g. ILUZero or AlgebraicMultigrid. These have
to depend on SparseArrays.
So the question then is: is there anything which can be backported from 1.11 to 1.10.9 which
resolves this problem ?

EDIT: this seems to be quite nontrivial: JuliaLang/julia#56204 (comment) - it appears that there was already a backport to 1.11 ...

[j-fu]

Just another info bit: Aqua disables the corresponding test for Julia <1.10:
https://github.com/JuliaTesting/Aqua.jl/blob/3e90d046f718efaf04108913f8fb385f44026498/src/persistent_tasks.jl#L78

So a band-aid could be to disable this test as well for 1.10 and to live with the cyclic dependency warnings...