Problems using Pkg with nightly

[ctkelley]

I just tried the latest nightly and got this. Tried to import Pkgand got similar results.

%julia --startup=no
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.11.0-DEV.586 (2023-10-01)
 _/ |\__'_|_|_|\__'_|  |  Commit e9d633f70a1 (0 days old master)
|__/                   |

julia> ]
┌ Info: Precompiling Pkg [44cfe95a-1eb2-52ea-b672-e2afdf69b78f]
└ @ Base loading.jl:2351
ERROR: LoadError: ArgumentError: Package LibGit2 does not have LibGit2_jll in its dependencies:
- You may have a partially installed environment. Try `Pkg.instantiate()`
  to ensure all packages in the environment are installed.
- Or, if you have LibGit2 checked out for development and have
  added LibGit2_jll as a dependency but haven't updated your primary
  environment's manifest file, try `Pkg.resolve()`.
- Otherwise you may need to report an issue with LibGit2
Stacktrace:
 [1] macro expansion
   @ Base ./loading.jl:1777 [inlined]
 [2] macro expansion
   @ Base ./lock.jl:267 [inlined]
 [3] __require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1754
 [4] #invoke_in_world#3
   @ Base ./essentials.jl:933 [inlined]
 [5] invoke_in_world
   @ Base ./essentials.jl:930 [inlined]
 [6] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1747
 [7] include
   @ Base ./Base.jl:497 [inlined]
 [8] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
   @ Base ./loading.jl:2220
 [9] top-level scope
   @ stdin:3
in expression starting at /Applications/Julia-1.11.app/Contents/Resources/julia/share/julia/stdlib/v1.11/LibGit2/src/LibGit2.jl:3
in expression starting at stdin:3
ERROR: LoadError: Failed to precompile LibGit2 [76f85450-5226-5b5a-8eaa-529ad045b433] to "/Users/ctk/.julia/compiled/v1.11/LibGit2/jl_R6EejV".
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:35
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
    @ Base ./loading.jl:2472
  [3] compilecache
    @ Base ./loading.jl:2338 [inlined]
  [4] (::Base.var"#998#999"{Base.PkgId})()
    @ Base ./loading.jl:1972
  [5] mkpidlock(f::Base.var"#998#999"{Base.PkgId}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64, wait::Bool})
    @ FileWatching.Pidfile /Applications/Julia-1.11.app/Contents/Resources/julia/share/julia/stdlib/v1.11/FileWatching/src/pidfile.jl:93
  [6] #mkpidlock#6
    @ FileWatching.Pidfile /Applications/Julia-1.11.app/Contents/Resources/julia/share/julia/stdlib/v1.11/FileWatching/src/pidfile.jl:88 [inlined]
  [7] trymkpidlock(::Function, ::Vararg{Any}; kwargs::@Kwargs{stale_age::Int64})
    @ FileWatching.Pidfile /Applications/Julia-1.11.app/Contents/Resources/julia/share/julia/stdlib/v1.11/FileWatching/src/pidfile.jl:111
  [8] #invokelatest#2
    @ Base ./essentials.jl:901 [inlined]
  [9] invokelatest
    @ Base ./essentials.jl:896 [inlined]
 [10] maybe_cachefile_lock(f::Base.var"#998#999"{Base.PkgId}, pkg::Base.PkgId, srcpath::String; stale_age::Int64)
    @ Base ./loading.jl:2984
 [11] maybe_cachefile_lock
    @ Base ./loading.jl:2981 [inlined]
 [12] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1968
 [13] __require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1813
 [14] #invoke_in_world#3
    @ Base ./essentials.jl:933 [inlined]
 [15] invoke_in_world
    @ Base ./essentials.jl:930 [inlined]
 [16] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1804
 [17] macro expansion
    @ Base ./loading.jl:1791 [inlined]
 [18] macro expansion
    @ Base ./lock.jl:267 [inlined]
 [19] __require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1754
 [20] #invoke_in_world#3
    @ Base ./essentials.jl:933 [inlined]
 [21] invoke_in_world
    @ Base ./essentials.jl:930 [inlined]
 [22] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1747
 [23] include(mod::Module, _path::String)
    @ Base ./Base.jl:497
 [24] include(x::String)
    @ Pkg /Applications/Julia-1.11.app/Contents/Resources/julia/share/julia/stdlib/v1.11/Pkg/src/Pkg.jl:3
 [25] top-level scope
    @ /Applications/Julia-1.11.app/Contents/Resources/julia/share/julia/stdlib/v1.11/Pkg/src/Pkg.jl:59
 [26] include
    @ Base ./Base.jl:497 [inlined]
 [27] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:2220
 [28] top-level scope
    @ stdin:3
in expression starting at /Applications/Julia-1.11.app/Contents/Resources/julia/share/julia/stdlib/v1.11/Pkg/src/GitTools.jl:3
in expression starting at /Applications/Julia-1.11.app/Contents/Resources/julia/share/julia/stdlib/v1.11/Pkg/src/Pkg.jl:3
in expression starting at stdin:3
┌ Error: Error in the keymap
│   exception = (ErrorException("Failed to precompile Pkg [44cfe95a-1eb2-52ea-b672-e2afdf69b78f] to \"/Users/ctk/.julia/compiled/v1.11/Pkg/jl_8mdno2\"."), Union{Ptr{Nothing}, Base.InterpreterIP}[Ptr{Nothing} @0x0000000115f77187, Ptr{Nothing} @0x00000001157cc16b, Ptr{Nothing} @0x000000011560a27f, Ptr{Nothing} @0x000000010977c0f7, Ptr{Nothing} @0x000000010977c1cf, Ptr{Nothing} @0x0000000104f93123, Ptr{Nothing} @0x0000000104cc80fb, Ptr{Nothing} @0x0000000104cd004f, Ptr{Nothing} @0x0000000104cd009f, Ptr{Nothing} @0x0000000104f93123, Ptr{Nothing} @0x0000000104fa1543, Ptr{Nothing} @0x0000000104cbc18f, Ptr{Nothing} @0x0000000104f93123, Ptr{Nothing} @0x0000000104fa1543, Ptr{Nothing} @0x0000000104cb4113, Ptr{Nothing} @0x0000000104f93123, Ptr{Nothing} @0x0000000104fa1c7b, Ptr{Nothing} @0x000000011602c45b, Ptr{Nothing} @0x00000001163bb22f, Ptr{Nothing} @0x0000000115b52d43, Ptr{Nothing} @0x0000000115f8e82f, Ptr{Nothing} @0x0000000104f931ab, Ptr{Nothing} @0x0000000104fa1d0b, Ptr{Nothing} @0x0000000115a49e53, Ptr{Nothing} @0x000000011e6afe2b, Ptr{Nothing} @0x0000000104f931ab, Ptr{Nothing} @0x0000000104fa1c7b, Ptr{Nothing} @0x000000011e637287, Ptr{Nothing} @0x000000011e5cba5f, Ptr{Nothing} @0x0000000104f931ab, Ptr{Nothing} @0x000000011e572383, Ptr{Nothing} @0x000000011e572cef, Ptr{Nothing} @0x000000011e6bff3f, Ptr{Nothing} @0x0000000104f931ab, Ptr{Nothing} @0x000000011e57126f, Ptr{Nothing} @0x000000011e5e90e3, Ptr{Nothing} @0x000000011e64d2f3, Ptr{Nothing} @0x0000000104f931ab, Ptr{Nothing} @0x0000000104fb31ff])
└ @ REPL.LineEdit /Applications/Julia-1.11.app/Contents/Resources/julia/share/julia/stdlib/v1.11/REPL/src/LineEdit.jl:2809

[jishnub]

You may need to remove ~/.Julia/environments/v1.11 and try again. I think this should fix it. Perhaps removing the manifest and resolving packages may be enough

[KristofferC]

It should work around it yes, but it is still unclear to me why Pkg would have to recompile at all in this situation.

[Seelengrab]

You may need to remove ~/.Julia/environments/v1.11 and try again. I think this should fix it. Perhaps removing the manifest and resolving packages may be enough

That does not work for me; just import Pkg in the default environment also triggers this for me.

EDIT: I had not removed the environment completely; Libgit2 was still in there. Seems like that indeed was the cause.

[Seelengrab]

This seems to be a bigger issue - I get the same kind of problem when I try to run the tests of one of my packages on nightly. The package has a Project.toml.

[KristofferC]

This seems to be a bigger issue - I get the same kind of problem when I try to run the tests of one of my packages on nightly. The package has a Project.toml.

Link to log?

[Seelengrab]

Working on a bisect, I'll link a gist when I'm done. I'm not sure if it happens in CI too, I've only noticed it when manually running ]test in the REPL so far.

[KristofferC]

The bisect is 99% when Pkg got moved out of the sysimage (and is in combination with #51294).

[Seelengrab]

The specific issue I'm seeing is bisected to #51399

Gist for the log I'm seeing is here. The package in question is this, but I suspect it doesn't matter; likely any testsuite based on Project.toml will do, when triggered from the REPL.

[vchuravy]

Duplicate of #51452?

[denius]

After applying the latest backport for version 1.10 #50971 the same problem occurs when building julia. Removing .julia/environments/v1.10/Manifest.toml is the solution.

[ctkelley]

Just tried

Version 1.11.0-DEV.590

on an Intel Mac (Sonoma) and all is well after I delete

.julia/environments/v1.11/Manifest.toml

Will try on an M2 Mac later today.

Just did it and it works. Look like this problem is fixed (at least for me).