Closed
Description
Reproduced on:
- Julia 1.8.5
- Julia 1.9.2
- Julia master (specifically, commit fd38d50)
MWE
Here is an MWE for Julia 1.8.5. Run the following commands in Bash.
Click to expand:
MY_DOWNLOAD_JULIA_FUNCTION () {
cd ${WORKING_DIRECTORY:?}
rm -rf ${MY_DOWNLOADS_DIRECTORY:?}
mkdir ${MY_DOWNLOADS_DIRECTORY:?}
cd ${MY_DOWNLOADS_DIRECTORY:?}
JULIA_TARBALL_FILENAME="${JULIA_DIRECTORY:?}-${JULIA_PLATFORM:?}.tar.gz"
JULIA_TARBALL_URL="https://julialang-s3.julialang.org/bin/${JULIA_OS:?}/${JULIA_ARCH:?}/${JULIA_MAJ:?}.${JULIA_MIN:?}/${JULIA_TARBALL_FILENAME:?}"
curl --fail -L -O ${JULIA_TARBALL_URL:?}
echo "${JULIA_TARBALL_SHA256:?} ${JULIA_TARBALL_FILENAME:?}" | shasum -a 256 -c
tar xzf ${JULIA_TARBALL_FILENAME:?}
cd ${WORKING_DIRECTORY:?}
}
cd $(mktemp -d)
export JULIA_DEBUG=loading
export JULIA_PKG_PRECOMPILE_AUTO=0
WORKING_DIRECTORY="$(pwd)"
export JULIA_DEPOT_PATH="${WORKING_DIRECTORY:?}/mydepot"
MY_DOWNLOADS_DIRECTORY="${WORKING_DIRECTORY:?}/mydownloads"
JULIA_MAJ=1
JULIA_MIN=8
JULIA_PATCH=5
JULIA_OS="mac"
JULIA_ARCH="x64"
JULIA_PLATFORM=mac64
JULIA_TARBALL_SHA256=a1a859eda7fb41a0b55467339a11c3c1c0df78b27d1e160e80bc6758b3d8dae0
JULIA_DIRECTORY="julia-${JULIA_MAJ:?}.${JULIA_MIN:?}.${JULIA_PATCH:?}"
JULIA_CMD="${MY_DOWNLOADS_DIRECTORY:?}/${JULIA_DIRECTORY:?}/bin/julia"
rm -rf ${MY_DOWNLOADS_DIRECTORY:?}
MY_DOWNLOAD_JULIA_FUNCTION
MY_MBEDTLSJLLSOURCEFILE="${MY_DOWNLOADS_DIRECTORY:?}/${JULIA_DIRECTORY:?}/share/julia/stdlib/v${JULIA_MAJ:?}.${JULIA_MIN:?}/MbedTLS_jll/src/MbedTLS_jll.jl"
# Print mtime
# On macOS, use `stat -f %m`
# On Linux, use `stat -c %Y`
MY_STAT_MTIME_CMD="stat -f %m"
${MY_STAT_MTIME_CMD:?} ${MY_MBEDTLSJLLSOURCEFILE:?}
${JULIA_CMD:?} --version
${JULIA_CMD:?} -e 'import InteractiveUtils; InteractiveUtils.versioninfo()'
${JULIA_CMD:?} -e 'import Pkg; Pkg.add(; name = "HTTP", uuid = "cd3eb016-35fb-5094-929b-558a96fad6f3", version = "1.9.14")'
${JULIA_CMD:?} -e 'import Pkg; Pkg.precompile(; strict = true)'
${JULIA_CMD:?} -e 'import HTTP'
# Subsequent invocations of `import HTTP` do not have any precompilation.
${JULIA_CMD:?} -e 'import HTTP'
${JULIA_CMD:?} -e 'import HTTP'
${MY_STAT_MTIME_CMD:?} ${MY_MBEDTLSJLLSOURCEFILE:?}
touch "${MY_MBEDTLSJLLSOURCEFILE:?}"
${MY_STAT_MTIME_CMD:?} ${MY_MBEDTLSJLLSOURCEFILE:?}
# Expected behavior: no precompilation occurs
# Observed behavior: precompilation occurs for the first `import HTTP`, because
# the `mtime` has changed for one of the `.jl` source files in the stdlib.
${JULIA_CMD:?} -e 'import HTTP'
# Subsequent invocations of `import HTTP` do not have any precompilation.
${JULIA_CMD:?} -e 'import HTTP'
${JULIA_CMD:?} -e 'import HTTP'Expected behavior
No precompilation occurs.
Observed behavior
Precompilation occurs for the first import HTTP, because the mtime has changed for one of the .jl source files in the stdlib.
Here are some examples of the observed log messages:
┌ Debug: Rejecting stale cache file [...]/mydepot/compiled/v1.8/MbedTLS_jll/u5NEn_aBpJv.ji (mtime 1.673165108e9) because file /private[...]/mydownloads/julia-1.8.5/share/julia/stdlib/v1.8/MbedTLS_jll/src/MbedTLS_jll.jl (mtime 1.692064837872149e9) has changed
└ @ Base loading.jl:2155
┌ Debug: Rejecting cache file [...]/mydepot/compiled/v1.8/HTTP/zXWya_aBpJv.ji because required dependency MbedTLS_jll [c8ffd9c3-330d-5841-b78e-0817d7145fa1] with build ID 2343483797027 is missing from the cache.
└ @ Base loading.jl:1010
┌ Debug: Rejecting stale cache file [...]/mydepot/compiled/v1.8/MbedTLS_jll/u5NEn_aBpJv.ji (mtime 1.673165108e9) because file /private[...]/mydownloads/julia-1.8.5/share/julia/stdlib/v1.8/MbedTLS_jll/src/MbedTLS_jll.jl (mtime 1.692064837872149e9) has changed
└ @ Base loading.jl:2155
┌ Debug: Rejecting cache file [...]/mydepot/compiled/v1.8/MbedTLS/bf9T0_aBpJv.ji because required dependency MbedTLS_jll [c8ffd9c3-330d-5841-b78e-0817d7145fa1] with build ID 2343483797027 is missing from the cache.
└ @ Base loading.jl:1010
┌ Debug: Rejecting stale cache file [...]/mydepot/compiled/v1.8/MbedTLS_jll/u5NEn_aBpJv.ji (mtime 1.673165108e9) because file /private[...]/mydownloads/julia-1.8.5/share/julia/stdlib/v1.8/MbedTLS_jll/src/MbedTLS_jll.jl (mtime 1.692064837872149e9) has changed
└ @ Base loading.jl:2155
Motivation
I have found this to be a problem in CI environments where the Julia installation may be coming from one place (e.g. baked into a Docker image, or maybe available via the GitHub Actions toolcache), but the Julia depot .julia (including the precompilation cache at .julia/compiled) is cached via a different mechanism.