-
Notifications
You must be signed in to change notification settings - Fork 278
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
conda: avoid py3.x-conda cross-contamination #38266
Conversation
Anything in the top-level directory of this package will end up getting installed into each version of the python module. We already use the `prevent-inclusion` parameter to prevent those files from being copied in. Now that we're using "cleanup" as a holding area, we need to include that as well. This was causing the usr/bin/* files from all previously built modules to be included in subsequent ones: # apk info -L py3.12-conda | grep cleanup WARNING: opening /work/packages: No such file or directory usr/lib/python3.12/site-packages/cleanup/3.10/bin/conda usr/lib/python3.12/site-packages/cleanup/3.11/bin/conda usr/lib/python3.12/site-packages/melange-out/py3.11-conda/usr/lib/python3.11/site-packages/cleanup/3.10/bin/conda And, because melange adds dependencies for paths in the shebangs, we were getting dependencies on additional python versions. Signed-off-by: dann frazier <dann.frazier@chainguard.dev>
- runs: | | ||
# We don't use this dir until further down, but py/pip-build-install | ||
# will fail if a parameter to its `prevent-inclusion:` does not exist | ||
mkdir -p ./cleanup |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we not modify py/pip-build-install
rather than potentially modifying multiple packages in the same way?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had considered it a feature that this pipeline failed when given bogus input because, in most cases, it seems like an absent path would be due to a mistake. What if instead we detect this case and emit a warning? Something like this not-yet-tested change:
diff --git a/pipelines/py/pip-build-install.yaml b/pipelines/py/pip-build-install.yaml
index 8cfa9d5a5..6b6a4bf01 100644
--- a/pipelines/py/pip-build-install.yaml
+++ b/pipelines/py/pip-build-install.yaml
@@ -92,13 +92,23 @@ pipeline:
fi
fi
+ add_prevent() {
+ if [ -e "$1" ]; then
+ ( vr tar -Apf "$2" && vr rm -rf "$1" ) ||
+ { echo "ERROR: failed adding $1 to prevent-inclusion.tar"; exit 1; }
+ echo "prevented-inclusion of $1"
+ return
+ fi
+ echo "WARNING: prevents-inclusion path $1 not found, ignoring."
+ }
+
+ pitar="$tmpd/prevent-inclusion.tar"
prevents="${{inputs.prevent-inclusion}}"
if [ -n "$prevents" ]; then
+ # initialize empty tar file
+ tar -cf "$pitar" -T /dev/null
# do not allow expansion of prevents
- ( set -f; vr tar -cpf "$tmpd/prevent-inclusion.tar" $prevents &&
- vr rm -rf $prevents ) ||
- { echo "ERROR: failed creation of prevent-inclusion.tar with $prevents"; exit 1; }
- echo "prevented-inclusion of $prevents"
+ ( set -f; for p in $prevents; do add_prevent "$p" "$pitar"; done )
fi
[ -d build ] && hadbuild=true || hadbuild=false
Anything in the top-level directory of this package will end up getting installed into each version of the python module. We already use the
prevent-inclusion
parameter to prevent those files from being copied in. Now that we're using "cleanup" as a holding area, we need to include that as well.This was causing the usr/bin/* files from all previously built modules to be included in subsequent ones:
And, because melange adds dependencies for paths in the shebangs, we were getting dependencies on additional python versions.