-
Notifications
You must be signed in to change notification settings - Fork 278
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
tensorflow-core: Remove build-dep on cython #38083
base: main
Are you sure you want to change the base?
Conversation
tensorflow-core is the last remaining package that uses cython~0, so I checked to see if it was compatible with current cython. But, as far as I could tell, cython is simply never used. To confirm, I did a build of tensorflow-core where I touched a reference file after the build-deps (including cython) had been installed and then, after each package build, ran: find $(apk info -L cython | grep -v ^cython | grep -v '^$' | sed 's/^/\//') -anewer /tmp/reference | sed 's/^/DANNF: /' I found no 'DANNF:' lines in the output. This tells me that nothing in the build accessed any files provided by the cython package. Signed-off-by: dann frazier <dann.frazier@chainguard.dev>
@@ -85,7 +84,7 @@ environment: | |||
TF_NEED_NGRAPH: 0 | |||
TF_NEED_IGNITE: 0 | |||
TF_NEED_ROCM: 0 | |||
TF_SYSTEM_LIBS: "boringssl,curl,gif,icu,libjpeg_turbo,nasm,png,zlib,cython" | |||
TF_SYSTEM_LIBS: "boringssl,curl,gif,icu,libjpeg_turbo,nasm,png,zlib" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doesn't this causes to use precompiled cython files or download old cython?
But yes it would be great to get rid of old cython!!!!! Would want to double check this analysis when I am on my laptop.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't see any references to cython in the logs, nor did I see any when I grep'd a local build log w/ bazel --subcommands
:
$ sed 's/TF_SYSTEM_LIBS=boringssl,curl,cython,gif,icu,libjpeg_turbo,nasm,png,zlib//' < tensorflow+subcommands.log | grep -i cython
2024/12/27 19:30:17 INFO packages: [alsa-lib=1.2.13-r0 apk-tools=2.14.4-r1 bash=5.2.37-r2 bazelisk-default=1.25.0-r0 bazelisk=1.25.0-r0 binutils=2.43.1-r2 brotli-dev=1.1.0-r4 brotli=1.1.0-r4 build-base=1-r8 busybox=1.37.0-r0 c-ares-dev=1.34.4-r0 c-ares=1.34.4-r0 ca-certificates-bundle=20241010-r2 ca-certificates=20241010-r2 clang-16-dev=16.0.6-r6 clang-16=16.0.6-r6 coreutils=9.5-r3 curl-dev=8.11.1-r0 cyrus-sasl=2.1.28-r5 cython-0=0.29.37.1-r2 e2fsprogs-dev=1.47.1-r1 e2fsprogs-libs=1.47.1-r1 fontconfig-config=2.15.0-r2 freetype=2.13.3-r0 gawk=5.3.1-r1 gcc-12-default=12.4.0-r7 gcc-12=12.4.0-r7 gdbm=1.24-r1 giflib-dev=5.2.2-r4 giflib=5.2.2-r4 git=2.47.1-r0 glib=2.83.2-r1 glibc-dev=2.40-r3 glibc-locale-posix=2.40-r3 glibc=2.40-r3 gmp=6.3.0-r2 hdf5-cpp=1.14.5-r0 hdf5-dev=1.14.5-r0 hdf5-fortran=1.14.5-r0 hdf5-hl-cpp=1.14.5-r0 hdf5-hl-fortran=1.14.5-r0 hdf5-hl=1.14.5-r0 hdf5=1.14.5-r0 heimdal-libs=7.8.0-r7 icu-data-full=75.1-r5 icu-dev=75.1-r5 icu-libs=75.1-r5 icu=75.1-r5 isl-dev=0.27-r0 isl=0.27-r0 java-cacerts=20240705-r0 java-common=0.1-r3 jitterentropy-library-dev=3.6.1-r0 jitterentropy-library=3.6.1-r0 keyutils-libs=1.6.3-r5 krb5-conf=1.0-r3 krb5-dev=1.21.3-r2 krb5-libs=1.21.3-r2 krb5-server-ldap=1.21.3-r2 lcms2=2.16-r3 ld-linux=2.40-r3 libLLVM-16=16.0.6-r3 libacl1=2.3.2-r4 libattr1=2.5.2-r4 libblkid=2.40.2-r1 libbrotlicommon1=1.1.0-r4 libbrotlidec1=1.1.0-r4 libbrotlienc1=1.1.0-r4 libbz2-1=1.0.8-r9 libclang-cpp-16=16.0.6-r6 libcom_err=1.47.1-r1 libcrypt1=2.40-r3 libcrypto3=3.4.0-r5 libcurl-openssl4=8.11.1-r0 libev=4.33-r6 libevent=2.1.12-r6 libexpat1=2.6.4-r0 libfdisk=2.40.2-r1 libffi=3.4.6-r5 libfontconfig1=2.15.0-r2 libgcc=14.2.0-r6 libgfortran=14.2.0-r6 libidn2=2.3.7-r3 libjpeg-turbo-dev=3.1.0-r0 libjpeg-turbo=3.1.0-r0 libldap=2.6.9-r0 liblzf=3.6-r1 libmount=2.40.2-r1 libnghttp2-14=1.64.0-r1 libpcre2-8-0=10.44-r2 libpng-dev=1.6.44-r3 libpng=1.6.44-r3 libpsl-dev=0.21.5-r4 libpsl=0.21.5-r4 libquadmath=14.2.0-r6 libsmartcols=2.40.2-r1 libssl3=3.4.0-r5 libstdc++-12-dev=12.4.0-r7 libstdc++-12=12.4.0-r7 libstdc++=14.2.0-r6 libsz=1.1.3-r1 libtasn1=4.19.0-r4 libunistring=1.3-r1 libuuid=2.40.2-r1 libverto-dev=0.3.2-r4 libverto-glib=0.3.2-r4 libverto-libev=0.3.2-r4 libverto-libevent=0.3.2-r4 libverto=0.3.2-r4 libx11=1.8.10-r0 libxau=1.0.12-r0 libxcb=1.17.0-r3 libxcrypt-dev=4.4.36-r8 libxcrypt=4.4.36-r8 libxdmcp=1.1.5-r4 libxext=1.3.6-r4 libxi=1.8.2-r1 libxml2=2.13.5-r0 libxrender=0.9.12-r0 libxtst=1.2.5-r1 libzstd1=1.5.6-r5 linux-headers=6.6.67-r0 llvm-lld-16-dev=16.0.6-r7 llvm-lld-16=16.0.6-r7 llvm16-dev=16.0.6-r3 llvm16=16.0.6-r3 make=4.4.1-r4 mpc-dev=1.3.1-r5 mpc=1.3.1-r5 mpdecimal=4.0.0-r3 mpfr=4.2.1-r5 ncurses-terminfo-base=6.5_p20241006-r4 ncurses=6.5_p20241006-r4 nghttp2-dev=1.64.0-r1 nghttp2=1.64.0-r1 nss-db=2.40-r3 nss-hesiod=2.40-r3 openblas=0.3.28-r1 openjdk-11-default-jdk=11.0.25-r2 openjdk-11-default-jvm=11.0.25-r2 openjdk-11-jre-base=11.0.25-r2 openjdk-11-jre=11.0.25-r2 openjdk-11=11.0.25-r2 openssf-compiler-options=20240627-r6 openssl-dev=3.4.0-r5 p11-kit-trust=0.25.5-r1 p11-kit=0.25.5-r1 patchelf=0.18.0-r1 perl=5.40.0-r3 pkgconf=2.3.0-r1 posix-cc-wrappers=1-r4 py3-pip-wheel=24.3.1-r1 py3-setuptools-wheel=75.6.0-r0 py3-supported-gpep517=16-r4 py3-supported-installer=0.7.0-r10 py3-supported-keras-applications=1.0.8-r3 py3-supported-keras-preprocessing=1.1.2-r4 py3-supported-packaging=24.2-r1 py3-supported-pybind11=2.13.6-r3 py3-supported-requests=2.32.3-r4 py3-supported-setuptools=75.6.0-r1 py3.10-certifi=2024.12.14-r0 py3.10-charset-normalizer=3.4.1-r0 py3.10-gpep517=16-r4 py3.10-h5py=3.12.1-r3 py3.10-idna=3.10-r3 py3.10-installer=0.7.0-r10 py3.10-keras-applications=1.0.8-r3 py3.10-keras-preprocessing=1.1.2-r4 py3.10-numpy=1.26.4-r7 py3.10-packaging=24.2-r1 py3.10-pybind11=2.13.6-r1 py3.10-pyparsing=3.2.0-r2 py3.10-requests=2.32.3-r4 py3.10-setuptools=75.6.0-r1 py3.10-six=1.17.0-r1 py3.10-tomli=2.2.1-r1 py3.10-urllib3=2.3.0-r0 py3.11-certifi=2024.12.14-r0 py3.11-charset-normalizer=3.4.1-r0 py3.11-gpep517=16-r4 py3.11-h5py=3.12.1-r3 py3.11-idna=3.10-r3 py3.11-installer=0.7.0-r10 py3.11-keras-applications=1.0.8-r3 py3.11-keras-preprocessing=1.1.2-r4 py3.11-numpy=1.26.4-r7 py3.11-packaging=24.2-r1 py3.11-pybind11=2.13.6-r1 py3.11-pyparsing=3.2.0-r2 py3.11-requests=2.32.3-r4 py3.11-setuptools=75.6.0-r1 py3.11-six=1.17.0-r1 py3.11-tomli=2.2.1-r0 py3.11-urllib3=2.3.0-r0 py3.12-certifi=2024.12.14-r0 py3.12-charset-normalizer=3.4.1-r0 py3.12-gpep517=16-r4 py3.12-h5py=3.12.1-r3 py3.12-idna=3.10-r3 py3.12-installer=0.7.0-r9 py3.12-keras-applications=1.0.8-r3 py3.12-keras-preprocessing=1.1.2-r3 py3.12-numpy=1.26.4-r7 py3.12-packaging=24.2-r1 py3.12-pybind11=2.13.6-r1 py3.12-pyparsing=3.2.0-r2 py3.12-requests=2.32.3-r4 py3.12-setuptools=75.6.0-r1 py3.12-six=1.17.0-r1 py3.12-tomli=2.2.1-r1 py3.12-urllib3=2.3.0-r0 py3.13-certifi=2024.12.14-r0 py3.13-charset-normalizer=3.4.1-r0 py3.13-gpep517=16-r4 py3.13-h5py=3.12.1-r3 py3.13-idna=3.10-r3 py3.13-installer=0.7.0-r10 py3.13-keras-applications=1.0.8-r3 py3.13-keras-preprocessing=1.1.2-r4 py3.13-numpy=1.26.4-r7 py3.13-packaging=24.2-r1 py3.13-pybind11=2.13.6-r3 py3.13-pyparsing=3.2.0-r2 py3.13-requests=2.32.3-r4 py3.13-setuptools=75.6.0-r1 py3.13-six=1.17.0-r1 py3.13-tomli=2.2.1-r1 py3.13-urllib3=2.3.0-r0 py3.13-wheel-bin=0.45.1-r1 py3.13-wheel=0.45.1-r1 python-3.10-base=3.10.16-r0 python-3.11-base=3.11.11-r0 python-3.12-base=3.12.8-r2 python-3.13-base-dev=3.13.1-r4 python-3.13-base=3.13.1-r4 python-3.13-dev=3.13.1-r4 python-3.13=3.13.1-r4 readline=8.2.13-r1 scanelf=1.3.8-r1 sqlite-dev=3.47.2-r0 sqlite-libs=3.47.2-r0 sqlite=3.47.2-r0 ttf-dejavu=2.37-r3 util-linux-dev=2.40.2-r1 util-linux=2.40.2-r1 wolfi-base=1-r6 wolfi-baselayout=20230201-r15 wolfi-keys=1-r8 xz=5.6.3-r2 zlib-dev=1.3.1-r4 zlib=1.3.1-r4]
2024/12/27 19:30:20 INFO installing cython-0 (0.29.37.1-r2)
Here's that full build log:
tensorflow+subcommands.log.gz
Note that the version of cython
this build would download, if needed, is 3.0.3:
tensorflow/workspace2.bzl: urls = tf_mirror_urls("https://github.com/cython/cython/archive/3.0.3.tar.gz"),
third_party/xla/third_party/tsl/workspace2.bzl: urls = tf_mirror_urls("https://github.com/cython/cython/archive/3.0.3.tar.gz"),
So a more cautious approach would be to leave the build-dep in place, but just drop the ~0
.
tensorflow-core is the last remaining package that uses cython~0, so I checked to see if it was compatible with current cython. But, as far as I could tell, cython is simply never used.
To confirm, I did a build of tensorflow-core where I touched a reference file after the build-deps (including cython) had been installed and then, after each (sub-)package build, ran:
I found no 'DANNF:' lines in the output. This tells me that nothing in the build accessed any files provided by the cython package.
See: #37272