Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tensorflow-core: Remove build-dep on cython #38083

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

dannf
Copy link
Contributor

@dannf dannf commented Dec 20, 2024

tensorflow-core is the last remaining package that uses cython~0, so I checked to see if it was compatible with current cython. But, as far as I could tell, cython is simply never used.

To confirm, I did a build of tensorflow-core where I touched a reference file after the build-deps (including cython) had been installed and then, after each (sub-)package build, ran:

find $(apk info -L cython | grep -v ^cython | grep -v '^$' | sed 's/^/\//') -anewer /tmp/reference | sed 's/^/DANNF: /'

I found no 'DANNF:' lines in the output. This tells me that nothing in the build accessed any files provided by the cython package.

See: #37272

tensorflow-core is the last remaining package that uses cython~0, so I
checked to see if it was compatible with current cython. But, as far
as I could tell, cython is simply never used.

To confirm, I did a build of tensorflow-core where I touched a reference
file after the build-deps (including cython) had been installed and then,
after each package build, ran:

find $(apk info -L cython | grep -v ^cython | grep -v '^$' | sed 's/^/\//') -anewer /tmp/reference | sed 's/^/DANNF: /'

I found no 'DANNF:' lines in the output. This tells me that nothing
in the build accessed any files provided by the cython package.

Signed-off-by: dann frazier <dann.frazier@chainguard.dev>
@dannf dannf self-assigned this Dec 20, 2024
@dannf dannf added eng:os approved-to-run A repo member has approved this external contribution labels Dec 20, 2024
@octo-sts octo-sts bot added the bincapz/pass bincapz/pass Bincapz (aka. malcontent) scan didn't detect any CRITICALs on the scanned packages. label Dec 20, 2024
@@ -85,7 +84,7 @@ environment:
TF_NEED_NGRAPH: 0
TF_NEED_IGNITE: 0
TF_NEED_ROCM: 0
TF_SYSTEM_LIBS: "boringssl,curl,gif,icu,libjpeg_turbo,nasm,png,zlib,cython"
TF_SYSTEM_LIBS: "boringssl,curl,gif,icu,libjpeg_turbo,nasm,png,zlib"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this causes to use precompiled cython files or download old cython?

But yes it would be great to get rid of old cython!!!!! Would want to double check this analysis when I am on my laptop.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't see any references to cython in the logs, nor did I see any when I grep'd a local build log w/ bazel --subcommands:

$ sed 's/TF_SYSTEM_LIBS=boringssl,curl,cython,gif,icu,libjpeg_turbo,nasm,png,zlib//' < tensorflow+subcommands.log  | grep -i cython
2024/12/27 19:30:17 INFO     packages:     [alsa-lib=1.2.13-r0 apk-tools=2.14.4-r1 bash=5.2.37-r2 bazelisk-default=1.25.0-r0 bazelisk=1.25.0-r0 binutils=2.43.1-r2 brotli-dev=1.1.0-r4 brotli=1.1.0-r4 build-base=1-r8 busybox=1.37.0-r0 c-ares-dev=1.34.4-r0 c-ares=1.34.4-r0 ca-certificates-bundle=20241010-r2 ca-certificates=20241010-r2 clang-16-dev=16.0.6-r6 clang-16=16.0.6-r6 coreutils=9.5-r3 curl-dev=8.11.1-r0 cyrus-sasl=2.1.28-r5 cython-0=0.29.37.1-r2 e2fsprogs-dev=1.47.1-r1 e2fsprogs-libs=1.47.1-r1 fontconfig-config=2.15.0-r2 freetype=2.13.3-r0 gawk=5.3.1-r1 gcc-12-default=12.4.0-r7 gcc-12=12.4.0-r7 gdbm=1.24-r1 giflib-dev=5.2.2-r4 giflib=5.2.2-r4 git=2.47.1-r0 glib=2.83.2-r1 glibc-dev=2.40-r3 glibc-locale-posix=2.40-r3 glibc=2.40-r3 gmp=6.3.0-r2 hdf5-cpp=1.14.5-r0 hdf5-dev=1.14.5-r0 hdf5-fortran=1.14.5-r0 hdf5-hl-cpp=1.14.5-r0 hdf5-hl-fortran=1.14.5-r0 hdf5-hl=1.14.5-r0 hdf5=1.14.5-r0 heimdal-libs=7.8.0-r7 icu-data-full=75.1-r5 icu-dev=75.1-r5 icu-libs=75.1-r5 icu=75.1-r5 isl-dev=0.27-r0 isl=0.27-r0 java-cacerts=20240705-r0 java-common=0.1-r3 jitterentropy-library-dev=3.6.1-r0 jitterentropy-library=3.6.1-r0 keyutils-libs=1.6.3-r5 krb5-conf=1.0-r3 krb5-dev=1.21.3-r2 krb5-libs=1.21.3-r2 krb5-server-ldap=1.21.3-r2 lcms2=2.16-r3 ld-linux=2.40-r3 libLLVM-16=16.0.6-r3 libacl1=2.3.2-r4 libattr1=2.5.2-r4 libblkid=2.40.2-r1 libbrotlicommon1=1.1.0-r4 libbrotlidec1=1.1.0-r4 libbrotlienc1=1.1.0-r4 libbz2-1=1.0.8-r9 libclang-cpp-16=16.0.6-r6 libcom_err=1.47.1-r1 libcrypt1=2.40-r3 libcrypto3=3.4.0-r5 libcurl-openssl4=8.11.1-r0 libev=4.33-r6 libevent=2.1.12-r6 libexpat1=2.6.4-r0 libfdisk=2.40.2-r1 libffi=3.4.6-r5 libfontconfig1=2.15.0-r2 libgcc=14.2.0-r6 libgfortran=14.2.0-r6 libidn2=2.3.7-r3 libjpeg-turbo-dev=3.1.0-r0 libjpeg-turbo=3.1.0-r0 libldap=2.6.9-r0 liblzf=3.6-r1 libmount=2.40.2-r1 libnghttp2-14=1.64.0-r1 libpcre2-8-0=10.44-r2 libpng-dev=1.6.44-r3 libpng=1.6.44-r3 libpsl-dev=0.21.5-r4 libpsl=0.21.5-r4 libquadmath=14.2.0-r6 libsmartcols=2.40.2-r1 libssl3=3.4.0-r5 libstdc++-12-dev=12.4.0-r7 libstdc++-12=12.4.0-r7 libstdc++=14.2.0-r6 libsz=1.1.3-r1 libtasn1=4.19.0-r4 libunistring=1.3-r1 libuuid=2.40.2-r1 libverto-dev=0.3.2-r4 libverto-glib=0.3.2-r4 libverto-libev=0.3.2-r4 libverto-libevent=0.3.2-r4 libverto=0.3.2-r4 libx11=1.8.10-r0 libxau=1.0.12-r0 libxcb=1.17.0-r3 libxcrypt-dev=4.4.36-r8 libxcrypt=4.4.36-r8 libxdmcp=1.1.5-r4 libxext=1.3.6-r4 libxi=1.8.2-r1 libxml2=2.13.5-r0 libxrender=0.9.12-r0 libxtst=1.2.5-r1 libzstd1=1.5.6-r5 linux-headers=6.6.67-r0 llvm-lld-16-dev=16.0.6-r7 llvm-lld-16=16.0.6-r7 llvm16-dev=16.0.6-r3 llvm16=16.0.6-r3 make=4.4.1-r4 mpc-dev=1.3.1-r5 mpc=1.3.1-r5 mpdecimal=4.0.0-r3 mpfr=4.2.1-r5 ncurses-terminfo-base=6.5_p20241006-r4 ncurses=6.5_p20241006-r4 nghttp2-dev=1.64.0-r1 nghttp2=1.64.0-r1 nss-db=2.40-r3 nss-hesiod=2.40-r3 openblas=0.3.28-r1 openjdk-11-default-jdk=11.0.25-r2 openjdk-11-default-jvm=11.0.25-r2 openjdk-11-jre-base=11.0.25-r2 openjdk-11-jre=11.0.25-r2 openjdk-11=11.0.25-r2 openssf-compiler-options=20240627-r6 openssl-dev=3.4.0-r5 p11-kit-trust=0.25.5-r1 p11-kit=0.25.5-r1 patchelf=0.18.0-r1 perl=5.40.0-r3 pkgconf=2.3.0-r1 posix-cc-wrappers=1-r4 py3-pip-wheel=24.3.1-r1 py3-setuptools-wheel=75.6.0-r0 py3-supported-gpep517=16-r4 py3-supported-installer=0.7.0-r10 py3-supported-keras-applications=1.0.8-r3 py3-supported-keras-preprocessing=1.1.2-r4 py3-supported-packaging=24.2-r1 py3-supported-pybind11=2.13.6-r3 py3-supported-requests=2.32.3-r4 py3-supported-setuptools=75.6.0-r1 py3.10-certifi=2024.12.14-r0 py3.10-charset-normalizer=3.4.1-r0 py3.10-gpep517=16-r4 py3.10-h5py=3.12.1-r3 py3.10-idna=3.10-r3 py3.10-installer=0.7.0-r10 py3.10-keras-applications=1.0.8-r3 py3.10-keras-preprocessing=1.1.2-r4 py3.10-numpy=1.26.4-r7 py3.10-packaging=24.2-r1 py3.10-pybind11=2.13.6-r1 py3.10-pyparsing=3.2.0-r2 py3.10-requests=2.32.3-r4 py3.10-setuptools=75.6.0-r1 py3.10-six=1.17.0-r1 py3.10-tomli=2.2.1-r1 py3.10-urllib3=2.3.0-r0 py3.11-certifi=2024.12.14-r0 py3.11-charset-normalizer=3.4.1-r0 py3.11-gpep517=16-r4 py3.11-h5py=3.12.1-r3 py3.11-idna=3.10-r3 py3.11-installer=0.7.0-r10 py3.11-keras-applications=1.0.8-r3 py3.11-keras-preprocessing=1.1.2-r4 py3.11-numpy=1.26.4-r7 py3.11-packaging=24.2-r1 py3.11-pybind11=2.13.6-r1 py3.11-pyparsing=3.2.0-r2 py3.11-requests=2.32.3-r4 py3.11-setuptools=75.6.0-r1 py3.11-six=1.17.0-r1 py3.11-tomli=2.2.1-r0 py3.11-urllib3=2.3.0-r0 py3.12-certifi=2024.12.14-r0 py3.12-charset-normalizer=3.4.1-r0 py3.12-gpep517=16-r4 py3.12-h5py=3.12.1-r3 py3.12-idna=3.10-r3 py3.12-installer=0.7.0-r9 py3.12-keras-applications=1.0.8-r3 py3.12-keras-preprocessing=1.1.2-r3 py3.12-numpy=1.26.4-r7 py3.12-packaging=24.2-r1 py3.12-pybind11=2.13.6-r1 py3.12-pyparsing=3.2.0-r2 py3.12-requests=2.32.3-r4 py3.12-setuptools=75.6.0-r1 py3.12-six=1.17.0-r1 py3.12-tomli=2.2.1-r1 py3.12-urllib3=2.3.0-r0 py3.13-certifi=2024.12.14-r0 py3.13-charset-normalizer=3.4.1-r0 py3.13-gpep517=16-r4 py3.13-h5py=3.12.1-r3 py3.13-idna=3.10-r3 py3.13-installer=0.7.0-r10 py3.13-keras-applications=1.0.8-r3 py3.13-keras-preprocessing=1.1.2-r4 py3.13-numpy=1.26.4-r7 py3.13-packaging=24.2-r1 py3.13-pybind11=2.13.6-r3 py3.13-pyparsing=3.2.0-r2 py3.13-requests=2.32.3-r4 py3.13-setuptools=75.6.0-r1 py3.13-six=1.17.0-r1 py3.13-tomli=2.2.1-r1 py3.13-urllib3=2.3.0-r0 py3.13-wheel-bin=0.45.1-r1 py3.13-wheel=0.45.1-r1 python-3.10-base=3.10.16-r0 python-3.11-base=3.11.11-r0 python-3.12-base=3.12.8-r2 python-3.13-base-dev=3.13.1-r4 python-3.13-base=3.13.1-r4 python-3.13-dev=3.13.1-r4 python-3.13=3.13.1-r4 readline=8.2.13-r1 scanelf=1.3.8-r1 sqlite-dev=3.47.2-r0 sqlite-libs=3.47.2-r0 sqlite=3.47.2-r0 ttf-dejavu=2.37-r3 util-linux-dev=2.40.2-r1 util-linux=2.40.2-r1 wolfi-base=1-r6 wolfi-baselayout=20230201-r15 wolfi-keys=1-r8 xz=5.6.3-r2 zlib-dev=1.3.1-r4 zlib=1.3.1-r4]
2024/12/27 19:30:20 INFO installing cython-0 (0.29.37.1-r2)

Here's that full build log:
tensorflow+subcommands.log.gz

Note that the version of cython this build would download, if needed, is 3.0.3:

tensorflow/workspace2.bzl:        urls = tf_mirror_urls("https://github.com/cython/cython/archive/3.0.3.tar.gz"),
third_party/xla/third_party/tsl/workspace2.bzl:        urls = tf_mirror_urls("https://github.com/cython/cython/archive/3.0.3.tar.gz"),

So a more cautious approach would be to leave the build-dep in place, but just drop the ~0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved-to-run A repo member has approved this external contribution bincapz/pass bincapz/pass Bincapz (aka. malcontent) scan didn't detect any CRITICALs on the scanned packages. eng:os
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants