Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pkg-config version in x86-linux and x86_64-linux images taking a long time to resolve re2.pc #121

Closed
mudge opened this issue Jun 3, 2024 · 4 comments

Comments

@mudge
Copy link

mudge commented Jun 3, 2024

RE2 2024-06-01 uses more of Abseil which has caused the amount of time spent running pkg-config --libs-only-L --static re2.pc within the rake-compiler-dock images for x86-linux and x86_64-linux to significantly increase.

Both images use CentOS 7's latest version of pkgconfig: 0.27.1 but the latest version available is 0.29.2.

By manually downloading, compiling and installing the latest version of pkg-config on ghcr.io/rake-compiler/rake-compiler-dock-image:1.5.0-mri-x86-linux we can compare the difference:

# /usr/bin/pkg-config --version
0.27.1
# time /usr/bin/pkg-config --libs-only-L --static re2.pc
-L/Users/mudge/Projects/re2/ports/arm64-apple-darwin23/abseil/20240116.2/lib -L/Users/mudge/Projects/re2/ports/arm64-apple-darwin23/libre2/2024-06-01/lib  

real	3m41.007s
user	3m40.990s
sys	0m0.025s
# /usr/local/bin/pkg-config --version
0.29.2
# time /usr/local/bin/pkg-config --libs-only-L --static re2.pc
-L/Users/mudge/Projects/re2/ports/arm64-apple-darwin23/libre2/2024-06-01/lib -L/Users/mudge/Projects/re2/ports/arm64-apple-darwin23/abseil/20240116.2/lib

real	0m0.101s
user	0m0.083s
sys	0m0.018s

Is there a way for us to provide a newer version of pkg-config in the rake-compiler-dock images to work around this slowdown?

@mudge
Copy link
Author

mudge commented Jun 3, 2024

If you want to reproduce this:

  1. Check out v2.12.0 of the re2 gem
  2. Run bundle install
  3. Run bundle exec rake compile
  4. Start a Docker container with ports mounted, e.g. from within the re2 clone: docker run -v "$(PWD):/re2" --platform linux/amd64 -it ghcr.io/rake-compiler/rake-compiler-dock-image:1.5.0-mri-x86-linux /bin/bash
  5. Set PKG_CONFIG_PATH inside the Docker container to include both Abseil and RE2, e.g. assuming you mounted the clone in /re2, export PKG_CONFIG_PATH=/re2/ports/arm64-apple-darwin23/abseil/20240116.2/lib/pkgconfig:/re2/ports/arm64-apple-darwin23/libre2/2024-06-01/lib/pkgconfig:$PKG_CONFIG_PATH
  6. Run time pkg-config --libs-only-L --static /re2/ports/arm64-apple-darwin23/libre2/2024-06-01/lib/pkgconfig/re2.pc

@flavorjones
Copy link
Collaborator

flavorjones commented Jun 3, 2024

Fascinatin, that's an incredible speedup for sure. I know that 0.28 and 0.29 addressed some recursive path issues, so I'll assume that's why it's faster.

Seems like we've got two options here ...

  1. on the manylinux_2014 images, try to upgrade pkg-config to a modern version
  2. drop manylinux_2014 in favor of a newer version which presumably will have a modern pkg-config

I was planning on doing (2) soon anyway, since the 2014 series is EOL as of 2024-06-30 (later this month!). See https://github.com/pypa/manylinux for details, and #122 for the issue describing this change in more detail.

Let me mess around with (1), though, since I could get that into a patch release on the 1.5.x branch, while (2) would really require a minor release.

@flavorjones
Copy link
Collaborator

Possible approach to (1) above is in #123

@flavorjones
Copy link
Collaborator

Closed in #123, will ship v1.5.1 promptly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants