Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

move x86_64-linux to be based on ubuntu:20.04 #126

Merged
merged 5 commits into from
Aug 21, 2024

Conversation

flavorjones
Copy link
Collaborator

@flavorjones flavorjones commented Jul 29, 2024

Exploration of a solution for #122

Changes:

  • Remove math.h patch.
    This should no longer be needed since it existed only to provide some additional compatibility for musl target systems in the "x86_64-linux" gems. Closes Drop the math.h patch in a future release #120.
  • Move the x86_64-linux-gnu and x86-linux-gnu images to be based off ubuntu:20.04
    Remove all manylinux-specific build steps from the dockerfile template.

@flavorjones flavorjones changed the title move x86_64-linux to ubuntu 20.04 move x86_64-linux to be based on ubuntu:20.04 Jul 29, 2024
- manylinux2014 is EOL
- manylinux_2_28 works for x86_64
  - but does not provide support for 32-bit linux
  - and non-x86 images are pinned to that platform so they don't easily cross-compile
- ubuntu 20.04 generates code compatible with glibc 2.29
  - not much worse than 2.28
  - much lower complexity since we unify the image build process

so let's give it a try
@flavorjones flavorjones force-pushed the flavorjones-update-all-to-ubuntu-20.04 branch from 2fbf528 to e236020 Compare July 29, 2024 21:13
@flavorjones flavorjones force-pushed the flavorjones-update-all-to-ubuntu-20.04 branch from 5dffd4d to 2ccf000 Compare August 7, 2024 19:04
which wasn't necessary on manylinux2014 but seems to be necessary on
ubuntu:20.04 (but just for intel+gnu? weird.) to resolve this error
building native extensions:

  LD_LIBRARY_PATH=.:/usr/local/rake-compiler/ruby/x86_64-linux-gnu/ruby-3.3.0-rc1/lib
  "x86_64-linux-gnu-gcc -o conftest
  -I/usr/local/rake-compiler/ruby/x86_64-linux-gnu/ruby-3.3.0-rc1/include/ruby-3.3.0+0/x86_64-linux-gnu
  -I/usr/local/rake-compiler/ruby/x86_64-linux-gnu/ruby-3.3.0-rc1/include/ruby-3.3.0+0/ruby/backward
  -I/usr/local/rake-compiler/ruby/x86_64-linux-gnu/ruby-3.3.0-rc1/include/ruby-3.3.0+0
  -I../../../../ext/mri -O1 -fno-omit-frame-pointer -fno-fast-math
  -fstack-protector-strong -s -fPIC conftest.c
  -L. -L/usr/local/rake-compiler/ruby/x86_64-linux-gnu/ruby-3.3.0-rc1/lib
  -Wl,-rpath,/usr/local/rake-compiler/ruby/x86_64-linux-gnu/ruby-3.3.0-rc1/lib
  -L. -pipe -s -fstack-protector-strong -rdynamic -Wl,-export-dynamic
  -Wl,--no-as-needed -lruby-static -lpthread -lrt -ldl -lcrypt
  -Wl,-rpath,/usr/local/rake-compiler/ruby/x86_64-linux-gnu/ruby-3.3.0-rc1/lib
  -lruby -lm -lpthread -lc"

  /usr/bin/ld: /usr/local/rake-compiler/ruby/x86_64-linux-gnu/ruby-3.3.0-rc1/lib/libruby-static.a(addr2line.o): in function `fill_lines':
  addr2line.c:(.text+0x1bc0): undefined reference to `uncompress'
  collect2: error: ld returned 1 exit status
@flavorjones flavorjones force-pushed the flavorjones-update-all-to-ubuntu-20.04 branch from 2ccf000 to fc03ad1 Compare August 7, 2024 19:07
@flavorjones
Copy link
Collaborator Author

@larskanis I'd love your opinion on this change if you get a few minutes to review.

@flavorjones
Copy link
Collaborator Author

I've got a fix for the x86 image almost ready to go.

- install i686 compilers on the x86-linux-gnu image
- tweak mkmf flags
@flavorjones flavorjones force-pushed the flavorjones-update-all-to-ubuntu-20.04 branch from c78cdcb to d6ef24a Compare August 7, 2024 23:32
@ianks
Copy link

ianks commented Aug 17, 2024

Would this mean that precompiled gems could no longer run on old libc?

@flavorjones
Copy link
Collaborator Author

flavorjones commented Aug 20, 2024

@ianks Correct. The min glibc version would update from 2.17 to 2.29, and versions older than 2.29 would not be supported. What platforms are you targetting?

We will need to update the min glibc version in some way at some point in time. If you can help me understand your requirements maybe we can make better decisions. Ideally I don't want to be supporting ancient systems for new gems and new versions of Ruby (especially if previous rake-compiler-dock images still support those ancient systems), for some definition of "ancient" that is informed by distro EOL and Python manylinux EOL.

@ianks
Copy link

ianks commented Aug 20, 2024

We will need to update the min glibc version in some way at some point in time. If you can help me understand your requirements maybe we can make better decisions. Ideally I don't want to be supporting ancient systems for new gems and new versions of Ruby (especially if previous rake-compiler-dock images still support those ancient systems), for some definition of "ancient" that is informed by distro EOL and Python manylinux EOL.

I feel somewhat unqualified here, and for me personally I would prefer to be on some ubuntu. But I do know there are companies out there on old un-upgradable servers and I worry we could break things for them... But then again, they can always compile from source so maybe it doesn't matter?

@flavorjones
Copy link
Collaborator Author

flavorjones commented Aug 20, 2024

@ianks Right, end users can still compile their own stuff on older systems if they need to -- there's an escape hatch if you consider precompiled gems to be an optimization and not a critical requirement for gem delivery.

Really, this comes down to: I'm willing to support old systems as long as it doesn't cost much in terms of effort and complexity. But we're at the point where continuing on manylinux2014 has been hard and just got harder.

@ianks
Copy link

ianks commented Aug 20, 2024

Think I agree with you here. My one request would be that we look into the cross-rs images, since they've solved a lot of the traditional cross toolchain problems and offer some nice builtin features like qemu emulation, etc

@flavorjones
Copy link
Collaborator Author

@ianks I'm happy to chat about cross-rs at some point, but since they are also based on 20.04 (and presumably mostly-compatible) maybe we can move that discussion to a separate issue?

@flavorjones
Copy link
Collaborator Author

@larskanis Do you have any thoughts about this change? Unless you have strong objections, I'd like to merge it to make it easier to start testing downstream gems against the snapshot images.

Copy link
Member

@larskanis larskanis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I never really liked the manylinux base images. For me it's questionable, if we could compile a gcc package or configure the default one, to build binaries with lower libc requirements.

@flavorjones
Copy link
Collaborator Author

Well, maybe we can start with this change, and if people ask for lower glibc requirements we can look into building/configuring the toolchain.

@flavorjones flavorjones merged commit d5a3d13 into main Aug 21, 2024
172 checks passed
@flavorjones flavorjones deleted the flavorjones-update-all-to-ubuntu-20.04 branch August 21, 2024 20:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Drop the math.h patch in a future release
3 participants