Skip to content

Commit

Permalink
Make native mvnd only require glibc 2.12 on ubuntu 22.04 (#825, fixes #…
Browse files Browse the repository at this point in the history
…823)

The modern glibc from 2.34 had two breaking changes: move all api in lib{pthread,dl,rt}.so into libc.so, new static start up code breaking the runtime compatibility for old glibc (<2.34). See https://developers.redhat.com/articles/2021/12/17/why-glibc-234-removed-libpthread for more info.

This commit has 3 changes to overcome these changes:
- override gcc path to redefine glibc symbols on the fly in the generated .o by graalvm when linking
- provide a dynamic startup code to support both old and modern runtime glibc
- add needed dynamic libraries: lib{pthread,rt,dl}.so.
  • Loading branch information
gzm55 authored Mar 23, 2023
1 parent 76f8139 commit da20df0
Show file tree
Hide file tree
Showing 8 changed files with 222 additions and 27 deletions.
40 changes: 28 additions & 12 deletions .github/workflows/early-access.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ jobs:
default-build:
name: 'Default build (without Graal)'
if: startsWith(github.event.head_commit.message, '[release] Release ') != true
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
steps:
- name: 'Checkout'
uses: actions/checkout@v3
Expand All @@ -52,7 +52,7 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ ubuntu-20.04, macOS-10.15, windows-2019 ]
os: [ ubuntu-22.04, macOS-10.15, windows-2019 ]
runs-on: ${{ matrix.os }}

steps:
Expand All @@ -79,22 +79,38 @@ jobs:
run: ./mvnw clean -Dmrm=false -B -ntp -e

- name: 'Patch Graal libs for only requiring glibc 2.12'
if: ${{ env.OS == 'linux' }}
shell: bash
run: |
if [[ $OS == linux ]] && [[ $GRAALVM_HOME ]] && [[ -d "$GRAALVM_HOME/lib/static/linux-amd64/glibc" ]]; then
mkdir -p client/target/graalvm-libs-for-glibc-2.12
echo "memcpy memcpy@GLIBC_2.2.5" >client/target/glibc.redef
echo "posix_spawn posix_spawn@GLIBC_2.2.5" >>client/target/glibc.redef
find "$GRAALVM_HOME/lib/static/linux-amd64/glibc" -name '*.a' | while IFS= read -r input; do
output="client/target/graalvm-libs-for-glibc-2.12/$(basename -- "$input")"
objcopy --redefine-syms=client/target/glibc.redef -- "$input" "$output" 2>/dev/null
done
find /usr/lib -name libz.a | xargs -r -I {} objcopy --redefine-syms=client/target/glibc.redef {} client/target/graalvm-libs-for-glibc-2.12/libz.a
fi
mkdir -p client/target/graalvm-libs-for-glibc-2.12
: patch common libraries
( find "$GRAALVM_HOME/lib/static/linux-amd64/glibc" -name '*.a'
ls -1 /lib/x86_64-linux-gnu/libz.a
ls -1 "$GRAALVM_HOME/lib/svm/clibraries/linux-amd64/libjvm.a"
ls -1 "$GRAALVM_HOME/lib/svm/clibraries/linux-amd64/liblibchelper.a"
) | while IFS= read -r input; do
output="client/target/graalvm-libs-for-glibc-2.12/$(basename -- "$input")"
objcopy --redefine-syms=client/src/main/resources/glibc/glibc.redef -- "$input" "$output" 2>/dev/null
done
: patch gcc startfile
gcc -O3 -Wall -Wextra -Werror -Wconversion -Wsign-conversion -Wcast-qual -pedantic -c -o client/target/dynamic-libc-start.o client/src/main/resources/glibc/dynamic-libc-start.c
ld -r /lib/x86_64-linux-gnu/Scrt1.o client/target/dynamic-libc-start.o -o client/target/graalvm-libs-for-glibc-2.12/Scrt1.o
objcopy --redefine-syms=client/src/main/resources/glibc/glibc.redef client/target/graalvm-libs-for-glibc-2.12/Scrt1.o 2>/dev/null
- name: 'Build native distribution'
run: ./mvnw verify -Pnative -Dmrm=false -B -ntp -e

- name: 'Verify native binary for only requiring glibc 2.12'
if: ${{ env.OS == 'linux' }}
shell: bash
run: |
(( 4 == "$(ldd client/target/mvnd | awk '{print $1}' | sort -u | grep -c 'lib\(c\|dl\|rt\|pthread\)\.so\.[0-9]')" )) || ( ldd client/target/mvnd && false )
err=0
objdump -T client/target/mvnd | grep GLIBC_ | grep -v 'GLIBC_\([01]\|2\.[0-9]\|2\.1[012]\)[^0-9]' || err=$?
(( err == 1 ))
- name: 'Upload daemon test logs'
if: always()
uses: actions/upload-artifact@v3
Expand Down
42 changes: 29 additions & 13 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ jobs:
strategy:
fail-fast: true
matrix:
os: [ ubuntu-20.04, macOS-10.15, windows-2019 ]
os: [ ubuntu-22.04, macOS-10.15, windows-2019 ]
runs-on: ${{ matrix.os }}

steps:
Expand All @@ -62,22 +62,38 @@ jobs:
run: ./mvnw clean -Dmrm=false -B -ntp -e

- name: 'Patch Graal libs for only requiring glibc 2.12'
if: ${{ env.OS == 'linux' }}
shell: bash
run: |
if [[ $OS == linux ]] && [[ $GRAALVM_HOME ]] && [[ -d "$GRAALVM_HOME/lib/static/linux-amd64/glibc" ]]; then
mkdir -p client/target/graalvm-libs-for-glibc-2.12
echo "memcpy memcpy@GLIBC_2.2.5" >client/target/glibc.redef
echo "posix_spawn posix_spawn@GLIBC_2.2.5" >>client/target/glibc.redef
find "$GRAALVM_HOME/lib/static/linux-amd64/glibc" -name '*.a' | while IFS= read -r input; do
output="client/target/graalvm-libs-for-glibc-2.12/$(basename -- "$input")"
objcopy --redefine-syms=client/target/glibc.redef -- "$input" "$output" 2>/dev/null
done
find /usr/lib -name libz.a | xargs -r -I {} objcopy --redefine-syms=client/target/glibc.redef {} client/target/graalvm-libs-for-glibc-2.12/libz.a
fi
mkdir -p client/target/graalvm-libs-for-glibc-2.12
: patch common libraries
( find "$GRAALVM_HOME/lib/static/linux-amd64/glibc" -name '*.a'
ls -1 /lib/x86_64-linux-gnu/libz.a
ls -1 "$GRAALVM_HOME/lib/svm/clibraries/linux-amd64/libjvm.a"
ls -1 "$GRAALVM_HOME/lib/svm/clibraries/linux-amd64/liblibchelper.a"
) | while IFS= read -r input; do
output="client/target/graalvm-libs-for-glibc-2.12/$(basename -- "$input")"
objcopy --redefine-syms=client/src/main/resources/glibc/glibc.redef -- "$input" "$output" 2>/dev/null
done
: patch gcc startfile
gcc -O3 -Wall -Wextra -Werror -Wconversion -Wsign-conversion -Wcast-qual -pedantic -c -o client/target/dynamic-libc-start.o client/src/main/resources/glibc/dynamic-libc-start.c
ld -r /lib/x86_64-linux-gnu/Scrt1.o client/target/dynamic-libc-start.o -o client/target/graalvm-libs-for-glibc-2.12/Scrt1.o
objcopy --redefine-syms=client/src/main/resources/glibc/glibc.redef client/target/graalvm-libs-for-glibc-2.12/Scrt1.o 2>/dev/null
- name: 'Build native distribution'
run: ./mvnw verify -Pnative -Dmrm=false -B -ntp -e -DskipTests -s .mvn/release-settings.xml

- name: 'Verify native binary for only requiring glibc 2.12'
if: ${{ env.OS == 'linux' }}
shell: bash
run: |
(( 4 == "$(ldd client/target/mvnd | awk '{print $1}' | sort -u | grep -c 'lib\(c\|dl\|rt\|pthread\)\.so\.[0-9]')" )) || ( ldd client/target/mvnd && false )
err=0
objdump -T client/target/mvnd | grep GLIBC_ | grep -v 'GLIBC_\([01]\|2\.[0-9]\|2\.1[012]\)[^0-9]' || err=$?
(( err == 1 ))
- name: 'Upload artifact'
uses: actions/upload-artifact@v3
with:
Expand All @@ -90,7 +106,7 @@ jobs:
source:
name: 'Build source distributions'
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
steps:
- name: 'Check out repository'
uses: actions/checkout@v3
Expand Down Expand Up @@ -123,7 +139,7 @@ jobs:
target/maven-mvnd-*.tar.gz
release:
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
needs: [build, source]

steps:
Expand Down
32 changes: 31 additions & 1 deletion client/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
<maven.compiler.release>11</maven.compiler.release>
<graalvm-native-static-opt />
<graalvm-native-glibc-opt />
<patchelf.skip>true</patchelf.skip>
</properties>

<dependencies>
Expand Down Expand Up @@ -199,7 +200,10 @@
</file>
</activation>
<properties>
<graalvm-native-glibc-opt>-H:CLibraryPath=${project.build.directory}/graalvm-libs-for-glibc-2.12</graalvm-native-glibc-opt>
<graalvm-native-glibc-opt>-H:CCompilerPath=${basedir}/src/main/resources/glibc/gcc
-H:CCompilerOption=-B${project.build.directory}/graalvm-libs-for-glibc-2.12
-H:CLibraryPath=${project.build.directory}/graalvm-libs-for-glibc-2.12</graalvm-native-glibc-opt>
<patchelf.skip>false</patchelf.skip>
</properties>
</profile>

Expand Down Expand Up @@ -235,6 +239,32 @@
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<configuration>
<skip>${patchelf.skip}</skip>
<executable>patchelf</executable>
<arguments>
<argument>--add-needed</argument>
<argument>libpthread.so.0</argument>
<argument>--add-needed</argument>
<argument>librt.so.1</argument>
<argument>--add-needed</argument>
<argument>libdl.so.2</argument>
<argument>${project.build.directory}/mvnd</argument>
</arguments>
</configuration>
<executions>
<execution>
<id>add-needed-glibc-so</id>
<goals>
<goal>exec</goal>
</goals>
<phase>package</phase>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
Expand Down
67 changes: 67 additions & 0 deletions client/src/main/resources/glibc/dynamic-libc-start.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

/* ref:
* https://elixir.bootlin.com/glibc/glibc-2.37.9000/source/csu/libc-start.c
* https://elixir.bootlin.com/glibc/glibc-2.33.9000/source/csu/elf-init.c#L68
*/

#define _GNU_SOURCE
#include <dlfcn.h>
#include <stdint.h>

__asm__(".symver dlsym,dlsym@GLIBC_2.2.5");
__asm__(".symver dlvsym,dlvsym@GLIBC_2.2.5");

/* These magic symbols are provided by the linker. */
extern void (*__init_array_start[])(int, char **, char **) __attribute__ ((visibility ("hidden")));
extern void (*__init_array_end[])(int, char **, char **) __attribute__ ((visibility ("hidden")));
extern void _init(void);

/* These functions are passed to __libc_start_main by the startup code.
These get statically linked into each program. */

static void
__libc_csu_init(const int argc, char **const argv, char **const envp)
{
_init();
const __auto_type size = __init_array_end - __init_array_start;
for (__auto_type i = 0; i < size; ++i)
(*__init_array_start[i])(argc, argv, envp);
}

int
__dynamic_libc_start_main(int (*const main)(int, char **, char **),
const int argc,
char ** const argv,
__typeof(&__libc_csu_init) init,
void (*const fini)(void),
void (*const rtld_fini)(void),
void (*const stack_end))
{
_Static_assert(sizeof(uintptr_t) >= sizeof(void*), "uintptr_t should contain a object pointer");
_Static_assert(sizeof(uintptr_t) <= sizeof(&__dynamic_libc_start_main), "function pointer should contain an uintptr_t");

const __auto_type __libc_start_main = (__typeof(&__dynamic_libc_start_main))(uintptr_t)dlsym(RTLD_DEFAULT, "__libc_start_main");
if (!dlvsym(RTLD_DEFAULT, "__libc_start_main", "GLIBC_2.34")) {
init = &__libc_csu_init; // old runtime glibc, ver < 2.34
}

return __libc_start_main(main, argc, argv, init, fini, rtld_fini, stack_end);
}
27 changes: 27 additions & 0 deletions client/src/main/resources/glibc/gcc
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
#!/bin/sh

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

set -euf

base=$(dirname -- "$0")

# fix glibc api version on the fly
find . -name '*.o' -print0 | xargs -0rn 1 objcopy --redefine-syms="$base/glibc.redef"

exec gcc "$@"
35 changes: 35 additions & 0 deletions client/src/main/resources/glibc/glibc.redef
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
__libc_start_main __dynamic_libc_start_main
clock_gettime clock_gettime@GLIBC_2.2.5
dlopen dlopen@GLIBC_2.2.5
dlsym dlsym@GLIBC_2.2.5
memcpy memcpy@GLIBC_2.2.5
posix_spawn posix_spawn@GLIBC_2.2.5
pthread_attr_getguardsize pthread_attr_getguardsize@GLIBC_2.2.5
pthread_attr_getstack pthread_attr_getstack@GLIBC_2.2.5
pthread_attr_setstacksize pthread_attr_setstacksize@GLIBC_2.2.5
pthread_condattr_setclock pthread_condattr_setclock@GLIBC_2.3.3
pthread_create pthread_create@GLIBC_2.2.5
pthread_getattr_np pthread_getattr_np@GLIBC_2.2.5
pthread_join pthread_join@GLIBC_2.2.5
pthread_kill pthread_kill@GLIBC_2.2.5
pthread_setname_np pthread_setname_np@GLIBC_2.12
sem_destroy sem_destroy@GLIBC_2.2.5
sem_init sem_init@GLIBC_2.2.5
sem_post sem_post@GLIBC_2.2.5
sem_wait sem_wait@GLIBC_2.2.5
1 change: 0 additions & 1 deletion native/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,6 @@
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>3.0.0</version>
<dependencies>
<dependency>
<groupId>info.picocli</groupId>
Expand Down
5 changes: 5 additions & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -485,6 +485,11 @@
<artifactId>maven-wrapper-plugin</artifactId>
<version>3.1.1</version>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>3.1.0</version>
</plugin>
</plugins>
</pluginManagement>

Expand Down

0 comments on commit da20df0

Please sign in to comment.