Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[HYD-132 + HYD-142] Fix Docker build failure #26

Merged
merged 2 commits into from
Oct 4, 2022
Merged

Conversation

owenthereal
Copy link
Collaborator

The docker build failed with the following:

ERROR: failed to solve: executor failed running [/bin/sh -c export DEBIAN_FRONTEND=noninteractive     && export MAKEFLAGS="-j $(grep -c ^processor /proc/cpuinfo)"     && set -ex     && sed -i 's/^#\s*\(deb.*universe\)$/\1/g' /etc/apt/sources.list     && apt-get update     && cd /builddeps     && BUILD_PACKAGES="devscripts equivs build-essential fakeroot debhelper git gcc libc6-dev make cmake libevent-dev libbrotli-dev libssl-dev libkrb5-dev"     && if [ "$DEMO" = "true" ]; then         export DEB_PG_SUPPORTED_VERSIONS="$PGVERSION"         && WITH_PERL=false         && rm -f *.deb         && apt-get install -y $BUILD_PACKAGES;     else         BUILD_PACKAGES="$BUILD_PACKAGES zlib1g-dev libpam0g-dev libcurl4-openssl-dev libicu-dev python libc-ares-dev pandoc pkg-config"         && apt-get install -y $BUILD_PACKAGES libcurl4         && git clone -b $PAM_OAUTH2 --recurse-submodules https://github.com/CyberDem0n/pam-oauth2.git         && make -C pam-oauth2 install         && git clone -b $PLPROFILER https://github.com/bigsql/plprofiler.git         && tar -xzf plantuner-${PLANTUNER_COMMIT}.tar.gz         && curl -sL https://github.com/sdudoladov/pg_mon/archive/$PG_MON_COMMIT.tar.gz | tar xz         && for p in python3-keyring python3-docutils ieee-data; do             version=$(apt-cache show $p | sed -n 's/^Version: //p' | sort -rV | head -n 1)             && echo "Section: misc\nPriority: optional\nStandards-Version: 3.9.8\nPackage: $p\nVersion: $version\nDescription: $p" > $p             && equivs-build $p;         done;     fi     && if [ "$WITH_PERL" != "true" ]; then         version=$(apt-cache show perl | sed -n 's/^Version: //p' | sort -rV | head -n 1)         && echo "Section: misc\nPriority: optional\nStandards-Version: 3.9.8\nPackage: perl\nSection:perl\nMulti-Arch: allowed\nReplaces: perl-base\nVersion: $version\nDescription: perl" > perl         && equivs-build perl;     fi     && if [ "$WITH_PERL" != "true" ] || [ "$DEMO" != "true" ]; then dpkg -i *.deb || apt-get -y -f install; fi     && curl -sL https://github.com/CyberDem0n/bg_mon/archive/$BG_MON_COMMIT.tar.gz | tar xz     && curl -sL https://github.com/sdudoladov/pg_auth_mon/archive/$PG_AUTH_MON_COMMIT.tar.gz | tar xz     && curl -sL https://github.com/cybertec-postgresql/pg_permissions/archive/$PG_PERMISSIONS_COMMIT.tar.gz | tar xz     && curl -sL https://github.com/x4m/pg_tm_aux/archive/$PG_TM_AUX_COMMIT.tar.gz | tar xz     && git clone -b $SET_USER https://github.com/pgaudit/set_user.git     && git clone https://github.com/timescale/timescaledb.git     && apt-get install -y postgresql-common libevent-2.1 libevent-pthreads-2.1 brotli libbrotli1 python3.6 python3-psycopg2     && sed -ri 's/#(create_main_cluster) .*$/\1 = false/' /etc/postgresql-common/createcluster.conf     && for version in $DEB_PG_SUPPORTED_VERSIONS; do             sed -i "s/ main.*$/ main $version/g" /etc/apt/sources.list.d/pgdg.list             && apt-get update             && if [ "$DEMO" != "true" ]; then                 EXTRAS="postgresql-pltcl-${version}                         postgresql-${version}-dirtyread                         postgresql-${version}-extra-window-functions                         postgresql-${version}-first-last-agg                         postgresql-${version}-hll                         postgresql-${version}-hypopg                         postgresql-${version}-pgaudit                         postgresql-${version}-pg-checksums                         postgresql-${version}-pgl-ddl-deploy                         postgresql-${version}-pglogical                         postgresql-${version}-pglogical-ticker                         postgresql-${version}-pgq-node                         postgresql-${version}-pldebugger                         postgresql-${version}-pllua                         postgresql-${version}-plpgsql-check                         postgresql-${version}-plproxy                         postgresql-${version}-postgis-${POSTGIS_VERSION%.*}                         postgresql-${version}-postgis-${POSTGIS_VERSION%.*}-scripts                         postgresql-${version}-repack                         postgresql-${version}-wal2json"                 && if [ "$WITH_PERL" = "true" ]; then                     EXTRAS="$EXTRAS postgresql-plperl-${version}";                 fi                 && if [ ${version%.*} -ge 10 ]; then                     EXTRAS="$EXTRAS postgresql-${version}-decoderbufs";                 fi                 && if [ ${version%.*} -lt 11 ]; then                     EXTRAS="$EXTRAS postgresql-${version}-amcheck";                 fi;             fi             && apt-get install --allow-downgrades -y postgresql-contrib-${version}                     postgresql-plpython3-${version} postgresql-server-dev-${version}                     postgresql-${version}-cron postgresql-${version}-pgq3                     postgresql-${version}-pg-stat-kcache $EXTRAS             && cd timescaledb             && for v in $TIMESCALEDB; do                 git checkout $v                 && sed -i "s/VERSION 3.11/VERSION 3.10/" CMakeLists.txt                 && if BUILD_FORCE_REMOVE=true ./bootstrap -DREGRESS_CHECKS=OFF -DWARNINGS_AS_ERRORS=OFF                     -DTAP_CHECKS=OFF -DPG_CONFIG=/usr/lib/postgresql/$version/bin/pg_config                     -DAPACHE_ONLY=$TIMESCALEDB_APACHE_ONLY -DSEND_TELEMETRY_DEFAULT=NO; then                         make -C build install                         && strip /usr/lib/postgresql/$version/lib/timescaledb*.so;                 fi                 && git reset --hard                 && git clean -f -d;             done             && cd ..             && if [ "$DEMO" != "true" ]; then                 EXTRA_EXTENSIONS="plantuner-${PLANTUNER_COMMIT} plprofiler"                 && if [ ${version%.*} -ge 10 ]; then                      EXTRA_EXTENSIONS="$EXTRA_EXTENSIONS pg_mon-${PG_MON_COMMIT}";                 fi;             else                 EXTRA_EXTENSIONS="";             fi             && for n in bg_mon-${BG_MON_COMMIT} pg_auth_mon-${PG_AUTH_MON_COMMIT} set_user pg_permissions-${PG_PERMISSIONS_COMMIT} pg_tm_aux-${PG_TM_AUX_COMMIT} $EXTRA_EXTENSIONS; do                 make -C $n USE_PGXS=1 clean install-strip;             done;     done     && apt-get install -y skytools3-ticker pgbouncer     && sed -i "s/ main.*$/ main/g" /etc/apt/sources.list.d/pgdg.list     && apt-get update     && apt-get install -y postgresql postgresql-server-dev-all postgresql-all libpq-dev     && for version in $DEB_PG_SUPPORTED_VERSIONS; do         apt-get install -y postgresql-server-dev-${version};     done     && if [ "$DEMO" != "true" ]; then         for version in $DEB_PG_SUPPORTED_VERSIONS; do             apt-get install -y postgresql-${version}-partman             && ln -s postgis-${POSTGIS_VERSION%.*}.so                 /usr/lib/postgresql/${version}/lib/postgis-2.5.so;         done;     fi     && for pkg in pgextwlist; do         apt-get source postgresql-13-${pkg}         && cd $(ls -d *${pkg%?}*-*/)         && if [ -f ../$pkg.patch ]; then patch -p1 < ../$pkg.patch; fi         && if [ "$pkg" = "pgextwlist" ]; then             sed -i '/postgresql-all/d' debian/control.in             && perl -ne 'print unless /PG_TRY/ .. /PG_CATCH/' pgextwlist.c > pgextwlist.c.f             && egrep -v '(PG_END_TRY|EmitWarningsOnPlaceholders)' pgextwlist.c.f > pgextwlist.c;         fi         && pg_buildext updatecontrol         && DEB_BUILD_OPTIONS=nocheck debuild -b -uc -us         && cd ..         && for version in $DEB_PG_SUPPORTED_VERSIONS; do             for deb in postgresql-${version}-${pkg}_*.deb; do                 if [ -f $deb ]; then dpkg -i $deb; fi;             done;         done;     done     && gcc -s -shared -fPIC -o /usr/local/lib/cron_unprivileged.so cron_unprivileged.c     && apt-get purge -y ${BUILD_PACKAGES} postgresql postgresql-all postgresql-server-dev-* libpq-dev=* libmagic1 bsdmainutils     && apt-get autoremove -y     && apt-get clean     && dpkg -l | grep '^rc' | awk '{print $2}' | xargs apt-get purge -y     && if [ "$DEMO" != "true" ]; then         cd /usr/lib/postgresql/$PGVERSION/bin         && for u in clusterdb pg_archivecleanup pg_basebackup pg_isready pg_recvlogical pg_test_fsync pg_test_timing pgbench psql reindexdb vacuumdb vacuumlo *.py; do             for v in /usr/lib/postgresql/*; do                 if [ "$v" != "/usr/lib/postgresql/$PGVERSION" ] && [ -f "$v/bin/$u" ]; then                     rm $v/bin/$u                     && ln -s ../../$PGVERSION/bin/$u $v/bin/$u;                 fi;             done;         done         && set +x         && for v1 in $(ls -1d /usr/share/postgresql/* | sort -Vr); do             cd $v1/extension             && for orig in $(ls -1 *.sql | grep -v -- '--'); do                 for f in ${orig%.sql}--*.sql; do                     if [ !
-L $f ] && diff $orig $f > /dev/null; then                         echo "creating symlink $f -> $orig"                         && rm $f && ln -s $orig $f;                     fi;                 done;             done             && for e in pgq pgq_node plproxy address_standardizer address_standardizer_data_us; do                 orig=$(basename "$(find -maxdepth 1 -type f -name "$e--*--*.sql" | head -n1)")                 && if [ "x$orig" != "x" ]; then                     for f in $e--*--*.sql; do                         if [ "$f" != "$orig" ] && [ ! -L $f ] && diff $f $orig > /dev/null; then                             echo "creating symlink $f -> $orig"                             && rm $f && ln -s $orig $f;                         fi;                     done;                 fi;             done             && started=0             && for v2 in $(ls -1d /usr/share/postgresql/* | sort -Vr); do                 if [ $v1 = $v2 ]; then                     started=1;                 elif [ $started = 1 ]; then                     for d1 in extension contrib contrib/postgis-$POSTGIS_VERSION; do                         cd $v1/$d1                         && d2="$d1"                         && d1="../../${v1##*/}/$d1"                         && if [ "${d2%-*}" = "contrib/postgis" ]; then                             if  [ "${v2##*/}" = "9.6" ]; then d2="${d2%-*}-$POSTGIS_LEGACY"; fi                             && d1="../$d1";                         fi                         && d2="$v2/$d2"                         && for f in *.html *.sql *.control *.pl; do                             if [ -f $d2/$f ] && [ ! -L $d2/$f ] && diff $d2/$f $f > /dev/null; then                                 echo "creating symlink $d2/$f -> $d1/$f"                                 && rm $d2/$f && ln -s $d1/$f $d2/$f;                             fi;                         done;                     done;                 fi;             done;         done         && set -x;     fi     && rm -rf /var/lib/apt/lists/*             /var/cache/debconf/*             /builddeps             /usr/share/doc             /usr/share/man             /usr/share/info             /usr/share/locale/??             /usr/share/locale/??_??             /usr/share/postgresql/*/man             /etc/pgbouncer/*             /usr/lib/postgresql/*/bin/createdb             /usr/lib/postgresql/*/bin/createlang             /usr/lib/postgresql/*/bin/createuser             /usr/lib/postgresql/*/bin/dropdb             /usr/lib/postgresql/*/bin/droplang             /usr/lib/postgresql/*/bin/dropuser             /usr/lib/postgresql/*/bin/pg_standby             /usr/lib/postgresql/*/bin/pltcl_*     && find /var/log -type f -exec truncate --size 0 {} \;]: exit code: 29
make: *** [docker_build] Error 1

CI: https://github.com/HydrasDB/hydra/actions/runs/3178034025.

This is due to the failure to build older versions of PG. We only need PG 13 & 14. This PR builds 13 & 14 by setting the PGOLDVERSIONS=13 (14 will be always built because it's the latest supported PG in Spilo)

At the same time, Spilo image is upgraded to 2.1-p7 to consume the latest bug fixes. Diff: zalando/spilo@2.1-p6...2.1-p7

Running into build failure locally:

```
```

Spilo diff: zalando/spilo@2.1-p6...2.1-p7
This is the issue why the build failed, and we don't need that many
older versions
@owenthereal owenthereal requested review from wuputah and ljcesca October 3, 2022 23:24
@owenthereal owenthereal changed the title [HYD-142] Docker build failure [HYD-142] Fix Docker build failure Oct 3, 2022
@ljcesca ljcesca changed the title [HYD-142] Fix Docker build failure [HYD-132 + HYD-142] Fix Docker build failure Oct 4, 2022
@owenthereal owenthereal merged commit b30d0fa into main Oct 4, 2022
@owenthereal owenthereal deleted the spilo_2_1_p_7 branch October 4, 2022 16:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants