Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OpenSUSE CI image builder fails to run dub tests #11773

Closed
eli-schwartz opened this issue May 8, 2023 · 6 comments · Fixed by #13517
Closed

OpenSUSE CI image builder fails to run dub tests #11773

eli-schwartz opened this issue May 8, 2023 · 6 comments · Fixed by #13517
Labels
CI Continuous integration: Travis/Appveyor/Actions language:D

Comments

@eli-schwartz
Copy link
Member 

=== d: 11 dub ===
 

(inprocess) $ setup --prefix /usr --libdir lib 'test cases/d/11 dub' '/meson/b dedabac16d' --backend=ninja
Build started at 2023-05-07T00:22:54.102213
Main binary: /usr/bin/python3
Build Options: -Dprefix=/usr -Dlibdir=lib -Dbackend=ninja
Python system: Linux
The Meson build system
Version: 1.1.99
Source dir: /meson/test cases/d/11 dub
Build dir: /meson/b dedabac16d
Build type: native build
Project name: dub-example
Project version: undefined
-----
Detecting compiler via: cc --version
compiler returned <Popen: returncode: 0 args: ['cc', '--version']>
compiler stdout:
cc (SUSE Linux) 13.0.1 20230421 (prerelease) [revision f980561c60b0446cc427595198d7f3f4f90e0924]
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


compiler stderr:

Running command: cc -E -dM -
-----
Detecting linker via: cc -Wl,--version
linker returned <Popen: returncode: 0 args: ['cc', '-Wl,--version']>
linker stdout:
GNU ld (GNU Binutils; openSUSE Tumbleweed) 2.40.0.20230412-4
Copyright (C) 2023 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.

linker stderr:
collect2 version 13.0.1 20230421 (prerelease) [revision f980561c60b0446cc427595198d7f3f4f90e0924]
/usr/lib64/gcc/x86_64-suse-linux/13/../../../../x86_64-suse-linux/bin/ld -plugin /usr/lib64/gcc/x86_64-suse-linux/13/liblto_plugin.so -plugin-opt=/usr/lib64/gcc/x86_64-suse-linux/13/lto-wrapper -plugin-opt=-fresolution=/tmp/ccFWsoY0.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --build-id --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 /usr/lib64/gcc/x86_64-suse-linux/13/../../../../lib64/crt1.o /usr/lib64/gcc/x86_64-suse-linux/13/../../../../lib64/crti.o /usr/lib64/gcc/x86_64-suse-linux/13/crtbegin.o -L/root/dlang/dmd-2.103.1/linux/lib64/../lib64 -L/usr/lib64/gcc/x86_64-suse-linux/13 -L/usr/lib64/gcc/x86_64-suse-linux/13/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/root/dlang/dmd-2.103.1/linux/lib64 -L/usr/lib64/gcc/x86_64-suse-linux/13/../../../../x86_64-suse-linux/lib -L/usr/lib64/gcc/x86_64-suse-linux/13/../../.. --version -lgcc --push-state --as-needed -lgcc_s --pop-state -lc -lgcc --push-state --as-needed -lgcc_s --pop-state /usr/lib64/gcc/x86_64-suse-linux/13/crtend.o /usr/lib64/gcc/x86_64-suse-linux/13/../../../../lib64/crtn.o

-----
Detecting linker via: dmd -L=--version /tmp/tmpnr3bqfjn.d -m64
linker returned <Popen: returncode: 0 args: ['dmd', '-L=--version', '/tmp/tmpnr3bqfjn.d', '-...>
linker stdout:
GNU ld (GNU Binutils; openSUSE Tumbleweed) 2.40.0.20230412-4
Copyright (C) 2023 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.

linker stderr:
collect2 version 13.0.1 20230421 (prerelease) [revision f980561c60b0446cc427595198d7f3f4f90e0924]
/usr/lib64/gcc/x86_64-suse-linux/13/../../../../x86_64-suse-linux/bin/ld -plugin /usr/lib64/gcc/x86_64-suse-linux/13/liblto_plugin.so -plugin-opt=/usr/lib64/gcc/x86_64-suse-linux/13/lto-wrapper -plugin-opt=-fresolution=/tmp/cceZEcxI.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --build-id --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o tmpnr3bqfjn /usr/lib64/gcc/x86_64-suse-linux/13/../../../../lib64/crt1.o /usr/lib64/gcc/x86_64-suse-linux/13/../../../../lib64/crti.o /usr/lib64/gcc/x86_64-suse-linux/13/crtbegin.o -L/root/dlang/dmd-2.103.1/linux/bin64/../lib64 -L/root/dlang/dmd-2.103.1/linux/lib64/../lib64 -L/usr/lib64/gcc/x86_64-suse-linux/13 -L/usr/lib64/gcc/x86_64-suse-linux/13/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/root/dlang/dmd-2.103.1/linux/lib64 -L/usr/lib64/gcc/x86_64-suse-linux/13/../../../../x86_64-suse-linux/lib -L/usr/lib64/gcc/x86_64-suse-linux/13/../../.. tmpnr3bqfjn.o --version --export-dynamic -Bstatic -lphobos2 -Bdynamic -lpthread -lm -lrt -ldl -lgcc --push-state --as-needed -lgcc_s --pop-state -lc -lgcc --push-state --as-needed -lgcc_s --pop-state /usr/lib64/gcc/x86_64-suse-linux/13/crtend.o /usr/lib64/gcc/x86_64-suse-linux/13/../../../../lib64/crtn.o

D compiler for the host machine: dmd (dmd 2.103.1 "DMD64 D Compiler v2.103.1")
D linker for the host machine: dmd ld.bfd 2.40.0.20230412-4
-----
Detecting compiler via: cc --version
compiler returned <Popen: returncode: 0 args: ['cc', '--version']>
compiler stdout:
cc (SUSE Linux) 13.0.1 20230421 (prerelease) [revision f980561c60b0446cc427595198d7f3f4f90e0924]
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


compiler stderr:

Running command: cc -E -dM -
-----
Detecting linker via: cc -Wl,--version
linker returned <Popen: returncode: 0 args: ['cc', '-Wl,--version']>
linker stdout:
GNU ld (GNU Binutils; openSUSE Tumbleweed) 2.40.0.20230412-4
Copyright (C) 2023 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.

linker stderr:
collect2 version 13.0.1 20230421 (prerelease) [revision f980561c60b0446cc427595198d7f3f4f90e0924]
/usr/lib64/gcc/x86_64-suse-linux/13/../../../../x86_64-suse-linux/bin/ld -plugin /usr/lib64/gcc/x86_64-suse-linux/13/liblto_plugin.so -plugin-opt=/usr/lib64/gcc/x86_64-suse-linux/13/lto-wrapper -plugin-opt=-fresolution=/tmp/cc49vRRt.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --build-id --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 /usr/lib64/gcc/x86_64-suse-linux/13/../../../../lib64/crt1.o /usr/lib64/gcc/x86_64-suse-linux/13/../../../../lib64/crti.o /usr/lib64/gcc/x86_64-suse-linux/13/crtbegin.o -L/root/dlang/dmd-2.103.1/linux/lib64/../lib64 -L/usr/lib64/gcc/x86_64-suse-linux/13 -L/usr/lib64/gcc/x86_64-suse-linux/13/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/root/dlang/dmd-2.103.1/linux/lib64 -L/usr/lib64/gcc/x86_64-suse-linux/13/../../../../x86_64-suse-linux/lib -L/usr/lib64/gcc/x86_64-suse-linux/13/../../.. --version -lgcc --push-state --as-needed -lgcc_s --pop-state -lc -lgcc --push-state --as-needed -lgcc_s --pop-state /usr/lib64/gcc/x86_64-suse-linux/13/crtend.o /usr/lib64/gcc/x86_64-suse-linux/13/../../../../lib64/crtn.o

-----
Detecting linker via: dmd -L=--version /tmp/tmps3cp2omm.d -m64
linker returned <Popen: returncode: 0 args: ['dmd', '-L=--version', '/tmp/tmps3cp2omm.d', '-...>
linker stdout:
GNU ld (GNU Binutils; openSUSE Tumbleweed) 2.40.0.20230412-4
Copyright (C) 2023 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.

linker stderr:
collect2 version 13.0.1 20230421 (prerelease) [revision f980561c60b0446cc427595198d7f3f4f90e0924]
/usr/lib64/gcc/x86_64-suse-linux/13/../../../../x86_64-suse-linux/bin/ld -plugin /usr/lib64/gcc/x86_64-suse-linux/13/liblto_plugin.so -plugin-opt=/usr/lib64/gcc/x86_64-suse-linux/13/lto-wrapper -plugin-opt=-fresolution=/tmp/cclLqH0J.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --build-id --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o tmps3cp2omm /usr/lib64/gcc/x86_64-suse-linux/13/../../../../lib64/crt1.o /usr/lib64/gcc/x86_64-suse-linux/13/../../../../lib64/crti.o /usr/lib64/gcc/x86_64-suse-linux/13/crtbegin.o -L/root/dlang/dmd-2.103.1/linux/bin64/../lib64 -L/root/dlang/dmd-2.103.1/linux/lib64/../lib64 -L/usr/lib64/gcc/x86_64-suse-linux/13 -L/usr/lib64/gcc/x86_64-suse-linux/13/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/root/dlang/dmd-2.103.1/linux/lib64 -L/usr/lib64/gcc/x86_64-suse-linux/13/../../../../x86_64-suse-linux/lib -L/usr/lib64/gcc/x86_64-suse-linux/13/../../.. tmps3cp2omm.o --version --export-dynamic -Bstatic -lphobos2 -Bdynamic -lpthread -lm -lrt -ldl -lgcc --push-state --as-needed -lgcc_s --pop-state -lc -lgcc --push-state --as-needed -lgcc_s --pop-state /usr/lib64/gcc/x86_64-suse-linux/13/crtend.o /usr/lib64/gcc/x86_64-suse-linux/13/../../../../lib64/crtn.o

D compiler for the build machine: dmd (dmd 2.103.1 "DMD64 D Compiler v2.103.1")
D linker for the build machine: dmd ld.bfd 2.40.0.20230412-4
Build machine cpu family: x86_64
Build machine cpu: x86_64
Host machine cpu family: x86_64
Host machine cpu: x86_64
Target machine cpu family: x86_64
Target machine cpu: x86_64
Program dub found: YES (/root/dlang/dmd-2.103.1/linux/bin64/dub)
Determining dependency 'urld' with DUB executable '/root/dlang/dmd-2.103.1/linux/bin64/dub'
ERROR: urld@2.1.1 not found
You may try the following command to install the necessary DUB libraries:
dub run dub-build-deep --yes -- urld --arch=x86_64 --compiler=dmd --build=debug
Run-time dependency urld found: NO 

test cases/d/11 dub/meson.build:8:11: ERROR: Dependency "urld" not found

=== d: 14 dub with deps ===
 

(inprocess) $ setup --prefix /usr --libdir lib 'test cases/d/14 dub with deps' '/meson/b 04877c7079' --backend=ninja
Build started at 2023-05-07T00:22:55.593610
Main binary: /usr/bin/python3
Build Options: -Dprefix=/usr -Dlibdir=lib -Dbackend=ninja
Python system: Linux
The Meson build system
Version: 1.1.99
Source dir: /meson/test cases/d/14 dub with deps
Build dir: /meson/b 04877c7079
Build type: native build
Project name: dub-with-deps-example
Project version: undefined
-----
Detecting compiler via: cc --version
compiler returned <Popen: returncode: 0 args: ['cc', '--version']>
compiler stdout:
cc (SUSE Linux) 13.0.1 20230421 (prerelease) [revision f980561c60b0446cc427595198d7f3f4f90e0924]
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


compiler stderr:

Running command: cc -E -dM -
-----
Detecting linker via: cc -Wl,--version
linker returned <Popen: returncode: 0 args: ['cc', '-Wl,--version']>
linker stdout:
GNU ld (GNU Binutils; openSUSE Tumbleweed) 2.40.0.20230412-4
Copyright (C) 2023 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
Detecting linker via: dmd -L=--version /tmp/tmpslumvuns.d -m64
linker returned <Popen: returncode: 0 args: ['dmd', '-L=--version', '/tmp/tmpslumvuns.d', '-...>
linker stdout:
GNU ld (GNU Binutils; openSUSE Tumbleweed) 2.40.0.20230412-4
Copyright (C) 2023 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.

linker stderr:
collect2 version 13.0.1 20230421 (prerelease) [revision f980561c60b0446cc427595198d7f3f4f90e0924]
/usr/lib64/gcc/x86_64-suse-linux/13/../../../../x86_64-suse-linux/bin/ld -plugin /usr/lib64/gcc/x86_64-suse-linux/13/liblto_plugin.so -plugin-opt=/usr/lib64/gcc/x86_64-suse-linux/13/lto-wrapper -plugin-opt=-fresolution=/tmp/ccZPOMyr.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --build-id --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o tmpslumvuns /usr/lib64/gcc/x86_64-suse-linux/13/../../../../lib64/crt1.o /usr/lib64/gcc/x86_64-suse-linux/13/../../../../lib64/crti.o /usr/lib64/gcc/x86_64-suse-linux/13/crtbegin.o -L/root/dlang/dmd-2.103.1/linux/bin64/../lib64 -L/root/dlang/dmd-2.103.1/linux/lib64/../lib64 -L/usr/lib64/gcc/x86_64-suse-linux/13 -L/usr/lib64/gcc/x86_64-suse-linux/13/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/root/dlang/dmd-2.103.1/linux/lib64 -L/usr/lib64/gcc/x86_64-suse-linux/13/../../../../x86_64-suse-linux/lib -L/usr/lib64/gcc/x86_64-suse-linux/13/../../.. tmpslumvuns.o --version --export-dynamic -Bstatic -lphobos2 -Bdynamic -lpthread -lm -lrt -ldl -lgcc --push-state --as-needed -lgcc_s --pop-state -lc -lgcc --push-state --as-needed -lgcc_s --pop-state /usr/lib64/gcc/x86_64-suse-linux/13/crtend.o /usr/lib64/gcc/x86_64-suse-linux/13/../../../../lib64/crtn.o

D compiler for the build machine: dmd (dmd 2.103.1 "DMD64 D Compiler v2.103.1")
D linker for the build machine: dmd ld.bfd 2.40.0.20230412-4
Build machine cpu family: x86_64
Build machine cpu: x86_64
Host machine cpu family: x86_64
Host machine cpu: x86_64
Target machine cpu family: x86_64
Target machine cpu: x86_64
Program dub found: YES (/root/dlang/dmd-2.103.1/linux/bin64/dub)
Running command: /root/dlang/dmd-2.103.1/linux/bin64/dub run dub-build-deep --yes -- xlsx --compiler dmd --arch x86_64
--- stdout ---
             Package 'dub-build-deep' was not found locally but is available online:
             ---
             Description: An utility to build a dub library and all its dependencies
             Version: 1.1.0
             ---
    Fetching dub-build-deep 1.1.0
             Building package dub-build-deep in /root/.dub/packages/dub-build-deep-1.1.0/dub-build-deep/
    Starting Performing "debug" build using /root/dlang/dmd-2.103.1/linux/bin64/dmd for x86_64.
    Building dub-build-deep 1.1.0: building configuration [application]
     Linking dub-build-deep
     Running ../../../../root/.dub/packages/dub-build-deep-1.1.0/dub-build-deep/dub-build-deep xlsx --compiler dmd --arch x86_64
    Fetching xlsx 0.0.8
    Finished xlsx fetched
             Please note that you need to use `dub run <pkgname>` or add it to dependencies of your package to actually use/run it. 
running dub describe xlsx --compiler dmd --arch 'x86_64'
Warning: xlsx does not appear to be present locally. Will try to fetch...
running dub fetch xlsx
running dub describe xlsx --compiler dmd --arch 'x86_64'
running dub build 'xlsx@0.0.8' --config library --compiler dmd --arch 'x86_64'
running dub build 'dxml@0.4.3' --config library --compiler dmd --arch 'x86_64'
running dub build 'archive@0.6.0' --config library --compiler dmd --arch 'x86_64'

--- stderr ---


Determining dependency 'xlsx' with DUB executable '/root/dlang/dmd-2.103.1/linux/bin64/dub'
ERROR: xlsx@0.0.8 not found
You may try the following command to install the necessary DUB libraries:
dub run dub-build-deep --yes -- xlsx --arch=x86_64 --compiler=dmd --build=debug
Run-time dependency xlsx found: NO 

test cases/d/14 dub with deps/meson.build:30:11: ERROR: Dependency "xlsx" not found

See https://github.com/mesonbuild/meson/actions/runs/4904266480/jobs/8757181437#step:5:2540

@rtbo as you've contributed quite a bit to this I'm hoping you might have an idea what's going on here? Your dub-build-deep program doesn't appear to be performing as expected in at least the latter case.
@eli-schwartz eli-schwartz added language:D CI Continuous integration: Travis/Appveyor/Actions labels May 8, 2023
@rtbo
Copy link
Contributor

rtbo commented May 16, 2023

It seems that latest Dub version has changed the cache location to ~/.dub/cache.
A bigger problem is that the new cache directories do not contain the compiler nor the arch in clear.
For example on my system, I can find a static library at:

~/.dub/cache/vibe-d/0.9.6/+http/build/library-debug-bUZ3sLErP9iQlvgdxNNYQg/libvibe-d_http.a

Dub computes the hash part from all build settings, including compiler, compiler version, architecture etc.
Generating reliable builds with meson from this is compromised.

@rtbo
Copy link
Contributor

rtbo commented May 16, 2023

All unit tests relying on finding a dub dependency in the cache will eventually fail at some point (when distro will move to this version of Dub). I suggest to skip these tests globally from now.

@rtbo
Copy link
Contributor

rtbo commented May 16, 2023

Relates to dlang/dub#2589

@eli-schwartz
Copy link
Member Author

Thanks for debugging this. You're right, it does look very serious. But that should probably be exposed as an explanatory error for users running meson, not just skipped in the testsuite. Hopefully a solution can be found either in meson or dub, so that dependency('foo', method: 'dub') starts working again for at least some versions of dub.

@rtbo
Copy link
Contributor

rtbo commented May 16, 2023

In meson, it would require to calculate the same hash as dub does. Hopefully it can be done from the data returned by dub describe, but we'd need all the same flags, versions etc. returned in the same order and for all future versions of Dub.
I've proposed a more reliable solution to the Dub team. See dlang/dub#2640.

@rtbo rtbo mentioned this issue May 17, 2023
Merged
@eli-schwartz
Copy link
Member Author

Awesome, the proposed fix has been merged for the next upcoming Dub release.

@rtbo rtbo mentioned this issue May 18, 2023
Closed
@rtbo rtbo mentioned this issue Aug 20, 2023
Closed
@eli-schwartz eli-schwartz mentioned this issue Aug 6, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
CI Continuous integration: Travis/Appveyor/Actions language:D
Projects
None yet
Milestone
No milestone
2 participants
@rtbo @eli-schwartz