cmd/link: regression on some distros for ppc64le with CL 130316

[laboger]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

upstream tip

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

ppc64le Debian 9

What did you do?

Built golang using all.bash

What did you expect to see?

All tests passed

What did you see instead?

~/golang/test/go/src$ go tool dist test testso

../misc/cgo/testso

./main.exe: error while loading shared libraries: R_PPC64_ADDR16_HA re113298fa4 for symbol `' out of range
2018/09/04 19:10:02 Failed: exit status 127

This does not fail in the go 1.11 branch, and I tracked it down the commit 6417e91.

The commit mentioned above is dropping the -no-pie option, which is the cause of the failure.

commit 247b034:

go tool dist test testso

##### ../misc/cgo/testso
# command-line-arguments
HEADER = -H5 -T0x11000 -D0x0 -R0x10000
 0.00 deadcode
 0.01 pclntab=247478 bytes, funcdata total 46718 bytes
 0.01 dodata
 0.01 dwarf
 0.02 symsize = 0
 0.03 reloc
 0.04 asmb
 0.04 datblk
 0.05 sym
 0.05 elfsym
 0.05 symsize = 32880
 0.05 symsize = 33312
 0.06 header
 0.17 host link: "gcc" "-m64" "-o" "/tmp/go-build620912334/b001/exe/a.out" "-rdynamic" "-Wl,--compress-debug-sections=zlib-gnu" "/tmp/go-link-627000385/go.o" "/tmp/go-link-627000385/000000.o" "/tmp/go-link-627000385/000001.o" "/tmp/go-link-627000385/000002.o" "/tmp/go-link-627000385/000003.o" "/tmp/go-link-627000385/000004.o" "/tmp/go-link-627000385/000005.o" "/tmp/go-link-627000385/000006.o" "/tmp/go-link-627000385/000007.o" "/tmp/go-link-627000385/000008.o" "/tmp/go-link-627000385/000009.o" "/tmp/go-link-627000385/000010.o" "/tmp/go-link-627000385/000011.o" "/tmp/go-link-627000385/000012.o" "/tmp/go-link-627000385/000013.o" "-g" "-O2" "-L/home/boger/golang/test/go/misc/cgo/testso" "-lcgosotest" "-g" "-O2" "-lpthread" "-no-pie"
 0.27 cpu time
23451 symbols
26908 liveness data

ALL TESTS PASSED (some were excluded)

commit 6417e91:

go tool dist test testso

##### ../misc/cgo/testso
# command-line-arguments
HEADER = -H5 -T0x11000 -D0x0 -R0x10000
 0.00 deadcode
 0.01 pclntab=247478 bytes, funcdata total 46718 bytes
 0.01 dodata
 0.01 dwarf
 0.02 symsize = 0
 0.05 reloc
 0.05 asmb
 0.05 datblk
 0.06 sym
 0.06 elfsym
 0.06 symsize = 32880
 0.06 symsize = 33312
 0.07 header
 0.22 host link: "gcc" "-m64" "-o" "/tmp/go-build377553058/b001/exe/a.out" "-rdynamic" "/tmp/go-link-215391806/go.o" "/tmp/go-link-215391806/000000.o" "/tmp/go-link-215391806/000001.o" "/tmp/go-link-215391806/000002.o" "/tmp/go-link-215391806/000003.o" "/tmp/go-link-215391806/000004.o" "/tmp/go-link-215391806/000005.o" "/tmp/go-link-215391806/000006.o" "/tmp/go-link-215391806/000007.o" "/tmp/go-link-215391806/000008.o" "/tmp/go-link-215391806/000009.o" "/tmp/go-link-215391806/000010.o" "/tmp/go-link-215391806/000011.o" "/tmp/go-link-215391806/000012.o" "/tmp/go-link-215391806/000013.o" "-g" "-O2" "-L/home/boger/golang/test/go/misc/cgo/testso" "-lcgosotest" "-g" "-O2" "-lpthread"
 0.29 cpu time
23451 symbols
26908 liveness data
./main.exe: error while loading shared libraries: R_PPC64_ADDR16_HA re10f2f8fa4 for symbol `' out of range
2018/09/05 09:53:14 Failed: exit status 127

We've seen similar problems in the past related to the use of the -pie option. In some distros, gcc uses -pie by default, so golang should be passing -no-pie to the external linker.

There were similar error messages with issue #21954.

[andybons]

@cherrymui @rsc @mdempsky @ianlancetaylor @mwhudson

[mwhudson]

Hmm this doesn't reproduce for me on Ubuntu 18.04. I can't really see why that commit would break things here, I guess one would need to dump out the command line the linker is using to test compat and see why it fails...

[laboger]

This only fails for me on Debian 9.
I am seeing binutils 2.28 and gcc 6.3.0 on that system.
Ubuntu 18.04 has binutils 2.30 and gcc 7.3.0.

The -no-pie option was passed to the linker due to this issue #17847 but now it is removed.

[laboger]

Hmm this doesn't reproduce for me on Ubuntu 18.04. I can't really see why that commit would break things here, I guess one would need to dump out the command line the linker is using to test compat and see why it fails...

As I show above, before this commit, gcc was passing -no-pie to the external linker but after this commit it does not. At some point gcc started passing -pie by default to ld so golang had to pass -no-pie to override it.

[ianlancetaylor]

Please use the go tool's -x option so that we can see exactly how cmd/link is being invoked, specifically so that we can see what the -extldflags option is.

CL 130316 was not supposed to change whether we pass -no-pie or not, it was just supposed to do a more accurate test of whether the external linker supports the option. Clearly that is breaking something for you, but we need to find out what is breaking and why.

[laboger]

Distros with binutils 2.29 or later don't fail. Debian 9 is the only one I found with 2.28. Perhaps a fix went in after 2.28 to fix a problem. I will try to track that down further.

This CL removes the -no-pie when invoking the external linker. Is that the correct/expected behavior?

[laboger]

I posted that last one before seeing your most recent comment.

Here is what I see when I hack into cmd/dist/test.go to add the -x option to the go build command (or is there any easier way to do that? Sure would help in debugging.)

/home/boger/golang/base/go/pkg/tool/linux_ppc64le/link -o $WORK/b001/exe/a.out -importcfg $WORK/b001/importcfg.link -buildmode=exe -buildid=yOqLpa4EXqY0Lp2pUyTE/7HIRw7u34JSv5mK_5IeT/KNTcbpoY3nfiB17VRqFN/yOqLpa4EXqY0Lp2pUyTE -extld=gcc $WORK/b001/pkg.a
/home/boger/golang/base/go/pkg/tool/linux_ppc64le/buildid -w $WORK/b001/exe/a.out # internal
mv $WORK/b001/exe/a.out main.exe
rm -r $WORK/b001/
./main.exe: error while loading shared libraries: R_PPC64_ADDR16_HA re138199fe4 for symbol `' out of range

I don't see -extldflags. I think -no-pie gets set in this code in cmd/link/internal/ld/lib.go, not because it was passed as an argument to the linker.

// When building a program with the default -buildmode=exe the
    // gc compiler generates code requires DT_TEXTREL in a
    // position independent executable (PIE). On systems where the
    // toolchain creates PIEs by default, and where DT_TEXTREL
    // does not work, the resulting programs will not run. See
    // issue #17847. To avoid this problem pass -no-pie to the
    // toolchain if it is supported.
    if ctxt.BuildMode == BuildModeExe {
            src := filepath.Join(*flagTmpdir, "trivial.c")
            if err := ioutil.WriteFile(src, []byte("int main() { return 0; }"), 0666); err != nil {
                    Errorf(nil, "WriteFile trivial.c failed: %v", err)
            }

            // GCC uses -no-pie, clang uses -nopie.
            for _, nopie := range []string{"-no-pie", "-nopie"} {
                    cmd := exec.Command(argv[0], nopie, "trivial.c")
                    cmd.Dir = *flagTmpdir
                    cmd.Env = append([]string{"LC_ALL=C"}, os.Environ()...)
                    out, err := cmd.CombinedOutput()
                    // GCC says "unrecognized command line option ‘-no-pie’"
                    // clang says "unknown argument: '-no-pie'"
                    supported := err == nil && !bytes.Contains(out, []byte("unrecognized")) && !bytes.Contains(out, []byte("unknown"))
                    if supported {
                            argv = append(argv, nopie)
                            break
                    }
            }
    }

[ianlancetaylor]

The only effect of CL 130316 that apparently caused this problem was to add the value of -extldflags to the external linker when testing whether -no-pie is supported. The code calls linkerFlagSupported to test whether -no-pie is supported. If it is, it is passed. CL 130316 changes linkerFlagSupported.

In the comment above you are quoting the old code. That's not what the linker code looks like now or in 1.11.

Can you change linkerFlagSupported to print exactly how it is invoking the external linker and what the linker is printing? Thanks.

[laboger]

When linkerFlagSupported was changed to pass extra linker options when linking trivial.c, this happens on Debian 9/binutils 2.28:

### linkerFlagSupported: gcc flags: [-g -O2 -L/home/boger/golang/bad/go/misc/cgo/testso -lcgosotest -g -O2 -lpthread -no-pie trivial.c] out: /home/boger/golang/bad/go/misc/cgo/testso/libcgosotest.so: undefined reference to `goCallback'
collect2: error: ld returned 1 exit status
 ###

So that causes linkerFlagSupported to return false for -no-pie since it contains the undefined string and -no-pie doesn't get added to the list of flags for ld.

I don't know why the linker error message from within linkerFlagSupported does not occur for other distros/binutils,. It is passing the same options to linkerFlagSupported on another system I tried.

[ianlancetaylor]

@laboger Thanks. Working on a patch.

[gopherbot]

Change https://golang.org/cl/134057 mentions this issue: cmd/link: don't pass all linker args when testing flag