runtime: gopls -v crashes immediately when linked with Xcode 15 beta

[mattmassicotte]

gopls version

Tested both 0.11.0 and 0.12.4. Cannot capture version output, as it crashes immediately on launch. Output is:

fatal: morestack on g0

go env

GO111MODULE=""
GOARCH="arm64"
GOBIN=""
GOCACHE="/Users/matt/Library/Caches/go-build"
GOENV="/Users/matt/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/matt/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/matt/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/opt/homebrew/Cellar/go/1.20.5/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/opt/homebrew/Cellar/go/1.20.5/libexec/pkg/tool/darwin_arm64"
GOVCS=""
GOVERSION="go1.20.5"
GCCGO="gccgo"
AR="ar"
CC="cc"
CXX="c++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/9_/q1rgv_js2rb518ql_svng32c0000gn/T/go-build3080324972=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Built gopls 0.12.4 with 1.20.5. Ran "gopls -v". Crashes immediately. Things were working fine on beta 2. Now, normally, I'd just wait because this is a beta. But, I always get worried when stuff like this happens, so I figured I'd at least make you aware.

[findleyr]

Hi, just to be sure I understand, these are the steps you followed:

1. install macOS 14 beta 3 (https://developer.apple.com/news/releases/?id=07052023c)
2. build gopls@v0.12.4 (latest), using the pre-compiled mac binaries for go 1.20.5
3. run gopls -v: crash

Is that right? Is there any output before the crash?

[mattmassicotte]

Those are the steps I followed. But, I had first tried a version of gopls I had previously built with an earlier Go version/macOS and that also wasn't able to launch.

All combinations produced the same output:

fatal: morestack on g0

[findleyr]

Thanks again (and sorry I missed that you'd already included the output).

CC @golang/runtime for awareness. I'm not sure if this is something we'd normally escalate to Apple.

Are you experiencing this type of problem with other Go binaries?

[mattmassicotte]

So far, I have not seen this occur with any other go binaries, but my testing has been pretty limited. I've also filed a bug with Apple about it.

[prattmic]

Would it be possible for you to clone https://github.com/golang/go, checkout tag go1.20.5 and then run src/all.bash to run all of the Go tests to see if any of those fail?

[mattmassicotte]

Just ran them, and in fact, they fail.

Building Go cmd/dist using /opt/homebrew/Cellar/go/1.20.5/libexec. (go1.20.5 darwin/arm64)
Building Go toolchain1 using /opt/homebrew/Cellar/go/1.20.5/libexec.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
Building packages and commands for darwin/arm64.

##### Test execution environment.
# GOARCH: arm64
# CPU: 
# GOOS: darwin
# OS Version: Darwin 23.0.0 Darwin Kernel Version 23.0.0: Fri Jun 30 17:50:12 PDT 2023; root:xnu-10002.0.168.505.3~1/RELEASE_ARM64_T6000 arm64

##### Testing packages.
ok  	archive/tar	0.189s
ok  	archive/zip	0.299s
ok  	bufio	0.143s
ok  	bytes	0.371s
ok  	compress/bzip2	0.291s
ok  	compress/flate	0.706s
ok  	compress/gzip	1.549s
ok  	compress/lzw	0.068s
ok  	compress/zlib	0.503s
ok  	container/heap	0.240s
ok  	container/list	0.337s
ok  	container/ring	0.289s
ok  	context	0.198s
ok  	crypto	0.385s
ok  	crypto/aes	0.445s
ok  	crypto/cipher	0.253s
ok  	crypto/des	0.258s
ok  	crypto/dsa	0.310s
ok  	crypto/ecdh	0.300s
ok  	crypto/ecdsa	0.087s
ok  	crypto/ed25519	0.203s
ok  	crypto/elliptic	0.220s
ok  	crypto/hmac	0.116s
ok  	crypto/internal/alias	0.055s
ok  	crypto/internal/bigmod	0.196s
ok  	crypto/internal/boring	0.144s
ok  	crypto/internal/boring/bcache	0.235s
ok  	crypto/internal/edwards25519	2.492s
ok  	crypto/internal/edwards25519/field	2.662s
ok  	crypto/internal/nistec	0.463s
ok  	crypto/internal/nistec/fiat	0.265s [no tests to run]
ok  	crypto/md5	0.297s
ok  	crypto/rand	0.283s
ok  	crypto/rc4	0.259s
ok  	crypto/rsa	0.432s
ok  	crypto/sha1	0.111s
ok  	crypto/sha256	0.063s
ok  	crypto/sha512	0.107s
ok  	crypto/subtle	0.126s
ok  	crypto/tls	0.330s
ok  	crypto/x509	0.330s
ok  	database/sql	0.322s
ok  	database/sql/driver	0.307s
ok  	debug/buildinfo	0.470s
ok  	debug/dwarf	0.423s
ok  	debug/elf	0.315s
ok  	debug/gosym	0.360s
ok  	debug/macho	0.333s
ok  	debug/pe	0.179s
ok  	debug/plan9obj	0.174s
ok  	embed	0.208s [no tests to run]
ok  	embed/internal/embedtest	0.252s
ok  	encoding/ascii85	0.277s
ok  	encoding/asn1	0.238s
ok  	encoding/base32	0.299s
ok  	encoding/base64	0.059s
ok  	encoding/binary	0.112s
ok  	encoding/csv	0.066s
ok  	encoding/gob	0.669s
ok  	encoding/hex	0.130s
ok  	encoding/json	0.194s
ok  	encoding/pem	0.639s
ok  	encoding/xml	0.186s
ok  	errors	0.227s
ok  	expvar	0.286s
ok  	flag	0.385s
ok  	fmt	0.114s
ok  	go/ast	0.070s
ok  	go/build	1.246s
ok  	go/build/constraint	0.252s
ok  	go/constant	0.173s
ok  	go/doc	0.338s
ok  	go/doc/comment	1.039s
ok  	go/format	0.414s
ok  	go/importer	0.590s
ok  	go/internal/gccgoimporter	0.394s
ok  	go/internal/gcimporter	1.296s
ok  	go/internal/srcimporter	7.546s
ok  	go/parser	0.271s
ok  	go/printer	0.267s
ok  	go/scanner	0.073s
ok  	go/token	0.073s
ok  	go/types	2.492s
ok  	hash	0.260s
ok  	hash/adler32	0.363s
ok  	hash/crc32	0.311s
ok  	hash/crc64	0.210s
ok  	hash/fnv	0.132s
ok  	hash/maphash	0.500s
ok  	html	0.412s
ok  	html/template	0.349s
ok  	image	0.114s
ok  	image/color	0.121s
ok  	image/draw	0.118s
ok  	image/gif	0.197s
ok  	image/jpeg	0.168s
ok  	image/png	0.209s
ok  	index/suffixarray	0.218s
ok  	internal/abi	1.070s
ok  	internal/buildcfg	0.207s
ok  	internal/coverage/cformat	0.306s
ok  	internal/coverage/cmerge	0.159s
ok  	internal/coverage/pods	0.267s
ok  	internal/coverage/slicereader	0.356s
ok  	internal/coverage/slicewriter	0.458s
ok  	internal/coverage/test	0.114s
ok  	internal/cpu	0.065s
ok  	internal/dag	0.269s
ok  	internal/diff	0.218s
ok  	internal/fmtsort	0.111s
ok  	internal/fuzz	0.168s
ok  	internal/godebug	0.373s
ok  	internal/intern	0.686s
ok  	internal/itoa	0.316s
ok  	internal/poll	0.111s
ok  	internal/profile	0.122s
ok  	internal/reflectlite	0.084s
ok  	internal/safefilepath	0.331s
ok  	internal/saferio	0.444s
ok  	internal/singleflight	0.401s
ok  	internal/testenv	0.175s
ok  	internal/trace	0.249s
ok  	internal/types/errors	0.337s
ok  	internal/unsafeheader	0.474s
ok  	internal/xcoff	0.279s
ok  	io	0.104s
ok  	io/fs	0.504s
ok  	io/ioutil	0.218s
ok  	log	0.165s
ok  	log/syslog	1.513s
ok  	math	0.072s
ok  	math/big	0.664s
ok  	math/bits	0.070s
ok  	math/cmplx	0.118s
ok  	math/rand	0.275s
ok  	mime	0.168s
ok  	mime/multipart	0.633s
ok  	mime/quotedprintable	0.306s
ok  	net	2.215s
ok  	net/http	2.578s
ok  	net/http/cgi	1.363s
ok  	net/http/cookiejar	0.250s
ok  	net/http/fcgi	0.463s
ok  	net/http/httptest	0.492s
ok  	net/http/httptrace	0.302s
ok  	net/http/httputil	1.084s
ok  	net/http/internal	0.639s
ok  	net/http/internal/ascii	0.589s
# plugin.test
ld: warning: -bind_at_load is deprecated on macOS
ld: warning: '/private/var/folders/9_/q1rgv_js2rb518ql_svng32c0000gn/T/go-link-4136702197/go.o' has malformed LC_DYSYMTAB, expected 91 undefined symbols to start at index 15225, found 97 undefined symbols starting at index 69
ok  	net/http/pprof	4.301s
ok  	net/internal/socktest	0.321s
ok  	net/mail	0.182s
ok  	net/netip	0.303s
ok  	net/rpc	0.330s
ok  	net/rpc/jsonrpc	0.153s
ok  	net/smtp	0.436s
ok  	net/textproto	0.495s
ok  	net/url	0.204s
ok  	os	0.844s
ok  	os/exec	0.328s
ok  	os/exec/internal/fdtest	0.287s
ok  	os/signal	2.894s
ok  	os/user	0.189s
ok  	path	0.335s
ok  	path/filepath	0.091s
ok  	plugin	0.091s
ok  	reflect	0.475s
ok  	regexp	0.402s
ok  	regexp/syntax	0.623s
ok  	runtime	28.409s
ok  	runtime/cgo	0.320s
ok  	runtime/coverage	0.266s
ok  	runtime/debug	0.624s
ok  	runtime/internal/atomic	0.258s
ok  	runtime/internal/math	0.363s
ok  	runtime/internal/sys	0.467s
ok  	runtime/metrics	0.411s
ok  	runtime/pprof	6.636s
ok  	runtime/trace	2.314s
ok  	sort	0.097s
ok  	strconv	0.394s
ok  	strings	0.165s
ok  	sync	0.369s
ok  	sync/atomic	0.636s
ok  	syscall	0.153s
ok  	testing	1.000s
ok  	testing/fstest	0.285s
ok  	testing/iotest	0.327s
ok  	testing/quick	0.191s
ok  	text/scanner	0.231s
ok  	text/tabwriter	0.386s
ok  	text/template	0.161s
ok  	text/template/parse	0.171s
ok  	time	2.602s
ok  	unicode	0.291s
ok  	unicode/utf16	0.127s
ok  	unicode/utf8	0.178s
ok  	cmd/addr2line	0.635s
ok  	cmd/api	5.450s
ok  	cmd/asm/internal/asm	0.557s
ok  	cmd/asm/internal/lex	0.174s
ok  	cmd/compile/internal/abt	0.210s
ok  	cmd/compile/internal/amd64	0.173s
ok  	cmd/compile/internal/base	0.218s
ok  	cmd/compile/internal/compare	0.086s
ok  	cmd/compile/internal/dwarfgen	0.518s
ok  	cmd/compile/internal/importer	1.700s
ok  	cmd/compile/internal/ir	0.248s
ok  	cmd/compile/internal/logopt	0.428s
ok  	cmd/compile/internal/noder	0.241s
ok  	cmd/compile/internal/reflectdata	0.157s [no tests to run]
ok  	cmd/compile/internal/ssa	33.958s
ok  	cmd/compile/internal/syntax	0.187s
ok  	cmd/compile/internal/test	10.216s
ok  	cmd/compile/internal/typecheck	0.762s
ok  	cmd/compile/internal/types	0.327s
ok  	cmd/compile/internal/types2	7.927s
ok  	cmd/covdata	0.579s
ok  	cmd/cover	2.490s
ok  	cmd/dist	0.443s
ok  	cmd/doc	0.511s
ok  	cmd/fix	3.800s
ok  	cmd/go	35.257s
ok  	cmd/go/internal/auth	0.094s
ok  	cmd/go/internal/cache	0.302s
ok  	cmd/go/internal/fsys	0.104s
ok  	cmd/go/internal/generate	0.139s
ok  	cmd/go/internal/get	0.249s
ok  	cmd/go/internal/imports	0.263s
ok  	cmd/go/internal/load	0.115s
ok  	cmd/go/internal/lockedfile	0.224s
ok  	cmd/go/internal/lockedfile/internal/filelock	0.116s
ok  	cmd/go/internal/modconv	0.067s
ok  	cmd/go/internal/modfetch	0.122s
ok  	cmd/go/internal/modfetch/codehost	0.112s
ok  	cmd/go/internal/modfetch/zip_sum_test	0.241s
ok  	cmd/go/internal/modindex	0.479s
ok  	cmd/go/internal/modload	0.125s
ok  	cmd/go/internal/mvs	0.217s
ok  	cmd/go/internal/par	0.270s
ok  	cmd/go/internal/str	0.166s
ok  	cmd/go/internal/test	0.144s
ok  	cmd/go/internal/vcs	0.128s
ok  	cmd/go/internal/vcweb	0.302s
ok  	cmd/go/internal/vcweb/vcstest	5.502s
ok  	cmd/go/internal/web	0.329s
ok  	cmd/go/internal/work	0.137s
ok  	cmd/gofmt	0.212s
ok  	cmd/internal/archive	10.283s
ok  	cmd/internal/buildid	0.458s
ok  	cmd/internal/cov	1.487s
ok  	cmd/internal/dwarf	0.135s
ok  	cmd/internal/edit	0.345s
ok  	cmd/internal/goobj	0.365s
ok  	cmd/internal/moddeps	3.465s
ok  	cmd/internal/notsha256	0.295s
ok  	cmd/internal/obj	0.794s
ok  	cmd/internal/obj/arm64	0.249s
ok  	cmd/internal/obj/ppc64	0.608s
ok  	cmd/internal/obj/riscv	0.244s
ok  	cmd/internal/obj/s390x	0.088s
ok  	cmd/internal/obj/x86	5.406s
ok  	cmd/internal/objabi	0.282s
ok  	cmd/internal/pkgpath	0.231s
ok  	cmd/internal/pkgpattern	0.227s
ok  	cmd/internal/quoted	0.351s
ok  	cmd/internal/src	0.387s
ok  	cmd/internal/test2json	0.468s
--- FAIL: TestDWARF (0.00s)
    --- FAIL: TestDWARF/testprogcgo (1.89s)
        dwarf_test.go:170: ErrUnknownPC
FAIL
FAIL	cmd/link	13.695s
ok  	cmd/link/internal/benchmark	0.071s
--- FAIL: TestRuntimeTypeAttrExternal (2.65s)
    dwarf_test.go:105: ## build output:
        # command-line-arguments
        ld: warning: '/private/var/folders/9_/q1rgv_js2rb518ql_svng32c0000gn/T/go-link-644603662/go.o' has malformed LC_DYSYMTAB, expected 44 undefined symbols to start at index 1548, found 50 undefined symbols starting at index 18
    dwarf_test.go:806: wanted 1 DIE named *main.X, found 0
FAIL
FAIL	cmd/link/internal/ld	13.489s
ok  	cmd/link/internal/loader	0.315s
ok  	cmd/nm	2.239s
ok  	cmd/objdump	7.029s
ok  	cmd/pack	3.898s
ok  	cmd/pprof	0.187s
ok  	cmd/trace	0.676s
ok  	cmd/vet	6.458s
FAIL
go tool dist: Failed: exit status 1

[cherrymui]

Most of the errors in #61190 (comment) should be addressed with CL stack https://golang.org/cl/505415 . I think they are caused by the (buggy or intentional or a mixture of) behavior changes of Apple's new linker. What version of the C toolchain and Xcode are you using? Could you share the output of ld -v? (I thought the new Apple linker has not been made default, maybe it does?)

fatal: morestack on g0

This is more concerning. Does this occur only with previously-built binaries, or it also occurs with new binaries?

I'll update my macOS machine to the beta and see tonight...

[findleyr]

Removing the gopls label to hide this from our triage dashboard. Please re-add it if this appears to be directly related to gopls in any way.