cmd/go: (spurious?) rebuild due to compiler mtime #3506
Labels
Milestone
Comments
The go command is rebuilding package runtime - you can see the built objects in the .tar.gz files you attached - but I don't understand why it is doing that. Worse, it is rebuilding package runtime using only the .go files. It omits all the C and assembly implementation objects. (The runtime.a that is built contains only _go_.8, no other .8 files.) Does /usr/lib/go/src/pkg/runtime have any .c or .s files in it? Not having the C or assembly objects in runtime.a is what causes the morestack undefined errors. Did you unpack the tree into /usr/lib/go or did you only copy certain files? Labels changed: added priority-later, removed priority-triage. Status changed to WaitingForReply. |
Hmm, I think I have found a culprit...
In strace there is:
12188 stat64("/usr/lib/go/src/pkg/fmt", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
12188 open("/usr/lib/go/src/pkg/fmt", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
12188 getdents64(3, /* 10 entries */, 4096) = 320
12188 getdents64(3, /* 0 entries */, 4096) = 0
12188 lstat64("/usr/lib/go/src/pkg/fmt/scan_test.go", {st_mode=S_IFREG|0644,
st_size=25834, ...}) = 0
12188 lstat64("/usr/lib/go/src/pkg/fmt/scan.go", {st_mode=S_IFREG|0644, st_size=30770,
...}) = 0
12188 lstat64("/usr/lib/go/src/pkg/fmt/stringer_test.go", {st_mode=S_IFREG|0644,
st_size=2156, ...}) = 0
12188 lstat64("/usr/lib/go/src/pkg/fmt/format.go", {st_mode=S_IFREG|0644, st_size=10125,
...}) = 0
12188 lstat64("/usr/lib/go/src/pkg/fmt/print.go", {st_mode=S_IFREG|0644, st_size=27721,
...}) = 0
12188 lstat64("/usr/lib/go/src/pkg/fmt/doc.go", {st_mode=S_IFREG|0644, st_size=7793,
...}) = 0
12188 lstat64("/usr/lib/go/src/pkg/fmt/export_test.go", {st_mode=S_IFREG|0644,
st_size=196, ...}) = 0
12188 lstat64("/usr/lib/go/src/pkg/fmt/fmt_test.go", {st_mode=S_IFREG|0644,
st_size=22484, ...}) = 0
12188 close(3) = 0
I guess this is the code you use to check if the package needs to be recompiled.
However even if I set timestamps on all directories and all source files, the fmt still
gets recompiled, which I think is wrong (or at least it doesn't really make sense to
have compiled objects in the code).
root@howl:~# ls -ld /usr/lib/go/pkg/linux_386/fmt.a
-rw-r--r-- 1 root root 817678 Apr 6 15:18 /usr/lib/go/pkg/linux_386/fmt.a
root@howl:~# ls -ld /usr/lib/go/src/pkg/fmt
drwxr-xr-x 2 root root 4096 Jan 1 01:01 /usr/lib/go/src/pkg/fmt
root@howl:~# ls -ld /usr/lib/go/src/pkg/fmt/*
-rw-r--r-- 1 root root 7793 Jan 1 01:01 /usr/lib/go/src/pkg/fmt/doc.go
-rw-r--r-- 1 root root 196 Jan 1 01:01 /usr/lib/go/src/pkg/fmt/export_test.go
-rw-r--r-- 1 root root 22484 Jan 1 01:01 /usr/lib/go/src/pkg/fmt/fmt_test.go
-rw-r--r-- 1 root root 10125 Jan 1 01:01 /usr/lib/go/src/pkg/fmt/format.go
-rw-r--r-- 1 root root 27721 Jan 1 01:01 /usr/lib/go/src/pkg/fmt/print.go
-rw-r--r-- 1 root root 30770 Jan 1 01:01 /usr/lib/go/src/pkg/fmt/scan.go
-rw-r--r-- 1 root root 25834 Jan 1 01:01 /usr/lib/go/src/pkg/fmt/scan_test.go
-rw-r--r-- 1 root root 2156 Jan 1 01:01 /usr/lib/go/src/pkg/fmt/stringer_test.go
This is a clearly regression from 0.60.x, because it worked before (but it didn't have
to 'go' command at all).
And one more observation:
If I remove src tree altogether, the go command don't even check if there are binary
libraries available and bail out:
ondrej@howl:~$ go build helloworld.go
helloworld.go:5:8: import "fmt": cannot find package
package runtime: import "runtime": cannot find package
But I can still compile the helloworld using 8g/8l combo just fine:
ondrej@howl:~$ /usr/lib/go/pkg/tool/linux_386/8g helloworld.go
ondrej@howl:~$ /usr/lib/go/pkg/tool/linux_386/8l helloworld.8
ondrej@howl:~$ ./8.out
Hello, world!
From my viewpoint there's something twisted in go command which needs to have sources
available at every possible moment. If that's your design decision I can live with that
(and copy the rest of the source tree to the package), but I little bit doubt about the
fact that you always recompile all needed libraries every time.
|
No special settings, I am testing the package inside clean chrooted Debian unstable. $ go env GOROOT="/usr/lib/go" GOBIN="" GOARCH="amd64" GOCHAR="6" GOOS="linux" GOEXE="" GOHOSTARCH="amd64" GOHOSTOS="linux" GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64" GOGCCFLAGS="-g -O2 -fPIC -m64 -pthread" CGO_ENABLED="1" |
If GOARCH is amd64, then the go command needs /usr/lib/go/pkg/linux_amd64/fmt.a. The existence of /usr/lib/go/pkg/linux_386/fmt.a doesn't help. When you are compiling code from a directory outside /usr/lib/go, the go command will not rebuild code from /usr/lib/go unless there are no binaries at all. Russ |
{6,8}{g,l} is able to produce binary even if the source tree is not installed at all:
ondrej@howl:~$ ls
fibonacci.6 fibonacci.go
ondrej@howl:~$ find -L /usr/lib/go/src
/usr/lib/go/src
/usr/lib/go/src/pkg
/usr/lib/go/src/pkg/runtime
/usr/lib/go/src/pkg/runtime/cgocall.h
/usr/lib/go/src/pkg/runtime/runtime.h
/usr/lib/go/src/Make.dist
ondrej@howl:~$ /usr/lib/go/pkg/tool/linux_amd64/6g fibonacci.go
ondrej@howl:~$ /usr/lib/go/pkg/tool/linux_amd64/6l fibonacci.6
ondrej@howl:~$ ls
6.out fibonacci.6 fibonacci.go
ondrej@howl:~$ ./6.out
1 1 2 3 5
It is just a problem of 'go' command not checking if it has binary libraries already
available.
|
> The go command requires source code to be present. > Eventually we will lift that restriction, but we did not lift it > for Go 1.0. Note that even though 6g works, tools like Ok, even though this is a change from previous stable release, I'll handle that somehow. > godoc still need source code to tell people how to use fmt, > so it does serve other purposes too. Yes, I know and this is handled. I will follow-up in next comment... |
I have tried one more thing... with binaries from Debian package I have replaced
/usr/lib/go with content from binary distribution. It works. Removed everything with
exception of /usr/lib/go/{src,pkg}. It works.
When I have a /usr/lib/go/pkg/ from package and /usr/lib/go/src/ from upstream binary
distribution. The list of files is exactly same. And it tries to recompile the libraries.
So, the question is what is the heuristics to detect that the file needs to be
recompiled?
|
1. If there's no source at all, it's not a real package. 2. If the source exists and the binary does not, compile the source. 3. If the source exists and is newer than the binary _and_ you are working in a directory inside the GOROOT, recompile the source. I believe you are working outside the GOROOT, so #3 does not apply. What does 'go env' print in both cases? In your earlier mail you were trying to run a 64-bit (amd64) Go with 32-bit (386) pre-compiled packages. My guess is that the same thing is happening here. If 'go env' says GOARCH="amd64" then you need pkg/linux_amd64. If 'go env' says GOARCH="386" then you need pkg/linux_386. To understand more about what the go command thinks of a given package (say, fmt), run 'go list -json fmt'. Does the "Target" exist? |
> You can create a dummy file in each directory I don't really need to do that, since I am already distribution "find src -name '*.go'" in the golang-src package, so it should work. The problem here is that it keeps recompiling stuff even if it doesn't need to and I am unable to understand why. |
> To understand more about what the go command thinks of a given package
> (say, fmt), run 'go list -json fmt'. Does the "Target" exist?
ondrej@howl:~$ go list -json runtime | grep Target
"Target": "/usr/lib/go/pkg/linux_amd64/runtime.a",
ondrej@howl:~$ ls -l /usr/lib/go/pkg/linux_amd64/runtime.a
-rw-r--r-- 1 root root 883732 Apr 11 14:35 /usr/lib/go/pkg/linux_amd64/runtime.a
So, it does exist, also all sources are in place:
$ find /usr/lib/go/src/pkg/runtime/ -type f | wc -l
220
But still, when I try to compile fibonacci.go:
WORK=/tmp/go-build260021805
runtime
mkdir -p $WORK/runtime/_obj/
cd /usr/lib/go/src/pkg/runtime
/usr/lib/go/pkg/tool/linux_amd64/6g -o $WORK/runtime/_obj/_go_.6 -p runtime -+ -D
_/usr/lib/go/src/pkg/runtime -I $WORK ./compiler.go ./debug.go ./error.go ./extern.go
./mem.go ./softfloat64.go ./type.go ./zgoarch_amd64.go ./zgoos_linux.go
./zruntime_defs_linux_amd64.go ./zversion.go
[...more files compiling...]
/usr/lib/go/pkg/tool/linux_amd64/pack grc $WORK/runtime.a $WORK/runtime/_obj/_go_.6
$WORK/runtime/_obj/alg.6 $WORK/runtime/_obj/atomic_amd64.6 $WORK/runtime/_obj/cgocall.6
$WORK/runtime/_obj/chan.6 $WORK/runtime/_obj/closure_amd64.6
$WORK/runtime/_obj/complex.6 $WORK/runtime/_obj/cpuprof.6 $WORK/runtime/_obj/float.6
$WORK/runtime/_obj/hashmap.6 $WORK/runtime/_obj/iface.6 $WORK/runtime/_obj/lock_futex.6
$WORK/runtime/_obj/mcache.6 $WORK/runtime/_obj/mcentral.6 $WORK/runtime/_obj/mem_linux.6
$WORK/runtime/_obj/mfinal.6 $WORK/runtime/_obj/mfixalloc.6 $WORK/runtime/_obj/mgc0.6
$WORK/runtime/_obj/mheap.6 $WORK/runtime/_obj/msize.6 $WORK/runtime/_obj/print.6
$WORK/runtime/_obj/proc.6 $WORK/runtime/_obj/rune.6 $WORK/runtime/_obj/runtime.6
$WORK/runtime/_obj/signal_linux_amd64.6 $WORK/runtime/_obj/signal_unix.6
$WORK/runtime/_obj/slice.6 $WORK/runtime/_obj/symtab.6 $WORK/runtime/_obj/thread_linux.6
$WORK/runtime/_obj/traceback_x86.6 $WORK/runtime/_obj/zmalloc_amd64.6
$WORK/runtime/_obj/zmprof_amd64.6 $WORK/runtime/_obj/zruntime1_amd64.6
$WORK/runtime/_obj/zsema_amd64.6 $WORK/runtime/_obj/zsigqueue_amd64.6
$WORK/runtime/_obj/zstring_amd64.6 $WORK/runtime/_obj/ztime_amd64.6
$WORK/runtime/_obj/asm_amd64.6 $WORK/runtime/_obj/memmove_amd64.6
$WORK/runtime/_obj/rt0_linux_amd64.6 $WORK/runtime/_obj/sys_linux_amd64.6
command-line-arguments
mkdir -p $WORK/command-line-arguments/_obj/
cd /home/ondrej
/usr/lib/go/pkg/tool/linux_amd64/6g -o $WORK/command-line-arguments/_obj/_go_.6 -p
command-line-arguments -D _/home/ondrej -I $WORK ./fibonacci.go
/usr/lib/go/pkg/tool/linux_amd64/pack grc $WORK/command-line-arguments.a
$WORK/command-line-arguments/_obj/_go_.6
cd .
/usr/lib/go/pkg/tool/linux_amd64/6l -o fibonacci -L $WORK $WORK/command-line-arguments.a
It should not really happen according to your description, but it does.
|
Attached in 10MB parts. And thank you that you care. Attachments:
|
Second part Attachments:
|
Just cat them together. Last part. Attachments:
|
ondrej@howl:~$ go list -json runtime
{
"Dir": "/usr/lib/go/src/pkg/runtime",
"ImportPath": "runtime",
"Name": "runtime",
"Doc": "Copyright 2011 The Go Authors.",
"Target": "/usr/lib/go/pkg/linux_amd64/runtime.a",
"Goroot": true,
"Standard": true,
"Stale": true,
"Root": "/usr/lib/go",
"GoFiles": [
"compiler.go",
"debug.go",
"error.go",
"extern.go",
"mem.go",
"softfloat64.go",
"type.go",
"zgoarch_amd64.go",
"zgoos_linux.go",
"zruntime_defs_linux_amd64.go",
"zversion.go"
],
"CFiles": [
"alg.c",
"atomic_amd64.c",
"cgocall.c",
"chan.c",
"closure_amd64.c",
"complex.c",
"cpuprof.c",
"float.c",
"hashmap.c",
"iface.c",
"lock_futex.c",
"mcache.c",
"mcentral.c",
"mem_linux.c",
"mfinal.c",
"mfixalloc.c",
"mgc0.c",
"mheap.c",
"msize.c",
"print.c",
"proc.c",
"rune.c",
"runtime.c",
"signal_linux_amd64.c",
"signal_unix.c",
"slice.c",
"symtab.c",
"thread_linux.c",
"traceback_x86.c",
"zmalloc_amd64.c",
"zmprof_amd64.c",
"zruntime1_amd64.c",
"zsema_amd64.c",
"zsigqueue_amd64.c",
"zstring_amd64.c",
"ztime_amd64.c"
],
"HFiles": [
"arch_amd64.h",
"cgocall.h",
"defs_linux_amd64.h",
"hashmap.h",
"malloc.h",
"os_linux.h",
"runtime.h",
"signals_linux.h",
"stack.h",
"type.h",
"zasm_linux_amd64.h"
],
"SFiles": [
"asm_amd64.s",
"memmove_amd64.s",
"rt0_linux_amd64.s",
"sys_linux_amd64.s"
],
"Imports": [
"unsafe"
],
"Deps": [
"unsafe"
],
"TestGoFiles": [
"export_test.go"
],
"XTestGoFiles": [
"append_test.go",
"chan_test.go",
"closure_test.go",
"gc_test.go",
"mfinal_test.go",
"proc_test.go",
"runtime_linux_test.go",
"runtime_test.go",
"softfloat64_test.go",
"stack_test.go",
"symtab_test.go"
],
"XTestImports": [
"io",
"math",
"math/rand",
"runtime",
"strings",
"sync",
"sync/atomic",
"syscall",
"testing",
"unsafe"
]
}
ondrej@howl:~$ go list -json fibonacci.go
{
"Dir": "/home/ondrej",
"ImportPath": "command-line-arguments",
"Name": "main",
"Stale": true,
"GoFiles": [
"fibonacci.go"
],
"Deps": [
"runtime",
"unsafe"
]
}
|
I believe I found it: the compiler and linker have time stamps newer than the object files, and the go command assumes that if the compiler is new, everything needs rebuilding. This is not a great assumption, but it is what it is for now. If you make the dates on the compiler older than the dates on the .a files I think everything will be happy. This is something we should decide for Go 1.1. Right now I think the only option is the workaround. Labels changed: added go1.1. Status changed to Thinking. |
I am having a similar problem here, but I suspect it is because I am using a tree of symlinks to point to sources in /usr/share. I have not fully dug into it, but go appears to be using lstat (instead of stat) so it gets the mtime of the symlink instead of target. If this is true, I can work around this by setting the mtime of the symlinks, but this seems wrong (and possibly not portable). Should go be using stat instead of lstat? (Am I on the right track here?) |
|
CL https://golang.org/cl/22432 mentions this issue. |
mk0x9
pushed a commit
to mk0x9/go
that referenced
this issue
Apr 27, 2016
It comes up every few months that we can't understand why the go command is rebuilding some package. Add diagnostics so that the go command can explain itself if asked. For golang#2775, golang#3506, golang#12074. Change-Id: I1c73b492589b49886bf31a8f9d05514adbd6ed70 Reviewed-on: https://go-review.googlesource.com/22432 Reviewed-by: Rob Pike <r@golang.org>
|
Essentially everything involved in this report has been rewritten. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Attachments:
The text was updated successfully, but these errors were encountered: