wasm: duplicate import in wasip1 compiled binary

[johanbrandhorst]

What version of Go are you using (go version)?

$ go version
go version devel go1.21-993707a9d6 Tue May 30 20:19:54 2023 +0000 linux/arm64

What did you do?

package main

import (
	"crypto/rand"
	"encoding/hex"
	"fmt"
)

func main() {
	b := [16]byte{}
	rand.Read(b[:])
	fmt.Println(hex.EncodeToString(b[:]))
}

GOARCH=wasm GOOS=wasip1 go build -o main.wasm main.go

wasm2wat main.wasm

The imports in the generated .wat file are:

  (import "wasi_snapshot_preview1" "sched_yield" (func (;0;) (type 1)))
  (import "wasi_snapshot_preview1" "proc_exit" (func (;1;) (type 2)))
  (import "wasi_snapshot_preview1" "args_get" (func (;2;) (type 3)))
  (import "wasi_snapshot_preview1" "args_sizes_get" (func (;3;) (type 3)))
  (import "wasi_snapshot_preview1" "clock_time_get" (func (;4;) (type 4)))
  (import "wasi_snapshot_preview1" "environ_get" (func (;5;) (type 3)))
  (import "wasi_snapshot_preview1" "environ_sizes_get" (func (;6;) (type 3)))
  (import "wasi_snapshot_preview1" "fd_write" (func (;7;) (type 5)))
  (import "wasi_snapshot_preview1" "random_get" (func (;8;) (type 3)))
  (import "wasi_snapshot_preview1" "poll_oneoff" (func (;9;) (type 5)))
  (import "wasi_snapshot_preview1" "fd_close" (func (;10;) (type 0)))
  (import "wasi_snapshot_preview1" "fd_read" (func (;11;) (type 5)))
  (import "wasi_snapshot_preview1" "fd_write" (func (;12;) (type 5)))
  (import "wasi_snapshot_preview1" "random_get" (func (;13;) (type 3)))
  (import "wasi_snapshot_preview1" "fd_fdstat_get" (func (;14;) (type 3)))
  (import "wasi_snapshot_preview1" "fd_fdstat_set_flags" (func (;15;) (type 3)))
  (import "wasi_snapshot_preview1" "fd_prestat_get" (func (;16;) (type 3)))
  (import "wasi_snapshot_preview1" "fd_prestat_dir_name" (func (;17;) (type 6)))

Both fd_write and random_get appear twice.

What did you expect to see?

Each import appears only once

This isn't an invalid wasm file and doesn't affect the end result much, but there is clearly some part of the compiler that isn't weeding out duplicate imports for wasm.

[HarikrishnanBalagopal]

Also happening for me with this code

The duplicate imports for fd_write in the compiled wasm (disassembled with wasm2wat)

(module
  (type (;0;) (func (param i32) (result i32)))
  (type (;1;) (func (result i32)))
  (type (;2;) (func (param i32)))
  (type (;3;) (func (param i32 i32) (result i32)))
  (type (;4;) (func (param i32 i64 i32) (result i32)))
  (type (;5;) (func (param i32 i32 i32 i32) (result i32)))
  (type (;6;) (func (param i32 i32 i32 i64 i32) (result i32)))
  (type (;7;) (func (param i32 i32 i32 i32 i32) (result i32)))
  (type (;8;) (func (param i32 i32 i32 i32 i32 i64 i64 i32 i32) (result i32)))
  (type (;9;) (func (param i32 i32 i32) (result i32)))
  (type (;10;) (func (param i64 i64 i64 i64) (result i64)))
  (type (;11;) (func (param i64 i64 i64) (result i64)))
  (type (;12;) (func (param i64) (result i64)))
  (type (;13;) (func (result i64)))
  (type (;14;) (func))
  (type (;15;) (func (param i64 i64) (result i64)))
  (type (;16;) (func (param f64) (result i64)))
  (import "wasi_snapshot_preview1" "sched_yield" (func (;0;) (type 1)))
  (import "wasi_snapshot_preview1" "proc_exit" (func (;1;) (type 2)))
  (import "wasi_snapshot_preview1" "args_get" (func (;2;) (type 3)))
  (import "wasi_snapshot_preview1" "args_sizes_get" (func (;3;) (type 3)))
  (import "wasi_snapshot_preview1" "clock_time_get" (func (;4;) (type 4)))
  (import "wasi_snapshot_preview1" "environ_get" (func (;5;) (type 3)))
  (import "wasi_snapshot_preview1" "environ_sizes_get" (func (;6;) (type 3)))
  (import "wasi_snapshot_preview1" "fd_write" (func (;7;) (type 5)))
  (import "wasi_snapshot_preview1" "random_get" (func (;8;) (type 3)))
  (import "wasi_snapshot_preview1" "poll_oneoff" (func (;9;) (type 5)))
  (import "wasi_snapshot_preview1" "fd_close" (func (;10;) (type 0)))
  (import "wasi_snapshot_preview1" "fd_read" (func (;11;) (type 5)))
  (import "wasi_snapshot_preview1" "fd_readdir" (func (;12;) (type 6)))
  (import "wasi_snapshot_preview1" "fd_filestat_get" (func (;13;) (type 3)))
  (import "wasi_snapshot_preview1" "fd_write" (func (;14;) (type 5)))
  (import "wasi_snapshot_preview1" "path_filestat_get" (func (;15;) (type 7)))
  (import "wasi_snapshot_preview1" "path_open" (func (;16;) (type 8)))
  (import "wasi_snapshot_preview1" "fd_fdstat_get" (func (;17;) (type 3)))
  (import "wasi_snapshot_preview1" "fd_fdstat_set_flags" (func (;18;) (type 3)))
  (import "wasi_snapshot_preview1" "fd_prestat_get" (func (;19;) (type 3)))
  (import "wasi_snapshot_preview1" "fd_prestat_dir_name" (func (;20;) (type 9)))
  (func $go_buildid (type 0) (param i32) (result i32)

Compiled with

$ go version
go version go1.21.0 darwin/amd64
$ CGO_ENABLED=0 GOOS=wasip1 GOARCH=wasm go build -o bin/test.wasm .
$ echo $?
0

The Golang code

package main

import (
	"os"

	"github.com/sirupsen/logrus"
)

func must(err error) {
	if err != nil {
		panic(err)
	}
}

func main() {
	logrus.Infof("start")
	pwd, err := os.Getwd()
	logrus.Infof("pwd: '%s'", pwd)
	must(err)
	fs, err := os.ReadDir(".")
	must(err)
	for _, f := range fs {
		logrus.Infof("f: %+v", f)
	}
	b, err := os.ReadFile("./dep.json")
	must(err)
	logrus.Infof("The contents are:\n%s", string(b))
	logrus.Infof("done")
}

The go.mod file

module foo.com/b

go 1.21

require github.com/sirupsen/logrus v1.9.4-0.20230606125235-dd1b4c2e81af

require golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8 // indirect

The go.sum file

github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/sirupsen/logrus v1.9.4-0.20230606125235-dd1b4c2e81af h1:Sp5TG9f7K39yfB+If0vjp97vuT74F72r8hfRpP8jLU0=
github.com/sirupsen/logrus v1.9.4-0.20230606125235-dd1b4c2e81af/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8 h1:0A+M6Uqn+Eje4kHMK80dtF3JCXC4ykBgQG4Fe06QRhQ=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

[elewis787]

This is likely known, but it appears to be related to the double import definition between os_wasip1.go and fs_wasip1.go.

go/src/runtime/os_wasip1.go

Line 74 in d90ce58

func fd_write(fd int32, iovs unsafe.Pointer, iovsLen size, nwritten *size) errno

go/src/syscall/fs_wasip1.go

Line 225 in d90ce58

func fd_write(fd int32, iovs *iovec, iovsLen size, nwritten *size) Errno

go/src/runtime/os_wasip1.go

Line 78 in d90ce58

func random_get(buf *byte, bufLen size) errno

go/src/syscall/fs_wasip1.go

Line 273 in d90ce58

func random_get(buf *byte, bufLen size) Errno

This surfaced for me when trying to adapt a wasip1 go module to a waspi2 component. There are likely other challenges to get to a p2 component but this was the first error.

[johanbrandhorst]

Good spot, we might be able to fix that by having the runtime call the syscall package import instead.

[elewis787]

I have a fix for this by using go linkname and having syscall calling the runtime fs write.

This felt backwards but was a pattern I saw in exec_libc.go

I can push a PR for reference.

[johanbrandhorst]

Using linkname between runtime and syscall is a time honored tradition. Please do submit a PR (or better, a CL to gerrit). Although shouldn't the dependency be the other way around? 🤔

[elewis787]

Using linkname between runtime and syscall is a time honored tradition. Please do submit a PR (or better, a CL to gerrit). Although shouldn't the dependency be the other way around? 🤔

Will do!

That is what I thought as well - but I found a similar pattern in

go/src/syscall/exec_libc.go

Line 60 in 5fdadff

func write1(fd uintptr, buf uintptr, nbyte uintptr) (n uintptr, err Errno)

It's a small change if it makes sense to update!

working on the CL now.

[johanbrandhorst]

That link just points to the write1 function, I don't see a go:linkname directive? Conversely, I searched for linkname syscall in the runtime package and found 86 results. I think the runtime package function should linkname to the syscall function, which should implement the import. Does that make sense?

[elewis787]

Yeah, that makes sense.

The exec_libc.go appeared to be pointing to https://github.com/golang/go/blob/master/src/runtime/syscall_aix.go#L232

my naive thought from the above was to say the runtime os ( wasip1 ) had an import fd_write that was linked through write1 or something similar and that the syscall pkg would leverage the runtime impl for it - this does feel backward tho!

[gopherbot]

Change https://go.dev/cl/629857 mentions this issue: syscall: point fd_write and random_get to runtime wasip1 implementations

[elewis787]

https://go-review.googlesource.com/c/go/+/629857 - happy to update this to reflect the above - but pushed what I have working to help outline the path so far.

[ydnar]

I think it’s important to point out that a duplicate import is not an error, as there are valid cases for a Go program having duplicate go:wasmimport entries that import the same function from the host. In the spirit of minimizing use of go:linkname, I’d argue that duplicate go:wasmimport entries might not only be acceptable, but preferable.

(The contrary would require coordination between all WebAssembly programs and the standard library, which would overly restrict the language from adding new capabilities to the standard library that are not exposed as exported symbols, without breaking backwards compatibility.)

That said:

1. It should probably be an error if the function signature for duplicate imports differ at the ABI level.
2. The compiler could resolve duplicate imports (which would address (1)).