MIPS on Linux support

[GTANAdam]

It seems that tinygo doesn't not support MIPS based embedded devices out of the box, although LLVM does support the following:

    mips       - MIPS (32-bit big endian)
    mips64     - MIPS (64-bit big endian)
    mips64el   - MIPS (64-bit little endian)
    mipsel     - MIPS (32-bit little endian)

My intent is to execute the tinygo compiled binary within a mipsel linux box, no bare metal support, so I assume that's feasible considering llvm support.
This would be extremely useful for routers with a limited set of storage and ram.

[niaow]

As far as I can tell, there are only a few things required to add support for such a target:

1. a runtime arch file - provides the equivalent GOARCH value, bitness of the cpu, etc. (example here)
2. a mapping from the LLVM arch name to the GOARCH name here
3. helper program selection guesses here

I can't think of anywhere else where changes would be necessary. After the appropriate changes are added, you should be able to build for MIPS by supplying an appropriate LLVM target to the -target flag.

[deadprogram]

Is there specific MIPS hardware you are looking to support?

[niaow]

@deadprogram This issue seems to be regarding a Linux target, rather than baremetal.

[GTANAdam]

Is there specific MIPS hardware you are looking to support?

Yeah, It would be great to have bare metal support for mips24kc out of the box since that's basically used by most routers.
I'll try tinkering around, might learn something from it.

UPDATE:

It seems that after manually building LLVM as mentioned in Getting Started doesn't contain mips arch as a target

LLVM (http://llvm.org/):
  LLVM version 10.0.1
  Optimized build.
  Default target: x86_64-unknown-linux-gnu
  Host CPU: skylake

  Registered Targets:
    aarch64    - AArch64 (little endian)
    aarch64_32 - AArch64 (little endian ILP32)
    aarch64_be - AArch64 (big endian)
    arm        - ARM
    arm64      - ARM64 (little endian)
    arm64_32   - ARM64 (little endian ILP32)
    armeb      - ARM (big endian)
    avr        - Atmel AVR Microcontroller
    riscv32    - 32-bit RISC-V
    riscv64    - 64-bit RISC-V
    thumb      - Thumb
    thumbeb    - Thumb (big endian)
    wasm32     - WebAssembly 32-bit
    wasm64     - WebAssembly 64-bit
    x86        - 32-bit X86: Pentium-Pro and above
    x86-64     - 64-bit X86: EM64T and AMD64

[aykevl]

Yeah, It would be great to have bare metal support for mips24kc out of the box since that's basically used by most routers.

My intent is to execute the tinygo compiled binary within a mipsel linux box, no bare metal support, so I assume that's feasible considering llvm support.

Are you interested in baremetal or Linux support?
The two are quite different. Running under Linux would be relatively easy, needing basically the changes that @jaddr2line mentioned above. Baremetal support will take a lot more work and will usually be very chip specific. To get an idea of what is needed, see this PR (but note that quite a few things changed since then to avoid duplicated code).

It seems that after manually building LLVM as mentioned in Getting Started doesn't contain mips arch as a target

Yes, you need to modify this line to add Mips to the enabled targets:

tinygo/Makefile

Line 157 in a9ba6eb

mkdir -p $(LLVM_BUILDDIR); cd $(LLVM_BUILDDIR); cmake -G Ninja $(TINYGO_SOURCE_DIR)/$(LLVM_PROJECTDIR)/llvm "-DLLVM_TARGETS_TO_BUILD=X86;ARM;AArch64;RISCV;WebAssembly" "-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=AVR" -DCMAKE_BUILD_TYPE=Release -DLIBCLANG_BUILD_STATIC=ON -DLLVM_ENABLE_TERMINFO=OFF -DLLVM_ENABLE_ZLIB=OFF -DLLVM_ENABLE_PROJECTS="clang;lld" -DLLVM_TOOL_CLANG_TOOLS_EXTRA_BUILD=OFF $(LLVM_OPTION)

It may be easier to instead use a prebuilt version of LLVM, which you can easily install on MacOS and most Linux distributions. It will have Mips already included as a target (no need to rebuild LLVM).

[GTANAdam]

Yes, you need to modify this line to add Mips to the enabled targets:

Thank you for the tip, I was already digging inside, getting that little mipsy mips support in LLVM 😄

Are you interested in baremetal or Linux support?

I am honestly interested in having it work in both Linux and bare metal but I do understand that bringing bare metal support is a bit out of my skill scope, but I am very eager to learn and contribute

UPDATE:

Well, after some tinkering around, I was successfully able to compile mipsel targeted binaries (statically linked) and as well as executing them

Here are the file size differences between golang compiled binaries and tinygo:

/opt/home/admin # ls -l
-rwxr-xr-x    1 admin    root        514908 Apr 28 06:33 hello_mipsel_tinygo
-rwxr-xr-x    1 admin    root          5704 Apr 28 06:17 hello_mipsel_tinygo_dynlink
-rwxr-xr-x    1 admin    root        851968 Apr 28 06:14 hello_mipsle_go

package main

func main() {
    print("HELLO\n")
}

Note: the binaries were compiled with omitted DWARF debugging information as well as symbol table generation using -ldflags "-s -w" to achieve the lowest sizes without using upx.

a brief ldd look into the dynamically linked binary:

/opt/home/admin # ldd hello_mipsel_tinygo_dynlink
checking sub-depends for 'not found'
        libc.so.6 => not found (0x00000000)
        /lib/ld.so.1 => /lib/ld.so.1 (0x00000000)

What's interesting here is the produced (by default) dynamically linked binary that is small in size albeit depending on libc.so.6 (glibC) which is missing by default on OpenWRT (muslC), so in order to fix this, an openwrt musl toolchain should be used but this out of the scope of this issue but I will eventually document that as soon as I'm finished.
This should allow the OpenWRT community to adopt tinygo.

[olarriga]

I'm also interrested on compiling for MTK7628NN with Linux support. With Go I select GOARCH=mipsle GOMIPS=softfloat.
It's possible with TinyGo ? Do you have any documents ?
Thanks

[niaow]

@olarriga this is not yet implemented, see my first comment on this issue

[GTANAdam]

I'm also interrested on compiling for MTK7628NN with Linux support. With Go I select GOARCH=mipsle GOMIPS=softfloat.
It's possible with TinyGo ? Do you have any documents ?
Thanks

Yes, it's possible and beautiful, I'm currently working on it, will have a PR and a documentation published soon.

UPDATE:

Currently the compilation is spewing a warning concerning wrong floating point setting but the binary is functioning properly, yet to figure out where to put the -msoft-float flag in tinygo/llvm given that the musl toolchain already uses -msoft-float

/home/adam/toolchain/toolchain-mips_24kc_gcc-8.4.0_musl/bin/../lib/gcc/mips-openwrt-linux-musl/8.4.0/../../../../mips-openwrt-linux-musl/bin/ld: warning: /tmp/tinygo498244004/main uses -msoft-float (set by /home/adam/toolchain/toolchain-mips_24kc_gcc-8.4.0_musl/bin/../lib/gcc/mips-openwrt-linux-musl/8.4.0/../../../../mips-openwrt-linux-musl/lib/crt1.o), /tmp/tinygo498244004/main.o uses -mhard-float

UPDATE:

Seems like -msoft-float flag is being ignored by llvm/clang, but considering that the binaries are working properly, it isn't critical atm.

UPDATE:

After having configured both the mips and mipsel toolchains, I went ahead and tried to compile a sample golang code using the fmt library, the compiler spewed the following:

../tinygo/tinygo build -target "mipsel-openwrt-linux" -o hello main.go
../../../usr/lib/go-1.13/src/syscall/zsyscall_linux_mipsle.go:310:21: unknown GOOS/GOARCH for syscall: linux/mipsle
../../../usr/lib/go-1.13/src/syscall/zsyscall_linux_mipsle.go:1716:23: unknown GOOS/GOARCH for syscall: linux/mipsle
../../../usr/lib/go-1.13/src/syscall/zsyscall_linux_mipsle.go:1048:21: unknown GOOS/GOARCH for syscall: linux/mipsle
../../../usr/lib/go-1.13/src/syscall/zsyscall_linux_mipsle.go:732:22: unknown GOOS/GOARCH for syscall: linux/mipsle
../../../usr/lib/go-1.13/src/syscall/zsyscall_linux_mipsle.go:1005:22: unknown GOOS/GOARCH for syscall: linux/mipsle

I figured that compiler/syscall.go needs some work as well in order for the go libraries to compile and by the looks of it, it became much more challenging..

UPDATE:

After hours of thorough studying of mips specs and docs, I have found out that the /o32 ABI only has 4 argument registers which is less than syscall6 arg length 🤔... additionally, the syscall manpage mentions that the syscall convention passes arguments 5+ through 8 on the user stack but that requires a different implementation in gollvm(?)..
I am currently limited by my lack of knowledge in this area so any input would be appreciated! 😄

UPDATE:

I had no luck in assembling a working solution so far so it's a roadblock at the moment, basically everything works but syscalls

[aykevl]

Currently the compilation is spewing a warning concerning wrong floating point setting but the binary is functioning properly, yet to figure out where to put the -msoft-float flag in tinygo/llvm given that the musl toolchain already uses -msoft-float

/home/adam/toolchain/toolchain-mips_24kc_gcc-8.4.0_musl/bin/../lib/gcc/mips-openwrt-linux-musl/8.4.0/../../../../mips-openwrt-linux-musl/bin/ld: warning: /tmp/tinygo498244004/main uses -msoft-float (set by /home/adam/toolchain/toolchain-mips_24kc_gcc-8.4.0_musl/bin/../lib/gcc/mips-openwrt-linux-musl/8.4.0/../../../../mips-openwrt-linux-musl/lib/crt1.o), /tmp/tinygo498244004/main.o uses -mhard-float

You probably need to set a flag (-soft-float maybe?) in the BuildTags property. This list of flags is for how TinyGo compiles the binary, and is independent of CFlags (which is used to build the C parts). You can figure out the needed flag with the Clang -v flag (look for -target-fetature).
While some programs may work, I do not think programs that have float operations will work without this change. Try running testdata/float.go for example.

After hours of thorough studying of mips specs and docs, I have found out that the /o32 ABI only has 4 argument registers which is less than syscall6 arg length thinking... additionally, the syscall manpage mentions that the syscall convention passes arguments 5+ through 8 on the user stack but that requires a different implementation in gollvm(?)..
I am currently limited by my lack of knowledge in this area so any input would be appreciated! smile

Ah really, they are passing syscall arguments on the stack? That complicates things a bit.
I think the easiest option is to write assembly functions that do the syscall itself. Calling those functions will make sure LLVM puts those values on the stack in the right place. There is a bit of a complication however with the success/error value (in $a3) so the function may need to do a bit more than just call syscall and return.

Source: https://www.linux-mips.org/wiki/Syscall

[ski7777]

Any update on this?