panics on darwin arm

[mangas]

Summary of Bug

Following the docs on mac arm (apple m1) causes a panic. The same instructions work on amd64.

I'm happy to help look into it if anyone is available to pair on it, I'll have a look in meantime but some guidance would appreciated.

starting ABCI with Tendermint
unexpected fault address 0x5f6c61697469a2bd
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x2 addr=0x5f6c61697469a2bd pc=0x1036a6eb0]

Version

v4.2.0

Steps to Reproduce

git clone -b v4.2.0 https://github.com/cosmos/gaia
cd gaia
make install
gaiad init chooseanicehandle
wget https://github.com/cosmos/mainnet/raw/master/genesis.cosmoshub-4.json.gz
gzip -d genesis.cosmoshub-4.json.gz
mv genesis.cosmoshub-4.json ~/.gaia/config/genesis.json
gaiad start --p2p.seeds bf8328b66dceb4987e5cd94430af66045e59899f@public-seed.cosmos.vitwit.com:26656,cfd785a4224c7940e9a10f6c1ab24c343e923bec@164.68.107.188:26656,d72b3011ed46d783e369fdf8ae2055b99a1e5074@173.249.50.25:26656,ba3bacc714817218562f743178228f23678b2873@public-seed-node.cosmoshub.certus.one:26656,3c7cad4154967a294b3ba1cc752e40e8779640ad@84.201.128.115:26656 --x-crisis-skip-assert-invariants

---

For Admin Use

• Not duplicate issue
• Appropriate labels applied
• Appropriate contributors tagged
• Contributor assigned/self-assigned

[gsora]

I wonder if Tendermint makes some running architecture assumptions...

What Go version are you compiling with?

Could you post the complete stack trace?

[faddat]

I do not have an M1 yet but have run a bunch of chains on arm64 Linux.

Arm64 Linux works quite well and so I bet that it is an issue with macos.

If you sent me a DM on twitter, I would be super happy to do a google meet + screenshare.

@mangas

I've never seen anything like that segfault before. Interesting.

[tac0turtle]

I have a m1, I run Tendermint all the time on my machine for testing. Will test gaia shortly

[tac0turtle]

I used two separate config files. One that was previously generated and one that was newly generated the old one works and the new comes with the same errors. @shahankhatch was there any sort of change that could cause this?

[mangas]

@marbar3778 could you post the files? I'm specifically interested in the database backend and possibly the indexer.

[mangas]

It works when I compile with the command below and set the database to badgerdb:

This is the default command generated by make build and I added badgerdb to the tags

go build -mod=readonly -tags "netgo ledger badgerdb" -ldflags '-X github.com/cosmos/cosmos-sdk/version.Name=gaia -X github.com/cosmos/cosmos-sdk/version.AppName=gaiad -X github.com/cosmos/cosmos-sdk/version.Version=v4.2.0 -X github.com/cosmos/cosmos-sdk/version.Commit=535be14a8bdbfeb0d950914b5baa2dc72c6b081c -X "github.com/cosmos/cosmos-sdk/version.BuildTags=netgo,ledger" -X github.com/tendermint/tendermint/version.TMCoreSemVer=v0.34.8 -w -s' -trimpath -o /Users/fa/git/gaia/build/ ./...

This is off tag v4.2.0.

[shahankhatch]

@marbar3778 Aside from sdk/tm version bumps, and some Makefile, doc and Docker updates, Gaia itself hasn't had changes that seem to impact building on an M1.

[faddat]

@shahankhatch correct.

It builds just fine, but well, I wanted to confirm and I love toys so I picked up one of these M1 MBP's.

Can fully confirm @mangas issue.

10:48AM INF Starting EventBus service impl=EventBus module=events
10:48AM INF Starting PubSub service impl=PubSub module=pubsub
unexpected fault address 0x5f6c61697469a2bd
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x2 addr=0x5f6c61697469a2bd pc=0x104923010]

goroutine 26 [running]:
runtime.throw(0x10523fd01, 0x5)
        runtime/panic.go:1117 +0x54 fp=0x1400054d5a0 sp=0x1400054d570 pc=0x104237a74

[faddat]

@mangas I don't know the "why" of this, but build it with the badgerdb flag and it should run fine.

PS:

Actually it runs fine for about 5 minutes, and then it crashes in a different way.

[cyborgshead]

@faddat I have the same issue on M1, also on Nvidia Jetson AGX Xavier and Jetson Nano. Trace

This happens when trying to sync from other nodes in the network that already launched (e.g not local setup). Also tried with badgerdb - same issue

[faddat]

Thank you, it's helpful to know that we now have three confirmed.

Rpi works great... So now I'm very curious why the Jetson does not work.

[faddat]

@litvintech Do you have a raspberry pie model 4?

I think you would find that works on the RPI but not M1 ...which is weird...

[cyborgshead]

@faddat I think that RPi is too slow compared to others and that's why the chain is not halted. On M1 chain stop sync in less than 30 seconds, on Xavier it stops in around 1 min and a couple of minutes on Nano. (approx time as I remembered as tested 2 months ago)

Anyway, I cannot understand is it a general problem cause there is not enough reports. Does nobody run gaia and their chains on ARMs?