Geth 1.8.20 causes panics when geth and parity run at the same time on macOS #18317
Description
System information
Geth version: 1.8.20-stable
OS & Version: OSX
Expected behaviour
Run both geth and parity on different ports at the same time on macOS (10.14.2). Have done this for a long time but after upgrading to Geth v1.8.20 both geth and parity panic. Have switched back to Geth/v1.8.19-stable.
Actual behaviour
Parity and Geth panic
Steps to reproduce the behaviour
Start Parity first
./parity --mode=active --tracing=off --pruning=fast --db-compaction=ssd --cache-size=2048 --max-peers=25 --port=30305 --no-ws --no-jsonrpc --no-serve-light
Followed by Geth
./geth --syncmode=full --maxpeers=25 --lightpeers=0 --verbosity "3" console 2>>x
Backtrace
Geth log 1
INFO [12-15|11:22:03.748] Maximum peer count ETH=25 LES=0 total=25
INFO [12-15|11:22:03.760] Starting peer-to-peer node instance=Geth/v1.8.20-stable-24d727b6/darwin-amd64/go1.11.2
INFO [12-15|11:22:03.761] Allocated cache and file handles database=/Users/x/Library/Ethereum/geth/chaindata cache=512 handles=4611686018427387903
INFO [12-15|11:22:25.065] Initialised chain configuration config="{ChainID: 1 Homestead: 1150000 DAO: 1920000 DAOSupport: true EIP150: 2463000 EIP155: 2675000 EIP158: 2675000 Byzantium: 4370000 Constantinople: 7080000 Engine: ethash}"
INFO [12-15|11:22:25.065] Disk storage enabled for ethash caches dir=/Users/x/Library/Ethereum/geth/ethash count=3
INFO [12-15|11:22:25.065] Disk storage enabled for ethash DAGs dir=/Users/x/.ethash count=2
INFO [12-15|11:22:25.068] Initialising Ethereum protocol versions="[63 62]" network=1
INFO [12-15|11:22:25.096] Loaded most recent local header number=6888077 hash=e60136…07278b td=8281817682095179481503 age=10h57m38s
INFO [12-15|11:22:25.097] Loaded most recent local full block number=6888077 hash=e60136…07278b td=8281817682095179481503 age=10h57m38s
INFO [12-15|11:22:25.097] Loaded most recent local fast block number=6888077 hash=e60136…07278b td=8281817682095179481503 age=10h57m38s
INFO [12-15|11:22:25.100] Loaded local transaction journal transactions=0 dropped=0
INFO [12-15|11:22:25.100] Regenerated local transaction journal transactions=0 accounts=0
INFO [12-15|11:22:25.357] New local node record seq=77 id=51cc22b3c031fbcd ip=127.0.0.1 udp=30303 tcp=30303
INFO [12-15|11:22:25.357] Started P2P networking self=enode://2575814e73d6f9e2900e280da586686da5075daca27a2ea505fcec7609d6c1c750fe7630a91974bd828eb458e89aea65383ac94d2c0e5354ee2aa3b8f83ef471@127.0.0.1:30303
INFO [12-15|11:22:25.367] IPC endpoint opened url=/Users/x/Library/Ethereum/geth.ipc
INFO [12-15|11:22:27.472] Mapped network port proto=tcp extport=30303 intport=30303 interface="UPNP IGDv1-PPP1"
INFO [12-15|11:22:27.550] Mapped network port proto=udp extport=30303 intport=30303 interface="UPNP IGDv1-PPP1"
INFO [12-15|11:22:27.680] New local node record seq=78 id=51cc22b3c031fbcd ip=211.27.216.80 udp=30303 tcp=30303
INFO [12-15|11:24:15.361] Block synchronisation started
INFO [12-15|11:24:35.791] Imported new chain segment blocks=2 txs=282 mgas=15.961 elapsed=3.760s mgasps=4.244 number=6888079 hash=9e2c57…e0b8b4 age=10h59m23s cache=1.77mB
INFO [12-15|11:24:45.604] Imported new chain segment blocks=7 txs=624 mgas=47.878 elapsed=9.811s mgasps=4.880 number=6888086 hash=cea25e…3cc87d age=10h58m17s cache=6.86mB
ERROR[12-15|11:24:46.120]
########## BAD BLOCK #########
Chain config: {ChainID: 1 Homestead: 1150000 DAO: 1920000 DAOSupport: true EIP150: 2463000 EIP155: 2675000 EIP158: 2675000 Byzantium: 4370000 Constantinople: 7080000 Engine: ethash}
Number: 6888088
Hash: 0xc24d482271a4743cfcd423f9ce83cd65835c6a9a6fa563e82312d9dfe21c4129
Error: nonce too high
##############################
Geth log example 2
INFO [12-14|01:07:45.757] Importing sidechain segment start=6875669 end=6875680
INFO [12-14|01:07:54.630] Imported new chain segment blocks=10 txs=539 mgas=61.908 elapsed=8.872s mgasps=6.977 number=6875678 hash=a0a61b…0684fd age=1d1h46m cache=6.02mB
ERROR[12-14|01:07:55.192]
########## BAD BLOCK #########
Chain config: {ChainID: 1 Homestead: 1150000 DAO: 1920000 DAOSupport: true EIP150: 2463000 EIP155: 2675000 EIP158: 2675000 Byzantium: 4370000 Constantinople: 7080000 Engine: ethash}
Number: 6875679
Hash: 0xf7bf39b803247b972964002e6d0c8a268801d47fa3169a6db00dc5a86524ecd7
Error: insufficient balance to pay for gas
##############################
panic: runtime error: index out of range
goroutine 350 [running]:
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).importBlockResults(0xc0000b0d00, 0xc0106b91a0, 0x2, 0x2, 0x2, 0x0)
/Users/travis/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1491 +0x657
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).processFullSyncContent(0xc0000b0d00, 0x0, 0xc00f4ef798)
/Users/travis/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1464 +0x79
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).processFullSyncContent-fm(0x8, 0x4c49670)
/Users/travis/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:479 +0x2a
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync.func1(0xc0000b0d00, 0xc0114b1500, 0xc0105dd030)
/Users/travis/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:491 +0x59
created by github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync
/Users/travis/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:491 +0xaf
INFO [12-14|01:08:17.327] Maximum peer count ETH=25 LES=0 total=25
INFO [12-14|01:08:17.338] Starting peer-to-peer node instance=Geth/v1.8.20-stable-24d727b6/darwin-amd64/go1.11.2
INFO [12-14|01:08:17.339] Allocated cache and file handles database=/Users/x/Library/Ethereum/geth/chaindata cache=512 handles=4611686018427387903
INFO [12-14|01:08:35.930] Initialised chain configuration config="{ChainID: 1 Homestead: 1150000 DAO: 1920000 DAOSupport: true EIP150: 2463000 EIP155: 2675000 EIP158: 2675000 Byzantium: 4370000 Constantinople: 7080000 Engine: ethash}"
INFO [12-14|01:08:35.930] Disk storage enabled for ethash caches dir=/Users/x/Library/Ethereum/geth/ethash count=3
INFO [12-14|01:08:35.930] Disk storage enabled for ethash DAGs dir=/Users/x/.ethash count=2
INFO [12-14|01:08:35.930] Initialising Ethereum protocol versions="[63 62]" network=1
WARN [12-14|01:08:35.966] Head state missing, repairing chain number=6875678 hash=a0a61b…0684fd
INFO [12-14|01:08:35.974] Rewound blockchain to past state number=6875668 hash=83c852…81a74c
INFO [12-14|01:08:35.974] Loaded most recent local header number=6875678 hash=a0a61b…0684fd td=8254324096627868960673 age=1d1h47m
INFO [12-14|01:08:35.974] Loaded most recent local full block number=6875668 hash=83c852…81a74c td=8254302001155502695091 age=1d1h49m
INFO [12-14|01:08:35.974] Loaded most recent local fast block number=6875678 hash=a0a61b…0684fd td=8254324096627868960673 age=1d1h47m
INFO [12-14|01:08:35.976] Loaded local transaction journal transactions=0 dropped=0
INFO [12-14|01:08:35.976] Regenerated local transaction journal transactions=0 accounts=0
INFO [12-14|01:08:36.393] New local node record seq=65 id=51cc22b3c031fbcd ip=127.0.0.1 udp=30303 tcp=30303
INFO [12-14|01:08:36.394] Started P2P networking self=enode://2575814e73d6f9e2900e280da586686da5075daca27a2ea505fcec7609d6c1c750fe7630a91974bd828eb458e89aea65383ac94d2c0e5354ee2aa3b8f83ef471@127.0.0.1:30303
INFO [12-14|01:08:36.395] IPC endpoint opened url=/Users/x/Library/Ethereum/geth.ipc
INFO [12-14|01:08:38.358] New local node record seq=66 id=51cc22b3c031fbcd ip=211.27.216.80 udp=30303 tcp=30303
INFO [12-14|01:08:38.367] Mapped network port proto=udp extport=30303 intport=30303 interface="UPNP IGDv1-PPP1"
INFO [12-14|01:08:38.454] Mapped network port proto=tcp extport=30303 intport=30303 interface="UPNP IGDv1-PPP1"
INFO [12-14|01:12:26.397] Block synchronisation started
ERROR[12-14|01:12:50.210] Failed to generate mapped ethash cache epoch=229 err="open /Users/x/Library/Ethereum/geth/ethash/cache-R23-5bedd237f48f4f08.7582468639515319993: too many open files"
ERROR[12-14|01:12:51.937] Failed to generate mapped ethash cache epoch=230 err="open /Users/x/Library/Ethereum/geth/ethash/cache-R23-86551490879e95ff.3238642280712712661: too many open files"
INFO [12-14|01:12:51.951] Importing sidechain segment start=6875669 end=6875682
ERROR[12-14|01:12:52.023]
########## BAD BLOCK #########
Chain config: {ChainID: 1 Homestead: 1150000 DAO: 1920000 DAOSupport: true EIP150: 2463000 EIP155: 2675000 EIP158: 2675000 Byzantium: 4370000 Constantinople: 7080000 Engine: ethash}
Number: 6875669
Hash: 0x560c647af44ea81cb11dc6951817282d993b7239988dbfcd6be0ae7b19f5203e
Error: insufficient balance to pay for gas
##############################
WARN [12-14|01:12:52.025] Synchronisation failed, dropping peer peer=b1fb004f3e4f407d err="retrieved hash chain is invalid"
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x1c0 pc=0x4652fa3]
goroutine 1829 [running]:
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).findAncestor(0xc0001bf1e0, 0xc02f14dc20, 0xc005fea240, 0x0, 0x0, 0xc00003e570)
/Users/travis/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:825 +0x15d3
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).syncWithPeer(0xc0001bf1e0, 0xc02f14dc20, 0xe3654dc33cda4f1d, 0x80cda6d527d7dca, 0x297784117d6a5f39, 0x9339eaed437d1508, 0xc034dfb2e0, 0x0, 0x0)
/Users/travis/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:437 +0x37d
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).synchronise(0xc0001bf1e0, 0xc0024a9310, 0x10, 0xe3654dc33cda4f1d, 0x80cda6d527d7dca, 0x297784117d6a5f39, 0x9339eaed437d1508, 0xc034dfb2e0, 0x0, 0x0, ...)
/Users/travis/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:406 +0x362
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).Synchronise(0xc0001bf1e0, 0xc0024a9310, 0x10, 0xe3654dc33cda4f1d, 0x80cda6d527d7dca, 0x297784117d6a5f39, 0x9339eaed437d1508, 0xc034dfb2e0, 0x0, 0x4c49670, ...)
/Users/travis/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:324 +0x91
github.com/ethereum/go-ethereum/eth.(*ProtocolManager).synchronise(0xc0000a81a0, 0xc0219086c0)
/Users/travis/gopath/src/github.com/ethereum/go-ethereum/eth/sync.go:200 +0x313
created by github.com/ethereum/go-ethereum/eth.(*ProtocolManager).syncer
/Users/travis/gopath/src/github.com/ethereum/go-ethereum/eth/sync.go:155 +0x1df
Parity log
2018-12-15 11:15:09 UTC Starting Parity-Ethereum/v2.2.5-beta-7fbcdfeed-20181213/x86_64-macos/rustc1.31.0
2018-12-15 11:15:09 UTC Keys path /Users/x/Library/Application Support/io.parity.ethereum/keys/ethereum
2018-12-15 11:15:09 UTC DB path /Users/x/Library/Application Support/io.parity.ethereum/chains/ethereum/db/906a34e69aec8c0d
2018-12-15 11:15:09 UTC State DB configuration: fast
2018-12-15 11:15:09 UTC Operating mode: active
2018-12-15 11:15:10 UTC Configured for Foundation using Ethash engine
2018-12-15 11:15:12 UTC Public node URL: enode://18f211dde0458515b26985add187363dcb233f94a514f56d4ddc8f0e060acde33b75fe5633144f1fe869c1500bd774388a4222282e30588868c2424fc483f723@192.168.1.7:30305
2018-12-15 11:15:13 UTC Updated conversion rate to Ξ1 = US$83.85 (56790750 wei/gas)
...
2018-12-15 11:24:37 UTC Syncing #6888770 0xc6a3…e1fd 0.80 blk/s 80.8 tx/s 5.6 Mgas/s 0+ 634 Qed #6889409 25/25 peers 16 MiB chain 100 MiB db 51 MiB queue 59 MiB sync RPC: 0 conn, 0 req/s, 0 µs
2018-12-15 11:24:42 UTC Syncing #6888772 0x2893…bf60 0.40 blk/s 23.0 tx/s 3.2 Mgas/s 0+ 634 Qed #6889409 25/25 peers 16 MiB chain 100 MiB db 51 MiB queue 59 MiB sync RPC: 0 conn, 0 req/s, 0 µs
====================
stack backtrace:
0: 0x10274835e - <unknown>
1: 0x10274912c - <unknown>
2: 0x1027491ad - <unknown>
3: 0x102747c3b - <unknown>
4: 0x100bb79aa - <unknown>
5: 0x1027b8268 - <unknown>
6: 0x1027b7d7c - <unknown>
7: 0x1027b7cd0 - <unknown>
8: 0x101942b23 - <unknown>
9: 0x1019248cd - <unknown>
10: 0x1016034eb - <unknown>
11: 0x1016176fc - <unknown>
12: 0x1015f7869 - <unknown>
13: 0x101664cac - <unknown>
14: 0x101691b2f - <unknown>
15: 0x101696c75 - <unknown>
16: 0x101692384 - <unknown>
17: 0x1027c3a1e - <unknown>
18: 0x10163c4a2 - <unknown>
19: 0x1027b4ab7 - <unknown>
20: 0x102798ab8 - <unknown>
21: 0x7fff7042f304 - __pthread_body
22: 0x7fff7043226e - __pthread_start
Thread 'IO Worker #2' panicked at 'db get failed, key: [0, 48, 68, 19, 30, 211, 81, 50, 47, 185, 147, 93, 121, 71, 64, 78, 253, 248, 62, 166, 203, 215, 125, 50, 77, 128, 202, 171, 239, 22, 250, 139, 2], err: Custom { kind: Other, error: StringError("IO error: While open a file for random read: /Users/x/Library/Application Support/io.parity.ethereum/chains/ethereum/db/906a34e69aec8c0d/overlayrecent/db/728436.sst: Too many open files in system") }', ethcore/src/db.rs:235
This is a bug. Please report it at:
https://github.com/paritytech/parity-ethereum/issues/new
2018-12-15 11:24:46 UTC Finishing work, please wait...