Stage 5 Error Block InvalidStateRoot Mismatch

[nicolashardy]

Hello Everyone !

• Parity Ethereum version: 2.5.1
• Operating system: Linux
• Installation: Docker container (one line)
• Fully synchronized: no
• Network: Private blockchain
• Restarted: yes

My problem is that after migrated from parity 1.11 to parity 2.5.1 my private chain cannot catch up previous blocks anymore

My error is :

Stage 5 block verification failed for #1140 (0x6c24…48e8)
Error: Error(Block(InvalidStateRoot(Mismatch { expected: xxx(hidden for security concerns), found: yyy(hidden for security concerns) }))

I understood by reading similar issues that this sounds like a consensus problem, so here is mine : (using Aura PoA)

Here is my config file. It was working on my previous parity version(I just had to move eipTransition to global params key) but it doesn't work anymore

{
  "engine": {
    "authorityRound": {
      "params": {
        "stepDuration": "2",
        "validators": {
          "list": [
            "HIDDEN1",
            "HIDDEN2",
            "HIDDEN3"
          ]
        }
      }
    }
  },
  "params": {
    "maximumExtraDataSize": "0x20",
    "minGasLimit": "0x1388",
    "gasLimitBoundDivisor": "0x400",
    "networkID": "0x11",
    "maxCodeSize": "0xffffffffff",
    "eip155Transition": "0x0",
    "eip98Transition": "0x7fffffffffffffff",
    "eip140Transition": "0x0",
    "eip211Transition": "0x0",
    "eip214Transition": "0x0",
    "eip658Transition": "0x0"
  },
  "genesis": {
    "seal": {
      "authorityRound": {
        "step": "0x0",
        "signature": "0x0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
      }
    },
    "difficulty": "0x20000",
    "gasLimit": "0x1312D00"
  },
  "accounts": {
    "0000000000000000000000000000000000000001": {
      "builtin": {
        "name": "ecrecover",
        "pricing": {
          "linear": {
            "base": 3000,
            "word": 0
          }
        }
      }
    },
    "0000000000000000000000000000000000000002": {
      "builtin": {
        "name": "sha256",
        "pricing": {
          "linear": {
            "base": 60,
            "word": 12
          }
        }
      }
    },
    "0000000000000000000000000000000000000003": {
      "builtin": {
        "name": "ripemd160",
        "pricing": {
          "linear": {
            "base": 600,
            "word": 120
          }
        }
      }
    },
    "0000000000000000000000000000000000000004": {
      "builtin": {
        "name": "identity",
        "pricing": {
          "linear": {
            "base": 15,
            "word": 3
          }
        }
      }
    },
    "0000000000000000000000000000000000000005": {
      "builtin": {
        "name": "modexp",
        "activate_at": 0,
        "pricing": {
          "modexp": {
            "divisor": 20
          }
        }
      }
    },
    "0000000000000000000000000000000000000006": {
      "builtin": {
        "name": "alt_bn128_add",
        "activate_at": 0,
        "pricing": {
          "linear": {
            "base": 500,
            "word": 0
          }
        }
      }
    },
    "0000000000000000000000000000000000000007": {
      "builtin": {
        "name": "alt_bn128_mul",
        "activate_at": 0,
        "pricing": {
          "linear": {
            "base": 2000,
            "word": 0
          }
        }
      }
    },
    "0000000000000000000000000000000000000008": {
      "builtin": {
        "name": "alt_bn128_pairing",
        "activate_at": 0,
        "pricing": {
          "alt_bn128_pairing": {
            "base": 100000,
            "pair": 80000
          }
        }
      }
    },
  },
  "name": "PoA-local"
}

Thank you VERY much and thanks for you hard work, parity team !

[jam10o-new]

This particular error only really happens if your nodes are propagating bad blocks, and there isn't really any solution other than taking the problematic nodes down (or killing their db) and resyncing.

[nicolashardy]

After more investigation, seems like the eip86Transition key doesn't exist on the newest parity version. Our old transactions were generated using this eip so now we are kind of screwed

We are trying to find a way to sync without warp because we need deep access of old nodes (Warping works)

[jam10o-new]

( for future reference, eip86 was removed in #9140)

[jam10o-new]

The only clean long-term solution here is to export state of your current chain and create a new genesis/chainspec with it.

You would need to export state with a fat-db node running your old version because you can't warp with fat-db though.

[nicolashardy]

Exporting state leads to

Thread 'main' panicked at 'slice index starts at 8 but ends at 1', src/libcore/slice/mod.rs:2419

This is a bug. Please report it at:

    https://github.com/paritytech/parity-ethereum/issues/new

Exporting blocks works however ! importing the file that was exported ends in the same error

Stage 5 block verification failed for #1140 (0x6c24…48e8)
Error: Error(Block(InvalidStateRoot(Mismatch { expected: xxx(hidden for security concerns), found: yyy(hidden for security concerns) }))

[jam10o-new]

Thread 'main' panicked at 'slice index starts at 8 but ends at 1', src/libcore/slice/mod.rs:2419

This is a known issue apparently: #10599 (comment)

[nicolashardy]

We are going to lose all the transactions history with this method..any way to replay all transactions ?