Description
In the spirit of pragmatic solutions to provide higher assurance of correctness relatively quickly, I suggest we consider adding a --assert-invariants
mode to gaiad
(perhaps even enabled by default). When run in this mode, gaiad
asserts all the specified invariants at the end of each block and panics if any fail.
This is easy to implement since we can just reuse the existing invariants from the simulation (and any future invariants which might be added to the simulation) and adds a reasonable degree of assurance from certain classes of bugs - if at least 1/3 of stake runs with this invariant assertion mode enabled, and an invariant breaks, the chain will safely halt and we can figure out what's going on & fix it.
This is not a long-term solution as it isn't necessarily incentive-compatible (extra compute by validators which they aren't getting paid for) - but in the short term I think enough might do it. We'd need to ensure the current invariants aren't too slow (< ~0.1s
maybe) to cause problems.
Activity
rigelrozanski commentedon Nov 6, 2018
This makes great sense for testnets
zmanian commentedon Nov 6, 2018
I think it make sense to run nodes on mainnet in this mode.
zmanian commentedon Nov 6, 2018
@jessysaurusrex FYI
gamarin2 commentedon Nov 6, 2018
As I mentioned to @cwgoes, this should be renamed, as this does not relate to the concept of Circuit breaker. Circuit breaker original goal is to maintain liveness if bugs in specific modules are found by temporarily deactivating these modules. This issue is not about maintaining liveness but halting the chain altogether in case of extreme emergency.
As for the proposal itself, I think all the concerns you put forward are valid. If we do it, we need to hope validators will run their node with this mode as there is no clear incentives to do so.
alexanderbez commentedon Nov 6, 2018
At the very least, we should benchmark it. If the overhead is relatively minimal, maybe we can enforce it. Otherwise, yes, incentives come into play.
cwgoes commentedon Nov 12, 2018
What if we make it mandatory (put the checks at the end of
EndBlock
)? Imposing an extra few fractions of a second per block seems a small price to pay for the additional security.Some invariants will be
O(n)
in a state that can be altered (accounts, validators) but as long as those cost gas to create I think it's reasonably safe in the short term.Of course, validators or nodes could elect to remove the code themselves - but I think making the invariants default might be enough to cause wide adoption.
8 remaining items