Bitcoin reorgs can impact run loop synchronization, stalling bitcoin downloads

[kantai]

A specific interaction between target_block_height and bitcoin reorgs can cause a node to stall in downloading Bitcoin blocks. The only remedy if a node gets caught in this situation is to restart the node with a fresh chainstate (in theory, the chainstate could be manually fixed, but it would require changes across a few different SQLite dbs).

The interaction in question:

https://github.com/blockstack/stacks-blockchain/blob/master/src/burnchains/burnchain.rs#L1223

Here, the sync_with_indexer function will correctly "shorten" the range for block downloads if passed a target block height. However, this does not stop the SPV client from downloading the headers from the original range. Once it has done this, subsequent invocations of sync_with_indexer will set the start_height initially to the end of that range. Naively, this would mean the indexer would never download, parse and process the blocks in that shortened portion of the range. In normal operation (and even in the presence of short-lived Bitcoin forks), this isn't an issue because the current height in burnchain_db is used to reduce the start_height. However, in the event of a bitcoin reorg, the burnchain_db height may already be higher than the blocks needed to download. This would cause the indexer to never download those blocks.

Solving this could be done in a number of different ways:

1. On reorgs, make entries in the burnchain_db as reorged, and ignore them in canonical_burnchain_tip lookups. (I think this is a bad approach).
2. Apply target_block_height to the SPV client's sync_headers command.
3. Track which blocks have been processed by the burnchain_db in the SPV headers db. Use that information to determine start_height.

[gregorycoppola]

Fixed.

[blockstack-devops]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.