chain: emit 'reset' when reorganizing SPV chain

[pinheadmz]

Closes #655

The bugfix here was to set the silent flag to false when _reset() is called by reorganizeSPV().

@braydonf found the commit that introduced the silent flag here: 025a5b9 but it is unclear to me in the current state of the codebase whether or not it is still necessary.

Either way, by setting silent to false here, chain will emit a 'reset' event during a reorg, which is caught by pool, which responds by re-syncing the chain. Without this change, SPV nodes that detect a chain reorg will reset to the fork-point of the reorg, and then stop. Wheras full nodes at this point would reconnect blocks from the fork point back up to the new tip, SPV nodes sit here and wait for a new block to be added to the [newly correct] chain before downloading blocks from the network.

SPV must re-download blocks after a reorg because it doesn't store any [filter-matched] transactions from "alternate" blocks.

This PR also includes a comprehensive SPV-sync test. It's an integration test, and it's slow, connecting a Full node to SPV node.

The test is based on the existing node-test.js script which tests a fullnode against chain reorgs. There a few other open PR's with SPV test but none have been merged yet. I think this is good place to start, it would have caught the regression in the commit listed above.

[codecov-io]

Codecov Report

Merging #755 into master will increase coverage by 5.79%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #755      +/-   ##
==========================================
+ Coverage   56.64%   62.43%   +5.79%     
==========================================
  Files         112      106       -6     
  Lines       28140    27860     -280     
  Branches     4800     4756      -44     
==========================================
+ Hits        15939    17394    +1455     
+ Misses      12201    10466    -1735

Impacted Files	Coverage Δ
lib/blockchain/chain.js	74.86% <100%> (+13.97%)	⬆️
lib/mempool/fees.js	52.5% <0%> (-8.84%)	⬇️
lib/blockstore/records.js
lib/blockstore/common.js
lib/blockstore/level.js
lib/blockstore/file.js
lib/blockstore/index.js
lib/blockstore/abstract.js
lib/blockstore/layout.js
lib/node/spvnode.js	59.43% <0%> (ø)
... and 24 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d601b6a...7df8db2. Read the comment docs.

[braydonf]

I should add this to test/util/common.js in #748, so it'll be available for other tests.

[tynes]

I agree, it is a good test utility function

[pinheadmz]

If you think #748 will get merged first, I'll wait and rebase and use common as a dependency. Or if this gets merged first, you can move it to common in #748 ?

[braydonf]

Okay, I moved it to /test/util/common.js as from #748

[braydonf]

Maybe we can wait for an event with a specific value?

[pinheadmz]

Tried a few things with while loops waiting for a certain chain height, can't really figure out a more elegant way to do this... any thoughts?

[braydonf]

What about something like this?

async function eventValue(obj, name, expected, timeout = 10000) {
  assert(typeof obj === 'object');
  assert(typeof name === 'string');

  return new Promise((resolve, reject) => {
    let timer = null;

    const check = (value) => {
      if (value === expected) {
        obj.removeListener(name, check);
        clearTimeout(timer);
        resolve();
      }
    };

    obj.addListener(name, check);

    timer = setTimeout(() => {
      obj.removeListener(name, check);
      reject(new Error('Timeout waiting for event value.'));
    }, timeout);
  });
}

I have been using this in #758 to wait for values:

async function forValue(obj, key, val, timeout = 60000) {
  assert(typeof obj === 'object');
  assert(typeof key === 'string');

  const ms = 10;
  let interval = null;
  let count = 0;

  return new Promise((resolve, reject) => {
    interval = setInterval(() => {
      if (obj[key] === val) {
        clearInterval(interval);
        resolve();
      } else if (count * ms >= timeout) {
        clearInterval(interval);
        reject(new Error('Timeout waiting for value.'));
      }
      count += 1;
    }, ms);
  });
};

[pinheadmz]

@braydonf I also want to copy this test exactly (once we're happy here) but connecting two full nodes.
Having regression tests that involve network messages and reorgs will be really important for testing headers-first eventually.

[braydonf]

This issue is fixed in the https://github.com/bcoin-org/bcoin/tree/chain-expansion branch.