Allow optional pre BIP-37 behavior (the original satoshi protocol)

[chjj]

I realize bcoin is supposed to be used as an spv, but it wouldn't be that hard to add an option to allow a more traditional blockchain downloader. What do you think, @indutny? Is there anything I missed here?

[indutny]

You should verify merkle root here

[chjj]

Alright, I wrote a function to build a merkle tree and verify the merkle root correctly (modeled after the bitcoind code). I also added some verification checks that bitcoind does on blocks (maybe we might want to do some of them on merkleblocks too?).

Anyway, this all seems to work in practice. Let me know what you think.

[chjj]

I can also revert all the CheckBlock stuff if you don't want it.

[indutny]

why not utils.dsha256?

[chjj]

I thought the same thing eventually. It now concatenates the two merkleTree hashes into utils.dsha256.

When I initially wrote it, I was essentially porting the awful c++ of bitcoind to javascript (to clarify: I'm not sure if I dislike the bitcoind codebase or just C++ in general), so it was confusing to say the least. The fuction looked too complex to be handled with just utils.dsha256. I was wrong.

[chjj]

I think the pull request is, more or less ready, but I kind of want to discuss usage.

I've been messing around adding some code to pool._addPeer, and although I can start receiving full blocks from peers, pool.chain.add(block) doesn't seem to like them. They don't get added to the blockchain, which is strange, becuase if I just instead do a _probeIndex on the prevBlock, it exists and I can then do an _addIndex on the block. I can't figure out why chain.add(block) is not working.

Even more puzzling, I can't seem to get more than ~6000 blocks from my peers, even if I do a getblocks loop of the last block hash I saw. The original protocol is a headache. :)

[chjj]

I definitely think there is something wrong with chain.add/chain.has. I cannot get a full blockchain by just relying on chain.has, chain.add, and pool.block.lastHash.

The only way I can get a full blockchain download is by creating my own version of pool.block.lastHash which simply checks a hash table of every block hash received so far, and then sets myownobj._lastHash to the last known hash received by the blocks event (INV). I then send peer.loadBlocks([myownobj._lastHash], 0) in every call.

With the former method, I'll usually get about 500-5000 unique blocks before bcoin hangs and stops requesting/receiving blocks completely. With the latter method, I can get a full blockchain download. I really can't figure it out. Does the Chain object need to be rewritten to accomodate these new features?

[indutny]

Hm... it really shouldn't be. Unfortunately, I'm busy optimizing elliptic.js right now. Will look into it somewhere later.

[chjj]

I understand. I'll continue to look into it. In the meantime, could you merge this branch? I think it's ready despite the Chain issue (I can still get a full blockchain download with it).

Could you also merge the version packet pull request? I added a test like you said, and now the relay=false option actually has a purpose (to trigger the pre-bip-37 behavior).

[chjj]

Awesome. Thank you.