pool.js::getHost() is CPU hungry

[rsmarples]

This is best observable on regtest where constant connections to 127.0.0.1:14038 are observed and always fail.

Near the top of the function we have this code:

for (let i = 0; i < 100; i++) {
      const entry = this.hosts.getHost();

All 100 iteratrions are made. This takes on average almost 2 seconds on my machine. Then multiply this by the need to fill 8 spots causes the CPU to peg 100% and many rpc calls then fail. My test case is wallet creation - generally the hsw rpc will timeout way before hsd actually logs the request to create it.

this.hosts.getHost() looks expensive to call 100 times. Maybe a better approach would be to have a function this.hosts.getRandomList(maxHosts) which does a similar job but just returns a list of a maximal number of hosts and iterate that rather than a forced 100 loop alll the time.

[tynes]

Definitions

• Pool - hsd/lib/net/pool
• Peer - Another node that implements the Handshake Protocol
• Loader Peer - The peer used to download blocks
• hosts.json - The file that contains the peer info
• Identity Key - Brontide p2p Identity key

Summary

In it's current form, the Pool will continuously try to connect to peers when it does not
have a minimum number of peers. When the network is sparse or running on a local regtest,
the node will be in an infinite loop trying to add more peers. This loop has caused
problems where the machine's full resources are used in this loop, making requests timeout.
If the node is also trying to stay in consensus, this would add to the problem.

A way to fix this problem is to use less iterations in Pool.getHost. That function is is used
when adding an outbound connection and also for selecting the loader peer. It is currently set at
100 and larger values are better when there are more byzantine entries in the hosts.json file.

It requires an IP address to get an entry into the hosts.json file. If peers were tracked
as IP + Identity key, it would be much cheaper to get byzantine entries into the hosts.json
file, as it would be as easy as generating a new secp256k1 keypair.

Based on personal experience, I would never see more than ~25 iterations on Bitcoin mainnet
using bcoin.

Solutions

• Config option to set the number of iterations in the loop
• Dynamically calculate the number of iterations in the loop
• Hard code in a smaller number of iterations