orderwatch: store networkId in DB

[fabioberger]

Currently Mesh does not store the networkID persistently corresponding to the block headers stored. If a Mesh operator is on Ganache (very low block numbers) and then switches to mainnet, Mesh will attempt to sync up to the latest block (potentially 8,000,000+ blocks) since it assumes we were previously on mainnet as well.

By persistently storing the networkID, we can throw an error requiring the operator to remove the orders and blocks stored in the DB as they correspond to a different network.

[hrharder]

Hoping to (try) to help out with this starting next week, but want to clarify a few things and get your input before I dig in!

1. Location of networkID field

You reference storing the networkID in the block headers stored in the DB, do you think this seems like a logical place for that? If not, where you would you suggest? Is it repetitive to store the networkID with each header?

My (perhaps naive) intuition would be to instead store the networkId only once per database (perhaps in a metadata collection), and then expose a new method on MeshDB to load a stored network ID, if found. In that case, I assume it would be relatively trivial to add a check in a constructor somewhere above, perhaps the OrderWatcher. Again -- I don't believe I am familiar enough with the current implementation, so I'll be looking for your input here.

Re. where to load networkID from/to; should it be passed as an additional argument to NewMeshDB?

2. Regarding orderwatch

The title of the issue is orderwatch: so I assume you expect much of this logic will expand that package.

The networkID seems to be available here, is the implication this a good place to access the ID we should be storing?

3. Desired behavior in case of ID mismatch

By persistently storing the networkID, we can throw an error requiring the operator to remove the orders and blocks stored in the DB as they correspond to a different network.

Hoping to get a little more clarity here on the desired behavior in the case of DB loaded with orders from mainnet/ropsten/ganache being loaded to a node on a different network. When you say "throw" I assume you don't mean a literal panic but rather some sort of helpful error message ("switch to compatable networkID or purge your database") and a graceful exit. Am I missing anything? What do you think?

[hrharder]

Another thought... would it be less complex to simply infer the current networkID based off the exchangeAddress of the orders in the DB?

In that case, for non-first-time startups (a db already exists) we could simply examine the exchangeAddress from say, the first/oldest order we are storing, and compare which network it corresponds to (based off this or a variant) to the detected networkID from an Ethereum client call.

It may be preferable (for any number of reasons) to store the configured ID explicitly 🤷‍♂

[fabioberger]

Thanks for the clarifying questions @hrharder.

1. I prefer the idea of a separate metadata collection that would initially store just the networkID but could eventually include other one-off pieces of persisted state. This collection should be added to MeshDB.

2/3. Since the networkID is parsed out of the env vars in core, we should probably check that the networkID supplied matches the one persisted to the DB on start (ideally right after instantiating MeshDB in core). At this point we should log a helpful error message and gracefully exit IF the two do not match. The user should be instructed to either start Mesh on the networkID persisted, or rm -rf the db before booting up Mesh with a new networkID.

4. Although we could infer the networkID from the order exchangeAddresses, this doesn't cover the edge-case where Mesh was started on a network but no orders added. This is extremely likely when connecting to Ganache, which is also the network with very low block numbers. To definitively avoid syncing 8M blocks, we should persist the networkID separately as mentioned above.

[hrharder]

I prefer the idea of a separate metadata collection that would initially store just the networkID but could eventually include other one-off pieces of persisted state. This collection should be added to MeshDB.

Great -- me as well. Should be useful in the future.

Re. 2/3, sounds good. All makes sense.

Re. 4, great point -- that idea was really just an after-thought!

I should be set to get something started, and will hopefully have a draft PR up Thursday/Friday.

Thanks for the clarifications :)

[albrow]

@hrharder feel free to ping me directly if you have questions about our db package. It is a little rough around the edges at the moment. There are some restrictions in the design based on our goal to be able to run it in the browser.

[hrharder]

@albrow Thanks! Let me know what you think of #301, I was able to get the desired behavior without issue, but I want to ensure I stick with your established conventions, etc.

I don't think anything I've done extends beyond the patterns that are established in the package already, but hopefully you'll be able to confirm that.