Build into Indy VDR the "best practices" around getting, caching and updating a pool

[swcurran]

After meeting to discuss #239 (more outputs from that meeting to be produced), we agreed that some updates to Indy VDR should be made to relieve the wrapper or caller (e.g., ACA-Py, AFJ, etc.) from having to implement the management.

Note: The first draft of this issue is my interpretation of the meeting results. We'll update the list to refine that is really needed!

The following items should be added to Indy VDR to simplify pool management:

• A mechanism to allow Indy VDR to directly maintain a cache of the current nodes for an Indy Network.
  • The cache would ideally only contain the currently active nodes, vs. all of the transaction in the Indy Pool ledger (example)
  • The currently active nodes are determined by retrieving and processing all of the transactions from the pool ledger -- transactions that add/remove nodes from the network..
  • The end result of the processing the transactions is a list of active node.
  • The cache should be accessible after a restart, so it is not recreated each restart.
  • TBD if the cache is managed consistently across instances of a server -- e.g. across many ACA-Py instances.
• Although the "genesis_file" transactions might be passed in by the caller, it should only be used if the full set of pool ledger transactions have not be retrieved.
  • The "genesis_file" transactions are a point in time list of pool ledger transactions published by the network.
  • After publication, additional pool ledger transactions may have been published that add/remove nodes from the ledger. It is that full state that we want to use whenever possible.
  • The "genesis_file" transactions should only be used when there is no other information available.
• Indy VDR can recognize when a pool ledger transaction has been recorded by seeing a change of the Merkle root in a state proof.
  • Indy VDR should check for such a change, and when found, automatically trigger a refresh of the pool transactions -- getting the new pool ledger transaction(s), and hence, the updated list of current nodes.

The knowledge of the caller and wrapper about the pool cache should be "minimized". We'll update this description to define what "minimum" means.

[swcurran]

@andrewwhitehead @cvarjao @jleach @WadeBarnes -- did I get that right?

[WadeBarnes]

One thing to add. The pool transactions can also define a change to other node settings such as a node's IP address(es) and port information without affecting the node's role on the network. Therefore it is important the resulting cached summary of the pool reflect the current state of all of the nodes on the network.

[KimEbert42]

If we are looking for Indy VDR to still be a library without storage, we could modify the interface to provide better caching of results.

If Indy VDR has an automated refresh, perhaps we expose the results by being able to ask for the updated results before the agent shuts down. It would simplify the workflow for the agent. Provide the cache on startup, before shutdown, fetch the updated list and store in cache. This would simplify the interface to indy vdr.

So the workflow would look like.

1. Agent starts Indy VDR and passes in cache or genesis file.
2. Indy VDR will periodically update itself as needed based upon merkle roots.
3. Before shutdown, or at regular interval, Agent will ask Indy VDR for updated cache values, and store.
4. Agent shuts down Indy VDR.

Alternatively, IndyVDR would have a callback to notify the agent of an updated cache.

1. Agent start Indy VDR and passes in cache or genesis file.
2. Indy VDR will periodically update itself as needed based upon merkle roots.
3. Indy VDR callback calls Agent to provide updated cache.
4. Agent stores new cache values.
5. Agent shuts down at some point.

Indy VDR has a problem on startup if it can't connect with enough nodes. I think it by default tries to connect to at least half of the nodes? If enough nodes aren't available at startup, it can lead to failure, even if other nodes in the network are available. With an outdated genesis file, this could be problematic. Keeping the genesis file up to date can avoid this, but perhaps a small change to the algorithm may be helpful. If a node is unavailable, Indy VDR should probably try to contact other nodes instead of failing, when some nodes are unavailable. I've seen this with aca-py with a smaller genesis file ( 4 nodes, with 1 or 2 no longer available ). The right behavior from a network perspective is to correct the genesis file, but an adaptation to Indy VDR would be to contact additional nodes instead of crashing.

An alternative algorithm on startup would be to connect to more nodes than requested, and when the required number of nodes are connected, disconnect the remaining nodes. This would encourage the mobile agent to use the fastest responding nodes.

[andrewwhitehead]

The (opt-in) genesis transactions cache is addressed in #243. Subsequent work could look at automatically triggering a refresh when a change in the pool state root hash is detected.

[swcurran]

I think opt-out would be better. What do you think? I think it makes sense it just happens.

[andrewwhitehead]

The tricky part there is just automatically finding a writable cache directory. It's fairly easy on say, a Windows machine, but there are circumstances where that might be more difficult (on mobile devices, or on Github actions for example). I'm not sure if it's appropriate at this level or not.

[swcurran]

OK — sounds good.

[swcurran]

Closing this issue as implemented in #243 #244 and in the latest release -- 0.4.1