Failover when an originator becomes unresponsive, and return to originator once responsive

[mkysel]

No description provided.

[mkysel]

This might already be done. To be investigated.

[mkysel]

We will re-establish a connection in the syncworker if an originator crashes. I have seen this work plenty of times. I don't think that we have any explicit testing for it.

Is there anything else that needs to happen here @richardhuaaa?

[richardhuaaa]

I think there's two more cases we need to handle:

1. If the originator is in the registry, but for some reason we are unable to connect to it even on retry, we should pull that originator's payloads from other nodes on the network (perhaps multiple).

2. if the originator was recently removed from the registry (e.g. within 1 day), or they have been removed long ago but we are a brand new node that has no history, we should pull that originator's payloads from other nodes on the network (perhaps multiple)

The XIP could def do a better job of going through these

[mkysel]

Gotcha. Thanks for the details