Authority active net to use ArcSwap

[lxfind]

Make the net field of authority active an ArcSwap. This will make it easy to swap with a new value latter.

[gdanezis]

Have a look at the initial draft for active services that restart in #2388 . I think we should at the start of each service get a consistent set of Arcs, and use these until service restart?

[gdanezis]

I do not think we should do that: it encourages each access to do a load() potentially, leading to different parts of the code using different nets / state / health. Instead: we can have a number of arcwaps to the inner Arcs, and upon restart we make an ActiveAuthority with a consistent set of Arcs, and use that for the lifetime of the service?

[lxfind]

What we could do is instead of passing around AuthorityActive in all the active functions, we pass around net. Then we don't have to restart.
Another general question I have is this: why are the active processes unsafe with potential committee changes? Don't they need to deal with potential byzantine validator anyway?

[bmwill]

Passive processes should be able to handle a swap but active once are going to need some sort of notification in order to do internal book keeping.

[bmwill]

So really I think we're going to need to have a reconfig notification go out on a broadcast channel as well to these arc swaps

[bmwill]

Not to mention that we cannot rely on ArcSwap in the future if we end up having multiple processes

[gdanezis]

we simply have to store the LifecycleSignalSender struct, and signal a Restart or Exit, and the services will restart (and this is when they should re-load the config etc) or exit.

[gdanezis]

Another general question I have is this: why are the active processes unsafe with potential committee changes? Don't they need to deal with potential byzantine validator anyway?

yeah, but they make the assumption that they know the correct committee and that 2/3 of it is correct. So if the committee changes half way through some task, and half the state is according to the old one, and the other half the new one, unknown things may happen.

[lxfind]

That's a reasonable assumption by checkpoint process, but I assume the gossip process doesn't have that assumption?

[lxfind]

Reminder for myself: we probably should also add some safety guards around all validator requests on the epoch number: that is, regardless how we do this, we should be able to handle the case where you are in epoch X, talking to a committee member who thinks it's epoch X + 1.

[gdanezis]

I assume the gossip process doesn't have that assumption?

It kind of does: for example, it computes how long to wait for responses before trying to reconnect on the basis of the size and stake distribution of the committee to not spam everyone with connection requests before the network is up.

[gdanezis]

Bottom line to summarize my position on the use of ArcSwaps:

Yes, lets have an ArcSwap warp the contents of the active authority. BUT we read + clone this into a swap at the start of the active service. And rely on active service re-start to read a new version. Then we use the cloned arc consistently for the lifetime of the service.

[lxfind]

Bottom line to summarize my position on the use of ArcSwaps:

Yes, lets have an ArcSwap warp the contents of the active authority. BUT we read + clone this into a swap at the start of the active service. And rely on active service re-start to read a new version. Then we use the cloned arc consistently for the lifetime of the service.

Yes I agree.

[gdanezis]

Ok, cool -- tell me how to want me to handle #2388 . One option is to use it for restart on epoch boundary. But we can also just use it for clean shutdown.

[lxfind]

Ok, cool -- tell me how to want me to handle #2388 . One option is to use it for restart on epoch boundary. But we can also just use it for clean shutdown.

Do we want to use it here though? At epoch boundaries, most of the gossip peers will likely remain the same. It seems inefficient if we have to wait to drain all the existing peers and recreate new peers if most of them will be the same anyway