Ping or Heartbeat for partner sites, detect if a site has lost a follower

[c0xc]

Describe the problem to be solved

It seems like an upstream site could remove a mirror instance if the downstream site has been offline for maintenance work for too long (see bug 3847).

The upstream instance might log something like 'Removed bad %d actor follows.', although I'm not sure if that's always the case.

When the downstream instance comes back online, it still thinks it's following the upstream site but the upstream site doesn't list it in the followers section anymore. This also appears to prevent new videos from being mirrored.

Describe the solution you would like:

This suggestion is for a ping (from the upstream site, interval and action configurable) and, more importantly, heartbeat from the downstream site, i.e., mirror, when it comes back online. So if the upstream site has removed it after the configured timeout, the mirror site's heartbeat should recognize that and at least produce an error message, maybe show the upstream site as "connection broken" under "instances you follow". This state should be returned by the api so that an admin can be notified that the mirror instance has stopped mirroring the upstream instance.

Going one step further, it might be an idea to have a "i am following you" heartbeat rather than a generic "i am alive" one and then restore this follower in the upstream instance automatically (if it's known that it used to be a a follower which was accepted) or notify the admin of the upstream instance about a new follower.

Either way, a lost follow relationship shouldn't go unnoticed, especially if it renders a mirror instance useless.

Describe alternatives you have considered

[c0xc]

This is still an open issue, it keeps happening and I don't even see a clear pattern anymore. Sometimes X still shows up as follower under Y even though instance X has been offline or showing some error for weeks, then some other X2 is following Y2 which wasn't down and lists all videos of Y2 but isn't listed as follower on Y2 anymore, then again X3 follows Y3 and it has stopped showing new videos of Y3.
I think there needs to be a reliable heartbeat mechanism and some reporting, especially for other instances listed in the left column "followers" as these appear to disappear too often for unknown reasons.
@Chocobozzz
Could there be any other reason why some followes disappears from the "followers" list?
Are there any initial thoughts on how this should be implemented other than a configurable heartbeat?