Closed
Description
This issue was originally created by @recht at matrix-org/dendrite#3177.
Background information
- Dendrite version or git SHA: 0.13.1
- SQLite3 or Postgres?: Postgres
- Running in Docker?: Yes
go version
: 1.19.7- Client used (if applicable):
Description
- What is the problem: Dendrite stops processing events if nats is restarted.
- Who is affected: Everybody
- How is this bug manifesting: Restart the entire nats cluster in a non-rolling fashion. This will trigger a
nats: Server Shutdown
error which exits the JetStreamConsumers, but leaves the rest of the system running. Only remediation seems to be to restart the server. - When did this first appear:
Steps to reproduce
- Start Dendrite
- Kill all nats servers simultaneously
- Messages are no longer processed
Ideally Dendrite would reconnect once nats is available again. Alternatively the service should exit, or at the very list trigger a signal somewhere that streams are no longer being processed.