Closed
Description
Issue
We have noticed some database locks occurring on the GCP Aggregator which these error messages:
cannot start a transaction within a transaction
database is locked
⚠ The problem occurred after activating the pruning
feature of the stores #516
Here are the full logs:
mithril-aggregator_1 | {"msg":"STATE MACHINE: an error occurred: ","v":0,"name":"slog-rs","level":50,"time":"2022-09-20T14:44:11.436860471Z","hostname":"1db9e5f989b2","pid":1,"error":"StoreError(AdapterError(QueryError(Error { code: Some(1), message: Some(\"cannot start a transaction within a transaction\") })))"}
mithril-aggregator_1 | {"msg":"… Cycle finished, Sleeping for 60000 ms","v":0,"name":"slog-rs","level":30,"time":"2022-09-20T14:44:11.436936131Z","hostname":"1db9e5f989b2","pid":1}
mithril-aggregator_1 | {"msg":"⇄ HTTP SERVER: certificate_pending","v":0,"name":"slog-rs","level":20,"time":"2022-09-20T14:44:11.436978054Z","hostname":"1db9e5f989b2","pid":1}
mithril-aggregator_1 | {"msg":"certificate_pending::error","v":0,"name":"slog-rs","level":40,"time":"2022-09-20T14:44:11.437023811Z","hostname":"1db9e5f989b2","pid":1,"error":"AdapterError(ParsingDataError(Error { code: Some(5), message: Some(\"database is locked\") }))"}
mithril-aggregator_1 | {"msg":"⇄ HTTP SERVER: certificate_pending","v":0,"name":"slog-rs","level":20,"time":"2022-09-20T14:44:12.626648308Z","hostname":"1db9e5f989b2","pid":1}
mithril-aggregator_1 | {"msg":"certificate_pending::error","v":0,"name":"slog-rs","level":40,"time":"2022-09-20T14:44:12.626705143Z","hostname":"1db9e5f989b2","pid":1,"error":"AdapterError(ParsingDataError(Error { code: Some(5), message: Some(\"database is locked\") }))"}
Workaround
Restart the container