Statefile gets corrupted and hydra-node cannot continue #2253
Description
Context & versions
Using the latest release (0.22.4) we have seen two kinds of corrupted statefiles.
- Nulls written into the last line:
- Simply incomplete JSON line:
I think this one may be due to not-enough-space-left-on-device error while writing; but this needs to be tested.
Steps to reproduce
???
Ideas
It seems like there might be a simple fix for both of these cases; namely to just log a warning and ignore the line. But the deeper question is how this is possible in the first place?