… (#557) Two follow-ups split out of #322 / PR #554. #555 -- a failed backup write left an orphaned "<path>.part" staging file in backup storage. LocalBackend.WriteReader preserves it deliberately so the file-replication puller can resume from the last committed byte, but backup has no resume path: a retried backup starts over under a fresh backup ID. The staging file was therefore unreferenced garbage -- never read, never listed as a backup (no manifest.json), and holding disk equal to the bytes transferred before the failure. streamBackupFile now removes it on write failure. Cleanup is best-effort: the write already failed, so the delete very likely fails too, and a cleanup error must not mask or replace the original write error. #556 -- skipping a file exists to tolerate one narrow race: a file removed by compaction or retention between the listing and the copy, which touches a handful of files at the tail of a run. A large fraction of the backup failing to read is a different event (throttling, credential expiry, a storage outage), and returning a fraction of the data as a successful backup is how an operator discovers the gap at restore time instead of at backup time. Backups now fail when more than maxSkipRatio (10%) of files are unreadable. The skip count is still recorded on the failed run. The ratio check subsumes the previous all-skipped guard -- 100% exceeds any ratio -- so total-loss behavior is unchanged; a test pins it explicitly. maxSkipRatio is a constant rather than a config key: no operator has asked to tune it, and a knob nobody sets is a knob nobody tests. Co-authored-by: Ignacio Van Droogenbroeck <ignacio@vandroogenbroeck.net>

medium(auth): Backup reads entire files into memory #322

Description

Activity

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions