You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
A long but legal source key makes every backup fail permanently, because the destination key overruns the limit #761
A source key that the storage contract fully accepts can make every backup fail, permanently, and leave partial data in the backup destination with no manifest.
internal/backup/backup.go:288 builds the destination as:
backupID is 31 bytes (manager.go:100-104) and /data/ is 6, so the destination is 37 bytes longer than the source key. MaxUsableKeyLen is 1019, so any source key over 982 bytes produces a destination key the backup backend refuses.
That failure is not classified as skippable. streamBackupFile only wraps source read failures with errBackupRead (backup.go:295-297), and the write to backup storage is deliberately fatal, so the whole run aborts:
[source key len=984, ValidateKey=<nil>]
CreateBackup ERROR: failed to back up db/sss…/f.parquet: failed to write to backup storage:
invalid path: storage: invalid path: key is 1021 bytes, over the 1019-byte limit
Every subsequent backup fails the same way, since nothing about the source changes. The destination is left holding whatever was copied before the abort, with no manifest written.
This is the same lesson MaxUsableKeyLen exists for, applied one layer up and missed. #744 reserved headroom in the key contract because LocalBackend appends .part to every key it stages; the backup path appends a 37-byte prefix to every key it copies and reserves nothing. Its own comment at backup.go:25-31 says the fatal classification exists so that "a failure mode added here later is fatal by default until someone marks it skippable", which is the right default and is exactly why this one is fatal.
Worth deciding explicitly rather than patching:
Treat a destination-key rejection as skippable, so the run completes and reports the file (it would then flow through the same accounting LocalBackend listings do not filter unusable keys, unlike S3 and Azure #756 adds for files that could not be copied). This keeps backups working and makes the gap visible.
Shorten the destination layout so the overhead is small and bounded, and document the reserved headroom the way MaxUsableKeyLen does.
Option 1 plus a documented headroom constant looks right: backups should not stop working because one measurement has a very long name, and an operator needs to know which file is affected.
Related: #756 (backup silently omitting files it could not address, where the accounting this would reuse is added).
Labelled good first issue. Option 1 above is the intended shape.
Where:internal/backup/backup.go:288 (destPath construction) and the skippable classification in streamBackupFile (:295-297).
What to change: before copying, check the destination key against storage.MaxUsableKeyLen. A key that would overrun is recorded as skipped with a reason, through the accounting #765 added for unaddressable files (progress.SkippedFiles, checkSkipRatio, the manifest's skipped list), instead of aborting the run. Add a documented headroom constant for the <backupID>/data/ prefix next to where backupID is built (manager.go:100-104) so the number is not implicit.
Worth knowing: keep every other write-to-backup-storage failure fatal; only the predictable length overrun becomes skippable. The comment at backup.go:25-31 explains why fatal is the default.
Scope: one package. Regression test: a legal source key of 983+ bytes is skipped and reported, the backup still completes with a manifest, and a shorter key is copied. go test ./internal/backup/.... Release-notes entry per CONTRIBUTING.
Found while working #756.
A source key that the storage contract fully accepts can make every backup fail, permanently, and leave partial data in the backup destination with no manifest.
internal/backup/backup.go:288builds the destination as:backupIDis 31 bytes (manager.go:100-104) and/data/is 6, so the destination is 37 bytes longer than the source key.MaxUsableKeyLenis 1019, so any source key over 982 bytes produces a destination key the backup backend refuses.That failure is not classified as skippable.
streamBackupFileonly wraps source read failures witherrBackupRead(backup.go:295-297), and the write to backup storage is deliberately fatal, so the whole run aborts:Every subsequent backup fails the same way, since nothing about the source changes. The destination is left holding whatever was copied before the abort, with no manifest written.
This is the same lesson
MaxUsableKeyLenexists for, applied one layer up and missed. #744 reserved headroom in the key contract becauseLocalBackendappends.partto every key it stages; the backup path appends a 37-byte prefix to every key it copies and reserves nothing. Its own comment atbackup.go:25-31says the fatal classification exists so that "a failure mode added here later is fatal by default until someone marks it skippable", which is the right default and is exactly why this one is fatal.Worth deciding explicitly rather than patching:
MaxUsableKeyLendoes.Option 1 plus a documented headroom constant looks right: backups should not stop working because one measurement has a very long name, and an operator needs to know which file is affected.
Related: #756 (backup silently omitting files it could not address, where the accounting this would reuse is added).