⚠️ Internal work only. Community contributions are not being accepted for this issue.
Basekick Labs is implementing this internally. Please do not open PRs against it; they will be closed without review. Discussion and questions in the comments are welcome.
Summary
A backup writes three kinds of object to the destination, and all three build a key as <backup_id>/data/<source key> or similar. Only one of them checks that the result fits.
| Copy path |
Where |
What it does with an over-long destination key |
| Data files |
internal/backup/backup.go:565 |
checks len(destPath) > storage.MaxUsableKeyLen, skips the file, counts it in skipped_overlong_keys and names it in the log |
| Compaction recovery state |
internal/backup/backup.go:1410 |
no check at all — hands the key straight to streamBackupFile |
| Iceberg warehouse files |
internal/backup/warehouse.go:307 |
storage.ValidateKey, and fails the whole backup |
So the same hazard produces a skip, a hard failure, or an unvalidated write depending on which copy loop reaches it.
The state path is the one that matters most. A compaction recovery manifest is what tells a restore that a compacted output replaced its inputs; copyStateFiles already refuses to let one vanish silently (backup.go:1417) precisely because a backup missing it can restore an output next to the inputs it replaced. An over-long key there would fail the write with a storage-layer error that does not name the cause the way the data path's message does.
Why now
Reachable today only with very long source keys: the reservation is storage.MaxUsableKeyLen (1019) minus the 37-byte backup-<ts>-<8hex>/data/ prefix, so a source key has to exceed 982 bytes. Arc's own partition layout stays far below that; #761 was filed when keys placed in the storage root by other tools hit it.
It gets closer with named backup targets (#1085): a target gains a configurable key prefix, so the reservation shrinks by the prefix length and the threshold moves down for every copy path at once.
Expected
One rule for all three paths. The data path's behaviour is the right one to converge on: check the destination key before writing, skip the file, count it, and name it with the byte figures the operator can act on. The warehouse path's fail-the-run is defensible for a warehouse (it is not routed and a partial warehouse is its own hazard) but should at least be stated as a deliberate difference rather than an accident.
Notes
Summary
A backup writes three kinds of object to the destination, and all three build a key as
<backup_id>/data/<source key>or similar. Only one of them checks that the result fits.internal/backup/backup.go:565len(destPath) > storage.MaxUsableKeyLen, skips the file, counts it inskipped_overlong_keysand names it in the loginternal/backup/backup.go:1410streamBackupFileinternal/backup/warehouse.go:307storage.ValidateKey, and fails the whole backupSo the same hazard produces a skip, a hard failure, or an unvalidated write depending on which copy loop reaches it.
The state path is the one that matters most. A compaction recovery manifest is what tells a restore that a compacted output replaced its inputs;
copyStateFilesalready refuses to let one vanish silently (backup.go:1417) precisely because a backup missing it can restore an output next to the inputs it replaced. An over-long key there would fail the write with a storage-layer error that does not name the cause the way the data path's message does.Why now
Reachable today only with very long source keys: the reservation is
storage.MaxUsableKeyLen(1019) minus the 37-bytebackup-<ts>-<8hex>/data/prefix, so a source key has to exceed 982 bytes. Arc's own partition layout stays far below that; #761 was filed when keys placed in the storage root by other tools hit it.It gets closer with named backup targets (#1085): a target gains a configurable key prefix, so the reservation shrinks by the prefix length and the threshold moves down for every copy path at once.
Expected
One rule for all three paths. The data path's behaviour is the right one to converge on: check the destination key before writing, skip the file, count it, and name it with the byte figures the operator can act on. The warehouse path's fail-the-run is defensible for a warehouse (it is not routed and a partial warehouse is its own hazard) but should at least be stated as a deliberate difference rather than an accident.
Notes
backupDataKeyHeadroomand the operator-facingmax_source_key_bytesfigure are pinned literally byinternal/backup/key_length_test.go:45andinternal/backup/skipped_sample_issue977_test.go:243; both become per-target in backup (stage B): named remote targets (local, S3, Azure) with per-database routing #1085.