Repository navigation
fix(wal): use io.ReadFull instead of f.Read for WAL header reads - #484
Conversation
f.Read may return fewer bytes than the buffer size without error, causing partial reads that cascade into misaligned subsequent reads. Replace both WAL file header and entry header reads with io.ReadFull which guarantees the buffer is filled completely or returns an error. Fixes #341
There was a problem hiding this comment.
Code Review
This pull request updates the WAL reader to use io.ReadFull instead of f.Read for fixed-size file and entry header reads to prevent partial reads. However, the review feedback highlights two critical issues: first, swallowing all read errors during the file header read is dangerous because transient I/O errors could be mistaken for an empty file, potentially leading to data loss; second, mapping io.ErrUnexpectedEOF to io.EOF when reading entry headers silently ignores truncated or corrupted entries at the end of the WAL file, bypassing corruption logging and metrics.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
…ader read - File header: separate io.EOF/ErrUnexpectedEOF (short/empty file, benign) from actual I/O errors (disk failure, must surface). - Entry header: route io.EOF cleanly to caller (0 bytes read = end of WAL) and io.ErrUnexpectedEOF (truncated entry) through corruption logging instead of silently treating it as clean EOF.
|
@gemini-code-assist please review |
There was a problem hiding this comment.
Code Review
This pull request improves the robustness of the WAL reader by replacing f.Read with io.ReadFull for fixed-size file and entry header reads, preventing potential corruption from partial reads. The review feedback suggests optimizing performance and reducing heap allocations by using stack-allocated fixed-size arrays instead of slices for the header buffers.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Replace make([]byte, WALFileHeaderSize) and make([]byte, WALEntryHeaderSize) with var header [WALFileHeaderSize]byte / [WALEntryHeaderSize]byte. Both sizes (7, 16) are known at compile time — no heap alloc needed in the hot recovery path.
|
@gemini-code-assist please review |
There was a problem hiding this comment.
Code Review
This pull request updates the WAL reader to use io.ReadFull with fixed-size arrays for reading file and entry headers, preventing partial reads from corrupting subsequent entries. The review feedback highlights a critical issue in readEntry where io.ErrUnexpectedEOF is not handled. If a crash results in a partially written entry header, the reader will fail with a hard error instead of gracefully stopping at the end of the valid entries. A code suggestion is provided to map io.ErrUnexpectedEOF to io.EOF to ensure robust recovery.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
io.ReadFull returns three outcomes: n=16, err=nil → full header, proceed n=0, err=io.EOF → clean end of WAL, stop 1≤n≤15, err=io.ErrUnexpectedEOF → crash during write, truncated header Without the io.ErrUnexpectedEOF→io.EOF mapping, case 3 hits fmt.Errorf, the caller's err==io.EOF check fails, and continue tries to read the next entry at a misaligned offset — the exact cascading corruption bug #341 describes. The file offset is mid-header and unrecoverable. The previous review incorrectly claimed io.EOF from io.ReadFull was dead code. It is not: ReadAtLeast preserves io.EOF when 0 bytes are read.
|
@gemini-code-assist please review |
There was a problem hiding this comment.
Code Review
This pull request updates the WAL reader to use io.ReadFull for fixed-size file and entry header reads to prevent partial reads from causing misaligned subsequent reads. It also updates the release notes to document this change. Feedback on the changes suggests that mapping io.ErrUnexpectedEOF to io.EOF when reading entry headers will cause truncated entry headers (e.g., from a crash mid-write) to be silently ignored instead of being flagged as corrupted. It is recommended to let io.ErrUnexpectedEOF propagate as an error so the caller can properly log and track the corruption.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Respectfully, this recommendation would reintroduce the cascading misalignment bug #341 describes. Here is the trace: If The file offset after a partial The Corruption tracking ( |
Fixes #341. The WAL reader used f.Read to read fixed-size headers. f.Read may return fewer bytes than the buffer size without error, causing partial reads that cascade into misaligned subsequent reads. Both header reads now use io.ReadFull.
All 40 WAL tests pass. gofmt/go vet clean. No configuration-interaction risk.