Skip to content

Robj security audit - #685

Open
rtjohnso wants to merge 107 commits into
mainfrom
robj-security-audit
Open

Robj security audit#685
rtjohnso wants to merge 107 commits into
mainfrom
robj-security-audit

Conversation

@rtjohnso

@rtjohnso rtjohnso commented Oct 11, 2026

Copy link
Copy Markdown
Collaborator

Implement durable_barrier by making log durable.

  • writeback_set mechanism

Make recovery guaranteed to be a prefix of operations in their linearization order.

  • log buffering
  • log groups

Allocation-state recovery

Add command-line tool to inspect disk image

Add benchmarking suite and infrastructure

Make disk image non-trusted

Optimize range query startup

Massive bug cleanup

rtjohnso added 30 commits July 31, 2026 12:11
@rtjohnso
cache layer of new writeback scheme
94888d3 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
Fix writeback races
bc9c1c1 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
fix some error handling in writeback
d997a56 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
definitely die on io error in writeback
1d4dd95 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
start to handle writeback errors
2e6fffa 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
formatting
0c54d7c 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
more writeback error handling
1ddcac3 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
formatting
19face6 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
still working on writeback
6807f75 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
add async_io_state pool
a8b7a0b 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
formatting
00e15e7 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
formatting
03bafcf 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
add buffering to the log
7552e2e 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
formatting
1d9500b 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
log groups
b4904dd 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
formatting
57c97e0 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
fix annoying clang options warning/error on github
89533e7 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
log cleanups
5e326b9 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
log bugfixes and playback end detection
75d8a77 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
handle log sealing errors
d7db242 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
formatting
822477d 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
clean up unmount
46ce935 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
formatting
fc77ea9 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
recovery space allocation work
a978e0c 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
complete recovery
6769a7f 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
formatting
e54edb9 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
minor cleanups
20b9f69 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
cleanup blob and log recovery
a417120 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
cleanup rc_allocator and replay bug in core
542e44f 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
make unmount return error only when there might be data loss
fbd2591 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
rtjohnso added 28 commits October 2, 2026 22:20
@rtjohnso
fix iterator cleanup refcounting bug, trunk serialization cleanup bug…
2ae5770 
…, flushing fail cleanup bug, stats off-by-one error

Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
fix trunk gc deadlock, trunk leaf-split allocation failure cleanup, c…
077aced 
…ompaction accounting bug, leaf init failure cleanup bug

Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
fix root growth double free on failure, leaf splitting dup pivot bug,…
0b8b7ba 
… io error handling in iterator reversal and iterator error handling in trunk splits

Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
fix root durable_barrier hang on incorp failure, btree_pack failure d…
aa7fdfa 
…isk leak, stats bugs

Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
fix iterator endpoint error handling in merge
aa3dd2c 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
add basic sanity check to allocator map on mount, fix trunk stat prin…
32950c1 
…ting race, fix task system init failure bug, improve trunk leaf splitting computation, fix core_print_lookup memstream leak

Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
add sanity checks to routing filter config
23f39b2 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
sanity check superblock, propagate errors in branch range counting, h…
f392551 
…andle errors in trunk lookup logging

Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
add some tests for recent bugs
3376a1a 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
fix merge iterator reversal bugs, trunk leaf key estimation bugs, btr…
b24980b 
…ee split/defrag computation bugs

Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
fix allocation failure handling, several integer overflow edge cases,…
c699dea 
… missing callback handling

Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
fix several small integer bugs
76632cd 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
make fetch-and-update succeed or fail completely. Cannot succed on in…
723f20a 
…sert but fail to return old value.

Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
ensure items are logged iff they are inserted into memtable
21eb256 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
formatting
d64385d 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
eliminate expensive, incomplete, unnecessary routing filter validation
7a6a652 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
eliminate some unnecessary validation of on-disk data structures
52d3aec 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
eliminate some unnecessary validation of on-disk data structures
0ba6768 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
experimental: inline btree validation checks
ab6d71a 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
experimental: inline trunk/routing-filter validation checks
dc11e7a 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
avoid rescanning routing filter encodings
def8c53
@rtjohnso
optimize mini allocator validity checks
1774df0 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
optimize point lookup validity checks
bdd9f65 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
add version and fp size to filters, bump default to 27 bits
cddb236 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
defer btree prefetch until extent boundary
b25c078
@rtjohnso
remove redundant validity checks
9469982 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
remove redundant validity checks
56f4fa5 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
tweak benchmarks
bf5023d 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@netlify

netlify Bot commented Oct 11, 2026 •
edited
Loading

Copy link
Copy Markdown

❌ Deploy Preview for splinterdb failed.

Name Link
🔨 Latest commit db3047a
🔍 Latest deploy log https://app.netlify.com/projects/splinterdb/deploys/6acb0a0077979b0008ffd75c

@rtjohnso
delete dead function
db3047a 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@rtjohnso