This repository has been archived by the owner on Jan 22, 2025. It is now read-only.
use bytemuck for disk bucket restart #33371
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jeffwashington
force-pushed
the
sp35
branch
from
a728692 to
7a2aa13
Compare
Codecov Report
@@ Coverage Diff @@
## master #33371 +/- ##
=======================================
Coverage 81.9% 81.9%
=======================================
Files 797 797
Lines 216013 216018 +5
=======================================
+ Hits 177039 177047 +8
+ Misses 38974 38971 -3 |
HaoranYi
approved these changes
Sep 22, 2023
| // In order to safely guarantee Header is Pod, it cannot have any padding. | ||
| const _: () = assert!( | ||
| std::mem::size_of::<Header>() == std::mem::size_of::<u128>() * 2, | ||
| "Header cannot have any padding" |
There was a problem hiding this comment.
nit: the error message is not accurate.
There was a problem hiding this comment.
I suppose it could be more clear if I said no automatic padding can be added to the header. Thoughts?
There was a problem hiding this comment.
Maybe, just say "incorrect size of header struct" generally.
There was a problem hiding this comment.
done in next pr
| // In order to safely guarantee Header is Pod, it cannot have any padding. | ||
| const _: () = assert!( | ||
| std::mem::size_of::<OneIndexBucket>() == std::mem::size_of::<u128>() * 2, | ||
| "Header cannot have any padding" |
There was a problem hiding this comment.
nit: inaccurate error message.
SwenSchaeferjohann
pushed a commit
to ananas-block/solana
that referenced
this pull request
Sep 23, 2023
Switch confirmed_unrooted_slots from Vec<_> to HashSet<_> (solana-labs#33311) The container is only used to check for inclusion of slots with the .contains() method. This method is O(n) on a Vec<_> but O(1) on a HashSet<_>. add Restart structs for disk index (solana-labs#33361) use bytemuck for disk bucket restart (solana-labs#33371) disk bucket: init restart path (solana-labs#33375) simple cleanup in bucket map (solana-labs#33376) add disk bucket get_restart_file (solana-labs#33373) * add disk bucket get_restart_file * add get_restartable_buckets pass RestartableBucket through disk index (solana-labs#33377) data bucket holds RestartableBucket (solana-labs#33381) Feature - better error codes for tx lamport check (solana-labs#33343) Replaces `TransactionError::InstructionError(0, InstructionError::UnbalancedInstruction)` with `TransactionError::UnbalancedTransaction`. Co-authored-by: Alexander Meißner <AlexanderMeissner@gmx.net> DiskIdx: reuse disk bucket file if possible (solana-labs#33379) diskidx: stats for created vs reused (solana-labs#33385) solana-program - altbn128: add g1 & g2 compression still fixing tests for point of infinity feat: proof compression syscall working add rust test to ci remove prints added c test added sycall pricing fixed ci checks refactored altbn128 and compression
jeffwashington
added a commit
to jeffwashington/solana
that referenced
this pull request
Sep 25, 2023
brooksprumo
reviewed
Sep 25, 2023
| #[repr(C)] | ||
| pub(crate) struct Header { | ||
| /// version of this file. Differences here indicate the file is not usable. | ||
| version: u64, | ||
| /// number of buckets these files represent. | ||
| buckets: usize, | ||
| buckets: u64, | ||
| /// u8 representing how many entries to search for during collisions. | ||
| /// If this is different, then the contents of the index file's contents are likely not as helpful. | ||
| max_search: u8, | ||
| /// padding to get header to u128 aligned | ||
| _dummy: [u8; 15], |
There was a problem hiding this comment.
As a nit, this padding doesn't change the alignment of the Header. Since the field with the largest alignment is a u64, the alignment will always be 8 bytes regardless of the amount of padding bytes.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
Speeding up restart.
Reusing disk index files on start.
Summary of Changes
Use bytemuck for manipulations.
Improve test.
Fixes #