feat: reduce size of segments containing fixed-size-list arrays#6180
feat: reduce size of segments containing fixed-size-list arrays#6180
Conversation
I also verified this on some fixed-size-list data I have locally and, indeed, I see more reasonable segment sizes (~1MiB). Signed-off-by: Daniel King <dan@spiraldb.com>
CodSpeed Performance ReportMerging this PR will degrade performance by 20.01%Comparing
|
| // `write_stream` from combining adjacent 0.9 MiB chunks into one 1.8 MiB chunk. | ||
| let max_rows = usize::try_from(block_size_target.div_ceil(elem_size as u64)) | ||
| .unwrap_or(usize::MAX); | ||
| self.block_len_multiple.min(max_rows).max(1) |
There was a problem hiding this comment.
this should help with large binary values too, no?
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
I also verified this on some fixed-size-list data I have locally and, indeed, I see more reasonable segment sizes (~1MiB). --------- Signed-off-by: Daniel King <dan@spiraldb.com>
I also verified this on some fixed-size-list data I have locally and, indeed, I see more reasonable segment sizes (~1MiB).