Create a function to read and sort a list of urls of car splits / validate car file splits #169
Description
- HEAD request to get size
- Read header
- Figure out CID for subset
- Read a chunk at the end to load the subset
This will give us:
- start and end slot
- plus ordering
- validation that the remote car file is what we expect
- validation that we have all the slots for the epoch
This can also be done when loading split car files from the file system so we can validate that we have every slot in the epoch.
Metadata
Metadata
Assignees
Labels
No labels