-
Notifications
You must be signed in to change notification settings - Fork 62
Open
Description
Block checksums are optional. From:
https://www.asdf-format.org/projects/asdf-standard/en/latest/file_layout.html#block-header
checksum (16-byte string): An optional MD5 checksum of the used data in the block. The special value of all zeros indicates that no checksum verification should be performed.
However:
- there is no way to disable writing checksums
- opening a file with an all 0 checksum and
validate_checksums=Trueresults in an error
If we have a file "bar.asdf" with an array with a checksum of all 0s.
>> af = asdf.open("bar.asdf", validate_checksums=True) # does not fail due to lazy loading
>> af._blocks.blocks[0].header
{'flags': 0,
'compression': b'\x00\x00\x00\x00',
'allocated_size': 800,
'used_size': 800,
'data_size': 800,
'checksum': b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'}
>> af["arr"][0]
ValueError: Block at 701 does not match given checksumThe expectation here would be:
- there is a way to disable checksum calculations for some or all blocks
- blocks with all 0 checksums would not fail checksum validation
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels