Skip to content

Conversation

@ivanovanton
Copy link
Contributor

According to documentation Frame_Content_Size_flag is a 2 bits value, not 1 bit as the current logic has.

@qkaiser
Copy link
Contributor

qkaiser commented Jun 21, 2024

Good catch ! How did you notice it ? It would be cool to have a ZSTD sample that triggers that part of the code, I'll see what I can do but if you have something you can upload it here and I'll put it at the right location in our integration tests.

@qkaiser qkaiser self-requested a review June 21, 2024 06:01
@qkaiser qkaiser added bug Something isn't working format:compression labels Jun 21, 2024
@ivanovanton
Copy link
Contributor Author

@qkaiser You just need to provide a file which decompressed size is long enough (> 65791), so FCS_Field_Size is 4 or 8, and Flag_Value is 2 (0b10) or 3 (0b11):
https://github.com/facebook/zstd/blob/dev/doc/zstd_compression_format.md#frame_content_size

tests.zip
1.zst - decompressed size 65791 (passing)
2.zst - decompressed size 65792 (failing)

@qkaiser
Copy link
Contributor

qkaiser commented Jun 21, 2024

@ivanovanton git LFS gets in the way of me modifying your forked branch. Could you do this ?

cp 2.zst unblob/tests/integration/compression/zstd/__input__/large_fcs.zstd
cd unblob
poetry run unblob -vvv -e tests/integration/compression/zstd/__output__/ -f -k tests/integration/compression/zstd/__input__/large_fcs.zstd
git add tests/integration/compression/zstd/__input__/large_fcs.zstd
git add tests/integration/compression/zstd/__output__/large_fcs.zstd_extract/zstd.uncompressed
git commit --fixup HEAD
git rebase -i --autosquash HEAD~2
git commit --amend -m 'fix(zstd): fix frame_content_size retrieval logic'

Then push your changes :)

@ivanovanton
Copy link
Contributor Author

@qkaiser hey, looks good?

@qkaiser qkaiser enabled auto-merge June 24, 2024 04:56
@qkaiser qkaiser merged commit 5b0f16f into onekey-sec:main Jun 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working format:compression

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants