Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Draft] Opencl lz4 #3900

Open
wants to merge 22 commits into
base: main
Choose a base branch
from
Open

[Draft] Opencl lz4 #3900

wants to merge 22 commits into from

Conversation

kif
Copy link
Member

@kif kif commented Jul 6, 2023

Changelog: LZ4 compressor on GPU

  • OpenCL code targeted to run on GPU
    • Find position of interest
    • Create segment descriptor (position in input, length of litterals and matches)
    • Concatenate several descriptor (between passes, in the case there are only litterals)
    • Write compressed stream, including tokens, offsets, overflows in litteral and matches
    • Manage the beginning of stream (4 bytes with uncompressed buffer size)
    • Manage end of stream (cannot end with matches, has to finish with litterals
    • Concatenate output stream to make it contiguous (1 or 2 kernel options ...)
  • Python code to call the OpenCL code
  • Non regression test
  • Profiling and performance assessment

@kif kif requested a review from pierrepaleo July 6, 2023 15:19
@t20100
Copy link
Member

t20100 commented Dec 11, 2023

I've added a commit to fix the conflict introduced by #3991.
A rebase would be best in order to sparse the merge of main in this branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants