Skip to content

[feature] Support Streaming Compression/Decompression with Dictionary #61

@AndrewCEmil

Description

@AndrewCEmil

Hi! I noticed that this library supports streaming compression/decompression and also dictionary compression/decompression. But it doesn't seem to support streaming compression/decompression using a dictionary.

I'd to add support for this. Please let me know if you have any thoughts on implementation or interface for this. I'm currently working through the changes in my forked repo: https://github.com/AndrewCEmil/zstd-ruby.

Ideally I would like to support:

  • Streaming compression/decompression using raw dictionary bytes
  • Streaming compression/decompression using processed dictionary objects (ZSTD_CDict and ZSTD_DDict)
  • The ability to process dictionary bytes. This would effectively allow a user to call ZSTD_CreateCDict and ZSTD_CreateDDict and receive the output in ruby. They could then store the processed and use it later rather than having to process it each time, which according to the zstd docs (http://facebook.github.io/zstd/zstd_manual.html) is a costly operation.

Initially, I'll probably just implement the first feature (raw dictionary bytes), but I'll try to leave room in the interface to support caching for later implementation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions