Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for LZMA compression #14

Open
mzaja opened this issue Nov 27, 2022 · 0 comments
Open

Add support for LZMA compression #14

mzaja opened this issue Nov 27, 2022 · 0 comments

Comments

@mzaja
Copy link

mzaja commented Nov 27, 2022

Would it be possible to add support for LZMA-compressed ZIP files? I did some benchmarking and found out that LZMA algorithm provides very similar compression ratios compared to BZ2, but decompression time is 3x faster. It is my algorithm of choice for compressing my own collected data and I would love to have this package support it.

Here are the results of some benchmarks. The numbers are: compression ratio, compression time, decompression time. Since the compression is done only once, the compression time is not relevant. However, the format is clearly superior to BZ2 which is what Betfair uses to provide data (probably because they don't bother to decompress it).

Collected data:
GZIP: 15.1 %, 5148 ms, 155 ms.
ZIP: 16.0 %, 1792 ms, 140 ms.
BZIP: 10.4 %, 4441 ms, 995 ms.
LZMA: 10.6 %, 15809 ms, 350 ms.

Betfair PRO data:
GZIP: 11.8 %, 2625 ms, 150 ms.
ZIP: 12.6 %, 884 ms, 121 ms.
BZIP: 7.5 %, 4676 ms, 841 ms.
LZMA: 8.3 %, 12055 ms, 266 ms.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant