Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This removes the custom zip file handling code and uses libzip instead. This addresses the missing Zip64 support (ie. the inability to write npz files with large arrays; #39). For ease of usage, the libzip++ C++ bindings are used. A submodule with the libzip++ header is included and cmake is set up to make sure the submodule is automatically initialized. This uses some C++14 features, but support should be sufficiently widespread by now I guess.
I believe it is simpler and more safe to rely on libzip instead of adding custom code to cnpy to handle Zip64 support while remaining compatible with the currently used Zip format. libzip handles this and other Zip extensions transparently.
This should also fix reading the header_length field in
parse_npy_header
on big-endian systems.If you're unhappy with introducing new dependencies and raising the requirement, feel free to close this. I will keep the code around as a fork for anyone that is willing to make the trade-off and requires Zip64 support. But of course it would be great if this can be merged in some way so that other people don't run into similar issues. Please let me know what you think. I may add some more documentation in case you would like to merge this at some point.