Browse zip file entries without loading data into memory

[manderson23]

From reading the API docs it doesn't seem possible to iterate though entry meta data (filename etc.) without reading the complete zip file into browser memory.

It would be useful to browse the zip file metadata without loading the complete file. Like getEntries in zip.js (http://gildas-lormeau.github.io/zip.js/core-api.html#zip-reading).

[dduponchel]

The missing feature seems to be the custom (async) reader. In the given link, only HttpRangeReader seems to not need everything in memory. Or do you see an other solution to avoid that in a browser ?

[manderson23]

Maybe just giving that link wasn't quite clear enough.

When testing zip.js I was able to do the something like:

zip.createReader(new zip.BlobReader(blob), function(reader) {

  // get all entries from the zip
  reader.getEntries(function(entries) {
    if (entries.length) {
      console.log(entries[0]);
    }
  });
});

to list the entries in the zip file. I was able to do this for a 3Gb zip file with browser memory usage staying pretty much flat.

From reading the jszip API documents I couldn't see an equivalent to this.

[dduponchel]

Indeed, we read the full content of the blob before parsing it. This requires some changes in the way we read blob objects but this would definitively be a nice thing to do.

[vincerubinetti]

I'm looking to do this. I have very large (like gigabytes) zip files that I'd like to get a file listing of (filenames, extensions, sizes if possible, etc.) without downloading them. I found this Python library that does it using Http ranges:

unzip-http

I think I'm going to try to port their infoiter function to JavaScript. I'll post here if I can come up with a snippet that works.

Unless anyone knows of any existing JS library that can already do this? I searched npm pretty thoroughly and couldn't find anything.

EDIT Nevermind, that zip.js library does exactly what I need. Didn't realize it because that docs link OP posted is a 404 now, and their current docs don't highlight or give an example of the HttpRangeReader. Had to find it in their tests:

await new ZipReader(new HttpRangeReader(url)).getEntries();

Thank god I don't have to do manual byte-by-byte parsing 😁