Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Node v6 breakage with gunzip #46

Closed
springmeyer opened this issue Sep 13, 2016 · 2 comments
Closed

Node v6 breakage with gunzip #46

springmeyer opened this issue Sep 13, 2016 · 2 comments

Comments

@springmeyer
Copy link
Contributor

Using file-sniff with node v6 leads to the inability to open valid gzip compressed files.

Errors look like:

# [tif+gz] Sniffing file: should return tif+gz filetype
not ok 89 no error
  ---
    operator: error
    expected: undefined
    actual:   { code: 'EINVALID' }
    stack:
      Error: Unknown filetype
        at invalid (/Users/dane/projects/mapbox-file-sniff/lib/invalid.js:2:52)
        at /Users/dane/projects/mapbox-file-sniff/index.js:80:34
        at Gunzip.onError (zlib.js:212:5)
        at emitOne (events.js:96:13)
        at Gunzip.emit (events.js:188:7)
        at Zlib._handle.onerror (zlib.js:373:10)
  ...
not ok 90 filetype: undefined
  ---
    operator: equal
    expected: 'tif+gz'
    actual:   undefined
  ...
# [serialtiles] Sniffing file: should return serialtiles filetype and serialtiles protocol
not ok 91 Error: Unknown filetype
  ---
    operator: error
    expected: undefined
    actual:   { code: 'EINVALID' }
    stack:
      Error: Unknown filetype
        at invalid (/Users/dane/projects/mapbox-file-sniff/lib/invalid.js:2:52)
        at /Users/dane/projects/mapbox-file-sniff/index.js:80:34
        at Gunzip.onError (zlib.js:212:5)
        at emitOne (events.js:96:13)
        at Gunzip.emit (events.js:188:7)
        at Zlib._handle.onerror (zlib.js:373:10)
  ...
# [serialtiles] Sniffing file: should return serialtiles filetype and serialtiles protocol
not ok 92 Error: Unknown filetype
  ---
    operator: error
    expected: undefined
    actual:   { code: 'EINVALID' }
    stack:
      Error: Unknown filetype
        at invalid (/Users/dane/projects/mapbox-file-sniff/lib/invalid.js:2:52)
        at /Users/dane/projects/mapbox-file-sniff/index.js:80:34
        at Gunzip.onError (zlib.js:212:5)
        at emitOne (events.js:96:13)
        at Gunzip.emit (events.js:188:7)
        at Zlib._handle.onerror (zlib.js:373:10)
  ...
# [tm2z] Sniffing file: should return tm2z filetype and tm2z protocol
not ok 93 Error: Unknown filetype
  ---
    operator: error
    expected: undefined
    actual:   { code: 'EINVALID' }
    stack:
      Error: Unknown filetype
        at invalid (/Users/dane/projects/mapbox-file-sniff/lib/invalid.js:2:52)
        at /Users/dane/projects/mapbox-file-sniff/index.js:80:34
        at Gunzip.onError (zlib.js:212:5)
        at emitOne (events.js:96:13)
        at Gunzip.emit (events.js:188:7)
        at Zlib._handle.onerror (zlib.js:373:10)
  ...
# [tm2z] Sniffing file with extended header: should return tm2z filetype and tm2z protocol
not ok 94 Error: Unknown filetype
  ---
    operator: error
    expected: undefined
    actual:   { code: 'EINVALID' }
    stack:
      Error: Unknown filetype
        at invalid (/Users/dane/projects/mapbox-file-sniff/lib/invalid.js:2:52)
        at /Users/dane/projects/mapbox-file-sniff/index.js:80:34
        at Gunzip.onError (zlib.js:212:5)
        at emitOne (events.js:96:13)
        at Gunzip.emit (events.js:188:7)
        at Zlib._handle.onerror (zlib.js:373:10)
  ...

The problem is that node v6 became more strict in how it handles and throws gunzip errors (nodejs/node#5883) and our method of only reading part of the gzip'd file is breaking now.

@springmeyer
Copy link
Contributor Author

By default, the zlib methods with throw an error when decompressing truncated data. However, if it is known that the data is incomplete, or the desire is to inspect only the beginning of a compressed file, it is possible to suppress the default error handling by changing the flushing method that is used to compressed the last chunk of input data:

from https://nodejs.org/api/zlib.html, indicates { finishFlush: zlib.Z_SYNC_FLUSH } is needed to do this, which we likely should have been doing all along.

@springmeyer
Copy link
Contributor Author

springmeyer commented Sep 13, 2016

Done, fix will land in #45 (0b46f11)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant