Accept-encoding not sent by MapBox Studio

[pnorman]

I finally did the packet logging to verify the MBS bug mentioned in mapbox/vector-tile-spec#27 (comment)

MapBox Studio does not send Accept-Encoding with its requests for vector tiles, but gzipped tiles served with Content-Encoding: gzip are expected (mapbox/vector-tile-spec#27 (comment)).

Without an Accept-Encoding field, a server SHOULD use the identity encoding, so most HTTP servers will require some setup to serve compressed data where there is no accept-encoding and include a content-encoding header indicating that it is compressed.

cc @bcamper @zerebubuth

[bcamper]

Thanks for the cc

[nyurik]

This bug has hit us as well - without the 'Accept-Encoding': 'gzip' in the tilejson/lib/tilejson.js - TileJSON.prototype.get, our caching server (Varnish) automatically sends an uncompressed response PBF tile, even though the backend performs the nasty hack of always returning the compressed pbf even if accept-encoding was not set.

From the looks of it, tilejson lib could use request lib's ability to uncompress transparently, unless the tile uncompression happens in c++, in which case it should simply send the accept-encoding for all pbf tile requests.

[wilhelmberg]

@nyurik We are already testing tilejson with request https://github.com/mapbox/node-tilejson/tree/request

Is there a public endpoint for me to try with the latest Mapbox Studio dev build?
You could also mail it to me: my-first-name AT mapbox DOT com

[nyurik]

@BergWerkGIS i have created a test point at http://ns512621.ip-167-114-156.net/osm/pbfinfo.json -- it points to the port 4000, which is NOT behind varnish. If you remove the port, it will go via varnish, and will uncompress the data on the fly. Thanks!
P.S. I am testing with the github master version of the studio, because the 2.7 removes the host name.

[nyurik]

One more thing - I enabled it via Varnish on https://maps.wikimedia.org/osm/pbfinfo.json, but because Varnish cached the old data for today, it has to be cache busted with https://maps.wikimedia.org/osm/pbfinfo.json?a -- but query params are not supported by mapbox studio for some reason.

[wilhelmberg]

@nyurik Thanks, I'm able to replicate:
via 4000 ✅
via 80: ❌

vs2015 is the branch I'm currently working on for the next release.
https://github.com/mapbox/mapbox-studio/tree/vs2015
Would be great, if you could take a look and let me know, if you stumble upon any other bumps with that branch.

[nyurik]

@BergWerkGIS I pulled that version, but its not working -- image_reader: can't determine type from input data (which is the same error i got when mapnik gets uncompressed data i think).

You can use https://maps.wikimedia.org/osm/a/pbfinfo.json as more stable (behind varnish)
ATTENTION: The link would not work with any non wmf referrer - so copy paste it instead of clicking it

P.S. Feel free to ping me on IRC -- "yurik" (in many channels on freenode)

[wilhelmberg]

@nyurik sorry, for any misunderstanding.
I'm able to replicate the problem, but haven't solved it yet.
vs2015 is the branch I'm currently working on for the next release.
I just thought you might be interested in trying it, as it contains some other updates, like node-mapnik@3.4.3 and others.

[nyurik]

Gotcha, will try it, thanks! Are you on IRC?

[wilhelmberg]

Not really on IRC, mainly communicating via Github issues.

[nyurik]

ok, thx. I have updated this server to always use nonVarnished (hacked) method, and the official maps.wikimedia.org will always use varnish