Rasterizing vector tiles on the server

[dwoznicki]

I have a slightly out-of-scope use case, but I was wondering if someone here might be able to point me in the right direction.

I have a tile server set up to serve vector tiles, and a client rasterizing those tiles in the browser using maplibre-gl-js (well, technically still on mapbox-gl-js, but not for long). If possible, I'd like to also be able to generate images from my vector tiles at request time on the server. This would allow me to serve both vector and raster tiles while maintaining a consistent styling.

It seems like one way I might be able to accomplish this is by executing the rasterization step from mapblibre-gl-js in NodeJS, but I'm not sure if that's actually feasible. Does anyone have any pointers/thoughts?

[HarelM]

Use tileserver-gl, it's destined to do just that.
@acalcutt is working on the migration to latest technologies in his branch.
Good luck! :-)

[dwoznicki]

Thanks for the quick reply! I was actually in the process of pivoting away from maptiler/tileserver-gl to go-spatial/tegola, but perhaps I'll give it another shot.

[acalcutt]

Like @HarelM mentioned I am working on an upgraded version of tileserver-gl here ( https://github.com/acalcutt/tileserver-gl ) , which uses maplibre-gl-native on the backend for the raster rendering and maplibre-gl-js and leaflet for the frontend. I can't quite say it is ready for the raster side of thing yet since there are still a few bugs with maplibre-gl-native I haven't been able to figure out

This is using the node package @acalcutt/maplibre-gl-native , which was built for several ubuntu/macos node versions ( https://github.com/acalcutt/maplibre-gl-native/releases/tag/node_v5.0.14 )

[dwoznicki]

@acalcutt Just curious, what's different between your tileserver-gl fork and the original? Also, it looks like you actually managed to build the node.js bindings in your maplibre-gl-native fork. Do you know if this is something that's possible in the current maplibre/maplibre-gl-native project? I've been trying to build the node + linux versions, but I haven't had much luck, and get the feeling that the READMEs are out of date.

[acalcutt]

The only difference in my maplibre-gl-native fork is I modified the node ci workflow to publish to github with node-pre-gyp-github and to do an npm publish. So that was basically some changes in package.json and the node-ci workflow. This was all based on conversation with @mnutt in maplibre/maplibre-native#217 and some of his suggestions and previous work.

In tileserver-gl I have made a bunch of changes. I have removed mapbox-gl-js and mapbox.js from the viewer pages in favor of maplibre-gl-js and the latest leaflet. after v4 I have converted the package to a esm module and replaced all the requires with imports and updated the syntax to make it work. The freshly built maplibre-gl-native allows it to be built on node versions later than node 10, so I have updated the docker build process linux and node version (it is now building with debian 11 and node 16). With the newer node I was also able to update all the dependencies to their latest version (though there's a few depreciated ones like request still in use). I've pulled in some maptiler pr's that seemed like they could be useful. I also created a workflow to automatically version and publish the docker/and node versions on push to main,

Pre v4, it was possible to swap the maplibre-gl-native with mapbox-gl-native and the one issue I had with the shields went away, so I think if i figure out the maplibre-gl-native issue, this would be in a pretty good state. After v4 it is no longer possible to switch back to mapbox-gl-native because mapbox doesn't have binaries for node 16 where I do for maplibre-gl-native

[acalcutt]

you can see an example of it running here https://tiles.wifidb.net/ . looks pretty much the same, but the style raster rendering is maplibre-gl-native and the viewers are maplibre-gl-js/leaflet

[nyurik]

Hi @dwoznicki , the server-side rasterization is a fairly well explored problem without a "perfect" solution :).

• (current) tileserver-gl already does that using various tricks, and is relatively successful. In my experience it keeps leaking memory and eventually crashes, so I run it using multiple docker self-restarting pods in kubernetes.
• (antiquated) Mapnik has traditionally been used by OSM servers for this, but it does not support any new features, so I would advise against it
• (future) mbtileserver-rs has recently joined MapLibre, and I hope we will integrate it with either maplibre-native or maplibre-rs in the future to support this usecase.

[acalcutt]

I've never experienced a tileserver-gl crash running it directly without docker/npm... but I also have a forced daily reboot to apply my updated mbtiles, so that could be why. I wonder if any of the updates in my branch will help with that...or at least where it isn't releasing memory.

[nyurik]

I observed it in production - we run server for months, and I saw a number of pod restarts because it was timing out requests

[dwoznicki]

@nyurik Thanks for the quick reply!

• We actually have a tileserver-gl-light instance running in production right now, and I was in the process of pivoting to go-spatial/tegola for unrelated reasons. Perhaps I'll take another look at tileserver-gl, though.
• What modern features does Mapnik not support?
• Do you know state is mablibre-rs in? It looks really promising, but as far as I can tell is still in proof of concept stage. If there's a chance that it'll be closer to production ready in the next several months, I might hold off for it.

[nyurik]

• Mapnik doesn't support the styling language maplibre uses, so you have to use the old XML-based or CSS-like old styling syntax.
• maplibre-rs won't be ready in the next few months for sure, but I think a better interim solution would be to create rust bindings for maplibre-native, and use that for rasterisation from maplibre/mbtileserver.