Add PMTiles plugin with MVT support#1469
Add PMTiles plugin with MVT support#1469AlaricBaraou wants to merge 1 commit intoNASA-AMMOS:masterfrom
Conversation
gkjohnson
left a comment
gkjohnson
left a comment
There was a problem hiding this comment.
This looks really great. It's feeling like a lot of this code could be shared with the GeoJSONImageSource for a general "vector overlay" system that can allow for drawing these kinds of shapes. We can look into that refactor after.
One architectural difference I would recommend is sticking more closely with the GeoJSONImageSource approach (using RegionImageSource) rather than XYZImageSource (which uses TiledImageSource). The difference in the way that these are being used is that the XYZ image plugin will draw / load images for every individual tile and then these tiles will be "composed" into a large texture covering the necessary region for overlays. This means we have texture memory for the original tiles and for the composed ones.
What GeoJSONImageSource is doing is drawing directly to those final region textures which allows for quick updates and redraws for styling changes. Explaining the architecture and how to implement this isn't so straight forward so maybe I can help with this if needed one things are closer to being merged.
|
|
||
| import { PMTiles } from 'pmtiles'; | ||
|
|
||
| export class PMTilesLoaderBase { |
There was a problem hiding this comment.
I think we can move anything from these "LoaderBase" classes to "ImageSource". They should only really be used in that one location and they're pretty small.
| // Generate a virtual URL for a tile (used by tiling scheme) | ||
| getUrl( z, x, y ) { | ||
|
|
||
| return `pmtiles://${z}/${x}/${y}`; | ||
|
|
||
| } | ||
|
|
||
| // Parse tile coordinates from a virtual URL (pmtiles://z/x/y) | ||
| static parseUrl( url ) { | ||
|
|
||
| const i2 = url.lastIndexOf( '/' ); | ||
| const i1 = url.lastIndexOf( '/', i2 - 1 ); | ||
| const i0 = url.lastIndexOf( '/', i1 - 1 ); | ||
|
|
||
| return { | ||
| z: parseInt( url.slice( i0 + 1, i1 ) ), | ||
| x: parseInt( url.slice( i1 + 1, i2 ) ), | ||
| y: parseInt( url.slice( i2 + 1 ) ), | ||
| }; | ||
|
|
||
| } |
There was a problem hiding this comment.
This logic I think should be in the plugin since it's specific to how the plugin is interpreting and handling tile urls, which the loaders and image sources shouldn't have to know about (they're tiles agnostic).
| ctx.beginPath(); | ||
|
|
||
| for ( const ring of geometry ) { | ||
|
|
||
| for ( let k = 0; k < ring.length; k ++ ) { | ||
|
|
||
| const p = ring[ k ]; | ||
| if ( k === 0 ) ctx.moveTo( p.x * scale, p.y * scale ); | ||
| else ctx.lineTo( p.x * scale, p.y * scale ); | ||
|
|
||
| } | ||
|
|
||
| ctx.closePath(); | ||
|
|
||
| } | ||
|
|
||
| ctx.fill(); |
There was a problem hiding this comment.
In order to support holes in the polygons you need to specify the "evenodd" rule - otherwise I think you'll just get solid shapes. See the geojson implementation.
An aside but I'm wondering if we can reuse this in the GeoJSON overlay since these shapes are all about the same 🤔
|
|
||
| _renderPoints( ctx, geometry, layerName, scale ) { | ||
|
|
||
| const isLabelLayer = ( layerName === 'place_label' ); |
There was a problem hiding this comment.
What is place_label here? And why is it preventing drawing points specifically?
| const x = p.x * scale; | ||
| const y = p.y * scale; | ||
|
|
||
| if ( ! isLabelLayer ) { |
There was a problem hiding this comment.
If this is blocking all point drawing then I think we can remove the label layer check from this class and just skip calling the "_renderPoints" function from the call site.
| const radius = ( layerName === 'poi' ) ? 3 : 2; | ||
|
|
||
| ctx.beginPath(); | ||
| ctx.moveTo( x + radius, y ); | ||
| ctx.arc( x, y, radius, 0, Math.PI * 2 ); | ||
| ctx.fill(); |
There was a problem hiding this comment.
Since the pixel ratios are not consistent over the surface of the globe just drawing a circle to the canvas will result in something that looks like an ellipsoid rather than a circle on the surface. The geojson layer scales the points by the aspect ratio at that point derived from the derivative of the lat / lon the given point.
You can see from the comments there, though, that it could still be improved a bit. So we can skip this for now but FYI.
See #1325 for an illustration of the issue and fix.
|
|
||
| const canvas = this._createCanvas( this.tileDimension, this.tileDimension ); | ||
| const ctx = canvas.getContext( '2d' ); | ||
| const scale = this.tileDimension / MVT_EXTENT; |
There was a problem hiding this comment.
Can we add some comments for some of these things - what is MVT_EXTENT? Why is it hardcoded to a 4k size? I'm wondering how much of the vector tiles-specific stuff we can move to the plugin and make this a more reusable class for use with GeoJSON plugin, as well
| constructor( options = {} ) { | ||
|
|
||
| this.filter = options.filter || ( () => true ); | ||
| this._layerOrder = options.layerOrder || DEFAULT_LAYER_ORDER; | ||
| this._styles = {}; | ||
|
|
||
| const colorsToSet = Object.assign( {}, LAYER_COLORS, options.styles || {} ); | ||
| for ( const key in colorsToSet ) { | ||
|
|
||
| _color.set( colorsToSet[ key ] ); | ||
| this._styles[ key ] = { | ||
| hex: _color.getHex(), | ||
| css: _color.getStyle() | ||
| }; | ||
|
|
||
| } | ||
|
|
||
| } |
There was a problem hiding this comment.
Can you give me a quick overview of how "features" and "styles" are handled in MVT and PMTiles? Does every shape etc only have a single type or class? Or is it possible for there to be multiple attributes associated with a single shape? As in "building", "city" and "building", "rural".
| // Intercept pmtiles:// URLs and fetch from the PMTiles archive | ||
| fetchData( url, options ) { | ||
|
|
||
| if ( url.startsWith( 'pmtiles://' ) ) { | ||
|
|
||
| const { z, x, y } = PMTilesLoaderBase.parseUrl( url ); | ||
|
|
||
| return this.imageSource.pmtilesLoader.getTile( z, x, y, options?.signal ) | ||
| .then( buffer => buffer || new ArrayBuffer( 0 ) ); | ||
|
|
||
| } | ||
|
|
||
| return null; | ||
|
|
||
| } |
There was a problem hiding this comment.
We should just be able to implement "fetchItem" here, as we do for "GeoJSONImageSource", which will get called with x, y, z, level - then we shouldn't have to deal with custom urls or anything.
|
|
||
| layersFolder.add( state.layers, key ) | ||
| .name( key.charAt( 0 ).toUpperCase() + key.slice( 1 ) ) | ||
| .onChange( createTiles ); |
There was a problem hiding this comment.
The GeoJSONImageSource includes a "redraw" function to rerender every texture in place in case something changes, allowing for styling updates, animations, etc. I'm thinking we should allow the same thing here rather than rebuilding the full set of tiles.
2 participants
Minimal PR extracted from this draft #990
Adds support for rendering PMTiles archives on the globe using HTTP range requests.
New files
Core:
MVTLoaderBase.js- Parses MVT formatPMTilesLoaderBase.js- PMTiles archive accessThree.js:
PMTilesPlugin.js- Main pluginPMTilesImageSource.js/MVTImageSource.js- Image sourcesVectorTileCanvasRenderer.js/VectorTileStyler.js- Canvas renderinglayerColors.js- Default layer stylesDemo:
pmtiles.html/pmtiles.js- Globe example with layer controlsDependencies
Adds optional peer dependencies:
pmtiles- Read tiles from PMTiles archives via range requests@mapbox/vector-tile- Parse MVT (Mapbox Vector Tile) formatpbf- Protocol buffer decoding (required by @mapbox/vector-tile)