Raster Field in Features

[flippmoke]

I know that @pnorman @pramsey and @ShibaBandit all commented in #39 about the experimental state of rasters. Here is the best clarification I can offer:

A decoder using v2 can simply ignore the raster field, we realize that a more detail specification for rasters is needed. We currently aren't sure how we think rasters should best be put into this field and some research/experimenting is required. We did want to include the field however, as we will be working in the mapnik-vector-tile repository to expand the capabilities for rasters in vector tiles. Additionally, @springmeyer has told me that some groups have already started to experiment with this as a feature. The goal is that we will have this as an experimental field for version 2 and it will lay the groundwork for the community to work on how specifically how the raster data will be stored.

[springmeyer]

Given things are still experimental and not well decribed, I propose we keep rasters out of v2 and plan to give them attention for a future spec.

[flippmoke]

👍 removing from milestone, tagging for future releases.

[mojodna]

The raster support is the main thing I have an investment in, particularly making it clear to use while retaining the ability to use it in creative and unanticipated ways. I'll keep an eye for discussion once it's time.

[markeberhart]

With "video" map tiles on the horizon, I'd recommend a "raster" format that is more akin to small FLV or SWF-like files for enhanced support of time-based motion.

[JesseCrocker]

It would be nice to be able to include a raster as an attribute on a feature. I imagine using this to put icons used to render points features in tiles, rather than in a sprite sheet.

[e-n-f]

I want to think through some ideas here about what rasters are for and how they should be represented.

• @JesseCrocker points out the use case for an icon type of raster: associated with a point, and with a fixed size, rather than scaled as the tile is zoomed
• There is the common situation of image tiles. Usually these are the same size as the tile, but there is no reason they would have to cover the entire tile.
• Digital elevation models are like images, but the pixels are numbers, not colors
• Tile-count is something like a DEM, but the values are densities, not heights.
• Another variation on the tile-count theme tracks things that are even more abstract, like a 3-tuple of (x offset, y offset, count) to log the average vector of travel from a given point.

For both of the image cases, I think we want an attribute type that is a binary blob that can contain JPEG or PNG data, and which can be used as part of compound attributes.

The icon attribute could be a compound attribute on a Point feature, something like

"type": "Point", "properties": { "icon": { "x_px": 16, "y_px": 16, "png": BLOB(…) } }

The tiled bitmap could be an attribute on a Polygon feature, just

"type": "Polygon", "properties": { "png": BLOB(…) }

which also generalizes to quadrilateral but not necessarily rectangular texture maps.

The downside of this representation of tiled bitmaps is that it requires some magic knowledge in the encoder during tiling, to know that if the feature as specified in the original dataset has a bitmap attribute, that bitmap should be split up when the feature is clipped into multiple tiles. This might be an argument that there should be a special grids section of the Feature that is explicitly known to be clippable:

"type": "Polygon", "grids": { "png": BLOB(…) }, "properties": { … }

instead of allowing the bitmap to be one of the properties, which are normally not modified during tiling.

For the DEM and count cases, compound attributes already give a natural notation with no other extensions:

"type": "Polygon", "properties": { "dem": [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7 8 9 ] ] }

If it turns out that this compresses badly, we could consider a special attribute representation for multidimensional arrays of numbers that compresses better, which any other numeric-array attribute would also benefit from.

For the same clipping reason as for images above, we might want to consider putting DEM-type data in a grids section as well, so it can be clipped during tiling.

"type": "Polygon", "grids": { "dem": [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ] }, "properties": { … }

The other thing to think about during tiling is whether there should be downsampling on these images by zoom level. For icons there shouldn't be, but for bitmaps and DEMs there probably should be. It probably makes sense to downsample using the tile resolution as the unit within the bitmap as well: if a tile is 4096x4096 and a bitmap feature occupies half of that area and has a higher resolution than 2048x2048 after clipping, the clipped data should be downsampled to 2048x2048.

With PNGs and JPEGs there is probably the expectation of bilinear or bicubic interpolation during downsampling. With DEM data this would probably also be appropriate, but there may be other cases where the encoder doesn't know the ultimate interpretation of the data, and can reasonably do nearest-neighbor samples from it but not do any other interpolation. Maybe we could establish a convention that raw numbers in the 2-dimensional array can be interpolated, but if the elements are nested one level deeper or contain strings:

"type": "Polygon", "grids": { "dem": [ [ [1], [2], [3] ], [ [4], [5], [6] ], [ [7], [8], [9] ] ] }, "properties": { … }
"type": "Polygon", "grids": { "dem": [ [ "1", "2", "3" ], [ "4", "5", "6" ], [ "7", "8", "9" ] ] }, "properties": { … }

then the grid is opaque and can be sampled but not interpolated. Alternately it could be in another special section of the feature to clarify what operations are legitimate.

[mojodna]

@jenningsanderson and I were discussing (ab)using the raster for storage of additional PBF-encoded data that doesn't necessarily make sense in the existing fields, so I'm in favor of either generalizing it (perhaps with a content-type for consumers) or including an alternative field so that opaque, application-specific blobs can be included in tiles.

[e-n-f]

Although it's not part of the vector tile spec itself, this also raises the question of how to talk about blobs in the JSON textual representation of a feature. The main options I see are:

• Invent a JSON syntax for blobs (or use someone else's if there is a precedent)
• Treat them as strings, but strings of bytes instead of UTF-8 strings
• Treat them as strings that can only contain the characters U+0000 through U+00FF
• Treat them as arrays of numbers
• Use Node's <Buffer 88 13 70 17> form
• Use Node's Buffer.from([1, 2, 3]) form

Additional option suggested by @sgillies:

• Treat them as base64 strings