Add support for custom tile matrix sets

[lindakarlovska]

Is your feature request related to a problem? Please describe.
I’m working with vector tiles generated from cadastral data in a custom tile matrix set, specifically based on EPSG:5514 (S-JTSK). The definition I’m using is published here:
🔗 OpenGeoLabs tile matrix sets (in Czech) — it’s the second TMS listed.

Tiles were created using GDAL (which supports custom tiling schemes) and stored in the standard {z}/{x}/{y} structure, served by a basic HTTP server.

I tested pygeoapi’s mvt-proxy plugin using the default Web Mercator TMS, and it works great for my case. However, I would like to serve tiles using my specific S-JTSK TMS.

Describe the solution you'd like
Introduce a new top-level tileMatrixSets: section in the servers part of the YAML configuration file. This would allow users to define one or more custom TMS definitions directly in their config at the beginning and use these TMS in following resources section.

Structure would follow the same format as the current hardcoded tile matrix sets in pygeoapi/models/provider/base.py.

tileMatrixSets:
  SJTSKQuad:
    tileMatrixSet: SJTSKQuad
    tileMatrixSetURI: https://github.com/OpenGeoLabs/tiles-cz/blob/main/03-tile-matrix-set.md
    crs: http://www.opengis.net/def/crs/EPSG/0/5514
    title: S-JTSK Quad (Czech cadastral)
    orderedAxes: [X, Y]
    tileMatrices:
      - id: 0
        scaleDenominator: 7315200
        cellSize: 2048.256
        cornerOfOrigin: topLeft
        pointOfOrigin: [-925000, -920000]
        matrixWidth: 1
        matrixHeight: 1
        tileWidth: 256
        tileHeight: 256
      - id: 1
        scaleDenominator: 3657600
        cellSize: 1024.128
        cornerOfOrigin: topLeft
        pointOfOrigin: [-925000, -920000]
        matrixWidth: 2
        matrixHeight: 2
        tileWidth: 256
        tileHeight: 256
      # ...
      - id: 19
        scaleDenominator: 13.95
        cellSize: 0.00390673
        cornerOfOrigin: topLeft
        pointOfOrigin: [-925000, -920000]
        matrixWidth: 524288
        matrixHeight: 262144
        tileWidth: 256
        tileHeight: 256

Then, in the provider configuration:

providers:
  - type: tile
    name: MVT-proxy
    data: http://localhost:8000/vector-tiles/parcels/{z}/{x}/{y}.pbf
    options:
      zoom:
        min: 10
        max: 14
      schemes:
        - SJTSKQuad
    format:
      name: pbf
      mimetype: application/vnd.mapbox-vector-tile

Additional context
Where does it make sense to implement it? Currently, custom TMS support seems most relevant for the mvt-proxy plugin (my focus), but may also make sense for WMTS-related use. All tile-related plugins are summed up in the following table:

📦 TMS Usage Across Plugins

Plugin	CRS Assumptions	Custom TMS Support?	Notes
mvt-postgres	WebMercator and WorldCRS84	❓ Unsure	Not sure if there is any option to use own tiling schemes, probably not?
mvt-elastic	WebMercator only	❌ Not needed	CRS is fixed, no custom projection support
mvt-tippecanoe	WebMercator only	❌ Not needed	Tippecanoe only supports Web Mercator output
mvt-proxy	Flexible (depends on tile source)	✅ Makes sense	Custom CRS and TMS make definitely sense here
wmts-facade	Potentially multiple	✅ Worth considering	Would depend on backend support and structure

I’ve already discussed this idea briefly on the pygeoapi mailing list, but I’d like to follow up here as well to get broader feedback.

I’d really appreciate any input — whether this idea makes sense, potential challenges to be aware of, or hints on how best to approach it within the existing architecture. I’d be happy to start working on an implementation soon if this direction seems reasonable.

[tomkralidis]

Hi @lindakarlovska thanks for the info. IMHO this could realized by:

1. externalizing tile matrix sets from code to configuration (i.e. moving WORLDCRS84QUAD as JSON file intopygeoapi/resources/tilematrixsets/worldcrs84quad.json. Default pygeoapi tile matrix definitions would draw from this file
2. providing an additional tilematrixset in provider configuration to point to a custom definition, i.e.:

providers:
    - type: tile
    - tilematrixset: /path/to/custom.json. # overrides default

Would this work for you @lindakarlovska ?

[lindakarlovska]

Hi @lindakarlovska thanks for the info. IMHO this could realized by:

1. externalizing tile matrix sets from code to configuration (i.e. moving WORLDCRS84QUAD as JSON file into`pygeoapi/resources/tilematrixsets/worldcrs84quad.json`.  Default pygeoapi tile matrix definitions would draw from this file

Great! That's actually what I suggested in the first email.

2. providing an additional `tilematrixset` in provider configuration to point to a custom definition, i.e.:

providers:
- type: tile
- tilematrixset: /path/to/custom.json. # overrides default

Currently, there seems to be only the schemes tag. @tomkralidis Do you have in mind creating the new tag called tilematrixset or using the current one? If I take the example from ogcapi-tiles.rst, it would be

providers:
  - type: tile
    name: MVT-proxy
    data: http://localhost:8000/vector-tiles/parcels/{z}/{x}/{y}.pbf
    options:
      zoom:
        min: 10
        max: 14
      schemes:
        - /path/to/SJTSKQuad.json
    format:
      name: pbf
      mimetype: application/vnd.mapbox-vector-tile

Would this work for you @lindakarlovska ?

[doublebyte1]

Hi @lindakarlovska, thank you for your comment.

I think there are two separate issues here:

• The ability to publish tiles using a particular TMS: this is depending on the provider, as you identified in your table.
• pygeoapi as a register for TMS definitions.

According to Requirement 17 of the OGC Two Dimensional Tile Matrix Set and Tile Set Metadata Standard :

"A tiled resource or dataset SHALL declare support for a tile matrix set by one of the following two methods: a link to a tile matrix set definition (e.g. one of the TileMatrixSet definitions from Annex D or Annex E) as one of the links in the links list, or by explicitly including a TileMatrixSet definition (as an object in the tileMatrixSet property)."

You can link to any TMS definition you like and there is absolutely no obligation for that definition to be hosted in the same server. In this example, instead of linking to the pygeoapi hosted definition of the TMS:

    "links":[
        {
            "href":"https://demo.pygeoapi.io/master/TileMatrixSets/WebMercatorQuad",
            "rel":"http://www.opengis.net/def/rel/ogc/1.0/tiling-scheme",
            "type_":"application/json",
            "title":"WebMercatorQuad tile matrix set definition"
        },

we could link to the version that is on the OGC definition server:

    "links":[
        {
            "href":"https://raw.githubusercontent.com/opengeospatial/2D-Tile-Matrix-Set/master/registry/json/WebMercatorQuad.json",
            "rel":"http://www.opengis.net/def/rel/ogc/1.0/tiling-scheme",
            "type_":"application/json",
            "title":"WebMercatorQuad tile matrix set definition"
        },

The second topic is whether pygeoapi should act as a register for TMS definitions and I am not convinced of that. In fact, I would be more in favour to point to an authorative source like the OGC rainbow.

[lindakarlovska]

https://demo.pygeoapi.io/master/TileMatrixSets/WebMercatorQuad"

application/json"

Hi @lindakarlovska, thank you for your comment.

I think there are two separate issues here:

* The ability to publish tiles using a particular TMS: this is depending on the provider, as you identified in your table.

* pygeoapi as a register for TMS definitions.

According to Requirement 17 of the OGC Two Dimensional Tile Matrix Set and Tile Set Metadata Standard :

"A tiled resource or dataset SHALL declare support for a tile matrix set by one of the following two methods: a link to a tile matrix set definition (e.g. one of the TileMatrixSet definitions from Annex D or Annex E) as one of the links in the links list, or by explicitly including a TileMatrixSet definition (as an object in the tileMatrixSet property)."

You can link to any TMS definition you like and there is absolutely no obligation for that definition to be hosted in the same server. In this example, instead of linking to the pygeoapi hosted definition of the TMS:

    "links":[
        {
            "href":"https://demo.pygeoapi.io/master/TileMatrixSets/WebMercatorQuad",
            "rel":"http://www.opengis.net/def/rel/ogc/1.0/tiling-scheme",
            "type_":"application/json",
            "title":"WebMercatorQuad tile matrix set definition"
        },

we could link to the version that is on the OGC definition server:

    "links":[
        {
            "href":"https://raw.githubusercontent.com/opengeospatial/2D-Tile-Matrix-Set/master/registry/json/WebMercatorQuad.json",
            "rel":"http://www.opengis.net/def/rel/ogc/1.0/tiling-scheme",
            "type_":"application/json",
            "title":"WebMercatorQuad tile matrix set definition"
        },

The second topic is whether pygeoapi should act as a register for TMS definitions and I am not convinced of that. In fact, I would be more in favour to point to an authorative source like the OGC rainbow.

Thanks for your comment, @doublebyte. I understand and agree that we shouldn’t register tile matrix definitions directly in pygeoapi – especially not in a hardcoded way as currently done in pygeoapi/models/provider/base.py.

If I understand your suggestion correctly, you propose that default pygeoapi tile matrix definitions should be drawn directly from the definitions hosted at https://github.com/opengeospatial/2D-Tile-Matrix-Set/tree/master/registry/json
That would mean not storing any definitions within pygeoapi itself. Am I getting this right?

If so, I’d be happy to implement this – though I’d consider it more of a refactoring or improvement of the current approach, as it wouldn’t yet address the handling of custom tile matrix sets.

[lindakarlovska]

Hi @lindakarlovska, thank you for your comment.

I think there are two separate issues here:

* The ability to publish tiles using a particular TMS: this is depending on the provider, as you identified in your table.

* pygeoapi as a register for TMS definitions.

According to Requirement 17 of the OGC Two Dimensional Tile Matrix Set and Tile Set Metadata Standard :

"A tiled resource or dataset SHALL declare support for a tile matrix set by one of the following two methods: a link to a tile matrix set definition (e.g. one of the TileMatrixSet definitions from Annex D or Annex E) as one of the links in the links list, or by explicitly including a TileMatrixSet definition (as an object in the tileMatrixSet property)."

You can link to any TMS definition you like and there is absolutely no obligation for that definition to be hosted in the same server. In this example, instead of linking to the pygeoapi hosted definition of the TMS:

    "links":[
        {
            "href":"https://demo.pygeoapi.io/master/TileMatrixSets/WebMercatorQuad",
            "rel":"http://www.opengis.net/def/rel/ogc/1.0/tiling-scheme",
            "type_":"application/json",
            "title":"WebMercatorQuad tile matrix set definition"
        },

we could link to the version that is on the OGC definition server:

    "links":[
        {
            "href":"https://raw.githubusercontent.com/opengeospatial/2D-Tile-Matrix-Set/master/registry/json/WebMercatorQuad.json",
            "rel":"http://www.opengis.net/def/rel/ogc/1.0/tiling-scheme",
            "type_":"application/json",
            "title":"WebMercatorQuad tile matrix set definition"
        },

The second topic is whether pygeoapi should act as a register for TMS definitions and I am not convinced of that. In fact, I would be more in favour to point to an authorative source like the OGC rainbow.

Btw, when comparing the WebMercatorQuad definition from the OGC registry and the one currently hardcoded in pygeoapi/models/provider/base.py, I noticed quite a few differences – not just in numerical precision, but also in tag naming conventions and different values for orderedAxes. E.g, our:

    WEBMERCATORQUAD = TileMatrixSetEnumType(
        tileMatrixSet="WebMercatorQuad",
        tileMatrixSetURI="http://www.opengis.net/def/tilematrixset/OGC/1.0/WebMercatorQuad",  # noqa
        crs="http://www.opengis.net/def/crs/EPSG/0/3857",
        title="Google Maps Compatible for the World",
        orderedAxes=["E", "N"],
        wellKnownScaleSet="http://www.opengis.net/def/wkss/OGC/1.0/GoogleMapsCompatible",
        tileMatrices=[
      {
         "id": "0",
         "scaleDenominator": 559082264.0287177562714,
         "cellSize": 156543.033928040968,
         "cornerOfOrigin": "topLeft",
         "pointOfOrigin": [-20037508.3427892439067, 20037508.3427892439067],
         "matrixWidth": 1,
         "matrixHeight": 1,
         "tileWidth": 256,
         "tileHeight": 256
      }, ....

vs registry:

{
   "id": "WebMercatorQuad",
   "title": "Google Maps Compatible for the World",
   "uri": "http://www.opengis.net/def/tilematrixset/OGC/1.0/WebMercatorQuad",
   "crs": "http://www.opengis.net/def/crs/EPSG/0/3857",
   "orderedAxes": ["X", "Y"],
   "wellKnownScaleSet": "http://www.opengis.net/def/wkss/OGC/1.0/GoogleMapsCompatible",
   "tileMatrices":
   [
      {
        "id": "0",
        "scaleDenominator": 559082264.028717,
        "cellSize": 156543.033928041,
        "pointOfOrigin": [-20037508.3427892,20037508.3427892],
        "tileWidth": 256,
        "tileHeight": 256,
        "matrixWidth": 1,
        "matrixHeight": 1
      }, ...

So to switch to external definitions properly, we would need to fully align with the OGC structure, including tag names and formats.
Did I understand your suggestion correctly? This the right path forward?

[doublebyte1]

Hi @lindakarlovska, thanks again for bringing this up.

The OGC definition server is an authoritative source, so I think that should take precedence. If we want to host our own definitions, we should make sure to align them with the definition server, which also means keeping our internal models up to date.

At the time that these models were implemented in pygeoapi, OGC was not hosting json definitions of the CRS, so there was really no option but to host them. This has changed, so I think it is a good moment to have that discussion now :-)

In any case, this is a separate issue and I think your particular use case could be solved by exposing the links from TileSet metadata as provider options on a collection level. Then you could link to any TMS definition, which could be for instance hosted in a GitHub repo.

[tomkralidis]

Currently, there seems to be only the schemes tag. @tomkralidis Do you have in mind creating the new tag called tilematrixset or using the current one? If I take the example from ogcapi-tiles.rst, it would be

providers:
  - type: tile
    name: MVT-proxy
    data: http://localhost:8000/vector-tiles/parcels/{z}/{x}/{y}.pbf
    options:
      zoom:
        min: 10
        max: 14
      schemes:
        - /path/to/SJTSKQuad.json
    format:
      name: pbf
      mimetype: application/vnd.mapbox-vector-tile

We could use schemes array, where the notation would be:

• WebMercatorQuad: "keyword" string whose definition resides on OGC Rainbow
• /path/to/my-def.json: file on disk for local definitions

Given the non-trivial architecture change, I've started working on this in a branch and should have a PR ready in 2-3 days, for review/comment.

[lindakarlovska]

Currently, there seems to be only the schemes tag. @tomkralidis Do you have in mind creating the new tag called tilematrixset or using the current one? If I take the example from ogcapi-tiles.rst, it would be

providers:
  - type: tile
    name: MVT-proxy
    data: http://localhost:8000/vector-tiles/parcels/{z}/{x}/{y}.pbf
    options:
      zoom:
        min: 10
        max: 14
      schemes:
        - /path/to/SJTSKQuad.json
    format:
      name: pbf
      mimetype: application/vnd.mapbox-vector-tile

We could use schemes array, where the notation would be:

* `WebMercatorQuad`: "keyword" string whose definition resides on OGC Rainbow

* `/path/to/my-def.json`: file on disk for local definitions

Given the non-trivial architecture change, I've started working on this in a branch and should have a PR ready in 2-3 days, for review/comment.

Thanks @tomkralidis 🙂 I’ve started on this as well — here’s the PR for externalizing tile matrix sets: #2077.
Once I started looking into the OGC registry definitions, I began preparing a branch that updates all TMS definitions via GitHub Actions: lindakarlovska@f2728ea.
It’s set to run once a month. The result will basically be a local copy of the OGC registry (https://github.com/opengeospatial/2D-Tile-Matrix-Set/tree/master/registry/json) — I think it’s important to store the definitions locally rather than fetch them on the fly, so things will be faster.
The branch still needs more testing, but happy to share it here if it’s conceptually going in the right direction.

[doublebyte1]

Hi @lindakarlovska ! Thanks for this PR 👍🏽

I still think there is a value in retrieving TMS definitions from a remote repository (rainbow or anywhere else). For instance a user that publishes a custom TMS could just use that, without having to copy anything to the pygeoapi tilematrixsets folder and remembering to keep it up to date.

What do you of shipping the two TMS definitions in your PR in the default pygeoapi installation and leaving up to the user, the option to add more, or to pull the definition from a remote url?

On the other hand, I think it could make sense to add your GitHub action to the pygeoapi demo instance :-)

[tomkralidis]

I think we should avoid any machinery to manage definitions locally beyond what we already have. A user deploying pygeoapi can download the definitions locally and configure them with volume mappings (if using Docker/containers, for example) into the forthcoming pygeoapi resources directory. Having said this, echo @doublebyte1, this approach, then, would be a great example once the functionality is implemented (still reviewing your PR @lindakarlovska).

[lindakarlovska]

Hi @lindakarlovska ! Thanks for this PR 👍🏽

I still think there is a value in retrieving TMS definitions from a remote repository (rainbow or anywhere else). For instance a user that publishes a custom TMS could just use that, without having to copy anything to the pygeoapi tilematrixsets folder and remembering to keep it up to date.

What do you of shipping the two TMS definitions in your PR in the default pygeoapi installation and leaving up to the user, the option to add more, or to pull the definition from a remote url?

On the other hand, I think it could make sense to add your GitHub action to the pygeoapi demo instance :-)

Ok @doublebyte1, I am gonna test it more and once have it I can push it there.

[lindakarlovska]

I think we should avoid any machinery to manage definitions locally beyond what we already have. A user deploying pygeoapi can download the definitions locally and configure them with volume mappings (if using Docker/containers, for example) into the forthcoming pygeoapi resources directory. Having said this, echo @doublebyte1, this approach, then, would be a great example once the functionality is implemented (still reviewing your PR @lindakarlovska).

I understand, my thought was that the OGC TMS registry will probably continue to grow, and maybe users would appreciate these definitions automatically available and updated in the default pygeoapi installation - so GitHub action may be just an idea for the future. Including the default two TMSs in the installation and letting users add their own definitions to resources/tilematrixsets as needed is completely sufficient now.