WMS prototype working #572

mindless-bureaucrat · 2023-01-16T04:57:36Z

This PR adds WMS functionality to the base TilerFactory at /wms

Notes:

WMS clients send params irregularly as UPPER or lower case. I was unable to find an elegant way for FastAPI to convert all params to the same case, so to mitigate this the lower case (and other WMS param handling) is taken care of in the process_wms_params static method.
The wms route is split in two parts: GetCapabilities and Get Map
GetCapabilities will generate a dynamic XML based on the params provided. In addition, multiple COGs can be built in a single XML by using multiple layers params. This behavior deviates from the standard url param as the primary COG input in the route.
GetMap is mostly a copy/paste of the /tiles route, but uses src_dst.part with bbox rather than .tile.
WMS route is optional, similar to preview
WMS XML templates were added for 1.0.0, 1.1.1, and 1.3.0. . I'm not familiar enough with inner workings of the format, but I suspect someone smarter than I could likely jam it in to one file with some complicated Jinja logic.
Additional WMS specific Enums were added

mindless-bureaucrat · 2023-01-16T05:02:56Z

Snip from QGIS:

vincentsarago · 2023-01-16T10:18:26Z

Wooot thank for the PR @mindless-bureaucrat 🙏

src/titiler/core/titiler/core/factory.py

src/titiler/core/titiler/core/resources/enums.py

src/titiler/core/titiler/core/factory.py

vincentsarago · 2023-01-16T17:23:14Z

@mindless-bureaucrat I've got some change I'll push later today which will do:

refactor the process_wms_params function as a dependency
some cleanup and simplification

Things to discuss:

because the wms do not work exactly like all other endpoint it might make more sense to create a titiler-wms plugin
what happens when multiple layers are passed to the GetMap request

mindless-bureaucrat · 2023-01-16T17:33:30Z

because the wms do not work exactly like all other endpoint it might make more sense to create a titiler-wms plugin

I initially had WMS as an extension, however the behavior seems similar to the wmts endpoint (in that we generate an XML definition and the client then reads from there). I also found maintaining a separate plug-in with the recent changes from TiTiler 0.7 to be cumbersome, however I'll defer to you on this.

what happens when multiple layers are passed to the GetMap request

At this time GetMap will pull the first layer specified (in the string-split list). I know in real WMS there's a way to pull and overlay multiple layers in a single call, however I think that is beyond the scope of just exposing a simple WMS protocol with this addition

vincentsarago · 2023-01-16T22:03:56Z

src/titiler/core/titiler/core/factory.py

+            "image/jp2",
+            "image/tiff; application=geotiff",
+        ]
+        supported_version: List[str] = ["1.0.0", "1.1.0", "1.3.0"]


For now this is hardcoded, let's discuss later how users could change ☝️

vincentsarago · 2023-01-16T22:05:13Z

src/titiler/core/titiler/core/factory.py

+                },
+            },
+            openapi_extra={
+                "parameters": [


👇 with openapi_extra we add entries to the openapi.json schema (swagger docs) but this is just for display 😄 (the api will accept lower and upper case)

vincentsarago · 2023-01-16T22:05:55Z

src/titiler/core/titiler/core/factory.py


            GetCapability will generate a WMS XML definition.

            GetMap is mostly copied from titiler.core.factory.TilerFactory.part.part
            """
-            processed_wms_params = process_wms_params(_request)
-            print(processed_wms_params)
+            req = {k.lower(): v for k, v in request.query_params.items()}


I've removed the middleware function and put everything here directly

vincentsarago · 2023-01-16T22:06:55Z

src/titiler/core/titiler/core/factory.py

                with rasterio.Env(**env):
                    with self.reader(src_path, **reader_params) as src_dst:
                        image = src_dst.part(
-                            buffer=buffer,


I removed buffer for now but we could add it back

src/titiler/core/titiler/core/factory.py

vincentsarago · 2023-01-16T22:08:33Z

src/titiler/core/titiler/core/factory.py

-                if processed_wms_params.get('format'):
-                    imagetype = ImageType(MediaType(processed_wms_params['format'])._name_)
-                else:
-                    imagetype = ImageType.jpeg if image.mask.all() else ImageType.png


the auto mode is not really part of the specification so I removed it

vincentsarago · 2023-01-16T22:09:59Z

src/titiler/core/titiler/core/templates/wms_1.0.0.xml

+      <DCPType>
+        <HTTP>
+          <Get onlineResource="{{ request_url }}" />
+          <Post onlineResource="{{ request_url }}" />


we don't accept POST request so I guess we should remove this

vincentsarago · 2023-01-16T22:10:26Z

src/titiler/core/titiler/core/templates/wms_1.0.0.xml

+        <Name>{{layer}}</Name>
+        <Title>{{layer}}</Title>
+        <Abstract>{{layers_dict[layer]['abstract']}}</Abstract>
+        <SRS>{{" ".join(available_epsgs)}}</SRS>


do we have to repeat the list of SRS here?

Not sure if it's needed, but this is what I saw in all the auto-generated WMS XMLs from Geoserver and ESRI

vincentsarago · 2023-01-17T09:17:27Z

src/titiler/core/titiler/core/factory.py

+                # Build information for the whole service
+                minx, miny, maxx, maxy = zip(
+                    *[layers_dict[layer]["bounds_wgs84"] for layer in layers_dict]
+                )


do we need to change the order here?

vincentsarago · 2023-01-17T13:48:16Z

with the latest commit I moved the /wms endpoint in its own factory and added support for multiple layers

src/titiler/core/titiler/core/factory.py

vincentsarago · 2023-01-17T14:57:03Z

@mindless-bureaucrat the more I think about this, the more I feel uncomfortable adding a WMS endpoint in the tiler factories. Here is why:

I think in the future we will try to support more OGC specification but the WMS is quite old and not really REST friendly (at least less than the WMTS specification)
Because it uses Layers and not url= and allow more than one dataset, it will be kinda odd to have the wms endpoint with the regular endpoint

I'm not quite sure we want to make a titiler.wms module or write just an example. I need to think more about this.

mindless-bureaucrat · 2023-01-17T17:13:59Z

@mindless-bureaucrat the more I think about this, the more I feel uncomfortable adding a WMS endpoint in the tiler factories. Here is why:

I think in the future we will try to support more OGC specification but the WMS is quite old and not really REST friendly (at least less than the WMTS specification)

Because it uses Layers and not url= and allow more than one dataset, it will be kinda odd to have the wms endpoint with the regular endpoint

I'm not quite sure we want to make a titiler.wms module or write just an example. I need to think more about this.

100% agree that WMS has seen better days. Unfortunately I (and likely others) am working in an environment with legacy web clients. This plugin/extension allows us to support modern clients (tile/rest based) along with dinosaurs on the same data definitions without having to run Geoserver (or similar WMS server) in addition to our beautiful TiTiler and FastAPI endpoints :) .

Happy to have this functionality as a separate factory! Separate library would be second best. Docs-only... 😭.

vincentsarago · 2023-01-18T12:47:46Z

Happy to have this functionality as a separate factory!

👍 ok then

ToDo

double check that we are OGC compliant
add tests
add WMSFactory for titiler.mosaic 😄

vincentsarago · 2023-01-18T21:39:11Z

so in parallel I've started #575, if it goes well I'll add a wmsExtension to the package with most of the code we added here.

then we could use it as

from fastapi import FastAPI
from titiler.core.factory import TilerFactory
from titiler.extensions import wmsExtension

app = FastAPI()

tiler = TilerFactory(
    extensions=[
        wmsExtension()
    ]
)

app.include_router(tiler.router)

vincentsarago · 2023-01-20T12:11:05Z

I've moved all the WMS related code to an extension.

TODO:

add getTile tests
add Extension documentation

vincentsarago · 2023-01-23T20:35:19Z

@mindless-bureaucrat would you be able to review the code before we merge? 🙏

I'll take care of the documentation in another PR

src/titiler/extensions/titiler/extensions/wms.py

mindless-bureaucrat

You are a machine! Great looking tests and cleanup. Have a couple comments regarding using the mosaic method rather than .part, and a few WMS spec related things.

src/titiler/extensions/tests/test_wms.py

src/titiler/extensions/titiler/extensions/wms.py

mindless-bureaucrat · 2023-01-24T22:52:14Z

src/titiler/extensions/titiler/extensions/wms.py

+                                **dataset_params,
+                            )
+
+                image, assets_used = mosaic_reader(


In theory we should only be passing a single layer in the layers qs param, although I supposed this could work for a very simple mosaic method, absent MosaicJson. Is this the intended purpose rather than just getting the image out with .part() directly? How would this work with the MosaicTilerFactory if a user specified a custom backend?

so here we use simple mosaic, meaning you just need to pass multiple file name, there is no idea of mosaic json or mosaic backend.

How would this work with the MosaicTilerFactory if a user specified a custom backend?

It won't work for mosaic backend because they don't implement .part() method (I might add it in titiler-pgstac)

mindless-bureaucrat · 2023-01-24T22:54:07Z

src/titiler/extensions/tests/test_wms.py

+            72.22979795551834,
+            -52.301598718454485,
+            74.66298001264106,
+        ]


Should we implement the mosaic method in this PR as well, or wait until a future version? I have a working prototype in my local rats nest, happy to wait or push to a later date :)

because mosaic backend do not implement part() method, I'm not planing to add it

…into HEAD

vincentsarago · 2023-01-26T10:17:03Z

@mindless-bureaucrat I'm going to merge this PR, please if you have more comments feel free to raise them before we make the next release 🙏

WMS prototype working

0d9af0b