Skip to content

Add Figure.tilemap to plot XYZ tile maps #2394

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 24 commits into from
Mar 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
adbd9ab
Add Figure.tilemap to plot XYZ tile maps
weiji14 Mar 4, 2023
1d24291
Add rioxarray to CI build matrix and include it as optional dependency
weiji14 Mar 4, 2023
fa8641b
Fix typo on rioxarray URL
weiji14 Mar 4, 2023
c51d9c6
Fix import-outside-toplevel for load_tile_map function
weiji14 Mar 4, 2023
6398829
Merge branch 'main' into tilemap
weiji14 Mar 4, 2023
56f5df8
Fix doctest for load_tile_map to include spatial_ref
weiji14 Mar 4, 2023
ff2fba0
Speed up test_tilemap by using zoom level 0
weiji14 Mar 4, 2023
c315609
Remove aliases img_out, cmap, img_in, bit_color, interpolation, colty…
weiji14 Mar 6, 2023
fd93c67
Merge branch 'main' into tilemap
weiji14 Mar 11, 2023
a1c29e4
Switch from conda install to mamba install
weiji14 Mar 11, 2023
2b20535
Add trailing comma to lists in pyproject.toml
weiji14 Mar 20, 2023
7efdbfd
Reword description of fig.tilemap
weiji14 Mar 20, 2023
d503268
Move load_tile_map call out of GMTTempFile context
weiji14 Mar 20, 2023
0a03a47
Merge branch 'main' into tilemap
weiji14 Mar 20, 2023
fde94ad
Reproject and plot map in lonlat coordinates by default
weiji14 Mar 20, 2023
9158c09
Switch from ModuleNotFoundError to ImportError
weiji14 Mar 20, 2023
17d0734
Ensure that plot is clipped to bounding box region when no_clip is False
weiji14 Mar 20, 2023
de74e3d
Web Mercator to Spherical Mercator and remove verbose statement
weiji14 Mar 23, 2023
125574e
Refactor no_clip if-condition to be a little more clear
weiji14 Mar 23, 2023
4d98102
Merge branch 'main' into tilemap
weiji14 Mar 23, 2023
10908d5
Merge branch 'main' into tilemap
yvonnefroehlich Mar 25, 2023
972c2c4
Edit docstring to say method instead of function
weiji14 Mar 27, 2023
e22d012
Merge branch 'main' into tilemap
weiji14 Mar 27, 2023
c3fca14
Use rio.set_crs instead of rio.write_crs
weiji14 Mar 27, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci_docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ jobs:
- name: Install dependencies
run: |
mamba install gmt=6.4.0 numpy pandas xarray netCDF4 packaging \
build ipython make myst-parser contextily geopandas \
build ipython make myst-parser contextily geopandas rioxarray \
sphinx sphinx-copybutton sphinx-design sphinx-gallery sphinx_rtd_theme

# Show installed pkg information for postmortem diagnostic
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci_tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ jobs:
optional-packages: ''
- python-version: '3.11'
numpy-version: '1.24'
optional-packages: 'contextily geopandas ipython'
optional-packages: 'contextily geopandas ipython rioxarray'
timeout-minutes: 30
defaults:
run:
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/ci_tests_dev.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -101,8 +101,9 @@ jobs:
geopandas ghostscript libnetcdf hdf5 zlib curl pcre make
pip install --pre --prefer-binary \
numpy pandas xarray netCDF4 packaging \
build contextily dvc ipython 'pytest>=6.0' pytest-cov \
pytest-doctestplus pytest-mpl sphinx-gallery
build contextily dvc ipython rioxarray \
'pytest>=6.0' pytest-cov pytest-doctestplus pytest-mpl \
sphinx-gallery

# Pull baseline image data from dvc remote (DAGsHub)
- name: Pull baseline image data from dvc remote
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci_tests_legacy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ jobs:
run: |
mamba install gmt=${{ matrix.gmt_version }} numpy \
pandas xarray netCDF4 packaging \
contextily geopandas ipython \
contextily geopandas ipython rioxarray \
build dvc make 'pytest>=6.0' \
pytest-cov pytest-doctestplus pytest-mpl sphinx-gallery

Expand Down
1 change: 1 addition & 0 deletions ci/requirements/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ dependencies:
# Optional dependencies
- contextily
- geopandas
- rioxarray
# Development dependencies (general)
- build
- ipython
Expand Down
1 change: 1 addition & 0 deletions doc/api/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ Plotting raster data
Figure.grdimage
Figure.grdview
Figure.image
Figure.tilemap

Configuring layout
~~~~~~~~~~~~~~~~~~
Expand Down
1 change: 1 addition & 0 deletions doc/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@
"python": ("https://docs.python.org/3/", None),
"pandas": ("https://pandas.pydata.org/pandas-docs/stable/", None),
"rasterio": ("https://rasterio.readthedocs.io/en/stable/", None),
"rioxarray": ("https://corteva.github.io/rioxarray/stable/", None),
"xarray": ("https://docs.xarray.dev/en/stable/", None),
"xyzservices": ("https://xyzservices.readthedocs.io/en/stable", None),
}
Expand Down
1 change: 1 addition & 0 deletions doc/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,7 @@ The following are optional dependencies:
* `IPython <https://ipython.org>`__: For embedding the figures in Jupyter notebooks (recommended).
* `Contextily <https://contextily.readthedocs.io>`__: For retrieving tile maps from the internet.
* `GeoPandas <https://geopandas.org>`__: For using and plotting GeoDataFrame objects.
* `RioXarray <https://corteva.github.io/rioxarray>`__: For saving multi-band rasters to GeoTIFFs.

Installing GMT and other dependencies
-------------------------------------
Expand Down
1 change: 1 addition & 0 deletions environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ dependencies:
- contextily
- geopandas
- ipython
- rioxarray
# Development dependencies (general)
- build
- dvc
Expand Down
10 changes: 5 additions & 5 deletions pygmt/datasets/tile_map.py
Original file line number Diff line number Diff line change
Expand Up @@ -103,16 +103,16 @@ def load_tile_map(region, zoom="auto", source=None, lonlat=True, wait=0, max_ret
Frozen({'band': 3, 'y': 256, 'x': 512})
>>> raster.coords
Coordinates:
* band (band) uint8 0 1 2
* y (y) float64 -7.081e-10 -7.858e+04 ... -1.996e+07 -2.004e+07
* x (x) float64 -2.004e+07 -1.996e+07 ... 1.996e+07 2.004e+07
* band (band) uint8 0 1 2
* y (y) float64 -7.081e-10 -7.858e+04 ... -1.996e+07 ...
* x (x) float64 -2.004e+07 -1.996e+07 ... 1.996e+07 2.004e+07
"""
# pylint: disable=too-many-locals
if contextily is None:
raise ImportError(
"Package `contextily` is required to be installed to use this function. "
"Please use `pip install contextily` or "
"`conda install -c conda-forge contextily` "
"`mamba install -c conda-forge contextily` "
"to install the package."
)

Expand Down Expand Up @@ -147,6 +147,6 @@ def load_tile_map(region, zoom="auto", source=None, lonlat=True, wait=0, max_ret

# If rioxarray is installed, set the coordinate reference system
if hasattr(dataarray, "rio"):
dataarray = dataarray.rio.write_crs(input_crs="EPSG:3857")
dataarray = dataarray.rio.set_crs(input_crs="EPSG:3857")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed from using .rio.write_crs to .rio.set_crs to prevent the CF coordinate spatial_ref from being written to the output xarray.DataArray. This fixes the int32 on Windows bu int64 on Linux/macOS issue mentioned in #2394 (comment).

If this looks ok (and the tests pass), I'll proceed to merge in this PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.


return dataarray
1 change: 1 addition & 0 deletions pygmt/figure.py
Original file line number Diff line number Diff line change
Expand Up @@ -523,6 +523,7 @@ def _repr_html_(self):
subplot,
ternary,
text,
tilemap,
timestamp,
velo,
wiggle,
Expand Down
1 change: 1 addition & 0 deletions pygmt/src/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@
from pygmt.src.surface import surface
from pygmt.src.ternary import ternary
from pygmt.src.text import text_ as text # "text" is an argument within "text_"
from pygmt.src.tilemap import tilemap
from pygmt.src.timestamp import timestamp
from pygmt.src.triangulate import triangulate
from pygmt.src.velo import velo
Expand Down
155 changes: 155 additions & 0 deletions pygmt/src/tilemap.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
"""
tilemap - Plot XYZ tile maps.
"""
from pygmt.clib import Session
from pygmt.datasets.tile_map import load_tile_map
from pygmt.helpers import (
GMTTempFile,
build_arg_string,
fmt_docstring,
kwargs_to_strings,
use_alias,
)

try:
import rioxarray
except ImportError:
rioxarray = None


@fmt_docstring
@use_alias(
B="frame",
E="dpi",
I="shading",
J="projection",
M="monochrome",
N="no_clip",
Q="nan_transparent",
# R="region",
Comment on lines +27 to +29
Copy link
Member Author

@weiji14 weiji14 Mar 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was playing with the zoom parameter just now, and it seems that setting lower zoom levels (e.g. 0) will actually cause the area covered by contextily.bounds2img to be bigger than the original bounding box region. This might come as a surprise for users, so maybe we should set no_clip to False by default? Would require a bit of extra work to get correct. Edit: Sorry, got a bit confused by the double negatives, no_clip (-N) is already False by default, just needed to pass the region arguments to grdimage properly.

Alternatively, we could also do the clipping in pygmt.datasets.load_tile_map.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added commit 17d0734 to ensure that:

  • The plot is clipped to the bounding box region when no_clip=False (default)
  • The plot extends to areas outside of the bounding box region when no_clip=True

V="verbose",
c="panel",
p="perspective",
t="transparency",
)
@kwargs_to_strings(c="sequence_comma", p="sequence") # R="sequence",
def tilemap(
self, region, zoom="auto", source=None, lonlat=True, wait=0, max_retries=2, **kwargs
):
r"""
Plots an XYZ tile map.

This method loads XYZ tile maps from a tile server or local file using
:func:`pygmt.datasets.load_tile_map` into a georeferenced form, and plots
the tiles as a basemap or overlay using :meth:`pygmt.Figure.grdimage`.

**Note**: By default, standard web map tiles served in a Spherical Mercator
(EPSG:3857) Cartesian format will be reprojected to a geographic coordinate
reference system (OGC:WGS84) and plotted with longitude/latitude bounds
when ``lonlat=True``. If reprojection is not desired, please set
``lonlat=False`` and provide Spherical Mercator (EPSG:3857) coordinates to
the ``region`` parameter.

{aliases}

Parameters
----------
region : list
The bounding box of the map in the form of a list [*xmin*, *xmax*,
*ymin*, *ymax*]. These coordinates should be in longitude/latitude if
``lonlat=True`` or Spherical Mercator (EPSG:3857) if ``lonlat=False``.

zoom : int or str
Optional. Level of detail. Higher levels (e.g. ``22``) mean a zoom
level closer to the Earth's surface, with more tiles covering a smaller
geographical area and thus more detail. Lower levels (e.g. ``0``) mean
a zoom level further from the Earth's surface, with less tiles covering
a larger geographical area and thus less detail [Default is
``"auto"`` to automatically determine the zoom level based on the
bounding box region extent].

**Note**: The maximum possible zoom level may be smaller than ``22``,
and depends on what is supported by the chosen web tile provider
source.

source : xyzservices.TileProvider or str
Optional. The tile source: web tile provider or path to a local file.
Provide either:

- A web tile provider in the form of a
:class:`xyzservices.TileProvider` object. See
:doc:`Contextily providers <contextily:providers_deepdive>` for a
list of tile providers [Default is
``xyzservices.providers.Stamen.Terrain``, i.e. Stamen Terrain web
tiles].
- A web tile provider in the form of a URL. The placeholders for the
XYZ in the URL need to be {{x}}, {{y}}, {{z}}, respectively. E.g.
``https://{{s}}.tile.openstreetmap.org/{{z}}/{{x}}/{{y}}.png``.
- A local file path. The file is read with
:doc:`rasterio <rasterio:index>` and all bands are loaded into the
basemap. See
:doc:`contextily:working_with_local_files`.

IMPORTANT: Tiles are assumed to be in the Spherical Mercator projection
(EPSG:3857).

lonlat : bool
Optional. If ``False``, coordinates in ``region`` are assumed to be
Spherical Mercator as opposed to longitude/latitude [Default is
``True``].

wait : int
Optional. If the tile API is rate-limited, the number of seconds to
wait between a failed request and the next try [Default is ``0``].

max_retries : int
Optional. Total number of rejected requests allowed before contextily
will stop trying to fetch more tiles from a rate-limited API [Default
is ``2``].

kwargs : dict
Extra keyword arguments to pass to :meth:`pygmt.Figure.grdimage`.

Raises
------
ImportError
If ``rioxarray`` is not installed. Follow
:doc:`install instructions for rioxarray <rioxarray:installation>`,
(e.g. via ``pip install rioxarray``) before using this function.
"""
kwargs = self._preprocess(**kwargs) # pylint: disable=protected-access

if rioxarray is None:
raise ImportError(
"Package `rioxarray` is required to be installed to use this function. "
"Please use `pip install rioxarray` or "
"`mamba install -c conda-forge rioxarray` "
"to install the package."
)

raster = load_tile_map(
region=region,
zoom=zoom,
source=source,
lonlat=lonlat,
wait=wait,
max_retries=max_retries,
)

# Reproject raster from Spherical Mercator (EPSG:3857) to
# lonlat (OGC:CRS84) if bounding box region was provided in lonlat
if lonlat and raster.rio.crs == "EPSG:3857":
raster = raster.rio.reproject(dst_crs="OGC:CRS84")
raster.gmt.gtype = 1 # set to geographic type
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setting raster.gmt.gtype = 1 is useless, because raster.rio.to_raster doesn't understand it, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, rioxarray's .rio.to_raster won't understand .gmt.gtype, but I don't want to forget to add this after #2398 is merged 🙂


# Only set region if no_clip is None or False, so that plot is clipped to
# exact bounding box region
if kwargs.get("N") in [None, False]:
kwargs["R"] = "/".join(str(coordinate) for coordinate in region)

with GMTTempFile(suffix=".tif") as tmpfile:
raster.rio.to_raster(raster_path=tmpfile.name)
with Session() as lib:
lib.call_module(
module="grdimage", args=build_arg_string(kwargs, infile=tmpfile.name)
)
Comment on lines +152 to +155
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this is not the full implementation of PyGMT's fig.grdimage, which is actually a bit more complicated to support shading (-I) with an xarray.DataArray grid

pygmt/pygmt/src/grdimage.py

Lines 179 to 192 in 8f31706

with Session() as lib:
file_context = lib.virtualfile_from_data(check_kind="raster", data=grid)
with contextlib.ExitStack() as stack:
# shading using an xr.DataArray
if kwargs.get("I") is not None and data_kind(kwargs["I"]) == "grid":
shading_context = lib.virtualfile_from_data(
check_kind="raster", data=kwargs["I"]
)
kwargs["I"] = stack.enter_context(shading_context)
fname = stack.enter_context(file_context)
lib.call_module(
module="grdimage", args=build_arg_string(kwargs, infile=fname)
)

Question is: should we support shading of the XYZ tiles with xarray.DataArray grids? Or is there a way to use the grdimage implementation above directly without code duplication?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently I would say no to shading.

4 changes: 4 additions & 0 deletions pygmt/tests/baseline/test_tilemap_no_clip_False.png.dvc
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
outs:
- md5: 9317080021b0ce6f3b9ea6d17feece00
size: 23275
path: test_tilemap_no_clip_False.png
4 changes: 4 additions & 0 deletions pygmt/tests/baseline/test_tilemap_no_clip_True.png.dvc
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
outs:
- md5: 83e6119b2351f9d472ca7e3cc45388c3
size: 60984
path: test_tilemap_no_clip_True.png
4 changes: 4 additions & 0 deletions pygmt/tests/baseline/test_tilemap_ogc_wgs84.png.dvc
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
outs:
- md5: 3de0555d86aca49b92425c8d5272a934
size: 59286
path: test_tilemap_ogc_wgs84.png
4 changes: 4 additions & 0 deletions pygmt/tests/baseline/test_tilemap_web_mercator.png.dvc
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
outs:
- md5: a76d9a9a1890d6b1345305eaea598bc3
size: 122195
path: test_tilemap_web_mercator.png
54 changes: 54 additions & 0 deletions pygmt/tests/test_tilemap.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
"""
Tests Figure.tilemap.
"""
import pytest
from pygmt import Figure

contextily = pytest.importorskip("contextily")
rioxarray = pytest.importorskip("rioxarray")


@pytest.mark.mpl_image_compare
def test_tilemap_web_mercator():
"""
Create a tilemap plot in Spherical Mercator projection (EPSG:3857).
"""
fig = Figure()
fig.tilemap(
region=[-20000000.0, 20000000.0, -20000000.0, 20000000.0],
zoom=0,
lonlat=False,
frame="afg",
)
return fig


@pytest.mark.mpl_image_compare
def test_tilemap_ogc_wgs84():
"""
Create a tilemap plot using longitude/latitude coordinates (OGC:WGS84),
centred on the international date line.
"""
fig = Figure()
fig.tilemap(
region=[-180.0, 180.0, -90, 90], zoom=0, frame="afg", projection="R180/5c"
)
return fig


@pytest.mark.mpl_image_compare
@pytest.mark.parametrize("no_clip", [False, True])
def test_tilemap_no_clip(no_clip):
"""
Create a tilemap plot clipped to the Southern Hemisphere when no_clip is
False, but for the whole globe when no_clip is True.
"""
fig = Figure()
fig.tilemap(
region=[-180.0, 180.0, -90, 0.6886],
zoom=0,
frame="afg",
projection="H180/5c",
no_clip=no_clip,
)
return fig
7 changes: 4 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ keywords = [
"geophysics",
"geospatial",
"oceanography",
"seismology"
"seismology",
]
classifiers = [
"Development Status :: 4 - Beta",
Expand All @@ -36,15 +36,16 @@ dependencies = [
"pandas",
"xarray",
"netCDF4",
"packaging"
"packaging",
]
dynamic = ["version"]

[project.optional-dependencies]
all = [
"contextily",
"geopandas",
"ipython"
"ipython",
"rioxarray",
]

[project.urls]
Expand Down