MRB-422 Generate plots for inference output

[cosunae]

• showcase a range of domains
  • plot global domain
  • plot european domain
  • plot swiss domain
•  showcase a range of parameters
  • plot surface temperature
  • plot surface wind speed
  • plot precipitation
• add polygon of the LAM domain
• fix shift
• fix param names mapping in _CMAP_DEFAULTS

[clairemerker]

I started working on using earthkit-plots to plot the data and also using the colortable files from NLC, building on the code that @cosunae wrote.

I didn't manage to finish the work before my holidays, here is the status, hope that helps :) :

Color tables

• parsing and using NCL colortables
• parsing and using the defined color bounds
• option to specify units corresponding to the parameter and colormap
• fallback if parameter has no specified colormap in CMAP_ DEFAULTS
• CMAP_DEFAULTS works with cmap/norm or cmap/vmin/vmax or only cmap
• added some unit tests for the colormap_loader and colormap_defaults

Plots with earthkit-plots

• I managed to produce the figure and plot using earthkit-plots, but with some caveats/unfinished things
• The data is not wrapped properly at the dateline and this causes a white line through the domains (I think this might be solved by transforming the coordinates outside of the plotting function, I commented that out to simplify the workflow while trying things)
• There is still something wrong with the projection/transformation, especially with the global view that i didn't manage to get to work for some projections... The transformation as such seems to be handled ok, I did a test comparing PlateCarree with a very distorted AzimuthalEquidistant for the smaller domain and that seems fine (see attached images)
• I think cropping the data would speed up plotting, I started implementing something but then stopped because I didn't have time to handle all the use cases with domains crossing datelines and so on
• I suspect a bug in earthkit-plots when calling tripcolor/tricontourf. Those functions ca handle a Triangulation when used directly, for some reason this doesn't work when using it with earthkit-plot, my guess is that the earthkit-plots check throwing: "ValueError: x and y arrays must have the same length" is incorrect. Therefore I had to use x/y (which makes plotting slower)

2t, Orthographic projection, Europe

2t, Orthographic projection, Globe

2t, PlateCarree projection, Europe

2t, PlateCarree projection, Globe

2t, PlateCarree projection, Switzerland

2t, AzimuthalEquidistant, Switzerland

[dnerini]

@clairemerker I merged #54 into this branch to refactor the plotting code and lean on Snakemake’s strengths. The result is hopefully a cleaner workflow that showcases how Snakemake can simplify orchestration, remove boilerplate, and make the pipeline easier to maintain and extend.

[OpheliaMiralles]

• No more white bar (global data had longitude 0-360 and local -180 180 which might have played a role in this bug);
• earthkit-plot defaults the "transform" kwargs for matplotlib axes objects to the cartopy PlateCarree() crs when the source is identified as a "NumpySource". The line "subplot._plot_kwarg: lambda source: {}" should fix this behaviour, but in the long run it would be nice if it could get fixed in earthkit plots (so that numpy sources can have different transforms). Now projections should behave as expected, although I only tried orthographic and platecarree and we might need to create the triangulation for other CRS but it should be straightforward.

[frazane]

I am not very happy with how earthkit-plots does not allow reusing a pre-computed Triangulation object. In my original implementation here the triangulation was only performed once and reused for every field, but now it is recomputed every time which is wasteful. Am I understanding this correctly @clairemerker @cosunae ? Would it make sense to open an issue on earthkit-plots?

[dnerini]

@frazane @OpheliaMiralles I'm getting a keyerror with the current version of the code:

2025-10-14 13:06:23,666 - meteodatalab.grib_decoder - INFO - Retrieving request: {'param': ['2t']}
Traceback (most recent call last):
  File "/users/ned/src/evalml/workflow/scripts/plot_forecast_frame.mo.py", line 199, in <module>
[...]
  File "/users/ned/src/evalml/src/plotting/compat.py", line 20, in load_state_from_grib
    lats = ds[paramlist[0]].lat.data.flatten()
           ~~^^^^^^^^^^^^^^
KeyError: '2t'

despite the variable 2t clearly being there in the GRIB file:

grib_ls output/data/runs/d0846032f/202002030000/grib/202002030000_000.grib
output/data/runs/d0846032f/202002030000/grib/202002030000_000.grib
edition      centre       date         dataType     gridType     stepRange    typeOfLevel  level        shortName    packingType  
2            lssw         20200203     fc           rotated_ll   0            heightAboveGround  10           10u          grid_simple 
2            lssw         20200203     fc           rotated_ll   0            heightAboveGround  10           10v          grid_simple 
2            lssw         20200203     fc           rotated_ll   0            heightAboveGround  2            2d           grid_simple 
2            lssw         20200203     fc           rotated_ll   0            heightAboveGround  2            2t           grid_simple 

However, I can load the right temperature variable using "T_2M" notation, which is a bit confusing. Do you know what's going on here? Is it meteodata-lab automatically renaming to COSMO/ICON variable names?

[OpheliaMiralles]

• Fixed shift + east boundary artefact
• Added LAM polygon

[jonasbhend]

Do we need baselines and analysis for the showcases?

[jonasbhend]

config handling clearly needs some more attention. Currently we require everything in all the configs (i.e. baseline, analysis, stratification even for showcases). This doesn't make a lot of sense. Instead we should either combine experiment and showcase configs, or allow showcase configs to be a lot simpler.

[frazane]

In my opinion we should have different config schemas for different commands. It's a minor inconvenience to have different configs (we will have max 3-4 commands in the end, I suppose), but we get:

• better separation of concerns
• clearer config schema-driven development
• simpler configs for each command

[jonasbhend]

Maybe we leave it as is and discuss how the we expect the showcases command to evolve. #74 introduces meteograms for which baselines and analyses would be a useful addition and reference. If we integrate this with the showcases command, we are back to the full config (minus the stratification by region).

[jonasbhend]

I broke the showcase by merging main with the plotting by region into this branch. I can have a look later, sorry :-/

[OpheliaMiralles]

Added precip, no global data yet

[jonasbhend]

Great, thanks @OpheliaMiralles for integrating precip.

When I run this with the M-1 forecaster only, I get a very small crop over switzerland for the global domain (e.g. output/showcases/d0846032f/202002030000/202002030000_tp_globe.gif):

This is my config:

# yaml-language-server: $schema=../workflow/tools/config.schema.json
description: |
  Basic config for showcasing M-1 forecaster.

dates:
  - 2020-02-03T00:00 # Storm Petra
  - 2020-02-07T00:00 # Storm Sabine
  - 2020-10-01T00:00 # Storm Brigitte

runs:
  - forecaster:
      mlflow_id: d0846032fc7248a58b089cbe8fa4c511
      label: M-1 forecaster
      steps: 0/120/6

baselines:
  - baseline:
      baseline_id: COSMO-E
      label: COSMO-E
      root: /store_new/mch/msopr/ml/COSMO-E
      steps: 0/120/6

stratification:
  regions:
    - jura
    - mittelland
    - voralpen
    - alpennordhang
    - innerealpentaeler
    - alpensuedseite
  root: /scratch/mch/bhendj/regions/Prognoseregionen_LV95_20220517

analysis:
  label: COSMO KENDA
  analysis_zarr: /scratch/mch/fzanetta/data/anemoi/datasets/mch-co2-an-archive-0p02-2015-2020-6h-v3-pl13.zarr

locations:
  output_root: output/
  mlflow_uri:
    - https://servicedepl.meteoswiss.ch/mlstore
    - https://mlflow.ecmwf.int

profile:
  executor: slurm
  global_resources:
    gpus: 15
  default_resources:
    slurm_partition: "postproc"
    cpus_per_task: 1
    mem_mb_per_cpu: 1800
    runtime: "1h"
    gpus: 0
  jobs: 50
  batch_rules:
    plot_forecast_frame: 32

[frazane]

This happens because earthkit plos automatically adjusts the extend of the displayed domain to the presence of data (note how it's an exact bounding box of the values).

[OpheliaMiralles]

This happens because earthkit plos automatically adjusts the extend of the displayed domain to the presence of data (note how it's an exact bounding box of the values).

Yes in the case of "globe" region, no cropping is done so it is adjusted to coordinates in the data. @jonasbhend are you sure this is tp though? it looks like the temperature

[OpheliaMiralles]

I have self-reviewed and it looks fine! @dnerini at this stage I suggest to ask review from someone else maybe

[dnerini]

Thanks Ophélia! I suggest to ask for a final review from @clairemerker next week

[frazane]

Suggested change

	{ name = "MeteoSwiss" },

[jonasbhend]

Maybe we leave it as is and discuss how the we expect the showcases command to evolve. #74 introduces meteograms for which baselines and analyses would be a useful addition and reference. If we integrate this with the showcases command, we are back to the full config (minus the stratification by region).

[jonasbhend]

Here we could specify the extent, no? So that the globe projection is the same irrespective of the presence of global data or not.

[dnerini]

I think this is good to go. There are still a few open points and certainly a lot of room for improvements, but the basic functionality is there and performs well. I suggest to merge as it is.