-
Notifications
You must be signed in to change notification settings - Fork 322
Introduce composite tags #3347
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Introduce composite tags #3347
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -495,6 +495,164 @@ image) for both of the static images:: | |
| min_stretch: [0, 0, 0] | ||
| max_stretch: [255, 255, 255] | ||
|
|
||
| .. _composite_variants: | ||
|
|
||
| Composite variants | ||
| ------------------ | ||
|
|
||
| .. versionadded:: 0.60 | ||
|
|
||
| Satpy supports defining multiple *variants* of a composite (e.g., one that | ||
| applies WMO-recommended recipe and one that does not). This feature is controlled by optional | ||
| fields in the composite YAML configuration. | ||
|
|
||
| Tagging composite variants | ||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
|
||
| A composite can carry a list of **tags** that describe which processing variant | ||
| it represents. Tags are plain strings (e.g., ``wmo``, ``crefl``, | ||
| ``nocorr``) and have no special meaning to Satpy beyond being matched during | ||
| compositor lookup. | ||
|
|
||
| .. code-block:: yaml | ||
|
|
||
| sensor_name: visir | ||
|
|
||
| composites: | ||
| true_color_wmo: | ||
| compositor: !!python/name:satpy.composites.SomeCompositor | ||
| prerequisites: | ||
| - name: red | ||
| modifiers: [rayleigh_corrected_wmo] | ||
| - name: green | ||
| - name: blue | ||
| standard_name: true_color | ||
| tags: [wmo] | ||
|
|
||
| true_color_crefl: | ||
| compositor: !!python/name:satpy.composites.SomeCompositor | ||
| prerequisites: | ||
| - name: red | ||
| modifiers: [rayleigh_corrected_crefl] | ||
| - name: green | ||
| - name: blue | ||
| standard_name: true_color | ||
| tags: [crefl] | ||
|
|
||
| true_color: # default, no tags | ||
| compositor: !!python/name:satpy.composites.SomeCompositor | ||
| prerequisites: | ||
| - name: red | ||
| - name: green | ||
| - name: blue | ||
| standard_name: true_color | ||
|
|
||
| The ``standard_name`` field acts as the *base name* shared by all variants. | ||
| Tag-based resolution searches for a compositor whose ``standard_name`` matches | ||
| and whose ``tags`` list contains the requested tag. | ||
|
|
||
| Loading a tagged variant explicitly | ||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
|
||
| Append ``:<tag>`` to the composite name when calling :meth:`~satpy.scene.Scene.load`: | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| scene.load(["true_color:wmo"]) # loads true_color_wmo | ||
|
|
||
| This syntax is interpreted as: "find a compositor with | ||
| ``standard_name='true_color'`` that has ``'wmo'`` in its ``tags``". It never | ||
| performs a plain string match, so ``true_color:wmo`` will not accidentally | ||
| resolve to a compositor named ``true_color`` or ``true_color_wmo``. | ||
|
|
||
| Multiple tags can be combined with additional colons. All listed tags must be | ||
| present on the compositor (AND semantics): | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| scene.load(["true_color:wmo:pyspectral"]) # compositor must carry wmo AND pyspectral | ||
| scene.load(["true_color:wmo"]) # also matches a compositor with tags [wmo, pyspectral] | ||
|
|
||
| A single-tag request is a subset match, so requesting ``"true_color:wmo"`` | ||
| will find a compositor tagged ``["wmo", "pyspectral"]`` just as readily as one | ||
| tagged ``["wmo"]`` alone. | ||
|
|
||
| Session-wide tag preferences | ||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
|
||
| You can configure an **ordered** list of preferred tags so that loading an | ||
| unqualified name like ``"true_color"`` automatically selects a tagged variant | ||
| when one is available: | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| import satpy | ||
| with satpy.config.set(preferred_composite_tags=["crefl", "wmo"]): | ||
| scene.load(["true_color"]) # picks true_color_crefl (crefl listed first) | ||
|
|
||
| Resolution order for ``preferred_composite_tags``: | ||
|
|
||
| 1. Try each tag in the list, in order, looking for a compositor with matching | ||
| ``standard_name`` and that tag. | ||
| 2. If no tagged variant is found, fall back to the normal name-based lookup. | ||
|
|
||
| An explicit ``name:tag`` in the load call always overrides the session-wide | ||
| preference for that specific dataset. | ||
|
|
||
| The setting can also be provided as an environment variable (comma-separated): | ||
|
|
||
| .. code-block:: bash | ||
|
|
||
| export SATPY_PREFERRED_COMPOSITE_TAGS=crefl,wmo | ||
|
|
||
| Or as a YAML configuration key: | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In which config file should this be defined?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. at the top of this file, you have your answer :) YAML Configuration YAML files that include these parameters can be in any of the following
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. sorry, it was actually in config.rst
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh, I didn't even know we had a |
||
|
|
||
| .. code-block:: yaml | ||
|
|
||
| preferred_composite_tags: | ||
| - crefl | ||
| - wmo | ||
|
|
||
| Enhancements for tagged variants | ||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
|
||
| No extra configuration is needed on the enhancement side. The enhancement | ||
| decision tree already matches by composite ``name`` (the YAML key) *before* | ||
| it falls back to ``standard_name``. A composite named ``true_color_wmo`` will | ||
| therefore automatically pick up an enhancement entry keyed ``true_color_wmo`` | ||
| if one exists, and fall back to the ``standard_name: true_color`` entry | ||
| otherwise. | ||
|
|
||
| Deprecating a composite | ||
| ------------------------ | ||
|
|
||
| To emit a warning when a composite is used, add a ``warnings`` mapping to the | ||
| composite YAML entry. Each key is a Python warning category name and the | ||
| value is the warning message: | ||
|
|
||
| .. code-block:: yaml | ||
|
|
||
| composites: | ||
| old_true_color: | ||
| compositor: !!python/name:satpy.composites.SomeCompositor | ||
| prerequisites: | ||
| - name: red | ||
| - name: green | ||
| - name: blue | ||
| standard_name: true_color | ||
| warnings: | ||
| DeprecationWarning: "old_true_color is deprecated, use true_color_wmo instead." | ||
|
|
||
| The warning is emitted only when the compositor is actually *loaded* (i.e. | ||
| when :meth:`~satpy.scene.Scene.load` is called with the deprecated name), not | ||
| during composite discovery calls such as | ||
| :meth:`~satpy.scene.Scene.available_dataset_names`. | ||
|
|
||
| Supported warning categories are any that exist in the Python ``builtins`` | ||
| module (e.g., ``DeprecationWarning``, ``FutureWarning``, | ||
| ``PendingDeprecationWarning``, ``UserWarning``). If a category name is not | ||
| recognised, ``UserWarning`` is used as a fallback. | ||
|
|
||
| .. _enhancing-the-images: | ||
|
|
||
| Enhancing the images | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Which variant is used if the tag is not defined in the load and the global env variable is not set and all/some of the following composites are defined?
foo_wmofoo_creflfooEach of them have the same
standard_name: foodefined, nonameis set. Only the first line in the YAML definition is different and for the first two there are tags set.So what would I get with
scn.load(["foo"])call in the following cases:foovariant without any tags)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If tags are not provided, neither at load time or in the configuration, the current behaviour is preserved, ie we will use the name (not standard name) to choose the composite to load.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So something like this for the first case? I'm not sure how this works, but hopefully it's close enough:
For case 2. the failure might need a completely separate test?