Unable to control subfolders with LayoutTemplate

[scottyhq]

the generate_subcatalogs() command is very handy for creating new catalogs with a layout based on STAC item properties! But it seems there is a bug or missing documentation. All the examples seem to map 1 property to 1 subfolder (https://pystac.readthedocs.io/en/latest/api.html?highlight=template#templating).

In many cases it can be useful to merge properties names into a single subfolder name or do some other customization. For example I was expecting the template string to behave like a python f'string'. In fact it looks like the '/' and all other characters outside ${} are ignored

template = '${mgrs:utm_zone}${mgrs:latitude_band}${mgrs:grid_square}/${year}'
collection.generate_subcatalogs(template=template)
collection.describe()

Expected Output:

* <Collection id=sentinel1-rtc-aws>
    * <Catalog id=10UCU>
        * <Catalog id=2017>
          * <Item id=S1A_20170101_10UCU_ASC>
          * <Item id=S1A_20170125_10UCU_ASC>
          * <Item id=S1B_20170102_10UCU_ASC>

Actual Output:

* <Collection id=sentinel1-rtc-aws>
    * <Catalog id=10>
        * <Catalog id=U>
            * <Catalog id=CU>
                * <Catalog id=2017>
                  * <Item id=S1A_20170101_10UCU_ASC>
                  * <Item id=S1A_20170125_10UCU_ASC>
                  * <Item id=S1B_20170102_10UCU_ASC>

[scottyhq]

@lossyrob @fnattino is there a workaround for this that I'm overlooking? Pinging you both b/c of your work and comments on this function in #241

[fnattino]

Hi @scottyhq @lossyrob, apologies for the late answer, I wanted to have a look at the issue but could not find time until now.

It looks to me that there is no workaround at the moment, but I think this could be fixed rather easily.

At the moment, the subcatalogs that are added to the structure are taken from LayoutTemplate.get_template_values(item):

pystac/pystac/catalog.py

Line 668 in e744897

item_parts = layout_template.get_template_values(item)

Iterating over that dictionary leads to one subdirectory being added for each property in the template, regardless of the separator between the property names.

Using instead LayoutTemplate.substitute(item), should lead to the correct behaviour:

subcats = LayoutTemplate.substitute(item)
item_parts = subcats.split("/")

I will open a pull request with a tentative fix!

[duckontheweb]

Closed via #595