Skip to content

Allow to save a catalog at a specific location #565

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 22, 2021
Merged

Allow to save a catalog at a specific location #565

merged 1 commit into from
Jul 22, 2021

Conversation

volaya
Copy link
Contributor

@volaya volaya commented Jul 17, 2021

This PR allows to specify a folder where a catalog should be saved, to be used instead of the pth taken for the catalog self href

This will allow implementing the functionaity requested at stac-utils/stactools#12

It is still a work in progress. I am creating this PR to share the approach and gather some feedback about it. I will add unit tests and implement the corresponding code in StacTools later.

@codecov-commenter
Copy link

codecov-commenter commented Jul 17, 2021

Codecov Report

Merging #565 (95aeeff) into main (e4059c8) will increase coverage by 0.33%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #565      +/-   ##
==========================================
+ Coverage   94.35%   94.68%   +0.33%     
==========================================
  Files          71       75       +4     
  Lines       10409    10758     +349     
  Branches     1089     1052      -37     
==========================================
+ Hits         9821    10186     +365     
+ Misses        419      396      -23     
- Partials      169      176       +7     
Impacted Files Coverage Δ
pystac/catalog.py 91.93% <100.00%> (+0.29%) ⬆️
tests/test_catalog.py 99.17% <100.00%> (+0.01%) ⬆️
pystac/item.py 90.50% <0.00%> (-2.73%) ⬇️
pystac/link.py 90.06% <0.00%> (-2.42%) ⬇️
pystac/extensions/eo.py 93.11% <0.00%> (-1.30%) ⬇️
pystac/extensions/sar.py 98.56% <0.00%> (-0.40%) ⬇️
pystac/extensions/scientific.py 94.90% <0.00%> (-0.19%) ⬇️
pystac/extensions/raster.py 89.70% <0.00%> (-0.15%) ⬇️
pystac/extensions/version.py 96.07% <0.00%> (-0.15%) ⬇️
pystac/extensions/file.py 92.06% <0.00%> (-0.13%) ⬇️
... and 24 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e4059c8...95aeeff. Read the comment docs.

@duckontheweb
Copy link
Contributor

I think this will also help with #425

Comment on lines 724 to 725
This is olnly used if the catalog type is
``CatalogType.ABSOLUTE_PUBLISHED``.
Copy link
Contributor

@duckontheweb duckontheweb Jul 17, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor typo here:

Suggested change
This is olnly used if the catalog type is
``CatalogType.ABSOLUTE_PUBLISHED``.
This is only used if the catalog type is
``CatalogType.ABSOLUTE_PUBLISHED``.

It might also be good to add a check at the beginning of this method that raises a STACError if dest_href is not None and root.catalog_type != CatalogType.ABSOLUTE_PUBLISHED. I could see users being confused if they pass in a dest_href that just gets ignored silently.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I restricted it to CatalogType.ABSOLUTE_PUBLISHED, because the original issue in the stactools repo mentioned the functionality for that case, but it's true that here it makes sense for other types as well (If anyone has a different opinion here or I am missing something, please comment here). I have removed that check and now the dest_href parameter is used regardless of the catalog type

Copy link
Contributor

@duckontheweb duckontheweb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tackling this @volaya! This approach looks good to me, but I'm wondering if we really need to restrict dest_href to only apply to "absolute published" catalogs. I'm admittedly not as familiar with the nuances of the different catalog types, but it seems like if someone has to explicitly pass in the dest_href argument then they may have a good reason to do it for other catalog types. Maybe that would require much more complicated updating of the catalog links, though, not sure 🤷🏻

include_self_link=items_include_self_link
)
item = cast(pystac.Item, item_link.target)
item_dest_href = None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
item_dest_href = None

This ends up just being unused

Copy link
Contributor

@duckontheweb duckontheweb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me @volaya, thanks for contributing this!

You mentioned in the original description that this is a WIP. Is that still true or is it ready to merge?

@volaya
Copy link
Contributor Author

volaya commented Jul 21, 2021

@duckontheweb This should be now fine to merge. I added the matching functionality in StacTools here: stac-utils/stactools#175

@duckontheweb
Copy link
Contributor

Coverage upload seems to be failing due to codecov/codecov-action#437, but everything else passes, so I'll merge this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants