Merge branch 'develop' of github.com:unity-sds/unity-data-services into fastapi

Author: wphyojpl

CHANGELOG.md

@@ -5,6 +5,38 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [5.2.2] - 2023-07-21
+### Changed
+- [#188](https://github.com/unity-sds/unity-data-services/pull/188) fix: Update stage out task to read STAC items from STAC catalog
+
+
+# [Unity Release 23.2] - 2023-07-20
+
+### Repository Tags
+- [unity-data-services](https://github.com/unity-sds/unity-data-services/) : [5.2.1](https://github.com/unity-sds/unity-data-services/releases/tag/v5.2.1)
+
+### Added
+- [#119](https://github.com/unity-sds/unity-data-services/issues/119) Update data download task to allow staging data from DAAC HTTPS to local work directory
+- [#123](https://github.com/unity-sds/unity-data-services/issues/123) Update metadata reader to parse metadata from CHIRP xml files
+- [#157](https://github.com/unity-sds/unity-data-services/issues/157) Update stage-in to support HTTP/HTTPS download that doesn't require EDL
+- [#170](https://github.com/unity-sds/unity-data-services/issues/170) Parallelize data download in stage in task
+### Changed
+- [#125](https://github.com/unity-sds/unity-data-services/issues/125) Update metadata parser/transformer names to reflect metadata format instead of data processing level
+- [#128](https://github.com/unity-sds/unity-data-services/issues/128) UDS search docker image to perform CMR search + pagination 
+- [#129](https://github.com/unity-sds/unity-data-services/issues/129) UDS search docker image to perform UDS search + pagination
+- [#130](https://github.com/unity-sds/unity-data-services/issues/130) Update stage out (upload data to S3) to read catalog.json 
+- [#133](https://github.com/unity-sds/unity-data-services/issues/133) Update stage in task to take as input STAC input file
+- [#141](https://github.com/unity-sds/unity-data-services/issues/141) Update stage in to modify input STAC JSON to point to local urls
+- [#147](https://github.com/unity-sds/unity-data-services/issues/147) UDS tasks to support optional output file parameter
+- [#151](https://github.com/unity-sds/unity-data-services/issues/151) Update catalog task to take stac input file
+- [#155](https://github.com/unity-sds/unity-data-services/issues/155) Update UDS catalog task to wait for granules to be registered 
+- [#158](https://github.com/unity-sds/unity-data-services/issues/158) Update stage in task to use relative path for href in STAC 
+- [#159](https://github.com/unity-sds/unity-data-services/issues/159) Update stage out task to not require integration with UDS DAPA
+- [#160](https://github.com/unity-sds/unity-data-services/issues/160) Update stage in to require FeatureCollection STAC as input and only download specific assets
+### Fixed
+- [#167](https://github.com/unity-sds/unity-data-services/issues/167) Add retry logic for temporary failure in name resolution in Earth Data Login
+- [#181](https://github.com/unity-sds/unity-data-services/issues/181) Update stage in task to auto retry upon 502 errors
+
 ## [5.2.1] - 2023-07-10
 ### Added
 - [#182](https://github.com/unity-sds/unity-data-services/pull/182) fix: Retry if Download Error in DAAC

cumulus_lambda_functions/cumulus_stac/granules_catalog.py

@@ -7,12 +7,12 @@
 
 class GranulesCatalog:
 
-    def get_child_link_hrefs(self, catalog_file_path: str):
+    def get_child_link_hrefs(self, catalog_file_path: str, rel_name: str = 'item'):
         if not FileUtils.file_exist(catalog_file_path):
             raise ValueError(f'missing file: {catalog_file_path}')
         catalog = FileUtils.read_json(catalog_file_path)
         catalog = Catalog.from_dict(catalog)
-        child_links = [k.href for k in catalog.get_links(rel='child')]
+        child_links = [k.href for k in catalog.get_links(rel=rel_name)]
         catalog_dir = os.path.dirname(catalog_file_path)
         new_child_links = []
         for each_link in child_links:

setup.py

@@ -22,7 +22,7 @@
 
 setup(
     name="cumulus_lambda_functions",
-    version="5.2.1",
+    version="5.2.2",
     packages=find_packages(),
     install_requires=install_requires,
     tests_require=['mock', 'nose', 'sphinx', 'sphinx_rtd_theme', 'coverage', 'pystac', 'python-dotenv', 'jsonschema'],

tests/integration_tests/test_docker_entry.py

@@ -1775,7 +1775,8 @@ def test_03_upload_complete_catalog(self):
                 id='NA',
                 description='NA')
             catalog.set_self_href(os.environ['CATALOG_FILE'])
-            catalog.add_link(Link('child', os.path.join('some_granules', 'test_file01.nc.stac.json'), 'application/json'))
+            catalog.add_link(Link('item', os.path.join('some_granules', 'test_file01.nc.stac.json'), 'application/json'))
+            print(json.dumps(catalog.to_dict(False, False)))
             with open(os.environ['CATALOG_FILE'], 'w') as ff:
                 ff.write(json.dumps(catalog.to_dict(False, False)))