Skip to content

[Bug]: stage-out fails when trying to overwrite stac.json #574

Closed
@ngachung

Description

@ngachung

Checked for duplicates

Yes

Describe the bug

stage-out uploads files to s3 but runs into the following bug

[2025-05-08, 20:02:00 UTC] {pod_manager.py:471} INFO - [base] 2025-05-08 20:02:00,662 [ERROR] [mdps_ds_lib.stage_in_out.upload_granules_by_complete_catalog_s3::104] error while processing: /tmp/outputs/./datum/TROPESS_Standard/TRPSDL2ALLCRS1MGLOS.2/2025/01/08/TROPESS_CrIS-JPSS1_L2_Standard_HDO_20250108_MUSES_R1p23_megacity_los_angeles_MGLOS_F2p5_J0.json
[2025-05-08, 20:02:00 UTC] {pod_manager.py:471} INFO - [base] Traceback (most recent call last):
[2025-05-08, 20:02:00 UTC] {pod_manager.py:471} INFO - [base]   File "/usr/local/lib/python3.9/site-packages/mdps_ds_lib/stage_in_out/upload_granules_by_complete_catalog_s3.py", line 97, in __exec_actual_job
[2025-05-08, 20:02:00 UTC] {pod_manager.py:471} INFO - [base]     FileUtils.write_json(os.path.join(os.path.dirname()))
[2025-05-08, 20:02:01 UTC] {pod_manager.py:471} INFO - [base] TypeError: dirname() missing 1 required positional argument: 'p'

Probably just need to remove https://github.com/unity-sds/mdps-ds-lib/blob/develop/mdps_ds_lib/stage_in_out/upload_granules_by_complete_catalog_s3.py#L97

NOTE: After fixing this bug, there are issues downstream with having / in the granule ID. If there is a use case for organizing files in s3 by paths, that is a separate feature.

What did you expect?

stage-out to upload files to s3 in subdirectories and upload successful_features.json to s3

Reproducible steps

Run stage out with files located in subdirectories

What is your environment?

ghcr.io/unity-sds/unity-data-services:9.11.5

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions