Create a data upload task

[ngachung]

Input:

• Local work dir containing files to upload to S3
• Output collection

Output:

• S3 locations / STAC

See Data Upload task depicted in https://lucid.app/lucidchart/8cfdfeb4-f244-4a38-b574-fde8c7c1feed/edit?page=jSXkb6Xvop-p

[wphyojpl]

Existing data upload task perform the following steps

• search collection details from Cumulus using Collection_id
• search valid files based on regex from collection details json
• upload files to appropriate location in S3.
• call register endpoint (ingest w/ CNM) with the files json.

In our new task, this would be the updated steps

• search collection details from Cumulus using Collection_id
• search valid files based on regex from collection details json
• upload files to appropriate location in S3.
• return the files json which is basically body for register endpoint.

Data Catalog & Data Notification would be pretty light weighted.

Does this sound ok? @ngachung

[ngachung]

Sounds good. Thanks @wphyojpl.