Auditing GIBS delivery status

[frankinspace]

Need to ensure we have continued monitoring in place for communication between PO.DAAC and GIBS

[frankinspace]

May need updates to dashboards.

Will need ops documentation. Need to answer these main questions for a configured collection and time range:

• What granules have no CNM sent to GIBS?
• What granules have CNM sent to GIBS but no CNM-R?
• What CNM-R have error response?
• What is the latency between CNM sent to GIBS and CNM-R received?

[torimcd]

@frankinspace will look at dashboards to determine if any changes required for OPERA HLS collection

[frankinspace]

I think the best way to address this right now is to create a new lambda in bignbit called 'granule_report' or similar.

This lambda should take as input:

1. Collection short name
2. Start/End data time to search in CMR

The output of the lambda will be a report that looks something like:

Granule UR	CMR Concept ID	Browse Image Workflow Execution ID	CMA Message Path	CNM Message Path	CNM Identifier	CNM-R Message Path	CNM-R Status
OPERA_L3_DSWx-HLS_T48SUE_20190302T034350Z_20230131T222341Z_L8_30_v0.0	G1256529439-POCLOUD	2f4af846-9368-4046-90d3-6ffc222d29f2	N/A	s3://svc-sit-bignbit-internal/pobit-cma-output/OPERA_L3_DSWX-S1_V1_none/OPERA_L3_DSWx-S1_T38KMG_20240408T023537Z_20240408T180618Z_S1A_30_v0.1.2024-08-16T20:20:29.584Z.cnm.json	OPERA_L3_DSWx-S1_T38KMG_20240408T023537Z_20240408T180618Z_S1A_30_v0.1_B01_WTR_2024099_G1263045014-POCLOUD	s3://svc-sit-bignbit-internal/pobit-cma-output/OPERA_L3_DSWX-S1_V1_none/OPERA_L3_DSWx-S1_T38KMG_20240408T023537Z_20240408T180618Z_S1A_30_v0.1.2024-06-05T16:06:57.955Z.cnm-r.json	SUCCESS

• This is just an example, the values here are taken from various test data.
• The same granule may have multiple rows if multiple CNM messages need to be sent per-granule.
• If data for some columns is not available, some indication as to why the data is not available should be filled in it's place. For example, if the browse image workflow did not get executed for this granule, none of the information except CMR information can be filled in which can be indicated by filling in 'Missing' or something similar.

[frankinspace]

Might be able to use this for retrieving workflow executions by granule ID
https://nasa.github.io/cumulus-api/v18.4.0/#search-executions-by-granules

There is a python client for the API as well https://github.com/nasa/cumulus-api/tree/master/api-client-py but it looks pretty sparse so not sure if it is usable or not