[Bug]: DAAC SNS publish updates

[wphyojpl]

Checked for duplicates

Have you checked for duplicate issue tickets?

Yes - I've already checked

Describe the bug

A clear and concise description of what the bug is. Plain-text snippets preferred but screenshots welcome.

SNS message publishing to DAAC failed

As it says in the following section:
<doc>
TROPESS provider needs to assume the STS role from GES DISC Cumulus:
aws sts assume-role --role-arn arn:aws:iam::xxx:role/xxx --role-session-name xxx --duration-seconds 43200

What did you expect?

Publishing SNS message.

Reproducible steps

Run this:
https://api.test.mdps.mcp.nasa.gov/am-uds-dapa/docs#/Granules%20Archive%20CRUD%20API/archive_single_granule_dapa_am_uds_dapa_collections__collection_id__archive__granule_id___put

What is your environment?

Unity Test

Solution
Add this to lambda role:

        {
            "Action": [
                "sts:AssumeRole",
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:iam::*:role/*"
            ]
        }

• TODO: Is this too broad? But how to control it as the role name can be anything created by DAAC.

Add this to SNS class

        sts_client = self.get_client('sts')

        assumed_role = sts_client.assume_role(
            RoleArn='arn:aws:iam::xxx:role/xxx',
            RoleSessionName='xxx',
            DurationSeconds=900  # 12 hours max
        )

        credentials = assumed_role['Credentials']

        self.__special_sns_client = self.get_session().client(
            "sns",
            aws_access_key_id=credentials['AccessKeyId'],
            aws_secret_access_key=credentials['SecretAccessKey'],
            aws_session_token=credentials['SessionToken'],
        )

Use the __special_sns_client with a flag to differentiate to send to DAAC.

• TODO The RoleArn and RoleSessionName may be dynamic which also needs to come when the request is made.

[wphyojpl]

This is what we are sending.

{
    'collection': 'TRPSDL2ALLCRS1MGLOS', 
    'identifier': 'URN:NASA:UNITY:unity:test:TRPSDL2ALLCRS1MGLOS___2:TROPESS_CrIS-JPSS1_L2_Standard_NH3_20250108_MUSES_R1p23_megacity_los_angeles_MGLOS_F2p5_J0', 
    'submissionTime': '2025-05-15T20:04:17.105578Z', 
    'provider': 'unity', 
    'version': '1.6.0', 
    'product': {
        'name': 'TRPSDL2ALLCRS1MGLOS___2', 
        'dataVersion': '2', 
        'files': [{
            'type': 'data', 
            'name': 'TROPESS_CrIS-JPSS1_L2_Standard_NH3_20250108_MUSES_R1p23_megacity_los_angeles_MGLOS_F2p5_J0.nc', 
            'uri': 'https://uds-distribution-placeholder/unity-test-unity-storage/URN:NASA:UNITY:unity:test:TRPSDL2ALLCRS1MGLOS___2/URN:NASA:UNITY:unity:test:TRPSDL2ALLCRS1MGLOS___2:TROPESS_CrIS-JPSS1_L2_Standard_NH3_20250108_MUSES_R1p23_megacity_los_angeles_MGLOS_F2p5_J0/TROPESS_CrIS-JPSS1_L2_Standard_NH3_20250108_MUSES_R1p23_megacity_los_angeles_MGLOS_F2p5_J0.nc', 
            'checksumType': 'md5', 
            'checksum': '00000000000000000000000000000000', 
            'size': 0
        }]}}

This is the provided schema although it seems to be for v1.3, and we use v1.6

    "product": {
        "files": [
            {{#files}}
            {
                "name": "{{name_of_granule_file}}",
                "type": "{{type_of_granule_file}}",
                "uri": "{{s3_uri_for_granule_data}}",
                "checksumType": "{{checksum_type}}",
                "checksum": "{{checksum_of_file}}",
                "Size": "{{size_in_bytes_of_file}}"
            }
            {{/files}}
        ],
        "name": "{{name_of_granule}}"
    },
    "identifier": "{{unique_identifier_for_message}}",
    "collection": {
        "name": "{{collection_name}}",
        "version": "{{collection_version}}"
    },
    "Provider": "TROPESS_CLOUD",
    "Version": "1.3",
    "submissionTime": "{{time_when_message_was_created}}"
}

This is an example of what they are expecting.

{
    "product": {
      "files": [
        {
            "name":"TROPESS_CrIS-JPSS1_L2_Standard_CH4_20250108_MUSES_R1p23_megacity_los_angeles_MGLOS_F2p5_J0.nc",
            "type":"data",
            "uri":"s3://unity-test-unity-storage/URN:NASA:UNITY:unity:test:TRPSDL2ALLCRS1MGLOS___2/URN:NASA:UNITY:unity:test:TRPSDL2ALLCRS1MGLOS___2:datum/TROPESS_Standard/TRPSDL2ALLCRS1MGLOS.2/2025/01/08/TROPESS_CrIS-JPSS1_L2_Standard_CH4_20250108_MUSES_R1p23_megacity_los_angeles_MGLOS_F2p5_J0/TROPESS_CrIS-JPSS1_L2_Standard_CH4_20250108_MUSES_R1p23_megacity_los_angeles_MGLOS_F2p5_J0.nc",
            "size":280595
         }
      ],
      "name": "TROPESS_CrIS-JPSS1_L2_Standard_CH4_20250108_MUSES_R1p23_megacity_los_angeles_MGLOS_F2p5_J0"
    },
    "identifier ":"testIdentifier123456",
    "collection": {
        "name": "TRPSDL2ALLCRS1MGLOS",
        "version": "2"
    },
    "provider":"tropess_testing"
}

These are their comments, and some reasonings on each of them.

1. The uri for the file appears to be from a https provider called uds-distribution-placeholder. The example files we've been provided have been from s3 endpoints. (i.e. s3://unity-test-unity-storage/URN:NASA:UNITY:unity:test:TRPSDL2ALLCRS1MGLOS___2/URN:NASA:UNITY:unity:test:TRPSDL2ALLCRS1MGLOS___2:datum/TROPESS_Standard/TRPSDL2ALLCRS1MGLOS.2/2025/01/08/TROPESS_CrIS-JPSS1_L2_Standard_CH4_20250108_MUSES_R1p23_megacity_los_angeles_MGLOS_F2p5_J0/TROPESS_CrIS-JPSS1_L2_Standard_CH4_20250108_MUSES_R1p23_megacity_los_angeles_MGLOS_F2p5_J0.nc). I was wondering if this was to be expected? We currently have it configured for s3 endpoints.
   • UDS store s3 URL as is. example: "href": "s3://unity-test-unity-storage/URN:NASA:UNITY:unity:test:TRPSDL2ALLCRS1MGLOS___2/URN:NASA:UNITY:unity:test:TRPSDL2ALLCRS1MGLOS___2:TROPESS_CrIS-JPSS1_L2_Standard_NH3_20250108_MUSES_R1p23_megacity_los_angeles_MGLOS_F2p5_J0/TROPESS_CrIS-JPSS1_L2_Standard_NH3_20250108_MUSES_R1p23_megacity_los_angeles_MGLOS_F2p5_J0.nc",
   • cnm changes it to "uri": "https://uds-distribution-placeholder/unity-test-unity-storage/URN:NASA:UNITY:unity:test:TRPSDL2ALLCRS1MGLOS___2/URN:NASA:UNITY:unity:test:TRPSDL2ALLCRS1MGLOS___2:TROPESS_CrIS-JPSS1_L2_Standard_NH3_20250108_MUSES_R1p23_megacity_los_angeles_MGLOS_F2p5_J0/TROPESS_CrIS-JPSS1_L2_Standard_NH3_20250108_MUSES_R1p23_megacity_los_angeles_MGLOS_F2p5_J0.nc",
   • Looks like DAAC won't accept this?
2. There's no file size provided in the files attribute of the message. Please include the file size in bytes.
   • both UDS and cnm has
     "checksumType": "md5",
     "checksum": "00000000000000000000000000000000",
     "size": 0
   • Looks like the application should add checksum and size, but I guess they probably push back to UDS.
3. There should be a collection field should be comprised of two separate variables: name and version.
   Using this reference: https://github.com/podaac/cloud-notification-message-schema?tab=readme-ov-file#notification-message-fields,
   • It can be either a string for the collection short name (e.g. GLDAS_NOAH025_3H ), or a JSON.
4. There some attributes outside of the product field.
   Using this reference: https://github.com/podaac/cloud-notification-message-schema?tab=readme-ov-file#notification-message-fields,
   • versions number/string of the granule.
     I just use the "version" and put it there. I can remove it if required.

[wphyojpl]

One more thing:
When there are duplicate cnm files for the same granule, need to grab the latest one instead of earliest one.