Case sensitivity issues with collection ID prefix

[mcduffie]

There are some inconsistencies in collection ID prefix case sensitivity consistent.

Using unity-sds-client I can only create collection IDs that begin wtih

urn:nasa:unity

Trying to capitalize this portion will result in this error:

UnityException: Collection Identifiers must start with urn:nasa:unity:unity:test

But when not giving an explicit prefix to stage out it will use the following before project/venue:

URN:NASA:UNITY

And if I query using get_collection_data from unity-sds-client with, for instance:

urn:nasa:unity:unity:test:MUSES-CRS1-MGLOS___1

I will get no results. But using the prefix capitalized like stage-out adds does result in hits:

URN:NASA:UNITY:unity:test:MUSES-CRS1-MGLOS___1

Looks like the STAC browser is also case sensitive in terms of prefix. So the products do not show up in the browser if they were registered with the lowercase urn:nasa:unity but submitted for ingestion by stage out using the uppercase prefix: URN:NASA:UNITY.

[ngachung]

@wphyojpl will investigate to see how easy it is to make search case insensitive for the prefix urn:nasa:unity. Note that S3 object keys are case sensitive.

aws s3 ls s3://my-venue-bucket/URN:NASA:UNITY:unity:test:my-collection___1/

is not the same as

aws s3 ls s3://my-venue-bucket/urn:nasa:unity:unity:test:my-collection___1/

[ngachung]

https://d3vc8w9zcq658.cloudfront.net/am-uds-dapa/catalog/ returns different count from https://d3vc8w9zcq658.cloudfront.net/am-uds-dapa/collections/ probably due to case handling differences

Changing auth configuration from

{
  "actions": [
    "READ",
    "CREATE"
  ],
  "resources": [
    "urn:nasa:unity:unity:dev:.*"
  ],
  "tenant": "unity",
  "venue": "dev",
  "group_name": "Unity_Viewer"
}

to

{
  "actions": [
    "READ",
    "CREATE"
  ],
  "resources": [
    "URN:NASA:UNITY:unity:dev:.*"
  ],
  "tenant": "unity",
  "venue": "dev",
  "group_name": "Unity_Viewer"
}

results in /catalog returning different collections

[wphyojpl]

This is the logic in UDS collection class.

        authorized_collection_ids_dsl = {
            'query': {
                'bool': {
                    'should': [
                        {'regexp': {DBConstants.collection_id: k}} for k in collection_regex
                    ]
                }
            },
            'sort': [
                {DBConstants.collection_id: {'order': 'asc'}}
            ]
        }

Simply updating it to

 {'regexp': {DBConstants.collection_id: {'value': k, 'case_insensitive': True}}}

might solve it.

[mcduffie]

Does this mean that ALL parts of the collection ID will be case insensitive, not just the prefix?

[wphyojpl]

yea. the whole thing would be insensitive.

[ngachung]

Moving forward we will require URN:NASA:UNITY:<tenant>:<venue> to be all uppercase.