[Bug]: DAPA API for collections does not return more than 50 results

[ngachung]

Cumulus limit is cap at 100 per https://nasa.github.io/cumulus-api/unreleased/#cumulus-api

However, trying to list all collections returns at most 50 even when setting limit to 100

https://1gp9st60gd.execute-api.us-west-2.amazonaws.com/dev/am-uds-dapa/collections?limit=100

[wphyojpl]

This is weird.

[DEBUG]	2023-09-15T18:01:00.548Z	7be698b0-ed17-4f71-92c2-f421fca8b4f1	payload: {'httpMethod': 'GET', 'resource': '/{proxy+}', 'path': '/collections', 'queryStringParameters': {'limit': '100', 'page': '1'}}
--
2023-09-15 18:01:00,804 [DEBUG] [cumulus_lambda_functions.cumulus_wrapper.query_collections::295] json query_result: {'meta': {'name': 'cumulus-api', 'stack': 'uds-dev-cumulus', 'table': 'collection', 'limit': 50, 'page': 1, 'count': 51, 'searchContext': '%5B1660029909667%5D'}, 'results': [{'createdAt': 1694106616717, 'updatedAt': 1694106616717, 'reportToEms': True, 'duplicateHandling': 'skip', 'granuleId': '^test_file.*, 'granuleIdExtraction': '(^test_file.*)(\\.nc\|\\.nc\\.cas\|\\.cmr\\.xml)', 'process': 'snpp.level1', 'name': 'ROBT_COLLECTION_EXAMPLE_TEST', 'version': '1', 'url_path': '{cmrMetadata.Granule.Collection.ShortName}___{cmrMetadata.Granule.Collection.VersionId}', 'sampleFileName': 'test_file01.nc', 'files': [{'reportToEms': True, 'type': 'data', 'sampleFileName': 'test_file01.nc', 'bucket': 'protected', 'regex': '^test_file.*\\.nc}, {'reportToEms': True, 'type': 'metadata', ...

It is calling cumulus with limit = 100. But it only returns 50.

[wphyojpl]

@ngachung
I think it is caused by this line: https://github.com/nasa/cumulus/blob/master/packages/es-client/collections.js#L16
The above line is called in this code: https://github.com/nasa/cumulus/blob/177fa68e951c2f26ba69d9b641c684e7c9eb5740/packages/api/endpoints/collections.js#L53

[ngachung]

Oh wow. Good finding! I'll let Cumulus team know. In the meantime, should we also set our collections limit to 50?

[wphyojpl]

default is only 10 if there is no mention.
We can add the logic so that max is always 50.

[ngachung]

Yes, let's make max 50 for now.

[ngachung]

Related Cumulus ticket: https://bugs.earthdata.nasa.gov/browse/CUMULUS-3428