Skip to content

feat: elasticsearch branch #95

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 344 commits into from
Closed

feat: elasticsearch branch #95

wants to merge 344 commits into from

Conversation

wphyojpl
Copy link
Collaborator

@wphyojpl wphyojpl commented Oct 20, 2022

Error while running terraform

Error: Error putting IAM role policy am-uds-dev-cumulus_lambda_processing_role_get_secrets_policy: AccessDenied: User: arn:aws:sts::884500545225:assumed-role/power_user/wai.phyo@jpl.nasa.gov is not authorized to perform: iam:PutRolePolicy on resource: role am-uds-dev-cumulus-lambda-processing with an explicit deny in an identity-based policy
        status code: 403, request id: f452538a-92cb-4008-99b3-a5a4b5b2a5c7


wphyojpl and others added 30 commits July 28, 2022 11:20
fix: l1A granule id is `<collection-id>___<collection-version>:<granule-id>` not to duplicate re-runs
* fix: add err msg when next & prev url does not generate + update makefile for local

* feat: update to STAC specific schema ref from CMR

* feat: make pagination class
… + update collection STAC schema update (#71)

* breaking: testing with pystac + make changes accordingly

* chore: convert manual test to unittest case

* fix: update test case

* fix: update upload granule to match new pystac collection schema

* chore: stashing collection transformer to use pystac (in-progress)

* chore: remove unused file

* chore: update gitignore file

* fix: update test cases
* chore: add open_api doc

* fix: update open api yaml errors

* chore: add test yaml

* chore: add test yaml

* chore: add test yaml

* chore: add test yaml

* feat: add other venue urls
* fix: disable building upon pull request

* chore: bump version
* fix: loop all files before returning metadata file

* chore: bump version
* fix: not assigning limit and offset from api endpoint

* chore: bump version
chore: update develop to latest master
* chore: move the unit test

* feat: add create collection for cumulus

* feat: adding stac to cumulus for json

* feat: finished transforming from stac to cumulus

* fix: try creating collection with min requirement

* feat: add class to create dapa collection obj

* fix: add class to create dapa collection object for unity

* feat: add class to create STAC collection

* feat: update to python3.9 + new lambda for collection creation

* chore: python3.9 for github action

* fix: ci/cd on develop branch with additional release postfix

* fix: testing different output number

* fix: testing different output number

* fix: testing different output number

* fix: testing different output number

* fix: testing different output number

* fix: testing different output number

* fix: testing different output number

* fix: testing different output number

* fix: testing different output number

* chore: add testcase.. in progress

* fix: wrong handler for new lambda

* fix: require libraries at the worng location

* fix: add mock url if url_path is misisng

* feat: add rule when creating a collection

* fix: wrong quotes in terraform

* fix: use thread to run the actual collection creation in background thread

* fix: revert background process. does not work as expected in lambda

* fix: addig facade for collection creation

* fix: api gateway wants string for body. not dict

* fix: no need to hash the dict

* feat: add single collection details

* feat: add single collection detail logic

* fix: update testcase + yaml for api gateway

* feat: delete collection if rule creation fails

* chore: update version
* feat: add completed status on granules

* fix: check start & end datetime before pushing add them to stac + defaulted it to unix 0
wphyojpl and others added 26 commits August 7, 2023 14:05
* fix: retry 5 times + wait 45 sec for DAAC

* feat: make re-try configurable

* feat: adding fastapi (in-progress)

* fix: get collection creation working (in-progress)

* feat: add collection folder in s3 upload

* fix: update note

* fix: update wrong PR number

* fix: update missed file during merge

* fix: remove old test case
…uck when there are large number of files (#194)

* fix: retry 5 times + wait 45 sec for DAAC

* feat: make re-try configurable

* feat: adding fastapi (in-progress)

* fix: get collection creation working (in-progress)

* feat: default poll checking to 0 times + conflict bug in dapa client

* feat: add facade endpoint + real endpoint for cnm

* fix: update version

* fix: need to call PUT. not POST

* fix: update test case

* chore: add large data catalog testcase

* fix: use Manager().Q instead of normal Q to fix it for download stuck for large granules

* chore: adding support files for test

* fix: calling catalog api in batch

* chore: update changelog

* chore: add missing test file

* fix: mismatch version
@ngachung
Copy link
Collaborator

Changes from this PR is included in #201. Will merge #201 when ready.

@ngachung ngachung closed this Sep 27, 2023
@ngachung ngachung mentioned this pull request Oct 16, 2023
@wphyojpl wphyojpl deleted the elasticsearch.lib branch November 8, 2023 17:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants