Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(etls): add interreg 2014TC16RFCB014 - EUBFR-258 #203

Merged
merged 7 commits into from
Apr 24, 2019

Conversation

kalinchernev
Copy link
Contributor

PR description

Input file is in CSV format, however it's necessary to aggregate rows before they go to harmonized storage and value store (respectively), thus read and write are not streamed in this ETL.

QA Checklist

When you add a new ETL/producer, please check for the following:

  • Ensure the ETL has been correctly setup following the guide in services/ingestion/etl/README.md
  • There is at least 1 sample record.json and 1 unit test with a jest snapshot for the result of the transform function of the ETL.
  • Ensure there is (flow/jsdocs) documentation in the transform function
  • Generate the necessary documentation pages for the new ETL by yarn docs:md

Copy link
Contributor

@yhuard yhuard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deployed and tested, all 97 projects have been imported ✅

Too bad we get a streamable file (CSV) but can't actually stream it (recordsAgg). I don't see any better solution for now.

@yhuard yhuard removed their assignment Apr 24, 2019
@kalinchernev kalinchernev merged commit 731057f into master Apr 24, 2019
@kalinchernev kalinchernev deleted the feat/etl-interreg-2014TC16RFCB014-EUBFR-258 branch April 24, 2019 13:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants