Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[backend/frontend] Enable CSV Feed Ingester (#4569) #5404

Merged
merged 45 commits into from
Feb 1, 2024
Merged

Conversation

Goumies
Copy link
Member

@Goumies Goumies commented Jan 5, 2024

Proposed changes

  • Add CSV Ingestion Entity
  • Single-column CSV from URL content / Direct Download CSV from URL content / CSV With # Header
  • Add Creation and Edition forms
  • Test submitted URL and CSV Mapper

Related issues

Checklist

  • I consider the submitted work as finished
  • I tested the code for its functionality
  • I wrote test cases for the relevant uses case
  • I added/update the relevant documentation (either on github or on notion)
  • Where necessary I refactored code to improve the overall quality

Further comments

To test functionally, your opencti-worker python must be started. Then:

Here is some CSV Example that are available to test (single column, with header # , direct download):

@Goumies Goumies marked this pull request as ready for review January 8, 2024 17:20
@SarahBocognano SarahBocognano changed the title [backend / frontend] Enable CSV Feed Ingester [backend/frontend] Enable CSV Feed Ingester (#4569) Jan 9, 2024
Copy link

codecov bot commented Jan 10, 2024

Codecov Report

Attention: 168 lines in your changes are missing coverage. Please review.

Comparison is base (45aabee) 64.76% compared to head (4e93ce5) 64.83%.

Files Patch % Lines
...rm/opencti-graphql/src/manager/ingestionManager.ts 19.23% 84 Missing ⚠️
...phql/src/modules/ingestion/ingestion-csv-domain.ts 64.28% 40 Missing ⚠️
...encti-platform/opencti-graphql/src/types/work.d.ts 0.00% 22 Missing ⚠️
...aphql/src/modules/ingestion/ingestion-converter.ts 15.78% 16 Missing ⚠️
...cti-graphql/src/modules/ingestion/ingestion-csv.ts 90.47% 4 Missing ⚠️
...pencti-platform/opencti-graphql/src/domain/work.js 0.00% 1 Missing ⚠️
...-platform/opencti-graphql/src/parser/csv-parser.ts 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5404      +/-   ##
==========================================
+ Coverage   64.76%   64.83%   +0.06%     
==========================================
  Files         516      521       +5     
  Lines       60973    61355     +382     
  Branches     4871     4904      +33     
==========================================
+ Hits        39489    39779     +290     
- Misses      21484    21576      +92     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@labo-flg

This comment was marked as resolved.

@Goumies

This comment was marked as resolved.

@Goumies Goumies force-pushed the issue/4569 branch 3 times, most recently from caeabd4 to 387d5fa Compare January 29, 2024 15:00
labo-flg

This comment was marked as resolved.

@Goumies Goumies force-pushed the issue/4569 branch 2 times, most recently from 56a54fd to a9dba38 Compare January 31, 2024 10:55
@Goumies Goumies merged commit 8d0da12 into master Feb 1, 2024
8 checks passed
@Goumies Goumies deleted the issue/4569 branch February 1, 2024 14:38
frapuks pushed a commit that referenced this pull request Feb 5, 2024
Co-authored-by: BocognanoSarah <sarah.bocognano@filigran.io>
Co-authored-by: Laurent Bonnet <146674147+labo-flg@users.noreply.github.com>
Archidoit pushed a commit that referenced this pull request Jun 6, 2024
Co-authored-by: BocognanoSarah <sarah.bocognano@filigran.io>
Co-authored-by: Laurent Bonnet <146674147+labo-flg@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
filigran team use to identify PR from the Filigran team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants