Change the way TAXII ingester paginates requests

[yassine-ouaamou]

Proposed changes

Extracted from https://docs.oasis-open.org/cti/taxii/v2.1/os/taxii-v2.1-os.html#_Toc31107519

• If the more property is set to true and the next property is populated then the client can paginate through the remaining records using the next URL parameter along with the same original query options.

• If the more property is set to true and the next property is empty then the client may paginate through the remaining records by using the added_after URL parameter with the date/time value from the X-TAXII-Date-Added-Last header along with the same original query options.

Related issues

• Problem in the way the TAXII ingester paginates requests #5681

Checklist

• I consider the submitted work as finished
• I tested the code for its functionality
• I wrote test cases for the relevant uses case
• I added/update the relevant documentation (either on github or on notion)
• Where necessary I refactored code to improve the overall quality

Further comments

If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc...

[codecov]

Codecov Report

Attention: 4 lines in your changes are missing coverage. Please review.

Comparison is base (d32bf29) 66.10% compared to head (7a01c29) 66.10%.
Report is 5 commits behind head on master.

Files	Patch %	Lines
...rm/opencti-graphql/src/manager/ingestionManager.ts	42.85%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #5691      +/-   ##
==========================================
- Coverage   66.10%   66.10%   -0.01%     
==========================================
  Files         513      513              
  Lines       60771    60776       +5     
  Branches     4422     4422              
==========================================
+ Hits        40174    40175       +1     
- Misses      20597    20601       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[yassine-ouaamou]

Changed the code as we said @richard-julien . We did some tests with @SouadHadjiat and we were able to retrieve all the pages