tenable_io: fix flakey test and timestamp handling

[efd6]

Proposed commit message

The comparison for determining whether the pagination had completed was based
on the response's size and the batch size (via state.batch_size, but could
also have been the response's params.size without difference in behaviour).
This was not correct since the size value is the current response's result
set and so may match the batch size even when the pagination is complete, in
the case that total size mod batch size is zero. Instead keep a running tally
of records returned throughout the pagination and compare this to the total
count reported by the API. Also fix the test case so that the response
matches the API docs.[1]

The timestamps were being examined for their maximum by string comparison.
This should approximately always work if the timestamp is formatted as
RFC3339, but the timestamps are rendered as variable precision RFC3339, so
make sure that the comparisons work by converting to timestamp values to
obtain the maximum.

[1]https://developer.tenable.com/reference/io-plugins-list

I don't now how it was working at all.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

• [ ]

How to test this PR locally

Related issues

• Fixes [Stack 8.16.0-SNAPSHOT] [tenable_io] Failing test daily: system test: (elastic-agent logs - default) in tenable_io.plugin #10626
• Fixes [Stack 8.15.0-SNAPSHOT] [tenable_io] Failing test daily: system test: (elastic-agent logs) in tenable_io.plugin #10370

Screenshots

[elasticmachine]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: ed82660

History

• 💔 Build #15216 failed 62867fb

cc @efd6

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
20.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[elasticmachine]

Package tenable_io - 3.2.1 containing this change is available at https://epr.elastic.co/search?package=tenable_io