fix: make EPSS behave like other data sources

[terriko]

This will make it so that -d EPSS will actually disable the EPSS data source, and should make it fail more gracefully when the source is not working for any reason.

Note that the EPSS source may not be working correctly even when not disabled; I'll file a separate issue.

• fixes fix: Timeout Error with CVE-Bin-Tool Database Update/Download #4083

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 66.66667% with 10 lines in your changes are missing coverage. Please review.

Project coverage is 80.79%. Comparing base (d6cbe40) to head (6cdd3be).
Report is 232 commits behind head on main.

Files	Patch %	Lines
cve_bin_tool/cvedb.py	66.66%	2 Missing and 3 partials ⚠️
cve_bin_tool/data_sources/epss_source.py	72.72%	3 Missing ⚠️
cve_bin_tool/cli.py	66.66%	0 Missing and 1 partial ⚠️
cve_bin_tool/data_sources/nvd_source.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4125      +/-   ##
==========================================
+ Coverage   75.41%   80.79%   +5.38%     
==========================================
  Files         808      820      +12     
  Lines       11983    12544     +561     
  Branches     1598     1702     +104     
==========================================
+ Hits         9037    10135    +1098     
+ Misses       2593     1973     -620     
- Partials      353      436      +83     

Flag	Coverage Δ
longtests	80.79% <66.66%> (+5.38%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[terriko]

@anthonyharrison and @Rexbeast2 do you have time to review this one? I can grab a colleague to review but I'd like someone who knows the EPSS code better if possible.

I'm working on fixing the "build wheel: test in another PR so you can ignore that failure. (It shouldn't be running on pull requests and needs to be fixed.)

[anthonyharrison]

Don't we still need a call to self.populate_epss() in the loop which stores the EPSS data?

[terriko]

It should be replaced by the default call to epss_source.get_cve_data() (which does the same thing: calls up update_epss(), only without the apparently extraneous database open because we weren't actually storing anything in the db in update_epss() even before?). We're populating cve_data instead of epss_data (which I should have removed) but then we shove it into the storage the same way.

[terriko]

Note to self: rebased against my branch with the test/ci fixes so hopefully all tests will pass this time.

[terriko]

I'm going to rebase this now that the CI fixes are in. I think I'd like to add a test to make sure disabling works, but I need to figure out how to do that without having to run basically cve-bin-tool -u now -d EPSS and wait for a full database refresh the way I did while testing it myself. Maybe I can combine it with a local data load? Or integrate it into an existing test that does the reload already?

It would also be nice to have some code to test the epss loading code I changed in cvedb.py, but again, I'm going to have to figure out how to hook into existing tests so that I'm not re-loading the whole database for a single test.

Not sure if these tests will make it into this PR, but I'll do a bit of reconnaissance to see how feasible they are and report back before asking for new reviews.

[terriko]

Rebase is done, will likely try for some tests tomorrow.

[alext-w]

@terriko, I'm not sure if it fits this PR (let me know if not, I can submit a separate issue or probably a PR straight away due to triviality), but while you're at it, would it be possible to add the trust_env=True piece for aiohttp sessions created for EPSS same as #923 did for cvedb.py? Otherwise it does time out for me behind a proxy trying to connect directly and adding that piece fixes it.

[terriko]

@alext-w Separate issue/pr please!

[terriko]

Okay, assuming that the long tests pass, I believe I've resolved the merge issues and other things that needed improvement here and this should be ready for re-review @anthonyharrison