Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve notices endpoint #146

Merged
merged 4 commits into from
Jun 13, 2024
Merged

Improve notices endpoint #146

merged 4 commits into from
Jun 13, 2024

Conversation

samhotep
Copy link
Member

@samhotep samhotep commented Jun 12, 2024

Done

Querying notices takes quite a while to complete, ~15s on average. This PR hopes to improve the db query by reworking how the notice json is built.

QA

  • Check out this feature branch
  • Run the site using the command ./run serve
  • View the site locally in your web browser at: http://0.0.0.0:8030/
  • Open /security/cves.json and /security/notices.json
  • Check that they both return in reasonable time

Notes

  • You'll need a database with a significant number of records to replicate the issue (~9000 notices, ~240000 cves)

@samhotep samhotep force-pushed the improve-request-performance branch from 95ac3c1 to 0fd7100 Compare June 12, 2024 13:24
Copy link

codecov bot commented Jun 12, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 85.85%. Comparing base (55311e3) to head (0fd7100).
Report is 10 commits behind head on main.

Current head 0fd7100 differs from pull request most recent head 6b7c018

Please upload reports for the commit 6b7c018 to get more accurate results.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #146      +/-   ##
==========================================
+ Coverage   85.00%   85.85%   +0.85%     
==========================================
  Files          29       29              
  Lines        1480     1570      +90     
==========================================
+ Hits         1258     1348      +90     
  Misses        222      222              
Flag Coverage Δ
python 85.85% <ø> (+0.85%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@samhotep samhotep changed the title Remove notices sorting Deafult notices sorting to ascending Jun 12, 2024
@samhotep samhotep changed the title Deafult notices sorting to ascending Sort notices using default order Jun 12, 2024
@samhotep samhotep changed the title Sort notices using default order Improve notices endpoint Jun 12, 2024
@samhotep samhotep marked this pull request as draft June 12, 2024 20:27
@hloeung
Copy link

hloeung commented Jun 12, 2024

FWIW, there's been a few timeout related issues in the past, some of it being resolved by reducing the query limit. Hopefully this fixes the DB query itself and for good. Here are some:

#129
#101
#59
#31
#20

I'm not sure if we can improve cacheability of these requests as well. This would reduce the backend load and have the fronting content-cache units service requests from their local cache.

@samhotep samhotep force-pushed the improve-request-performance branch from 29aa9ed to 6b7c018 Compare June 13, 2024 12:42
@mtruj013 mtruj013 marked this pull request as ready for review June 13, 2024 12:53
@samhotep samhotep merged commit 54969d3 into main Jun 13, 2024
5 checks passed
@samhotep samhotep deleted the improve-request-performance branch June 13, 2024 15:14
@sudhackar
Copy link

I suppose this is causing canonical/ubuntu.com#13956

@samhotep samhotep restored the improve-request-performance branch June 18, 2024 19:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants