Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Populate/update cache on ClusterCatalog reconcile #1284

Merged

Conversation

m1kola
Copy link
Member

@m1kola m1kola commented Sep 19, 2024

Description

Currently we fetch catalog data and populate cache on demand during ClusterExtension reconciliation. This works but the first reconciliation after ClusterCatalog creation or update is slow due to the need to fetch data.

With this change we proactively populate cache on ClusterCatalog creation and check if cache needs to be updated on ClusterCatalog update.

Reviewer Checklist

  • API Go Documentation
  • Tests: Unit Tests (and E2E Tests, if appropriate)
  • Comprehensive Commit Messages
  • Links to related GitHub Issue(s)

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 19, 2024
Copy link

netlify bot commented Sep 19, 2024

Deploy Preview for olmv1 ready!

Name Link
🔨 Latest commit 2ee4aa4
🔍 Latest deploy log https://app.netlify.com/sites/olmv1/deploys/6724a49a2282d8000812fd8f
😎 Deploy Preview https://deploy-preview-1284--olmv1.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Copy link

codecov bot commented Sep 19, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 73.45%. Comparing base (cd0b096) to head (2ee4aa4).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1284      +/-   ##
==========================================
+ Coverage   73.30%   73.45%   +0.15%     
==========================================
  Files          42       42              
  Lines        3057     3063       +6     
==========================================
+ Hits         2241     2250       +9     
+ Misses        643      640       -3     
  Partials      173      173              
Flag Coverage Δ
e2e 55.33% <62.06%> (-0.08%) ⬇️
unit 52.59% <86.20%> (+0.38%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 22, 2024
@m1kola m1kola force-pushed the populate-cache-on-from-controller branch from 28b3312 to 72e1586 Compare September 23, 2024 14:43
@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 23, 2024
@m1kola m1kola force-pushed the populate-cache-on-from-controller branch from 72e1586 to 4b76985 Compare September 23, 2024 14:45
@m1kola m1kola force-pushed the populate-cache-on-from-controller branch 5 times, most recently from 80763b1 to 482531d Compare October 10, 2024 13:53
@m1kola m1kola marked this pull request as ready for review October 10, 2024 14:01
@m1kola m1kola requested a review from a team as a code owner October 10, 2024 14:01
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 10, 2024

catalogFsys, err := r.CatalogCache.Get(existingCatalog.Name, existingCatalog.Status.ResolvedSource.Image.Ref)
if err != nil {
return ctrl.Result{}, fmt.Errorf("error retrieving cache for catalog %q: %v", existingCatalog.Name, err)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WDYT about propagating this error into the cache so that cache clients have more context about why the content is not available?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@joelanford it is currently implemented how we discussed here: #1284 (comment)

Cache client caches content fetch errors and r.CatalogCache.Get(...) will return fetch error to the clients. So here we will have something like error retrieving cache for catalog "my-catalog": network request timeout or other error from actual network call

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see now. I didn't realize most of the plumbing had already been done! Beautiful!

Currently we fetch catalog data and populate
cache on demand during ClusterExtension reconciliation.
This works but the first reconciliation after ClusterCatalog
creation or update is slow due to the need to fetch data.

With this change we proactively populate cache on ClusterCatalog
creation and check if cache needs to be updated
on ClusterCatalog update.

Signed-off-by: Mikalai Radchuk <mradchuk@redhat.com>
@m1kola m1kola force-pushed the populate-cache-on-from-controller branch from 482531d to 2ee4aa4 Compare November 1, 2024 09:51
@m1kola
Copy link
Member Author

m1kola commented Nov 1, 2024

Updated the error wording. @joelanford please take another look.

@joelanford joelanford added this pull request to the merge queue Nov 1, 2024
Merged via the queue into operator-framework:main with commit a0f12e6 Nov 1, 2024
18 checks passed
@m1kola m1kola deleted the populate-cache-on-from-controller branch November 1, 2024 12:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants