Skip to content

Perf updates #1523

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Sep 3, 2025
Merged

Perf updates #1523

merged 1 commit into from
Sep 3, 2025

Conversation

kfswain
Copy link
Collaborator

@kfswain kfswain commented Sep 3, 2025 •
edited
Loading

Small updates to logging and adding goroutine for function call that has high lock contention, but doesnt impact the request routine, so spinning up into another thread.

These were found during scale testing, can include an extensive write-up if needed, but the tl;dr is:

  • the logging issues were because golang still calls sprintf even if the log level isnt enabled, which was creating CPU hotspots, we are now letting the logging library handle all that under the hood.
  • The indexer.Add func can have a high level of lock contention if all the prefixes are different, so moving the call to its own routine allows requests to continue without being blocked by indexing.

@netlify Netlify
Copy link

netlify bot commented Sep 3, 2025 •
edited
Loading

✅ Deploy Preview for gateway-api-inference-extension ready!

Name Link
🔨 Latest commit 1e14400
🔍 Latest deploy log https://app.netlify.com/projects/gateway-api-inference-extension/deploys/68b8c0ae548a9c00075e6f9e
😎 Deploy Preview https://deploy-preview-1523--gateway-api-inference-extension.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Sep 3, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kfswain

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Sep 3, 2025
liu-cong
liu-cong reviewed Sep 3, 2025
View reviewed changes
Copy link
Contributor

@liu-cong liu-cong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/hold if you want to address the nit

pkg/epp/scheduling/framework/plugins/multi/prefix/plugin.go

p.indexer.Add(state.PrefixHashes, ServerID(targetPod.NamespacedName))
// This function is just adding data, it does not need to block other operations.
go func() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Actually the entire PreRequest can be done async. But this lgtm as well.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack, I'll add a TODO comment so we can figure this out later, I tested with this change, and I'm trying to make this minimally impactful since I plan to cherrypick this into the v1 branch.

You're probably right, but just in case.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated, PTAL

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Sep 3, 2025
@liu-cong
Copy link
Contributor

liu-cong commented Sep 3, 2025

/lgtm

Thanks! This is great!

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 3, 2025
@kfswain
Small updates to logging and adding gorouting for high lock contentio…
1e14400 
…n work, that doesnt need to block the request goroutine
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 3, 2025
@liu-cong
Copy link
Contributor

liu-cong commented Sep 3, 2025

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 3, 2025
@kfswain
Copy link
Collaborator Author

kfswain commented Sep 3, 2025

/unhold

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Sep 3, 2025
@k8s-ci-robot k8s-ci-robot merged commit 01e3627 into kubernetes-sigs:main Sep 3, 2025
12 checks passed
@nirrozenbaum nirrozenbaum mentioned this pull request Sep 4, 2025
Merged
@kfswain kfswain mentioned this pull request Sep 4, 2025
Closed
@nirrozenbaum nirrozenbaum mentioned this pull request Sep 5, 2025
Open
nirrozenbaum pushed a commit that referenced this pull request Sep 5, 2025
@kfswain @nirrozenbaum
Small updates to logging and adding gorouting for high lock contentio…
6023213 
…n work, that doesnt need to block the request goroutine (#1523)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ahg-g ahg-g Awaiting requested review from ahg-g

1 more reviewer

@liu-cong liu-cong liu-cong left review comments

Reviewers whose approvals may not affect merge requirements
Assignees

@liu-cong liu-cong

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@kfswain @k8s-ci-robot @liu-cong