Move filter and scorer plugins registration to a separate file #729

mayabar · 2025-04-23T08:23:20Z

fix issue [#718]

linux-foundation-easycla · 2025-04-23T08:23:25Z

The committers listed above are authorized under a signed CLA.

✅ login: mayabar / name: Maya Barnea (66c4cc2, e6b29fc, b4c6630, 7f3b120)

k8s-ci-robot · 2025-04-23T08:23:29Z

Welcome @mayabar!

It looks like this is your first PR to kubernetes-sigs/gateway-api-inference-extension 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/gateway-api-inference-extension has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

k8s-ci-robot · 2025-04-23T08:23:30Z

Hi @mayabar. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

netlify · 2025-04-23T08:23:37Z

✅ Deploy Preview for gateway-api-inference-extension ready!

Name	Link
🔨 Latest commit	`66c4cc2`
🔍 Latest deploy log	https://app.netlify.com/sites/gateway-api-inference-extension/deploys/680a3b8d164e9d0008bf0dd8
😎 Deploy Preview	https://deploy-preview-729--gateway-api-inference-extension.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

mayabar · 2025-04-23T09:11:00Z

/assign @nirrozenbaum

nirrozenbaum · 2025-04-23T09:27:54Z

/ok-to-test

kfswain · 2025-04-23T17:34:19Z

Looks like a noop PR & heremetic tests pass so the filters should be WAI, won't ask for benchmarking on this PR.
/lgtm
/approve

k8s-ci-robot · 2025-04-23T17:34:26Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kfswain, mayabar

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [kfswain]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

pkg/epp/scheduling/default_config.go

pkg/epp/scheduling/scheduler.go

nirrozenbaum · 2025-04-24T13:08:24Z

pkg/epp/scheduling/scheduler_test.go

 	for _, test := range tests {
 		t.Run(test.name, func(t *testing.T) {
-			scheduler := NewScheduler(&fakeDataStore{pods: test.input})
+			scheduler := NewSchedulerWithConfig(&fakeDataStore{pods: test.input}, schedConfig)


seems like schedulerConfig here is the default, you can init with NewScheduler :)

nirrozenbaum · 2025-04-24T13:09:08Z

overall looks awesome!
left just a few very minor comments

Signed-off-by: Maya Barnea <mayab@il.ibm.com>

nirrozenbaum · 2025-04-24T15:47:34Z

/lgtm

…netes-sigs#729) * Move filters and scorers registration to filter/scorer specific files * Default scheduler config contains empty list of scorers Signed-off-by: Maya Barnea <mayab@il.ibm.com> * Default plugin is not a scorer any more Signed-off-by: Maya Barnea <mayab@il.ibm.com> * fix scheduler test + lint comments Signed-off-by: Maya Barnea <mayab@il.ibm.com> --------- Signed-off-by: Maya Barnea <mayab@il.ibm.com>

* Add unit test coverage for pod APIs under datastore/pkg * Add unit test coverage for pod APIs under datastore/pkg * Add unit test coverage for pod APIs under datastore/pkg * Add unit test coverage for pod APIs under datastore/pkg * EPP Architecture proposal (#683) * initial changes * Adding to proposal to give a quick barebones definition to refactor * feedback changes * more feedback addressing * removed unused Fake struct (#723) Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * epp: return correct response for trailers (#726) This looks like a copy paste error. * Refactor scheduler to run plugins (#677) * Refactor scheduler to run plugins * Add scheduler plugin latency metric * Address comments * Address comments * Complete the InferencePool documentation (#673) * Initial guide for inference pool * Add extensionReference to the InferencePool spec * Fix list formatting * Remove unused labels * Autogenerate the spec * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <rob.scott87@gmail.com> * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <rob.scott87@gmail.com> * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <rob.scott87@gmail.com> * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <rob.scott87@gmail.com> * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <rob.scott87@gmail.com> * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <rob.scott87@gmail.com> * Rename llm-pool names in rollout example * Add use cases for replacing an inference pool * Rewording the background section * Create replacing-inference-pool.md * Replace instructions with a link for how to replace an inference pool * Update replacing-inference-pool.md * Update mkdocs.yml * Update replacing-inference-pool.md * Update inferencemodel_types.go * Update inferencepool.md * Update site-src/guides/replacing-inference-pool.md Co-authored-by: Rob Scott <rob.scott87@gmail.com> --------- Co-authored-by: Rob Scott <rob.scott87@gmail.com> * reduce log level in metrics logger not to trash the log (#708) * reduce log level in metrics logger not to trash the log Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * rename flush metrics to refresh metrics Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * revert log level Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> --------- Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * few updates in datastore (#713) * few updates in datastore Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * PoolSet documentation Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * error phrasing Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * removed unused pool arg from PodUpdateOrAddIfNotExist Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * linter Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> --------- Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * scheduler refactoring (#730) Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * filter irrelevant pod in pod_reconciler (#696) * EPP: Update GetRandomPod() to return nil if no pods exist (#731) Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io> * Move filter and scorer plugins registration to a separate file (#729) * Move filters and scorers registration to filter/scorer specific files * Default scheduler config contains empty list of scorers Signed-off-by: Maya Barnea <mayab@il.ibm.com> * Default plugin is not a scorer any more Signed-off-by: Maya Barnea <mayab@il.ibm.com> * fix scheduler test + lint comments Signed-off-by: Maya Barnea <mayab@il.ibm.com> --------- Signed-off-by: Maya Barnea <mayab@il.ibm.com> * Update issue templates (#738) * Update issue templates * Updates artifacts for v0.3.0-rc.1 release Signed-off-by: Kellen Swain <kfswain@google.com> * Updates bbr chart for v0.3.0-rc.1 release Signed-off-by: Kellen Swain <kfswain@google.com> * Updates artifacts for v0.3.0 release Signed-off-by: Kellen Swain <kfswain@google.com> * Adding blank issue template so that all issues start with label --------- Signed-off-by: Kellen Swain <kfswain@google.com> * Add unit test coverage for pod APIs under datastore/pkg * few updates in datastore (#713) * few updates in datastore Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * PoolSet documentation Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * error phrasing Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * removed unused pool arg from PodUpdateOrAddIfNotExist Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * linter Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> --------- Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * few updates in datastore (#713) * few updates in datastore Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * PoolSet documentation Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * error phrasing Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * removed unused pool arg from PodUpdateOrAddIfNotExist Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * linter Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> --------- Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * Add unit test coverage for pod APIs under datastore/pkg --------- Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io> Signed-off-by: Maya Barnea <mayab@il.ibm.com> Signed-off-by: Kellen Swain <kfswain@google.com> Co-authored-by: Kellen Swain <kfswain@google.com> Co-authored-by: Nir Rozenbaum <nirro@il.ibm.com> Co-authored-by: John Howard <john.howard@solo.io> Co-authored-by: Cong Liu <conliu@google.com> Co-authored-by: Nicole Xin <nxin@google.com> Co-authored-by: Rob Scott <rob.scott87@gmail.com> Co-authored-by: nayihz <smartczy@outlook.com> Co-authored-by: Daneyon Hansen <daneyon.hansen@solo.io> Co-authored-by: Maya Barnea <mayab@il.ibm.com>

…netes-sigs#729) * Move filters and scorers registration to filter/scorer specific files * Default scheduler config contains empty list of scorers Signed-off-by: Maya Barnea <mayab@il.ibm.com> * Default plugin is not a scorer any more Signed-off-by: Maya Barnea <mayab@il.ibm.com> * fix scheduler test + lint comments Signed-off-by: Maya Barnea <mayab@il.ibm.com> --------- Signed-off-by: Maya Barnea <mayab@il.ibm.com>

* Add unit test coverage for pod APIs under datastore/pkg * Add unit test coverage for pod APIs under datastore/pkg * Add unit test coverage for pod APIs under datastore/pkg * Add unit test coverage for pod APIs under datastore/pkg * EPP Architecture proposal (kubernetes-sigs#683) * initial changes * Adding to proposal to give a quick barebones definition to refactor * feedback changes * more feedback addressing * removed unused Fake struct (kubernetes-sigs#723) Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * epp: return correct response for trailers (kubernetes-sigs#726) This looks like a copy paste error. * Refactor scheduler to run plugins (kubernetes-sigs#677) * Refactor scheduler to run plugins * Add scheduler plugin latency metric * Address comments * Address comments * Complete the InferencePool documentation (kubernetes-sigs#673) * Initial guide for inference pool * Add extensionReference to the InferencePool spec * Fix list formatting * Remove unused labels * Autogenerate the spec * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <rob.scott87@gmail.com> * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <rob.scott87@gmail.com> * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <rob.scott87@gmail.com> * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <rob.scott87@gmail.com> * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <rob.scott87@gmail.com> * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <rob.scott87@gmail.com> * Rename llm-pool names in rollout example * Add use cases for replacing an inference pool * Rewording the background section * Create replacing-inference-pool.md * Replace instructions with a link for how to replace an inference pool * Update replacing-inference-pool.md * Update mkdocs.yml * Update replacing-inference-pool.md * Update inferencemodel_types.go * Update inferencepool.md * Update site-src/guides/replacing-inference-pool.md Co-authored-by: Rob Scott <rob.scott87@gmail.com> --------- Co-authored-by: Rob Scott <rob.scott87@gmail.com> * reduce log level in metrics logger not to trash the log (kubernetes-sigs#708) * reduce log level in metrics logger not to trash the log Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * rename flush metrics to refresh metrics Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * revert log level Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> --------- Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * few updates in datastore (kubernetes-sigs#713) * few updates in datastore Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * PoolSet documentation Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * error phrasing Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * removed unused pool arg from PodUpdateOrAddIfNotExist Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * linter Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> --------- Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * scheduler refactoring (kubernetes-sigs#730) Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * filter irrelevant pod in pod_reconciler (kubernetes-sigs#696) * EPP: Update GetRandomPod() to return nil if no pods exist (kubernetes-sigs#731) Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io> * Move filter and scorer plugins registration to a separate file (kubernetes-sigs#729) * Move filters and scorers registration to filter/scorer specific files * Default scheduler config contains empty list of scorers Signed-off-by: Maya Barnea <mayab@il.ibm.com> * Default plugin is not a scorer any more Signed-off-by: Maya Barnea <mayab@il.ibm.com> * fix scheduler test + lint comments Signed-off-by: Maya Barnea <mayab@il.ibm.com> --------- Signed-off-by: Maya Barnea <mayab@il.ibm.com> * Update issue templates (kubernetes-sigs#738) * Update issue templates * Updates artifacts for v0.3.0-rc.1 release Signed-off-by: Kellen Swain <kfswain@google.com> * Updates bbr chart for v0.3.0-rc.1 release Signed-off-by: Kellen Swain <kfswain@google.com> * Updates artifacts for v0.3.0 release Signed-off-by: Kellen Swain <kfswain@google.com> * Adding blank issue template so that all issues start with label --------- Signed-off-by: Kellen Swain <kfswain@google.com> * Add unit test coverage for pod APIs under datastore/pkg * few updates in datastore (kubernetes-sigs#713) * few updates in datastore Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * PoolSet documentation Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * error phrasing Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * removed unused pool arg from PodUpdateOrAddIfNotExist Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * linter Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> --------- Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * few updates in datastore (kubernetes-sigs#713) * few updates in datastore Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * PoolSet documentation Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * error phrasing Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * removed unused pool arg from PodUpdateOrAddIfNotExist Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * linter Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> --------- Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * Add unit test coverage for pod APIs under datastore/pkg --------- Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io> Signed-off-by: Maya Barnea <mayab@il.ibm.com> Signed-off-by: Kellen Swain <kfswain@google.com> Co-authored-by: Kellen Swain <kfswain@google.com> Co-authored-by: Nir Rozenbaum <nirro@il.ibm.com> Co-authored-by: John Howard <john.howard@solo.io> Co-authored-by: Cong Liu <conliu@google.com> Co-authored-by: Nicole Xin <nxin@google.com> Co-authored-by: Rob Scott <rob.scott87@gmail.com> Co-authored-by: nayihz <smartczy@outlook.com> Co-authored-by: Daneyon Hansen <daneyon.hansen@solo.io> Co-authored-by: Maya Barnea <mayab@il.ibm.com>