Add APIs for the instantiated plugins to the EPP Handle #1039

shmuelk · 2025-06-22T08:25:31Z

This PR adds a set of APIs for managing the instantiated plugins in the EPP. These APIs include:

Plugin - returns the named plugin instance
AddPlugin - Adds a plugin to the set of known plugin instances
GetAllPlugins - Returns all of the known plugins
GetAllPluginsWithNames - Returns all of the known plugins with their names

This PR is the first of two to solve issue #1011 which requires referencing other instantiated plugins.

netlify · 2025-06-22T08:25:35Z

✅ Deploy Preview for gateway-api-inference-extension ready!

Name	Link
🔨 Latest commit	`846a7cf`
🔍 Latest deploy log	https://app.netlify.com/projects/gateway-api-inference-extension/deploys/685a5315b5b6cb0008197ca5
😎 Deploy Preview	https://deploy-preview-1039--gateway-api-inference-extension.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

k8s-ci-robot · 2025-06-22T08:25:40Z

Hi @shmuelk. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

nirrozenbaum · 2025-06-22T18:26:22Z

pkg/epp/scheduling/scheduler_config.go

 }

-func LoadSchedulerConfig(configProfiles []v1alpha1.SchedulingProfile, references map[string]plugins.Plugin) (*SchedulerConfig, error) {
+func LoadSchedulerConfig(configProfiles []v1alpha1.SchedulingProfile, handle plugins.Handle) (*SchedulerConfig, error) {


ideally, I think it would be better if this function (and same one in requestcontrol config) gets out of this package and into a dedicated package for file configuration parsing.
this function is not part of the struct and is a free function. I think it would be better to define a configurationParser struct with receiver functions like LoadSchedulerConfig.

skipped this comment in previous PR cause it was too big and we had too many comments on it..

All of the functions in this file are static functions (without a receiver). Why is this so different from NewSchedulerConfig? This just an additional way to create a SchedulerConfig.

there are no functions other than NewSchedulerConfig :).
the main point I was trying to make is that as we make progress and add more extension points (e.g., in flow control) the code to iterate over the plugins and call some "AddPlugins" or similar function will be duplicated across different places. instead, I think we should have a single package and struct where we handle the file parsing and creation of the relevant configs from it.
this is a matter of decoupling between scheduler package and the configuration parsing.
IMO the scheduler or any other layer shouldn't care about the configuration file and it should get from outside its configuration. Additionally, scheduling package shouldn't have unit-tests to test configuration file parsing and validity.

All of the configuration loading code has been moved to pkg/epp/common/config/loader

nirrozenbaum · 2025-06-22T18:27:55Z

pkg/epp/scheduling/scheduler_config_test.go

 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/multi/prefix"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/picker"
 	"sigs.k8s.io/gateway-api-inference-extension/pkg/epp/scheduling/framework/plugins/profile"
+	"sigs.k8s.io/gateway-api-inference-extension/test/utils"


ditto about this file.
nothing in this file is testing SchedulerConfig. this is testing the configuration file parsing and the logic around it.

This file has been merged into config_loader_test.go in the new loader package.

pkg/epp/plugins/plugins.go

nirrozenbaum · 2025-06-23T18:07:38Z

/ok-to-test

ahg-g · 2025-06-23T18:48:40Z

/lgtm

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

nirrozenbaum · 2025-06-24T07:55:56Z

/lgtm
/approve

Thanks!

k8s-ci-robot · 2025-06-24T07:56:04Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: nirrozenbaum, shmuelk

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [nirrozenbaum]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

…igs#1039) * Added plugin instance APIs to plugins.Handle Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * An implementation of the new plugins.Handle APIs Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Moved all configuration loading code to new package Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Updates due to new and moved APIs Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Cleanup of old configuration loading code Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> --------- Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Add APIs for the instantiated plugins to the EPP Handle (kubernetes-sigs#1039) * Added plugin instance APIs to plugins.Handle Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * An implementation of the new plugins.Handle APIs Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Moved all configuration loading code to new package Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Updates due to new and moved APIs Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Cleanup of old configuration loading code Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> --------- Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> chore(deps): bump the kubernetes group with 6 updates (kubernetes-sigs#1050) Bumps the kubernetes group with 6 updates: | Package | From | To | | --- | --- | --- | | [k8s.io/api](https://github.com/kubernetes/api) | `0.33.1` | `0.33.2` | | [k8s.io/apiextensions-apiserver](https://github.com/kubernetes/apiextensions-apiserver) | `0.33.1` | `0.33.2` | | [k8s.io/apimachinery](https://github.com/kubernetes/apimachinery) | `0.33.1` | `0.33.2` | | [k8s.io/client-go](https://github.com/kubernetes/client-go) | `0.33.1` | `0.33.2` | | [k8s.io/code-generator](https://github.com/kubernetes/code-generator) | `0.33.1` | `0.33.2` | | [k8s.io/component-base](https://github.com/kubernetes/component-base) | `0.33.1` | `0.33.2` | Updates `k8s.io/api` from 0.33.1 to 0.33.2 - [Commits](kubernetes/api@v0.33.1...v0.33.2) Updates `k8s.io/apiextensions-apiserver` from 0.33.1 to 0.33.2 - [Release notes](https://github.com/kubernetes/apiextensions-apiserver/releases) - [Commits](kubernetes/apiextensions-apiserver@v0.33.1...v0.33.2) Updates `k8s.io/apimachinery` from 0.33.1 to 0.33.2 - [Commits](kubernetes/apimachinery@v0.33.1...v0.33.2) Updates `k8s.io/client-go` from 0.33.1 to 0.33.2 - [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md) - [Commits](kubernetes/client-go@v0.33.1...v0.33.2) Updates `k8s.io/code-generator` from 0.33.1 to 0.33.2 - [Commits](kubernetes/code-generator@v0.33.1...v0.33.2) Updates `k8s.io/component-base` from 0.33.1 to 0.33.2 - [Commits](kubernetes/component-base@v0.33.1...v0.33.2) --- updated-dependencies: - dependency-name: k8s.io/api dependency-version: 0.33.2 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: kubernetes - dependency-name: k8s.io/apiextensions-apiserver dependency-version: 0.33.2 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: kubernetes - dependency-name: k8s.io/apimachinery dependency-version: 0.33.2 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: kubernetes - dependency-name: k8s.io/client-go dependency-version: 0.33.2 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: kubernetes - dependency-name: k8s.io/code-generator dependency-version: 0.33.2 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: kubernetes - dependency-name: k8s.io/component-base dependency-version: 0.33.2 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: kubernetes ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> remove datastore dependency from the scheduler (kubernetes-sigs#1049) * remove datastore dependency from the scheduler Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * added back comments on snapshotting pods from datastore before calling schedule Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * removed fake datastore from conformance scheduler test Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> --------- Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> Add subsetting logic for epp (kubernetes-sigs#981) feat: Added a factory function for the DecisionTree filter (kubernetes-sigs#1053) * Added a factory function for the DecisionTreeFilter Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Added tests of the factory function of the DecisionTreeFilter Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Registered the factory function of the DecisionTreeFilter Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Refactored the configuration loading Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> --------- Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> Adding pprof endpoints to metrics port (kubernetes-sigs#1069) feat: Add a context.Context to the plugins.HAndle interface (kubernetes-sigs#1076) * Added a context.Context to the plugins.Handle interface Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Changes due to changes in internal APIs Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Changes to tests due to changes in internal APIs Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> --------- Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> convert subset filter from a plugin to logic in director (kubernetes-sigs#1088) * convert subset filter from a plugin to logic in director Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * replace interface{} with any Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * make linter happy Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * address code review comments Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> --------- Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> chore(deps): bump golang.org/x/sync from 0.14.0 to 0.15.0 (kubernetes-sigs#1096) Bumps [golang.org/x/sync](https://github.com/golang/sync) from 0.14.0 to 0.15.0. - [Commits](golang/sync@v0.14.0...v0.15.0) --- updated-dependencies: - dependency-name: golang.org/x/sync dependency-version: 0.15.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Introduce plugins.TypedName to be used for Plugin base implementation (kubernetes-sigs#1086) * introduce TypedName to reduce boilerplate, modify plugins Signed-off-by: Etai Lev Ran <elevran@gmail.com> * implement GetTypedName() Signed-off-by: Etai Lev Ran <elevran@gmail.com> * Remove Type() and Name() from Plugin interface Signed-off-by: Etai Lev Ran <elevran@gmail.com> * use TypedName as private field, not embedded Signed-off-by: Etai Lev Ran <elevran@gmail.com> --------- Signed-off-by: Etai Lev Ran <elevran@gmail.com> move the conversion from pod metrics to scheduler pod representation one level up (kubernetes-sigs#1104) * move the converstion from pod metrics to scheduler pod representation one level up Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * minor change in helper func Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> --------- Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> handle picking multiple destinations in scheduling layer (kubernetes-sigs#1059) * implement multiple destination as the output of the scheduler Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * updated max score picker unit tests to cover multiple pods Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * imports Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * unit-test fix Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> --------- Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> refactor: 🔨 use the more explicit singular form (kubernetes-sigs#1129)

k8s-ci-robot requested review from ahg-g and robscott June 22, 2025 08:25

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jun 22, 2025

k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 22, 2025

nirrozenbaum reviewed Jun 22, 2025

View reviewed changes

pkg/epp/plugins/plugins.go Show resolved Hide resolved

k8s-ci-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 23, 2025

shmuelk closed this Jun 23, 2025

shmuelk force-pushed the epp-handle-1 branch from b91fa6d to 5d4d3ec Compare June 23, 2025 15:05

k8s-ci-robot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jun 23, 2025

shmuelk reopened this Jun 23, 2025

k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jun 23, 2025

k8s-ci-robot assigned ahg-g Jun 23, 2025

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 23, 2025

k8s-ci-robot assigned nirrozenbaum Jun 23, 2025

k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jun 24, 2025

shmuelk added 5 commits June 24, 2025 10:22

Added plugin instance APIs to plugins.Handle

26eab51

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

An implementation of the new plugins.Handle APIs

df96c24

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Moved all configuration loading code to new package

ce1dea2

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Updates due to new and moved APIs

0da7c51

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Cleanup of old configuration loading code

846a7cf

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

shmuelk force-pushed the epp-handle-1 branch from 287dcc4 to 846a7cf Compare June 24, 2025 07:26

k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jun 24, 2025

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 24, 2025

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 24, 2025

k8s-ci-robot merged commit 4485f63 into kubernetes-sigs:main Jun 24, 2025
9 checks passed

shmuelk mentioned this pull request Jun 24, 2025

feat: Added a factory function for the DecisionTree filter #1053

Merged

shmuelk deleted the epp-handle-1 branch August 4, 2025 13:56

shmuelk mentioned this pull request Sep 25, 2025

REQUEST: New membership for shmuelk kubernetes/org#5864

Closed

11 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add APIs for the instantiated plugins to the EPP Handle #1039

Add APIs for the instantiated plugins to the EPP Handle #1039

Uh oh!

shmuelk commented Jun 22, 2025

Uh oh!

netlify bot commented Jun 22, 2025 •

edited

Loading

Uh oh!

k8s-ci-robot commented Jun 22, 2025

Uh oh!

nirrozenbaum Jun 22, 2025 •

edited

Loading

Uh oh!

shmuelk Jun 23, 2025

Uh oh!

nirrozenbaum Jun 23, 2025

Uh oh!

shmuelk Jun 23, 2025

Uh oh!

nirrozenbaum Jun 22, 2025

Uh oh!

shmuelk Jun 23, 2025

Uh oh!

Uh oh!

nirrozenbaum commented Jun 23, 2025

Uh oh!

ahg-g commented Jun 23, 2025

Uh oh!

nirrozenbaum commented Jun 24, 2025

Uh oh!

k8s-ci-robot commented Jun 24, 2025

Uh oh!

Uh oh!

Uh oh!

Add APIs for the instantiated plugins to the EPP Handle #1039

Add APIs for the instantiated plugins to the EPP Handle #1039

Uh oh!

Conversation

shmuelk commented Jun 22, 2025

Uh oh!

netlify bot commented Jun 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for gateway-api-inference-extension ready!

Uh oh!

k8s-ci-robot commented Jun 22, 2025

Uh oh!

nirrozenbaum Jun 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shmuelk Jun 23, 2025

Choose a reason for hiding this comment

Uh oh!

nirrozenbaum Jun 23, 2025

Choose a reason for hiding this comment

Uh oh!

shmuelk Jun 23, 2025

Choose a reason for hiding this comment

Uh oh!

nirrozenbaum Jun 22, 2025

Choose a reason for hiding this comment

Uh oh!

shmuelk Jun 23, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nirrozenbaum commented Jun 23, 2025

Uh oh!

ahg-g commented Jun 23, 2025

Uh oh!

nirrozenbaum commented Jun 24, 2025

Uh oh!

k8s-ci-robot commented Jun 24, 2025

Uh oh!

Uh oh!

Uh oh!

netlify bot commented Jun 22, 2025 •

edited

Loading

nirrozenbaum Jun 22, 2025 •

edited

Loading