Skip to content

Conversation

zetxqx
Copy link
Contributor

@zetxqx zetxqx commented Sep 20, 2025

What type of PR is this?
/kind bug

What this PR does / why we need it:

This is discovered by the Lohi Pipline, where we always use the main helm to install inferencePool and it failed at finding GCPBackendPolicy:
image

Current inferecePool Helm install will return warning and missing the GCPBackendPolicy for logging. The following as an exmple.

❯ export NAMESPACE=inference-demo
export HELM_RELEASE_NAME=infpool-gemma-2b
❯ helm upgrade -i $HELM_RELEASE_NAME \
  config/charts/inferencepool \
  -n $NAMESPACE \
  --create-namespace \
  --set inferencePool.modelServers.matchLabels.app=vllm-gemma2b \
  --set provider.name=gke \
  --set inferenceExtension.monitoring.gke.enabled=true
Release "infpool-gemma-2b" does not exist. Installing it now.
**I0920 00:19:58.644743 1622628 warnings.go:110] "Warning: unknown field \"spec\""**
NAME: infpool-gemma-2b
LAST DEPLOYED: Sat Sep 20 00:19:56 2025
NAMESPACE: inference-demo
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
InferencePool infpool-gemma-2b deployed.

W/ the change in this PR, the warning is gone and GCPBackendPolicy is creating again

❯ helm upgrade -i $HELM_RELEASE_NAME \
  config/charts/inferencepool \
  -n $NAMESPACE \
  --create-namespace \
  --set inferencePool.modelServers.matchLabels.app=vllm-gemma2b \
  --set provider.name=gke \
  --set inferenceExtension.monitoring.gke.enabled=true
Release "infpool-gemma-2b" has been upgraded. Happy Helming!
NAME: infpool-gemma-2b
LAST DEPLOYED: Sat Sep 20 00:39:38 2025
NAMESPACE: inference-demo
STATUS: deployed
REVISION: 6
TEST SUITE: None
NOTES:
InferencePool infpool-gemma-2b deployed.
❯ helm status infpool-gemma-2b -n inference-demo --show-resources
NAME: infpool-gemma-2b
LAST DEPLOYED: Sat Sep 20 00:39:38 2025
NAMESPACE: inference-demo
STATUS: deployed
REVISION: 6
RESOURCES:
==> v1/Secret
NAME                                     TYPE                                  DATA   AGE
infpool-gemma-2b-metrics-reader-secret   kubernetes.io/service-account-token   3      19m

==> v1/ConfigMap
NAME                   DATA   AGE
infpool-gemma-2b-epp   1      19m

==> v1/Role
NAME                                          CREATED AT
infpool-gemma-2b-metrics-reader-secret-read   2025-09-20T00:19:58Z
infpool-gemma-2b-epp   2025-09-20T00:19:58Z

==> v1/Service
NAME                   TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
infpool-gemma-2b-epp   ClusterIP   34.118.228.139   <none>        9002/TCP,9090/TCP   19m

==> v1/Deployment
NAME                   READY   UP-TO-DATE   AVAILABLE   AGE
infpool-gemma-2b-epp   1/1     1            1           19m

==> v1/Pod(related)
NAME                                    READY   STATUS    RESTARTS   AGE
infpool-gemma-2b-epp-845d58797f-858v8   1/1     Running   0          19m

==> v1/HealthCheckPolicy
NAME               AGE
infpool-gemma-2b   104s

==> v1/ServiceAccount
NAME                                 SECRETS   AGE
infpool-gemma-2b-metrics-reader-sa   0         19m
infpool-gemma-2b-epp   0     19m

==> v1/ClusterRole
NAME                                             CREATED AT
inference-demo-infpool-gemma-2b-metrics-reader   2025-09-20T00:19:58Z
infpool-gemma-2b-inference-demo-epp   2025-09-20T00:19:58Z

==> v1/ClusterRoleBinding
NAME                                                          ROLE                                                         AGE
inference-demo-infpool-gemma-2b-metrics-reader-role-binding   ClusterRole/inference-demo-infpool-gemma-2b-metrics-reader   19m
infpool-gemma-2b-inference-demo-epp   ClusterRole/infpool-gemma-2b-inference-demo-epp   19m

==> v1/RoleBinding
NAME                                                                              ROLE                                               AGE
gmp-system:collector:inference-demo-infpool-gemma-2b-metrics-reader-secret-read   Role/infpool-gemma-2b-metrics-reader-secret-read   19m
infpool-gemma-2b-epp   Role/infpool-gemma-2b-epp   19m

==> v1/GCPBackendPolicy
NAME               AGE
infpool-gemma-2b   104s

==> v1/InferencePool
infpool-gemma-2b   19m

==> v1/PodMonitoring
infpool-gemma-2b   19m


TEST SUITE: None
NOTES:
InferencePool infpool-gemma-2b deployed.

Which issue(s) this PR fixes:

NONE

Does this PR introduce a user-facing change?:

NONE

@k8s-ci-robot k8s-ci-robot added the kind/bug Categorizes issue or PR as related to a bug. label Sep 20, 2025
Copy link

netlify bot commented Sep 20, 2025

Deploy Preview for gateway-api-inference-extension ready!

Name Link
🔨 Latest commit 57f9aa6
🔍 Latest deploy log https://app.netlify.com/projects/gateway-api-inference-extension/deploys/68cdfa4b13ac1000088d007a
😎 Deploy Preview https://deploy-preview-1623--gateway-api-inference-extension.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Sep 20, 2025
@k8s-ci-robot k8s-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Sep 20, 2025
@zetxqx
Copy link
Contributor Author

zetxqx commented Sep 20, 2025

/assign @ahg-g @kfswain

unfortunately there is a bug. But we are lucky to have lohi pipeline to catch it.

this is also needed in #1616

@zetxqx zetxqx mentioned this pull request Sep 20, 2025
@zetxqx
Copy link
Contributor Author

zetxqx commented Sep 20, 2025

Don't know how the syntax work, but here is a more visualize diff using helm template.

image

@ahg-g
Copy link
Contributor

ahg-g commented Sep 20, 2025

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 20, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ahg-g, zetxqx

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 20, 2025
@k8s-ci-robot k8s-ci-robot merged commit 32970c0 into kubernetes-sigs:main Sep 20, 2025
13 checks passed
kfswain pushed a commit that referenced this pull request Sep 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants