Use dedicated namespace for e2e test code #661
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,126 @@ | ||
| apiVersion: inference.networking.x-k8s.io/v1alpha2 | ||
|
There was a problem hiding this comment. This looks like a duplication of the existing inferencepool-resources.yaml, excluding the namespace field. there is a big value in testing e2e our "Getting Started" files, so we are self testing the public documentation. There was a problem hiding this comment. How would you suggest getting the env var parsed in the YAML file? There was a problem hiding this comment. I suppose I could use a similar approach to the one you described above. However, that might complicate the getting started guide? How would the user of the guide make sure the env var was parsed in the file? There was a problem hiding this comment. in the e2e a similar approach would work. in getting started guide I’m not sure we have an agreement of the other maintainers about being able to choose the namespace (I’m in favor of it). for example - this discussion is relevant only for the subject serviceaccount in the ClusterRoleBinding, as the namespace in the Service and Deployment resources can be just removed and then you can decide the ns during e2e test. There was a problem hiding this comment. Should we tackle this in a follow-up then? If we change the getting started file in this PR its going to expand the scope considerably as we now need to update the guide with the envvar substitution command. cc @danehans There was a problem hiding this comment.
+1 on not changing the getting started guide in this PR. There was a problem hiding this comment.
I agree with this point. We may need to consider 2 different e2e tests, 1 for testing the quickstart and another that is more programmatic to support various inference use cases. Maybe we track this in a separate issue? There was a problem hiding this comment. I prefer to move away from yaml files and just create those objects in code,,, the envoy one is complex, so I don't mind it, but other than that I think it is more manageable to have the test create those objects in code. There was a problem hiding this comment. oh, well, creating the pool requires also creating all the epp and the rbac resources, so it is also complex... |
||
| kind: InferencePool | ||
| metadata: | ||
| labels: | ||
| name: vllm-llama3-8b-instruct | ||
| spec: | ||
| targetPortNumber: 8000 | ||
| selector: | ||
| app: vllm-llama3-8b-instruct | ||
| extensionRef: | ||
| name: vllm-llama3-8b-instruct-epp | ||
| namespace: $E2E_NS | ||
| --- | ||
| apiVersion: v1 | ||
| kind: Service | ||
| metadata: | ||
| name: vllm-llama3-8b-instruct-epp | ||
| namespace: $E2E_NS | ||
| spec: | ||
| selector: | ||
| app: vllm-llama3-8b-instruct-epp | ||
| ports: | ||
| - protocol: TCP | ||
| port: 9002 | ||
| targetPort: 9002 | ||
| appProtocol: http2 | ||
| type: ClusterIP | ||
| --- | ||
| apiVersion: apps/v1 | ||
| kind: Deployment | ||
| metadata: | ||
| name: vllm-llama3-8b-instruct-epp | ||
| namespace: $E2E_NS | ||
| labels: | ||
| app: vllm-llama3-8b-instruct-epp | ||
| spec: | ||
| replicas: 1 | ||
| selector: | ||
| matchLabels: | ||
| app: vllm-llama3-8b-instruct-epp | ||
| template: | ||
| metadata: | ||
| labels: | ||
| app: vllm-llama3-8b-instruct-epp | ||
| spec: | ||
| # Conservatively, this timeout should mirror the longest grace period of the pods within the pool | ||
| terminationGracePeriodSeconds: 130 | ||
| containers: | ||
| - name: epp | ||
| image: us-central1-docker.pkg.dev/k8s-staging-images/gateway-api-inference-extension/epp:main | ||
| imagePullPolicy: Always | ||
| args: | ||
| - -poolName | ||
| - "vllm-llama3-8b-instruct" | ||
| - -poolNamespace | ||
| - "$E2E_NS" | ||
| - -v | ||
| - "4" | ||
| - --zap-encoder | ||
| - "json" | ||
| - -grpcPort | ||
| - "9002" | ||
| - -grpcHealthPort | ||
| - "9003" | ||
| env: | ||
| - name: USE_STREAMING | ||
| value: "true" | ||
| ports: | ||
| - containerPort: 9002 | ||
| - containerPort: 9003 | ||
| - name: metrics | ||
| containerPort: 9090 | ||
| livenessProbe: | ||
| grpc: | ||
| port: 9003 | ||
| service: inference-extension | ||
| initialDelaySeconds: 5 | ||
| periodSeconds: 10 | ||
| readinessProbe: | ||
| grpc: | ||
| port: 9003 | ||
| service: inference-extension | ||
| initialDelaySeconds: 5 | ||
| periodSeconds: 10 | ||
| --- | ||
| kind: ClusterRole | ||
| apiVersion: rbac.authorization.k8s.io/v1 | ||
| metadata: | ||
| name: pod-read | ||
| rules: | ||
| - apiGroups: ["inference.networking.x-k8s.io"] | ||
| resources: ["inferencemodels"] | ||
| verbs: ["get", "watch", "list"] | ||
| - apiGroups: [""] | ||
| resources: ["pods"] | ||
| verbs: ["get", "watch", "list"] | ||
| - apiGroups: ["inference.networking.x-k8s.io"] | ||
| resources: ["inferencepools"] | ||
| verbs: ["get", "watch", "list"] | ||
| - apiGroups: ["discovery.k8s.io"] | ||
| resources: ["endpointslices"] | ||
| verbs: ["get", "watch", "list"] | ||
| - apiGroups: | ||
| - authentication.k8s.io | ||
| resources: | ||
| - tokenreviews | ||
| verbs: | ||
| - create | ||
| - apiGroups: | ||
| - authorization.k8s.io | ||
| resources: | ||
| - subjectaccessreviews | ||
| verbs: | ||
| - create | ||
| --- | ||
| kind: ClusterRoleBinding | ||
| apiVersion: rbac.authorization.k8s.io/v1 | ||
| metadata: | ||
| name: pod-read-binding | ||
| subjects: | ||
| - kind: ServiceAccount | ||
| name: default | ||
| namespace: $E2E_NS | ||
| roleRef: | ||
| kind: ClusterRole | ||
| name: pod-read | ||
Uh oh!
There was an error while loading. Please reload this page.