Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions conformance/reports/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ This directory stores conformance reports submitted by various implementations o

This folder stores conformance reports organized first by the version of the Gateway API Inference Extension specification they were tested against, and then by the specific conformance profile (e.g., Gateway, EPP, Model Server):

```
|-- conformance/reports
| |-- v0.3.0 # Example extension version
| | |-- gateway # Conformance profile/category
Expand All @@ -22,6 +23,7 @@ This folder stores conformance reports organized first by the version of the Gat
| | | |-- ...
| |-- v0.4.0 # Future extension version
| | |-- ...
```

## Implementation Submissions

Expand Down
3 changes: 3 additions & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,9 @@ nav:
- Configuration Guide:
- Prefix Cache Aware Plugin: guides/epp-configuration/prefix-aware.md
- Implementer's Guide: guides/implementers.md
- Implementer Guides:
- Getting started: guides/implementers.md
- Conformance Tests: guides/conformance-tests.md
- Performance:
- Benchmark: performance/benchmark/index.md
- Regression Testing: performance/regression-testing/index.md
Expand Down
46 changes: 46 additions & 0 deletions site-src/guides/conformance-tests.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@

# Test Setup and Execution

This document provides steps to run the Gateway API Inference Extension conformance tests.

## Prerequisites

1. You need a Kubernetes cluster with [LoadBalancer](https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer) support.

2. Choose an Implementation -
Install an [existing implementation](https://gateway-api-inference-extension.sigs.k8s.io/implementations/gateways/). For setup instructions, refer to the [The Quickstart Guide](https://gateway-api-inference-extension.sigs.k8s.io/guides/). Alternatively run tests against your implementation after completing the [implementer's guide](https://gateway-api-inference-extension.sigs.k8s.io/guides/implementers/#implementers-guide).

Note: Since the EPP (EndPoint Picker) takes the `InferencePool` name as an environment variable, each conformance test creates a corresponding EPP deployment for each `InferencePool` it defines. For conformance testing, the EPP is configured with the `HeaderBasedTestingFilter`. This is enabled by setting the `ENABLE_REQ_HEADER_BASED_SCHEDULER_FOR_TESTING=true` environment variable in the EPP deployment manifest.

## Running Conformance Tests

1. **Clone the Repository**:
Create a local copy of the Gateway API Inference Extension repository:
```bash
git clone https://github.com/kubernetes-sigs/gateway-api-inference-extension.git
cd gateway-api-inference-extension
```

2. **Execute Tests**:
Run the following command to execute all available tests. Replace `<your_gatewayclass_name>` with the GatewayClass used by the implementation under test.

```bash
go test ./conformance -args -gateway-class <your_gatewayclass_name>
```

### Test Execution Options

* **Speeding up Reruns**: For repeated runs, you can add the flag `-cleanup-base-resources=false`. This will preserve resources such as namespaces and gateways between test runs, speeding up the process.
```bash
go test ./conformance -args -gateway-class <your_gatewayclass_name> -cleanup-base-resources=false
```

* **Running Specific Tests**: To run a specific test, you can reference the test name by using the `-run-test` flag. For example:
```bash
go test ./conformance -args -gateway-class <your_gatewayclass_name> -run-test HTTPRouteMultipleGatewaysDifferentPools
```

* **Detailed Logging**: To view detailed logs, you can enable logging mode by adding the `-v` as well as `-debug` flags.
```bash
go test -v ./conformance -args -debug -gateway-class <your_gatewayclass_name> -cleanup-base-resources=false -run-test HTTPRouteMultipleGatewaysDifferentPools
```
2 changes: 1 addition & 1 deletion site-src/guides/implementers.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,4 +110,4 @@ Here are some tips for testing your controller end-to-end:

### Conformance Tests

A set of conformance tests will be developed soon to help verify that a controller is working as expected. This guide will be updated once we have more information. Stay tuned!
See [Conformance Test Setup and Execution](https://gateway-api-inference-extension.sigs.k8s.io/guides/conformance-tests).