Conformance Tests for v0.5 Release

[zetxqx]

As discussed in our recent community meeting, this issue will serve as a central tracking point for all conformance tests targeted for the v0.5 release (aiming for early July). This will help us monitor progress and ensure everything is ready on time.

Conformance Test Checklist

Here is a list of the conformance test cases, their current status, and the corresponding Pull Requests.

• inferencepool_accepted.go: Validates basic CRUD operations for the InferencePool resource (#772).
• inferencepool_epp_reference_non_existent.go: Confirms an error status is reported when an InferencePool references a non-existent external process service (feat(conformance): Add test for invalid EPP service reference #959).
• inferencepool_resolvedrefs_condition.go: Verifies that the "ResolvedRefs" condition on an InferencePool accurately reflects whether it's being referenced by other resources (#832).
• inferencepool_multiple_gateway_refs.go: Tests the "ResolvedRefs" condition when an InferencePool is referenced by multiple gateways (covered in #832).
• httproute_invalid_inferencepool_ref.go: Verifies the behavior of an HTTPRoute when it references a non-existent InferencePool (#807).
• inferencepool_httproute_port_validation.go: Checks HTTPRoute port matching when referencing an InferencePool (PR #911). Based on discussion How should implementations handle backendRef.TargetPort #918
• httproute_multiple_rules_different_pools.go: Validates that a single HTTPRoute can correctly route traffic to multiple different InferencePools using different rules (PR #838).
• httproute_multiple_gateways_different_pools.go: Validates that two separate HTTPRoutes on different gateways can successfully reference two different InferencePools (PR #838).
• gateway_following_epp_routing.go: Verify the gateway correctly selects an endpoint when the External Policy Processor (EPP) returns a list of multiple candidate endpoints. (feat(conformance): Add EPP conformance test for Gateway routing #961)
• EPP unavailable behavior:
  • FailureMode == FailOpen: Request should still hit an endpoint in the selector. feat(conformance) add EPP unavailable fail-open test #999
  • FailureMode == FailClose: Request should fail with a 50x response.

Please feel free to update this issue with new tests, PRs, or comments.

cc: @robscott @danehans @kfswain

[robscott]

/triage accepted

[robscott]

@danehans or @kfswain can we add this to the v0.5 milestone?

[LiorLieberman]

/cc

[zetxqx]

/asssign

[zetxqx]

/assign

[zetxqx]

Update: two things are needed

1. fix suite set up issue fix(conformance): fix conformance setup issue by not relying on suite.Setup from gateway-api #1060 refactor(conformance) merge similar utility functions. #1055 fix(conformance): change InferencePoolInvalidEPPService test to align with spec . #1061
2. have reports as example: refactor(conformance) merge similar utility functions. #1055

[nirrozenbaum]

@zetxqx @robscott is this issue completed?

[zetxqx]

@nirrozenbaum yes, let's close it, thanks!

[nirrozenbaum]

closing as completed.