Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

API: Support Circuit Breakers in BackendTrafficPolicy #2284

Merged
merged 4 commits into from
Dec 19, 2023
Merged

API: Support Circuit Breakers in BackendTrafficPolicy #2284

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
efe8f06
API: Support Circuit Breakers in BackendTrafficPolicy
guydc Dec 8, 2023
9c923bb
reivew fixes
guydc Dec 12, 2023
945ef27
Merge branch 'main' into circuit-breakers
zirain Dec 13, 2023
17964fe
Implement API changes
guydc Dec 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions api/v1alpha1/backendtrafficpolicy_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,12 @@ type BackendTrafficPolicySpec struct {
//
// +optional
TCPKeepalive *TCPKeepalive `json:"tcpKeepalive,omitempty"`

// Circuit Breaker settings for the upstream connections and requests.
// If not set, circuit breakers will be enabled with the default thresholds
//
// +optional
CircuitBreaker *CircuitBreaker `json:"circuitBreaker,omitempty"`
}

// BackendTrafficPolicyStatus defines the state of BackendTrafficPolicy
Expand Down
33 changes: 33 additions & 0 deletions api/v1alpha1/circuitbreaker_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
// Copyright Envoy Gateway Authors
// SPDX-License-Identifier: Apache-2.0
// The full text of the Apache license is available in the LICENSE file at
// the root of the repo.

package v1alpha1

// CircuitBreaker defines the Circuit Breaker configuration.
type CircuitBreaker struct {
// The maximum number of connections that Envoy will establish to the referenced backend defined within a xRoute rule.
//
// +kubebuilder:validation:Minimum=0
// +kubebuilder:validation:Maximum=4294967295
// +kubebuilder:default=1024
// +optional
MaxConnections *int64 `json:"maxConnections,omitempty"`

// The maximum number of pending requests that Envoy will queue to the referenced backend defined within a xRoute rule.
//
// +kubebuilder:validation:Minimum=0
// +kubebuilder:validation:Maximum=4294967295
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is 4294967295 ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the maximum value of uint32, but i think this Maximum validation can be optional

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, my bad, think about 2147483647.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can -1 pass if the type is uint32?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, it cannot

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's what it means.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @zirain, @shawnh2. Do note that the OpenAPI spec (used by K8s CRDs) doesn't really support unsigned ints: https://swagger.io/specification/. The controller-gen tools actually produce a schema that refers to these fields as int32 in the generated CRD. The actual K8s API server behavior, from my limited check, is to treat these fields as int64. I think that the actual go type (*uint32) mostly impacts the unmarshalling done by client go. So, guaranteeing that the value stored is actually safe to cast to uint32 could be useful...

Copy link
Contributor Author

@guydc guydc Dec 13, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another approach would be to use int64 explicitly in the go types layer and have uint32 as a representation in the IR layer and downwards. The value range validation can occur either using the schema or during the IR translation. WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

like Gateway API project, let's use *int32 with valiation min and max?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it make more sense to use *int64? MaxUInt32 > MaxInt32, so by using *int32 users would not able to use the full value range provided by Envoy.

// +kubebuilder:default=1024
// +optional
MaxPendingRequests *int64 `json:"maxPendingRequests,omitempty"`

// The maximum number of parallel requests that Envoy will make to the referenced backend defined within a xRoute rule.
//
// +kubebuilder:validation:Minimum=0
// +kubebuilder:validation:Maximum=4294967295
// +kubebuilder:default=1024
// +optional
MaxParallelRequests *int64 `json:"maxParallelRequests,omitempty"`
}
35 changes: 35 additions & 0 deletions api/v1alpha1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

33 changes: 33 additions & 0 deletions charts/gateway-helm/crds/generated/gateway.envoyproxy.io_backendtrafficpolicies.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,39 @@ spec:
spec:
description: spec defines the desired state of BackendTrafficPolicy.
properties:
circuitBreaker:
description: Circuit Breaker settings for the upstream connections
and requests. If not set, circuit breakers will be enabled with
the default thresholds
properties:
maxConnections:
default: 1024
description: The maximum number of connections that Envoy will
establish to the referenced backend defined within a xRoute
rule.
format: int64
maximum: 4294967295
minimum: 0
type: integer
maxParallelRequests:
default: 1024
description: The maximum number of parallel requests that Envoy
will make to the referenced backend defined within a xRoute
rule.
format: int64
maximum: 4294967295
minimum: 0
type: integer
maxPendingRequests:
default: 1024
description: The maximum number of pending requests that Envoy
will queue to the referenced backend defined within a xRoute
rule.
format: int64
maximum: 4294967295
minimum: 0
type: integer
type: object
loadBalancer:
description: LoadBalancer policy to apply when routing traffic from
the gateway to the backend endpoints
Expand Down
17 changes: 17 additions & 0 deletions site/content/en/latest/api/extension_types.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ _Appears in:_
| `loadBalancer` _[LoadBalancer](#loadbalancer)_ | LoadBalancer policy to apply when routing traffic from the gateway to the backend endpoints |
| `proxyProtocol` _[ProxyProtocol](#proxyprotocol)_ | ProxyProtocol enables the Proxy Protocol when communicating with the backend. |
| `tcpKeepalive` _[TCPKeepalive](#tcpkeepalive)_ | TcpKeepalive settings associated with the upstream client connection. Disabled by default. |
| `circuitBreaker` _[CircuitBreaker](#circuitbreaker)_ | Circuit Breaker settings for the upstream connections and requests. If not set, circuit breakers will be enabled with the default thresholds |



Expand Down Expand Up @@ -125,6 +126,22 @@ _Appears in:_
| `maxAge` _[Duration](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#duration-v1-meta)_ | MaxAge defines how long the results of a preflight request can be cached. |


#### CircuitBreaker



CircuitBreaker defines the Circuit Breaker configuration.

_Appears in:_
- [BackendTrafficPolicySpec](#backendtrafficpolicyspec)

| Field | Description |
| --- | --- |
| `maxConnections` _integer_ | The maximum number of connections that Envoy will establish to the referenced backend defined within a xRoute rule. |
| `maxPendingRequests` _integer_ | The maximum number of pending requests that Envoy will queue to the referenced backend defined within a xRoute rule. |
| `maxParallelRequests` _integer_ | The maximum number of parallel requests that Envoy will make to the referenced backend defined within a xRoute rule. |


#### ClaimToHeader


Expand Down
52 changes: 50 additions & 2 deletions test/cel-validation/backendtrafficpolicy_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,12 @@ package celvalidation
import (
"context"
"fmt"
egv1a1 "github.com/envoyproxy/gateway/api/v1alpha1"
"k8s.io/utils/pointer"
"strings"
"testing"
"time"

egv1a1 "github.com/envoyproxy/gateway/api/v1alpha1"

metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
gwapiv1a2 "sigs.k8s.io/gateway-api/apis/v1alpha2"
)
Expand Down Expand Up @@ -306,6 +306,54 @@ func TestBackendTrafficPolicyTarget(t *testing.T) {
"spec.loadBalancer: Invalid value: \"object\": Currently SlowStart is only supported for RoundRobin and LeastRequest load balancers.",
},
},
{
desc: " valid config: min, max, nil",
mutate: func(btp *egv1a1.BackendTrafficPolicy) {
valMax := pointer.Int64(4294967295)
valMin := pointer.Int64(0)
btp.Spec = egv1a1.BackendTrafficPolicySpec{
TargetRef: gwapiv1a2.PolicyTargetReferenceWithSectionName{
PolicyTargetReference: gwapiv1a2.PolicyTargetReference{
Group: gwapiv1a2.Group("gateway.networking.k8s.io"),
Kind: gwapiv1a2.Kind("Gateway"),
Name: gwapiv1a2.ObjectName("eg"),
},
},
CircuitBreaker: &egv1a1.CircuitBreaker{
MaxConnections: valMax,
MaxPendingRequests: valMin,
MaxParallelRequests: nil,
},
}
},
wantErrors: []string{},
},
{
desc: " invalid config: min and max valyues",
mutate: func(btp *egv1a1.BackendTrafficPolicy) {
valOverMax := pointer.Int64(4294967296)
valUnderMin := pointer.Int64(-1)
btp.Spec = egv1a1.BackendTrafficPolicySpec{
TargetRef: gwapiv1a2.PolicyTargetReferenceWithSectionName{
PolicyTargetReference: gwapiv1a2.PolicyTargetReference{
Group: gwapiv1a2.Group("gateway.networking.k8s.io"),
Kind: gwapiv1a2.Kind("Gateway"),
Name: gwapiv1a2.ObjectName("eg"),
},
},
CircuitBreaker: &egv1a1.CircuitBreaker{
MaxConnections: valOverMax,
MaxPendingRequests: valUnderMin,
MaxParallelRequests: valOverMax,
},
}
},
wantErrors: []string{
"spec.circuitBreaker.maxParallelRequests: Invalid value: 4294967296: spec.circuitBreaker.maxParallelRequests in body should be less than or equal to 4294967295",
"spec.circuitBreaker.maxPendingRequests: Invalid value: -1: spec.circuitBreaker.maxPendingRequests in body should be greater than or equal to 0",
"spec.circuitBreaker.maxConnections: Invalid value: 4294967296: spec.circuitBreaker.maxConnections in body should be less than or equal to 4294967295",
},
},
}

for _, tc := range cases {
Expand Down