generated from kubernetes/kubernetes-template-project
-
Notifications
You must be signed in to change notification settings - Fork 472
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Adding documentation for policy attachment model
- Loading branch information
Showing
4 changed files
with
366 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
/* | ||
Copyright 2021 The Kubernetes Authors. | ||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
http://www.apache.org/licenses/LICENSE-2.0 | ||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
*/ | ||
|
||
package v1alpha2 | ||
|
||
// PolicyTargetReference identifies an API object to apply policy to. | ||
type PolicyTargetReference struct { | ||
// Group is the group of the target resource. | ||
// | ||
// +kubebuilder:validation:MinLength=1 | ||
// +kubebuilder:validation:MaxLength=253 | ||
Group string `json:"group"` | ||
|
||
// Kind is kind of the target resource. | ||
// | ||
// +kubebuilder:validation:MinLength=1 | ||
// +kubebuilder:validation:MaxLength=253 | ||
Kind string `json:"kind"` | ||
|
||
// Name is the name of the target resource. | ||
// | ||
// +kubebuilder:validation:MinLength=1 | ||
// +kubebuilder:validation:MaxLength=253 | ||
Name string `json:"name"` | ||
|
||
// Namespace is the namespace of the referent. When unspecified, the local | ||
// namespace is inferred. Even when policy targets a resource in a different | ||
// namespace, it may only apply to traffic originating from the same | ||
// namespace as the policy. | ||
// | ||
// +kubebuilder:validation:MinLength=1 | ||
// +kubebuilder:validation:MaxLength=253 | ||
// +optional | ||
Namespace string `json:"namespace,omitempty"` | ||
|
||
// ClassName is the name of the class this policy should apply to. When | ||
// unspecified, the policy will apply to all classes that support it. | ||
// | ||
// +kubebuilder:validation:MinLength=1 | ||
// +kubebuilder:validation:MaxLength=253 | ||
// +optional | ||
ClassName string `json:"className,omitempty"` | ||
} |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,294 @@ | ||
# Policy Attachment | ||
|
||
Policies attached to Gateway API resources and implementations should use the | ||
following approach to ensure consistency across implementations of the API. | ||
There are three primary components of this pattern: | ||
|
||
* A shared means of attaching policy to resources. | ||
* Support for configuring both default and override values within policy | ||
resources. | ||
* A shared hierarchy to illustrate how default and override values should | ||
interact. | ||
|
||
This kind of standardization will not only enable consistent patterns, it will | ||
enable tooling such as kubectl plugins to be able to visualize all policies that | ||
have been applied to a given resource. | ||
|
||
## Policy Attachment for Ingress | ||
Attaching policy to Gateway resources for ingress use cases is relatively | ||
straightforward. A policy can reference the resource it wants to apply to. | ||
Access is granted with RBAC - anyone that has access to create a RetryPolicy in | ||
a given namespace can attach it to any resource within that namespace. | ||
|
||
![Simple Ingress Example](/geps/images/713-ingress-simple.png) | ||
|
||
To build on that example, it’s possible to attach policies to more resources. | ||
Each policy applies to the referenced resource and everything below it in terms | ||
of hierarchy. Although this example is likely more complex than many real world | ||
use cases, it helps demonstrate how policy attachment can work across | ||
namespaces. | ||
|
||
![Complex Ingress Example](/geps/images/713-ingress-complex.png) | ||
|
||
## Policy Attachment for Mesh | ||
Although there is a great deal of overlap between ingress and mesh use cases, | ||
mesh enables more complex policy attachment scenarios. For example, you may want | ||
to apply policy to requests from a specific namespace to a backend in another | ||
namespace. | ||
|
||
![Simple Mesh Example](/geps/images/713-mesh-simple.png) | ||
|
||
Policy attachment can be quite simple with mesh. Policy can be applied to any | ||
resource in any namespace but it can only apply to requests from the same | ||
namespace if the target is in a different namespace. | ||
|
||
At the other extreme, policy can be used to apply to requests from a specific | ||
workload to a backend in another namespace. A route can be used to intercept | ||
these requests and split them between different backends (foo-a and foo-b in | ||
this case). | ||
|
||
![Complex Mesh Example](/geps/images/713-mesh-complex.png) | ||
|
||
## Target Reference API | ||
|
||
Each Policy resource MUST include a single `targetRef` field. It MUST not | ||
target more than one resource at a time, but it can be used to target larger | ||
resources such as Gateways or Namespaces that may apply to multiple child | ||
resources. | ||
|
||
The `targetRef` field MUST be an exact replica of the `PolicyTargetReference` | ||
struct included in the Gateway API. Where possible, we recommend using that | ||
struct directly instead of duplicating the type. | ||
|
||
### Policy Boilerplate | ||
The following structure can be used as a starting point for any Policy resource | ||
using this API pattern. | ||
|
||
```go | ||
// ACMEServicePolicy provides a way to apply Service policy configuration with | ||
// the ACME implementation of the Gateway API. | ||
type ACMEServicePolicy struct { | ||
metav1.TypeMeta `json:",inline"` | ||
metav1.ObjectMeta `json:"metadata,omitempty"` | ||
|
||
// Spec defines the desired state of ACMEServicePolicy. | ||
Spec ACMEServicePolicySpec `json:"spec"` | ||
|
||
// Status defines the current state of ACMEServicePolicy. | ||
Status ACMEServicePolicyStatus `json:"status,omitempty"` | ||
} | ||
|
||
// ACMEServicePolicySpec defines the desired state of ACMEServicePolicy. | ||
type ACMEServicePolicySpec struct { | ||
// TargetRef identifies an API object to apply policy to. | ||
TargetRef gatewayv1a2.PolicyTargetReference `json:"targetRef"` | ||
|
||
// Override defines policy configuration that should override policy | ||
// configuration attached below the targeted resource in the hierarchy. | ||
// +optional | ||
Override *ACMEPolicyConfig `json:"override,omitempty"` | ||
|
||
// Default defines default policy configuration for the targeted resource. | ||
// +optional | ||
Default *ACMEPolicyConfig `json:"default,omitempty"` | ||
} | ||
|
||
// ACMEPolicyConfig contains ACME policy configuration. | ||
type ACMEPolicyConfig struct { | ||
// Add configurable policy here | ||
} | ||
|
||
// ACMEServicePolicyStatus defines the observed state of ACMEServicePolicy. | ||
type ACMEServicePolicyStatus struct { | ||
// Conditions describe the current conditions of the ACMEServicePolicy. | ||
// | ||
// +optional | ||
// +listType=map | ||
// +listMapKey=type | ||
// +kubebuilder:validation:MaxItems=8 | ||
Conditions []metav1.Condition `json:"conditions,omitempty"` | ||
} | ||
``` | ||
|
||
### Hierarchy | ||
Each policy MAY include default or override values. Default values are given | ||
precedence from the bottom up, while override values are top down. That means | ||
that a default attached to a Backend will have the highest precedence among | ||
default values while an override value attached to a GatewayClass will have the | ||
highest precedence overall. | ||
|
||
![Ingress and Sidecar Hierarchy](/geps/images/713-hierarchy.png) | ||
|
||
To illustrate this, consider 3 resources with the following hierarchy: | ||
A > B > C. When attaching the concept of defaults and overrides to that, the | ||
hierarchy would be expanded to this: | ||
|
||
A override > B override > C override > C default > B default > A default. | ||
|
||
Note that the hierarchy is reversed for defaults. The rationale here is that | ||
overrides usually need to be enforced top down while defaults should apply to | ||
the lowest resource first. For example, if an admin needs to attach required | ||
policy, they can attach it as an override to a Gateway. That would have | ||
precedence over Routes and Services below it. On the other hand, an app owner | ||
may want to set a default timeout for their Service. That would have precedence | ||
over defaults attached at higher levels such as Route or Gateway. | ||
|
||
If using defaults and overrides, each policy resource MUST include 2 structs | ||
within the spec. One with override values and the other with default values. | ||
|
||
In the following example, the policy attached to the Gateway requires cdn to | ||
be enabled and provides some default configuration for that. The policy attached | ||
to the Route changes the value for one of those fields (includeQueryString). | ||
|
||
```yaml | ||
kind: GKEServicePolicy # Example of implementation specific policy name | ||
spec: | ||
override: | ||
cdn: | ||
enabled: true | ||
default: | ||
cdn: | ||
cachePolicy: | ||
includeHost: true | ||
includeProtocol: true | ||
includeQueryString: true | ||
targetRef: | ||
kind: Gateway | ||
name: example | ||
--- | ||
kind: GKEServicePolicy | ||
spec: | ||
default: | ||
cdn: | ||
cachePolicy: | ||
includeQueryString: false | ||
targetRef: | ||
kind: HTTPRoute | ||
name: example | ||
``` | ||
In this final example, we can see how the override attached to the Gateway has | ||
precedence over the default drainTimeout value attached to the Route. At the | ||
same time, we can see that the default connectionTimeout attached to the Route | ||
has precedence over the default attached to the Gateway. | ||
![Hierarchical Policy Example](/geps/images/713-policy-hierarchy.png) | ||
#### Supported Resources | ||
It is important to note that not every implementation will be able to support | ||
policy attachment to each resource described in the hierarchy above. When that | ||
is the case, implementations MUST clearly document which resources a policy may | ||
be attached to. | ||
#### Attaching Policy to GatewayClass | ||
GatewayClass may be the trickiest resource to attach policy to. Policy | ||
attachment relies on the policy being defined within the same scope as the | ||
target. This ensures that only users with write access to a policy resource in a | ||
given scope will be able to modify policy at that level. Since GatewayClass is a | ||
cluster scoped resource, this means that any policy attached to it must also be | ||
cluster scoped. | ||
GatewayClass parameters provide an alternative to policy attachment that may be | ||
easier for some implementations to support. These parameters can similarly be | ||
used to set defaults and requirements for an entire GatewayClass. | ||
### Targeting External Services | ||
In some cases (likely limited to mesh) we may want to apply policies to requests | ||
to external services. To accomplish this, implementations can choose to support | ||
a refernce to a virtual resource type: | ||
```yaml | ||
kind: RetryPolicy | ||
apiVersion: networking.acme.io/v1alpha1 | ||
metadata: | ||
name: foo | ||
spec: | ||
default: | ||
maxRetries: 5 | ||
targetRef: | ||
group: networking.acme.io | ||
kind: ExternalService | ||
name: foo.com | ||
``` | ||
### Conflict Resolution | ||
It is possible for multiple policies to target the same resource. When this | ||
happens, merging is the preferred outcome. If multiple policy resources target | ||
the same resource _and_ have an identical field specified with different values, | ||
precedence MUST be determined in order of the following criteria, continuing on | ||
ties: | ||
* The oldest Policy based on creation timestamp. For example, a Policy with a | ||
creation timestamp of "2021-07-15 01:02:03" is given precedence over a Policy | ||
with a creation timestamp of "2021-07-15 01:02:04". | ||
* The Policy appearing first in alphabetical order by "<namespace>/<name>". For | ||
example, foo/bar is given precedence over foo/baz. | ||
For a better user experience, a validating webhook can be implemented to prevent | ||
these kinds of conflicts all together. | ||
### Kubectl Plugin | ||
To help improve UX and standardization, a kubectl plugin will be developed that | ||
will be capable of describing the computed sum of policy that applies to a given | ||
resource, including policies applied to parent resources. | ||
Each Policy CRD that wants to be supported by this plugin will need to follow | ||
the API structure defined above and add a `gateway.networking.k8s.io/policy: | ||
true` label to the CRD. | ||
|
||
### Status | ||
In the future, we may consider adding a new `Policies` field to status on | ||
Gateways and Routes. This would be a list of `PolicyTargetReference` structs | ||
with the fields instead used to refer to the Policy resource that has been | ||
applied. | ||
|
||
Unfortunately, this may create more confusion than it is worth, here are some of | ||
the key concerns: | ||
|
||
* When multiple controllers are implementing the same Route and recognize a | ||
policy, it would be difficult to determine which controller should be | ||
responsible for adding that policy reference to status. | ||
* For this to be somewhat scalable, we'd need to limit the status entries to | ||
policies that had been directly applied to the resource. This could get | ||
confusing as it would not provide any insight into policies attached above or | ||
below. | ||
* Since we only control some of the resources a policy might be attached to, | ||
adding policies to status would only be possible on Gateway API resources, not | ||
Services or other kinds of backends. | ||
|
||
Although these concerns are not unsolvable, they lead to the conclusion that | ||
a Kubectl plugin should be our primary approach to providing visibility here, | ||
with a possibility of adding policies to status at a later point. | ||
|
||
### Interaction with Custom Filters and other extension points | ||
There are multiple methods of custom extension in the Gateway API. Policy | ||
attachment and custom Route filters are two of these. Policy attachment is | ||
designed to provide arbitrary configuration fields that decorate Gateway API | ||
resources. Route filters provide custom request/response filters embedded inside | ||
Route resources. Both are extension methods for fields that cannot easily be | ||
standardized as core or extended fields of the Gateway API. The following | ||
guidance should be considered when introducing a custom field into any Gateway | ||
controller implementation: | ||
|
||
1. For any given field that a Gateway controller implementation needs, the | ||
possibility of using core or extended should always be considered before | ||
using custom policy resources. This is encouraged to promote standardization | ||
and, over time, to absorb capabilities into the API as first class fields, | ||
which offer a more streamlined UX than custom policy attachment. | ||
|
||
2. Although it's possible that arbitrary fields could be supported by custom | ||
policy, custom route filters, and core/extended fields concurrently, it is | ||
strongly recommended that implementations not use multiple mechanisms for | ||
representing the same fields. A given field should only be supported through | ||
a single extension method. An example of potential conflict is policy | ||
precedence and structured hierarchy, which only applies to custom policies. | ||
Allowing a field to exist in custom policies and also other areas of the API, | ||
which are not part of the structured hierarchy, breaks the precedence model. | ||
Note that this guidance may change in the future as we gain a better | ||
understanding for extension mechanisms of the Gateway API can interoperate. | ||
|
||
### Conformance Level | ||
This policy attachment pattern is associated with an "EXTENDED" conformance | ||
level. The implementations that support this policy attachment model will have | ||
the same behavior and semantics, although they may not be able to support | ||
attachment of all types of policy at all potential attachment points. |