-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AuthPolicy Atomic Overrides #525
Conversation
@@ -65,8 +65,17 @@ func (r *AuthPolicyReconciler) desiredAuthConfig(ctx context.Context, ap *api.Au | |||
|
|||
switch obj := targetNetworkObject.(type) { | |||
case *gatewayapiv1.HTTPRoute: | |||
ok, err := routeGatewayHasAuthOverrides(ctx, obj, r.Client()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if ok { | ||
logger.V(1).Info("targeted gateway has authpolicy with atomic overrides, skipping authorino authconfig for the HTTPRoute authpolicy") | ||
utils.TagObjectToDelete(authConfig) | ||
r.OverriddenPolicyMap.SetOverriddenPolicy(ap) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice. I wonder if we could do something similar (or exactly the same!) for the RLP as well.
The RLP doesn't have the same 1:1 relation to the underlying resources as the AP does with the AuthConfigs, so it would somewhere else of course. But useful for setting the status of the policy later on nevertheless.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I think we can and probably should do something similar for RLPs also 👍
// Mutual Exclusivity Validation | ||
// +kubebuilder:validation:XValidation:rule="!(has(self.defaults) && (has(self.routeSelectors) || has(self.patterns) || has(self.when) || has(self.rules)))",message="Implicit and explicit defaults are mutually exclusive" | ||
// +kubebuilder:validation:XValidation:rule="!(has(self.overrides) && (has(self.routeSelectors) || has(self.patterns) || has(self.when) || has(self.rules)))",message="Implicit defaults and explicit overrides are mutually exclusive" | ||
// +kubebuilder:validation:XValidation:rule="!(has(self.overrides) && has(self.defaults))",message="Explicit overrides and explicit defaults are mutually exclusive" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the RLP equivalent, I've opted to add a CEL validation to only allow overrides
block for policies targeting a gateway. Not sure do we want to do the same here
// +kubebuilder:validation:XValidation:rule="!(has(self.overrides) && has(self.defaults))",message="Explicit overrides and explicit defaults are mutually exclusive" | |
// +kubebuilder:validation:XValidation:rule="!(has(self.overrides) && has(self.defaults))",message="Explicit overrides and explicit defaults are mutually exclusive" | |
// +kubebuilder:validation:XValidation:rule="!(has(self.overrides) && self.targetRef.kind != 'Gateway')",message="Overrides are only allowed for policies targeting a Gateway resource" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was thinking about this, and I felt if we blocked the overrides on the route policies, we should block the defaults on the gateway policies. I do think we should be consistent on what we allow on both sets of policies and that it is easier to add access than remove it at a later date. What do you think @KevFan
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we should block the defaults on the gateway policies.
Sorry, @Boomatang. Maybe I'm not following. The defaults on the gateway is a valid use case. I don't think we want to block them. How would one set defaults for multiple routes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am working off my understand of the hierarchy chart, the defaults sets value for the resources above in the tree and overrides set values on resources lower in the tree. As we don't support setting these values on resources about the gateway, we don't need to allow defaults on it.
I did check the RFC0009 and it would allow setting the defaults as we currently have it but that seems to be different than the definition in the gateway-api GEP2649. I know that GEP is being up dated, so maybe that definition is also being updated.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not really. Both defaults and overrides are set for the resources lower in the tree. The arrows (up or down) tell which level have precedence in case of conflicting rules set at multiple levels. (The arrow pointing from higher precedence to lower.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
During the sync we cleared this up. I will be implementing the CEL to disable overrides on the policy attached to the HttpRoutes.
c67790f
to
5327d63
Compare
rebase commit
5327d63
to
17378e0
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Verified, changes looks good to me 👍
} | ||
|
||
refNetworkObject := &gatewayapiv1.HTTPRoute{} | ||
err = r.Client().Get(ctx, ref.TargetKey(), refNetworkObject) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe add a comment here that this only works because of single policy of a kind allowed per target object. Since the gateway policy was skipped in line 186, all the other refs are of policies targeting routes. In the future, that may no longer be the case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see your point, but I don't think a comment is worth it. I would assume there are other spots in the code base that assume only one type of resources is attached to the gateway. When the day comes when we support it, I would expect that would be a large change to the whole code base and not just here.
controllers/authpolicy_authconfig.go
Outdated
continue | ||
} | ||
anno := strings.Split(annotation, "/") | ||
ap := &api.AuthPolicy{} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe rename ap
here as otherAP
, to avoid confusion with the AP that triggered the reconciliation?
244f947
to
7efe3c9
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks very good, @Boomatang. Thank you!
Leaving here almost as a self reminder. In the future, we can revisit this and try to leverage more the gatewayapi.Topology
and gatewayapi.TopologyIndexes
(aka DAG) as an alternative to nested API reads such as:
kuadrant-operator/controllers/authpolicy_authconfig.go
Lines 192 to 204 in 7efe3c9
gw := &gatewayapiv1.Gateway{} | |
namespace := ptr.Deref(parentRef.Namespace, gatewayapiv1.Namespace(route.GetNamespace())) | |
err := c.Get(ctx, client.ObjectKey{Name: string(parentRef.Name), Namespace: string(namespace)}, gw) | |
if err != nil { | |
return false, err | |
} | |
annotation, ok := gw.GetAnnotations()[common.AuthPolicyBackRefAnnotation] | |
if !ok { | |
continue | |
} | |
otherAP := &api.AuthPolicy{} | |
err = c.Get(ctx, utils.NamespacedNameToObjectKey(annotation, gw.Namespace), otherAP) |
Some additional verification steps I tried, for the CEL validation piece:
|
AuthPolicy Atomic Overrides
Closes: #464
Verification
Base Setup
Create local cluster from this branch
Apply Kuadrant CR.
Install the toy store app
Add HTTP Route
Create secrets for Auth. The secrets are doubled to be created in two namespaces.
Set up gateway url
Set up a watch on AuthConfigs.
Case 1
There is an existing AuthPolicy attached to a HTTPRoute and the system admin attaches a AuthPolicy to the Gateway with atonic overrides.
Create HTTPRoute AuthPolicy. Expect watch to contain one AuthConfig
ap-default-toystore
Check auth is working. Note the name of the Authorization key,
TOYSTORE
. Expected result is200
.Add the Gateway AuthPolicy that contains overrides. Expect AuthConfig to change to
ap-istio-system-gateway
Curl the endpoint again with Route Key. Expected
401
Curl endpoint with key defined in the Gateway AuthPolicy,
APIKEY
. Expected200
.This show the Gateway overrides are working.
Case 2
In this case the Gateway AuthPolicy has being defined before any route AuthPolicies have being attached.
Set the cluster back to the configuration as per the Base Setup.
Install the gateway AuthPolicy. Expected AuthConfig on watch:
ap-istio-system-gateway
Curl endpoint with key defined in the Gateway AuthPolicy,
APIKEY
. Expected200
.Apply Auth Policy to route. Expected AuthConfig on watch:
ap-istio-system-gateway
Curl the endpoint with Route Key. Expected
401
Curl endpoint with key defined in the Gateway AuthPolicy,
APIKEY
. Expected200
.This show the Gateway overrides are working for routes added after the initial overrides.
Case 3
The cluster has exiting AuthPolicies attached to the routes and gateways. The system admin later defines the overrides on the gateway AuthPolicy.
Set the cluster back to the configuration as per the Base Setup.
Create the Route and Gateway AuthPolices. Note that the Gateway AuthPolicy does not define any overrides. Expected AuthConfig in watch:
ap-default-toystore
Curl the endpoint with Route Key. Expected
200
Curl endpoint with key defined in the Gateway AuthPolicy,
APIKEY
. Expected401
.Update the Gateway AuthPolicy to have
overrides
. Expected AuthConfig on watch:ap-istio-system-gateway
.Curl the endpoint with Route Key. Expected
401
Curl endpoint with key defined in the Gateway AuthPolicy,
APIKEY
. Expected200
.This proves that existing Gateway AuthPolicies can be updated to contain
overrides
.Case 4
The cluster in this case is configure to have
overrides
on the Gateway from the start. At a later time the system admin removes theoverrides
from the gateway AuthPolicy.Set the cluster back to the configuration as per the Base Setup.
Create the Route and Gateway AuthPolices. Note that the Gateway AuthPolicy does define overrides. Expected AuthConfig in watch:
ap-istio-system-gateway
Curl the endpoint with Route Key. Expected
401
Curl endpoint with key defined in the Gateway AuthPolicy,
APIKEY
. Expected200
.Update the Gateway AuthPolicy to remove the overrides. Expect AuthConfig in watch:
ap-default-toystore
Curl the endpoint with Route Key. Expected
200
Curl endpoint with key defined in the Gateway AuthPolicy,
APIKEY
. Expected401
.This proves AuthPolicies can be updated to remove
overrides
.````