Make auth-proxy query EventPolicies dynamically#8870
Make auth-proxy query EventPolicies dynamically#8870knative-prow[bot] merged 5 commits intoknative:mainfrom
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #8870 +/- ##
==========================================
- Coverage 51.11% 50.91% -0.21%
==========================================
Files 409 409
Lines 21381 21524 +143
==========================================
+ Hits 10928 10958 +30
- Misses 9602 9710 +108
- Partials 851 856 +5 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
IntegrationSink was baking EventPolicies into the AUTH_POLICIES env var, requiring deployment rollouts whenever policies changed. This caused test failures because old pods with stale policies continued serving traffic during RollingUpdate. Change auth-proxy to query EventPolicies dynamically using a namespace-scoped informer, similar to how Broker and Channel work. This eliminates deployment rollouts when EventPolicies change. - Add knative-eventing-eventpolicy-reader ClusterRole - Create namespace-scoped EventPolicy informer in auth-proxy - Add parent resource env vars to identify which resource to query policies for - Create RoleBinding in sink's namespace for EventPolicy access - Remove AUTH_POLICIES env var from deployment spec - Add test coverage for OIDC-enabled deployments with RoleBindings
a847b16 to
ff8caa6
Compare
|
/cc @simkam |
|
/cc @Cali0707 |
| } | ||
|
|
||
| // Create namespace-scoped informer factory | ||
| eventingInformerFactory := eventinginformers.NewSharedInformerFactoryWithOptions( |
| logger.Debugf("Found %d EventPolicies for %s/%s", len(policies), h.config.ParentNamespace, h.config.ParentName) | ||
|
|
||
| // Convert EventPolicies to SubjectsWithFilters | ||
| subjectsWithFilters := make([]auth.SubjectsWithFilters, 0, len(policies)) |
There was a problem hiding this comment.
Maybe we can cache this & recompute whenever the informer has a new event?
In a NS with a lot of policies this will probably really slow the proxy down (since it happens on every request as far as I can tell)
There was a problem hiding this comment.
I thought the informer cache should be enought. But you're right with the computation of the filters on each request. I updated it
| func (r *Reconciler) reconcileEventPolicyRoleBinding(ctx context.Context, sink *sinks.IntegrationSink) error { | ||
| expected := resources.MakeEventPolicyRoleBinding(sink) | ||
|
|
||
| rb, err := r.kubeClientSet.RbacV1().RoleBindings(expected.Namespace).Get(ctx, expected.Name, metav1.GetOptions{}) |
There was a problem hiding this comment.
Is there a reason we are using the kubeclient directly instead of an informer here?
There was a problem hiding this comment.
Good point. I'll update it
|
/cc @Cali0707 |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: Cali0707, creydr The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
* Make auth-proxy query EventPolicies dynamically IntegrationSink was baking EventPolicies into the AUTH_POLICIES env var, requiring deployment rollouts whenever policies changed. This caused test failures because old pods with stale policies continued serving traffic during RollingUpdate. Change auth-proxy to query EventPolicies dynamically using a namespace-scoped informer, similar to how Broker and Channel work. This eliminates deployment rollouts when EventPolicies change. - Add knative-eventing-eventpolicy-reader ClusterRole - Create namespace-scoped EventPolicy informer in auth-proxy - Add parent resource env vars to identify which resource to query policies for - Create RoleBinding in sink's namespace for EventPolicy access - Remove AUTH_POLICIES env var from deployment spec - Add test coverage for OIDC-enabled deployments with RoleBindings * Get resync period from context * Use rolebindingLister instead of kubeclient directly * Delete EventPolicy RBAC when OIDC gets disabled * Recreate subjectsWithFilters only on eventPolicy changes
* Make auth-proxy query EventPolicies dynamically (knative#8870) * Make auth-proxy query EventPolicies dynamically IntegrationSink was baking EventPolicies into the AUTH_POLICIES env var, requiring deployment rollouts whenever policies changed. This caused test failures because old pods with stale policies continued serving traffic during RollingUpdate. Change auth-proxy to query EventPolicies dynamically using a namespace-scoped informer, similar to how Broker and Channel work. This eliminates deployment rollouts when EventPolicies change. - Add knative-eventing-eventpolicy-reader ClusterRole - Create namespace-scoped EventPolicy informer in auth-proxy - Add parent resource env vars to identify which resource to query policies for - Create RoleBinding in sink's namespace for EventPolicy access - Remove AUTH_POLICIES env var from deployment spec - Add test coverage for OIDC-enabled deployments with RoleBindings * Get resync period from context * Use rolebindingLister instead of kubeclient directly * Delete EventPolicy RBAC when OIDC gets disabled * Recreate subjectsWithFilters only on eventPolicy changes * Add probes to IntegrationSource & -Sink deployments (knative#8867) * Add probes to IntegrationSource deployments * Fix auth-proxy SINK_URI missing value error The auth-proxy container was crashing with "required key SINK_URI missing value" due to a circular dependency in the reconciliation order. The deployment (with auth-proxy) was being created before the IntegrationSink status.Address was set, causing the auth-proxy to fail during startup. This commit fixes the issue by deriving the SINK_URI directly from the sink name and namespace (using network.GetServiceHostname()) instead of reading it from status.Address. This matches how the reconcileAddress() function constructs the URL, but without requiring the status to be set first. The same approach is now used for the SINK_AUDIENCE when OIDC authentication is enabled. This eliminates the circular dependency and ensures the auth-proxy always has a valid SINK_URI, regardless of reconciliation timing. * Add readiness check to auth-proxy * Fix unit tests * Use same timings for auth-proxy probes
IntegrationSink was baking EventPolicies into the AUTH_POLICIES env var, requiring deployment rollouts whenever policies changed. This required a redeploy when Policies changes (and lead to many flaky test failures because old pods with stale policies continued serving traffic during RollingUpdate).
This PR addresses it and changes auth-proxy to query EventPolicies dynamically using a namespace-scoped informer, similar to how Broker and Channel work. This eliminates deployment rollouts when EventPolicies change.
Makes tests more stable and required for #8867