feat: azure/aks workload identity support

[ehrnst]

Description

The PR is a small re-work of #1183 which I could not get to work since it was using azidentity.NewManagedIdentityCredential(managedIDOptions) which would leverage the underlying AKS node's identity. I added azidentity.NewWorkloadIdentityCredential(workloadIDOptions) that allows pods to run with a specific Azure identity, federated with the kubernetes cluster.

Related Issues/PRs (if applicable)

Related to PR #1183 (Azure Managed Identity support by @mattmancel)

Special notes for reviewers (if applicable)

• CRD Changes: The CRD file was manually updated to match API changes due to controller-gen access issues on Windows. you may want to regenerate the CRD to verify
• I tested this from all angles I could think of on a clean AKS cluster. Installing AI Gateway with the latest helm chart, patching the deployments with my local build. Then i tested annotating the helm install like this 👇 before making requests against the GW.

  --version 0.0.0-2861c1b5b182b278841eec609d0f2b497b150ce9 \
  --set controller.serviceAccount.annotations."azure\.workload\.identity/client-id"="d245ed76-8227-" \
  --set controller.serviceAccount.annotations."azure\.workload\.identity/tenant-id"="e5806cb5-f8f4-" \
  --set controller.image.repository="ehrnst.azurecr.io/ai-gateway-controller" \
  --set controller.image.tag="azure-managed-identity-v3"

I dont know if we should change to specifically creating a service account, or if we should continue with the patch approach during install or after.

• I noticed that the controller will create a secret (witht the bearer token) which is used by the extproc container during calls to Azure OpenAI. I did not inspect how this rotation work, but i did remove my identity and delete the secret to verify authentication agains azure stopped working.

[mathetake]

Thanks for the work! However, I have a suggestion that would drastically change the current code.

Right now, the impl is using the SA mounted by Envoy AI Gateway controller, and it distributes/copies that identity into the secret that is eventually mounted by extproc. That I think is a bad practice from security perspective since you have no idea to tell which pod is actually accessing the service since this makes potentially tens of envoy poids will access Azure sevice with the same identity.

So, I would suggest to do the same thing as #1394 like by using Azure SDK to get the token dynamically inside the extproc using the SA mounted by the Envoy pod, and insert the token in the header. That would mean you have to write more code in https://github.com/envoyproxy/ai-gateway/tree/main/internal/extproc/backendauth than controller package right now

[ehrnst]

Thanks for the work! However, I have a suggestion that would drastically change the current code.

Right now, the impl is using the SA mounted by Envoy AI Gateway controller, and it distributes/copies that identity into the secret that is eventually mounted by extproc. That I think is a bad practice from security perspective since you have no idea to tell which pod is actually accessing the service since this makes potentially tens of envoy poids will access Azure sevice with the same identity.

So, I would suggest to do the same thing as #1394 like by using Azure SDK to get the token dynamically inside the extproc using the SA mounted by the Envoy pod, and insert the token in the header. That would mean you have to write more code in https://github.com/envoyproxy/ai-gateway/tree/main/internal/extproc/backendauth than controller package right now

I agree, I was just trying to get the origina #1183 to work. But I guess this will affect the current Azure authentication implementation as well?

I can see if i can make something work with the help of some collegues and various agents.

[mathetake]

I agree, I was just trying to get the origina #1183 to work. But I guess this will affect the current Azure authentication implementation as well?

no, we can make it work without affecting it.