ExternalDNS generated policy lacks ECR authToken generator permissions

[loicvolle]

Is your request related to a new offering from AWS?

Is this functionality available in the AWS provider for Terraform? See CHANGELOG.md, too.

• No 🛑: please wait to file a request until the functionality is avaialble in the AWS provider
• Yes ✅: please list the AWS provider version which introduced this functionality

Is your request related to a problem? Please describe.

We are using External Secrets Generator for ECR. The goal is to have external Secret generating a secret with an ECR token, in order to use ArgoCD and helm charts stored on ECR.

We use :

module "eks_role" {
  source  = "terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks"
  version = "~> 5.41"

  attach_external_secrets_policy = true

  role_name = "${var.eks_cluster_name}_${var.k8s_namespace}"
  role_path = "/"


  allow_self_assume_role = true

  oidc_providers = {
    eks = {
      provider_arn               = data.aws_iam_openid_connect_provider.eks_provider.arn
      namespace_service_accounts = ["${var.k8s_namespace}:${var.k8s_namespace}-sa"]
    }
  }
  tags = var.aws_tags
}

The policy lacks some ECR permissions which should be

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ecr:GetDownloadUrlForLayer",
                "ecr:BatchGetImage",
                "ecr:DescribeImages",
                "ecr:GetAuthorizationToken",
                "ecr:BatchCheckLayerAvailability"
            ],
            "Resource": "*"
        }
    ]
}

Yaml used to use the feature :

---
apiVersion: v1
kind: Secret
metadata:
  name: ecr-eu-central-1-secret
  namespace: argocd
stringData:
  AWS_USERNAME: AWS
  AWS_PASSWORD: TOTO
---
apiVersion: generators.external-secrets.io/v1alpha1
kind: ECRAuthorizationToken
metadata:
  name: ecr-eu-central-1
  namespace: argocd
spec:
  region: eu-central-1
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: ecr-auth-token-external-secret
  namespace: argocd
spec:
  dataFrom:
  - sourceRef:
      generatorRef:
        apiVersion: generators.external-secrets.io/v1alpha1
        kind: ECRAuthorizationToken
        name: ecr-eu-central-1
  refreshInterval: 30m
  target:
    creationPolicy: Owner
    deletionPolicy: Retain
    name: ecr-eu-central-1-secret
    template:
      data:
        AWS_PASSWORD: '{{ .password }}'
        AWS_USERNAME: '{{ .username }}'
      engineVersion: v2
      metadata:
        labels:
          argocd.argoproj.io/secret-type: repo-creds

Describe the solution you'd like.

Have the policy used when the parameter attach_external_secrets_policy is set to true

Describe alternatives you've considered.

Adding a custom policy or add AWSAppRunnerServicePolicyForECRAccess AWS Managed policy through a new parameter external_secrets_use_ecr_auth_token

Additional context

Documentation to ECR generator.

[bryantbiggs]

this permission is already present, no? the nodes themselves need these permissions to pull the necessary images so therefore the IAM roles that run on the nodes inherit these permissions

[loicvolle]

they are present on the node, but since the service account is used, the IAM role from the node cannot be used

[loicvolle]

See snippet of logs on the pod

{"level":"error","ts":1725541882.6136096,"logger":"controllers.ExternalSecret","msg":"could not get secret data from provider","ExternalSecret":"argocd/ecr-auth-token-external-secret","error":"could not generate [0]: unable to get authorization token: AccessDeniedException: User: arn:aws:sts::1234567890:assumed-role/common-tools-external-secrets/1725541882585093008 is not authorized to perform: ecr:GetAuthorizationToken on resource: * because no identity-based policy allows the ecr:GetAuthorizationToken action\n\tstatus code: 400, request id: ??????","stacktrace":"github.com/external-secrets/external-secrets/pkg/controllers/externalsecret.(*Reconciler).Reconcile\n\t/home/runner/work/external-secrets/external-secrets/pkg/controllers/externalsecret/externalsecret_controller.go:189\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:122\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:235"}
{"level":"error","ts":1725541882.6269062,"msg":"Reconciler error","controller":"externalsecret","controllerGroup":"external-secrets.io","controllerKind":"ExternalSecret","ExternalSecret":{"name":"ecr-auth-token-external-secret","namespace":"argocd"},"namespace":"argocd","name":"ecr-auth-token-external-secret","reconcileID":"??????","","error":"could not generate [0]: unable to get authorization token: AccessDeniedException: User: arn:aws:sts::1234567890:assumed-role/common-tools-external-secrets/1725541882585093008 is not authorized to perform: ecr:GetAuthorizationToken on resource: * because no identity-based policy allows the ecr:GetAuthorizationToken action\n\tstatus code: 400, request id: ??????","","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:235"}
{"level":"info","ts":1725541882.6448529,"logger":"provider.aws","msg":"using aws session","region":"eu-central-1","credentials":null}

Then I added the permissions manualy and that worked instantly

[bryantbiggs]

do your nodes have the arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly policy attached? you also have the ability to attach this policy along with your role if that is required

[loicvolle]

do your nodes have the arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly policy attached? you also have the ability to attach this policy along with your role if that is required

yep they do have. That's the workaround I will use : attach that policy to my role