Issue with Installing Trivy-Operator on GKE Autopilot Cluster

[mohammadzaidsha]

Hi Team,

I am encountering an issue while trying to install the Trivy-Operator using the Helm chart on our GKE Autopilot cluster. Our Kubernetes cluster is created in Autopilot mode and configured using Terraform. Below is the error message I received in the log of 'trivy-operator' :

creating job: admission webhook "warden-validating.common-webhooks.networking.gke.io" denied the request: GKE Warden rejected the request because it violates one or more constraints.
Violations details: {
  "[denied by autogke-disallow-hostnamespaces]": [
    "enabling hostPID is not allowed in Autopilot."
  ],
  "[denied by autogke-no-write-mode-hostpath]": [
    "hostPath volume var-lib-etcd used in container node-collector uses path /var/lib/etcd which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].",
    "hostPath volume var-lib-kubelet used in container node-collector uses path /var/lib/kubelet which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].",
    "hostPath volume var-lib-kube-scheduler used in container node-collector uses path /var/lib/kube-scheduler which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].",
    "hostPath volume var-lib-kube-controller-manager used in container node-collector uses path /var/lib/kube-controller-manager which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].",
    "hostPath volume etc-systemd used in container node-collector uses path /etc/systemd which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].",
    "hostPath volume lib-systemd used in container node-collector uses path /lib/systemd which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].",
    "hostPath volume etc-kubernetes used in container node-collector uses path /etc/kubernetes which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].",
    "hostPath volume etc-cni-netd used in container node-collector uses path /etc/cni/net.d which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/]."
  ],
  "[denied by autogke-node-affinity-selector-limitation]": [
    "Key 'kubernetes.io/hostname' is not allowed with node selector; Autopilot only allows labels with keys: cloud.google.com/compute-class,cloud.google.com/machine-family,cloud.google.com/gke-ephemeral-storage-local-ssd,cloud.google.com/gke-spot,cloud.google.com/gke-placement-group,topology.kubernetes.io/region,topology.kubernetes.io/zone,failure-domain.beta.kubernetes.io/region,failure-domain.beta.kubernetes.io/zone,cloud.google.com/gke-os-distribution,kubernetes.io/os,kubernetes.io/arch,cloud.google.com/private-node,sandbox.gke.io/runtime,cloud.google.com/gke-accelerator,cloud.google.com/gke-accelerator-count,iam.gke.io/gke-metadata-server-enabled,cloud.google.com/reservation-name,cloud.google.com/gke-tpu-accelerator,cloud.google.com/gke-tpu-topology,cloud.google.com/reservation-project,cloud.google.com/reservation-affinity,cloud.google.com/gke-gpu-sharing-strategy,cloud.google.com/gke-max-shared-clients-per-gpu,cloud.google.com/gke-gpu-partition-size,cloud.google.com/pods-per-node,cloud.google.com/gke-boot-disk,cloud.google.com/gke-boot-disk-size,cloud.google.com/gke-gpu-driver-version,cloud.google.com/gke-nccl-fastsocket,cloud.google.com/pods-per-node,autoscaling.gke.io/provisioning-request."
  ]
}

Requested by user: 'system:serviceaccount:scanning:trivy-operator', groups: 'system:serviceaccounts,system:serviceaccounts:scanning,system:authenticated'.",

Here is my main.tf as configure in terraform code to deploy in my existing cluster using helm chart

resource "kubernetes_namespace" "scanning" {
  metadata {
    name = "scanning"
  }
}

resource "helm_release" "trivy" {
  name             = "trivy-operator"
  repository       = "oci://ghcr.io/aquasecurity/helm-charts"
  namespace        = kubernetes_namespace.scanning.id
  chart            = "trivy-operator"
  version          = "0.21.4"
  timeout          = 1200
  create_namespace = true

  set {
    name  = "trivy.ignoreUnfixed"
    value = "true"
  }
}

Environment:

• Trivy-Operator version 0.21.4:
• GKE version 1.30.5-gke.1443001

[itaysk]

these are all required by the node-collector. it is a component that is required for some parts of the CIS benchmarks. In trivy, you can disable the node collector using the --disable-node-collector flag. In trivy-operator you can set the same flag using Helm value: nodeCollector.useNodeSelector: false