Description
TL;DR
Attempting to configure IP Masquerading in an AutoPilot cluster doesn't actually work.
Terraform apply errors with:
Error configmaps is forbidden: User "some-super-admin-serviceaccount" cannot create resource "configmaps" in API Group "" in the namespace "kube-system": GKEAutopilot authz: the namespace "kube-system" is managed and the request's verb "create" is denied.
Expected behavior
I"m not actually sure because of how ip-masquerading is implemented in AutoPilot: From the Docs:
GKE enforces the IP masquerading behavior for Autopilot clusters through the following process:
GKE deploys the Egress NAT controller and the ip-masq-agent.
You create the Egress NAT policy.
The GKE controller translates the policy into the ip-masq-agent ConfigMap.
The ip-masq-agent DaemonSet reads the ConfigMap and then GKE enforces the IP masquerading behavior.
Observed behavior
Failure as listed above,
Terraform Configuration
pretty much same kindaas the example autopilot cluster code is with the following values set:
configure_ip_masq = true
non_masquerade_cidrs = [data.google_compute_subnet.subnetwork.ip_cidr_range] #This is our SharedVPC Node Subnet
#tested the below with/without and same results
disable_default_snat = trueTerraform Version
Terraform 1.3.7Additional information
Our workaround at the moment is to remove the 10.0.0.0/8 CIDR from the default egressnatpolicy, as our GKE Node subnet is a 10.x address. (actually done via ACM). But was hoping this could be handled better in the module. It's 100% possible i'm just doing this wrong too and if so would appreciate the guidance.