ibm_container_vpc_alb resource does not honor IAM permissions correctly

[wosteven]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform IBM Provider Version

terraform -v
Terraform v0.15.0
on linux_amd64

• provider registry.terraform.io/ibm-cloud/ibm v1.26.2

Affected Resource(s)

ibm_container_vpc_alb
https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/container_vpc_alb

Terraform Configuration Files

resource "ibm_container_vpc_cluster" "iks_cluster" {
  name              = "mycluster"
  vpc_id             = "VPC in RG A"
  kube_version  = "1.20"
  resource_group_id = "ID from RG A"

  flavor                = ""
  worker_count   = 2

  zones { 
    name = "zone1, e.g. eu-de-1"
    subnet_id = "id from a subnet in zone1 in this VPC"
  }
}
...
resource "ibm_container_vpc_alb" "iks_cluster_alb_enable_private_only" {
  for_each = {for i, v in data.ibm_container_vpc_cluster.iks_cluster.albs:  i => v} 
  alb_id = each.value.id
  enable = each.value.alb_type == "private" ? true : false
}

Debug Output

https://gist.github.com/wosteven/6caeffef41dfa6498a0a889fd4310571

Panic Output

Expected Behavior

ALB enablement/disable should work as it also works with CLI with same user.
ibmcloud ks ingress alb enable vpc-gen2 --alb XXX -c YYY

Actual Behavior

TF fails with permission error:

 Error: Error waiting for cluster resource availabilty () : Request failed with status code: 401, ServerErrorResponse: {"incidentID":"15bd324a-81ec-4d43-9554-9dbb995b885e,15bd324a-81ec-4d43-9554-9dbb995b885e","code":"A0007","description":"This action requires the Viewer role for the cluster in IBM Cloud Kubernetes Service. Contact the IBM Cloud account administrator and request the required Identity and Access user role.","type":"Authentication"}
│
│   on main.tf line 28, in resource "ibm_container_vpc_alb" "iks_cluster_alb_enable":
│   28: resource "ibm_container_vpc_alb" "iks_cluster_alb_enable" {

Says "This action requires the Viewer role for the cluster in IBM Cloud Kubernetes Service." but user has "Administrator" Role on this cluster, he even created it.

Doing same task with same user via IBM Cloud CLI succeeds:

ibmcloud ks ingress alb enable vpc-gen2 --alb XXX -c YYY

As CLI works,we assume TF does check permissions differently/wrong.

Steps to Reproduce

• Create a RG "A" containing a VPC
• Create a IAM Access Group "AG_A" granting platform role "Administrator" + Service Role "Manager" for IKS,CR,VPC scoped to RG "A" , like:

Policy ID:   1 (on RG itself)
Roles:       Operator, Editor, Viewer   
Resources:                               
             Resource Group Name   A      
                              
Policy ID:   2 (full access to VPC)
Roles:       Administrator, Editor, Operator, Viewer, VirtualServerConsoleAdmin, IPSpoofingOperator   
Resources:                               
             Resource Group Name   A      
             Service Name          is      
                
Policy ID:   3 (full access to CR)
Roles:       Administrator, Manager, Writer, Reader   
Resources:                               
             Resource Group Name   A
             Service Name          container-registry                      
                               
Policy ID:   4 (full access to IKS)
Roles:       Administrator, Manager, Writer, Reader   
Resources:                               
             Resource Group Name   A
             Service Name          containers-kubernetes 

• With a user which is member in "AG_A"

• Create a IKS Cluster in RG "A" using TF
• Try to enable / disable an ALB using TF with "ibm_container_vpc_alb" resource

Important Factoids

References

• #0000

[hkantare]

@wosteven Unlike API/CLI terraform is mainly used in doing an e2e automation.....As part of ibm_container_vpc_alb resource first we are making some set of calls to make sure cluster (worker) are in healthy state....

terraform-provider-ibm/ibm/resource_ibm_container_vpc_alb.go

Line 110 in 580715a

_, err = waitForVpcClusterAvailable(d, meta, albID, schema.TimeoutCreate)

terraform-provider-ibm/ibm/resource_ibm_container_vpc_alb.go

Line 282 in 580715a

workerPools, err := albClient.WorkerPools().ListWorkerPools(cluster, targetEnv)

terraform-provider-ibm/ibm/resource_ibm_container_vpc_alb.go

Line 287 in 580715a

workers, err := albClient.Workers().ListByWorkerPool(cluster, wpool.ID, false, targetEnv)

1)A call to retreive all worker pools assigned to a Cluster
2) A call to retreive workers in each worker pool
3)Then wait for each worker worker to be in desired state (normal state)

May be we need to confirm with IKS API team based on the incident ID why we need a Viewer role even if we have an Administrator role

[wosteven]

@hkantare from my perspective IKS permissions are very inconsistent. I guess the issue you see ("why we need a Viewer role") is that IKS wants an Account Wide Viewer Platform Role , so having Administator Platform role on the RG containing the cluster is not enough, but it should be enough.

[hkantare]

@wosteven OK...Can we take this up to IKS team ?

[wosteven]

@hkantare sure, if they need to fix this

[rrpolanco]

I suspect the authorization error is due to the X-Auth-Resource-Group header missing on the alb enable api.

Is it possible to pass the resource group id of the IKS cluster as a header to the enableAlb API via the container_vpc_alb TF resource?

[hkantare]

@rrpolanco I think its failing even before we are calling enable ALb API"s..It failing before that see the comments above
#2768 (comment)

[rrpolanco]

@hkantare I suspect the reason it's failing is due to the GetAlb API:

terraform-provider-ibm/ibm/resource_ibm_container_vpc_alb.go

Line 279 in 580715a

albInfo, err := albClient.Albs().GetAlb(albID, targetEnv)

The reason I state this is if your cluster resource is in a non-default resource group and you have at least viewer platform permission for container-kubernetes service, when you call the GetAlb API directly it fails, while the CLI equivalent ibmcloud ks ingress alb get --alb ALB_ID works. This is because the ibmcloud CLI adds the resource group ID as a header parameter to the request.

[kavya498]

We are working on the fix..

[hkantare]

We fixed the issue and added a optional argument
resource_group_id which can be set if the cluster is not in default resource group....

The fix will be available in up coming release v1.35.0