PowerVS VSI update in-place with 'ibm_pi_instance' times out when VSI status is SHUTDOWN

[jaywcarman]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform IBM Provider Version

$ terraform -v
Terraform v1.3.6
on linux_amd64
+ provider registry.terraform.io/ibm-cloud/ibm v1.48.0

Affected Resource(s)

• ibm_pi_instance

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

terraform {
  required_providers {
    ibm = {
      source  = "IBM-Cloud/ibm"
      version = "1.48.0"
    }
  }
}

provider "ibm" {
  ibmcloud_api_key = var.ibmcloud_api_key // export IC_API_KEY = "<api key>"
  region           = var.region           // export IBMCLOUD_REGION = "<region>"
  zone             = var.zone             // export IBMCLOUD_ZONE = "<zone>"
}

resource "ibm_pi_instance" "resize-test" {
  pi_cloud_instance_id = var.cloud_instance_id
  pi_instance_name     = var.instance_name
  pi_memory            = var.memory
  pi_processors        = var.processors
  pi_sys_type          = var.sys_type
  pi_proc_type         = var.proc_type
  pi_image_id          = var.image_id
  pi_network { network_id = var.network_id }
}

Debug Output

Note: The gist include output from both the working scenario (VSI in 'ACTIVE'
state) and hang scenario (VSI in 'SHUTOFF' state).

https://gist.github.com/jaywcarman/2dde2d72483861250b668aac7e9d78e6

Panic Output

No panic occurs.

Expected Behavior

The ibm_pi_instance update in-place action should complete once the VM status
is no longer 'RESIZE', whether the new state is 'ACTIVE', 'SHUTOFF', or
possibly others (I'm not familiar with all possible states, anything other than
error type states should be OK).

Actual Behavior

The ibm_pi_instance update in-place action times out polling for the VSI state
to be 'ACTIVE'.

Steps to Reproduce

The Terraform code required to recreate is available here:
https://github.com/jaywcarman/terraform-provider-ibm/tree/ibm-power-resize/examples/ibm-power-resize

1. Create a PowerVS VSI. See project above. I set variables in a var.tfvars
   file, export IBMCLOUD_API_KEY as an env var and apply via:

   terraform apply -var-file var.tfvars -var ibmcloud_api_key="$IBMCLOUD_API_KEY"
   

2. Shutdown the PowerVS VSI using ibmcloud cli or https://cloud.ibm.com web GUI.

3. Modify the ibm_pi_instance's pi_memory and/or pi_processors
   attributes. We want to update the VSI in place.

4. Wait an hour for the operation to time out ☹️

Important Factoids

The resize action is successful in all cases, even when Terroform exits with
Error: context deadline exceeded.

References

• #0000

[jaywcarman]

In the Terraform DEBUG logs I see the following:

2022-12-22T09:22:43.450-0600 [WARN]  Provider "registry.terraform.io/ibm-cloud/ibm" produced an unexpected new value for ibm_pi_instance.resize-test during refresh.
      - .status: was cty.StringVal("ACTIVE"), but now cty.StringVal("SHUTOFF")
2022-12-22T09:22:43.453-0600 [WARN]  Provider "registry.terraform.io/ibm-cloud/ibm" produced an invalid plan for ibm_pi_instance.resize-test, but we are tolerating it because it is using the legacy plugin SDK.

gist ref: https://gist.github.com/jaywcarman/2dde2d72483861250b668aac7e9d78e6#file-ibm_pi_instance_resize_shutoff_vsi-tf_debug-log-L155-L157

I also see when the VSI status transitions from VERIFY_RESIZE at line 332 to SHUTOFF at line 355.

All of these have a [WARN] unexpected data: registry.terraform.io/ibm-cloud/ibm:stderr=... message.

[jaywcarman]

Looks like it's "Waiting for PIInstance (db84e59e-a0df-4ff6-be9f-8571ec923171) to be available and active":
https://gist.github.com/jaywcarman/2dde2d72483861250b668aac7e9d78e6#file-ibm_pi_instance_resize_shutoff_vsi-tf_debug-log-L300

Here's the source code where this is called:
https://github.com/IBM-Cloud/terraform-provider-ibm/blob/master/ibm/service/power/resource_ibm_pi_instance.go#L644

I updated PR #4259 to fix this.