Azure blob storage transfer crash

[maitreya-source]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

• Can be reproduced in following versions :
  • v1.0.2
  • v0.14.5
  • v0.14.8

Affected Resource(s)

• google_storage_transfer_job

Terraform Configuration Files

provider "google-beta" {
}

data "google_storage_transfer_project_service_account" "default" {
  project       = "someproject"
}

resource "google_storage_bucket" "blob-backup-bucket" {
  name          = "somebucketname"
  storage_class = "NEARLINE"
  project       = "someproject"
}

resource "google_storage_bucket_iam_member" "blob-backup-bucket" {
  bucket     = google_storage_bucket.blob-backup-bucket.name
  role       = "roles/storage.admin"
  member     = "serviceAccount:${data.google_storage_transfer_project_service_account.default.email}"
  depends_on = [google_storage_bucket.blob-backup-bucket]
}

resource "google_storage_transfer_job" "azure-to-gcp-new" {
  description = "Storage Migrate from Az to GCP"
  project     = "someproject"

  transfer_spec {

    transfer_options {
      delete_objects_unique_in_sink = false
    } 

    azure_blob_storage_data_source {
      storage_account = "somenewstorageaccount"
      container = "somenewcontainer"
      path = ""

      azure_credentials {
        sas_token = var.sas_token
      } 
    }

    gcs_data_sink {
      bucket_name = google_storage_bucket.blob-backup-bucket.name
    }
  }

  schedule {
    schedule_start_date {
      year  = 2021
      month = 07
      day   = 9
    }
    schedule_end_date {
      year  = 2019
      month = 1
      day   = 15
    }
    start_time_of_day {
      hours   = 23
      minutes = 30
      seconds = 0
      nanos   = 0
    }
  }

  depends_on = [google_storage_bucket_iam_member.blob-backup-bucket]
}

Debug Output

• Debug logs

Panic Output

• Crash logs
• Crash logs after PR#4964

Expected Behavior

It should have been able to create the storage transfer job.

Actual Behavior

Panic / Terraform crash

Steps to Reproduce

1. terraform apply

Important Factoids

• Original issue - #9547
  • Same error after this issue has been closed.
• PR adding this feature
• The support for Azure Blob Storage Transfer in storage_transfer_job was added recently : Jun 5, 2021.

[maitreya-source]

Shouldn't the input argument of expandAzureCredentials() function be azureBlobStorageData["azure_credentials"] instead of azureBlobStorageData["sas_token"]? resource_storage_transfer_job.go#L854

[slevenick]

Hey @maitreya-source have you tested this with the current master version of the provider?

We have roughly weekly releases of the provider, and the most recent release will not contain the fix you linked in the bug report. You could build the most recent version of the provider from source and try it out yourself, or wait for the official release that is expected next week

[slevenick]

But yes you are correct, it should reference azure_credentials rather than sas_token as it is nested within that object. I'll get that fixed

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.