Support for content in azurerm_storage_blob data source

[gaui]

Is there an existing issue for this?

• I have searched the existing issues

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Currently the azurerm_storage_blob data source supports has the following outputs:

• id - The ID of the storage blob.
• url - The URL of the storage blob.
• type - The type of the storage blob
• access_tier - The access tier of the storage blob.
• content_type - The content type of the storage blob.
• content_md5 - The MD5 sum of the blob contents.
• metadata - A map of custom blob metadata.

I see that it supports content_md5 but no content. Is there any particular reason not to be able to fetch the content itself, but only the MD5 sum and the type? I would like to suggest that this would be made possible.

Reference:
https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/data-sources/storage_blob

New or Affected Resource(s)/Data Source(s)

azurerm_storage_blob

Potential Terraform Configuration

data "azurerm_storage_blob" "blob" {
  name                   = "blob.txt"
  storage_account_name   = azurerm_storage_account.account.name
  storage_container_name = azurerm_storage_container.container.name
}

output "blob_content" {
  value = data.azurerm_storage_blob.blob.content
}

References

No response

[magodo]

@gaui Thank you for submitting this! Currently, the azurerm_storage_blob data source is only exposing the properties of the blob via its get blob propertes API. In order to getting the content, it requires support on the storage SDK.

[gaui]

@magodo The API in the SDK is already supported: https://github.com/tombuildsstuff/giovanni/blob/master/storage/2020-08-04/blob/blobs/get.go

[tombuildsstuff]

hey @gaui

Thanks for opening this issue.

This issue's been requested previously (although unfortunately I can't find the issue/PR right now), and we've previously opted not to expose the contents from this Data Source since this would involve storing the file contents within the state file which would end up causing issues (both due to the size of the state file, and encoding, amongst other issues).

Instead it should be possible to retrieve an existing file using a SAS Token and the http data source - which would allow you to download this to a file, rather than storing this within the State File itself. Would you be able to take a look and see if that works for you here?

Thanks!

[tombuildsstuff]

hey @gaui

Since we've not heard back from you here I'm going to close this issue for now - but if that doesn't work for you then please let us know and we can take another look.

Thanks!

[jsok]

Instead it should be possible to retrieve an existing file using a SAS Token and the http data source - which would allow you to download this to a file, rather than storing this within the State File itself. Would you be able to take a look and see if that works for you here?

After attempting this, it's not exactly concise or easy:

data "azurerm_storage_account_blob_container_sas" "container" {
  connection_string = data.azurerm_storage_account.account.primary_connection_string
  container_name    = "my-container"

  start  = timestamp()
  expiry = timeadd(timestamp(), "1h")

  permissions {
    read   = true
    add    = false
    create = false
    write  = false
    delete = false
    list   = false
  }

  content_type = "application/json"
}

data "http" "artifacts" {
  url = "${data.azurerm_storage_account.account.primary_blob_endpoint}my-container/path/to/file.ext${data.azurerm_storage_account_blob_container_sas.cicd_metadata.sas}"

  lifecycle {
    postcondition {
      condition     = self.status_code == 200
      error_message = "Could not fetch: ${self.response_body}"
    }
  }
}

Surely proper data source support for this would be better.
In some cases the file is small and you'd be more than happy to keep it in the state file.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.