Refresh Dataflow flex template job options when reading

[andremarianiello]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment. If the issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If the issue is assigned to a user, that user is claiming responsibility for the issue. If the issue is assigned to "hashibot", a community member has claimed the issue already.

Description

When reading a flex template job, none of the pipeline options are pulled from the job response. This means that if the tf state gets out of sync with the running jobs options, refresh does not sync them up again. It would be better behavior for those options to be read from the response and stored in the tf state. The options can be found in environment.sdkPipelineOptions.options in the job response.

New or Affected Resource(s)

• google_dataflow_flex_template_job

Potential Terraform Configuration

resource "google_dataflow_flex_template_job" "job" {
  provider                = google-beta
  name                    = "myjob"
  container_spec_gcs_path = "gs://${data.google_storage_bucket_object.flex_template.bucket}/${data.google_storage_bucket_object.flex_template.name}"
  on_delete               = "cancel"
  parameters = {
    schemaLocation = "gs://mybucket/schema.json"
    qps            = "1"
    topic          = "projects/myproject/topics/mytopic"
  }
}

Currently, nothing refreshes the parameters attribute in this resource.

References