Allow "count" for non-null check on resource attributes that are always present

[sgrimm]

Current Terraform Version

Terraform v0.13.4
+ provider registry.terraform.io/hashicorp/aws v3.10.0
+ provider registry.terraform.io/hashicorp/local v1.4.0
+ provider registry.terraform.io/hashicorp/template v2.2.0

Use-cases

I want a module to conditionally create a particular AWS Route53 entry if a zone ID is passed in. I'm using count and checking whether the variable is null. I'd like Terraform to be able to create everything, including the zone, in one plan/apply so a CI system can do the actual plan application.

variable "public_zone_id" {
  type    = string
  default = null
}

resource "aws_route53_record" "public_cname" {
  count = var.public_zone_id != null ? 1 : 0

  zone_id = var.public_zone_id
  name    = "${var.hostname}"
  type    = "CNAME"
  ttl     = 1800
  records = [aws_instance.this.public_dns]
}

And then in the calling project, I pass in the zone ID, like so:

module "foo" {
  source = "my_module"

  hostname       = "example"
  public_zone_id = module.vpc.public_zone_id
}

But because the count depends on a value from a resource that isn't created yet, terraform plan gives me

Error: Invalid count argument

  on ../modules/ec2/main.tf line 40, in resource "aws_route53_record" "public_cname":
  40:   count = var.public_zone_id != null ? 1 : 0

The "count" value depends on resource attributes that cannot be determined
until apply, so Terraform cannot predict how many instances will be created.
To work around this, use the -target argument to first apply only the
resources that the count depends on.

Attempted Solutions

In the calling project, I have tried various tricks to make it impossible for the variable to ever be null regardless of whether or not the original resource exists, all with the same result as above.

public_zone_id = coalesce(module.vpc.public_zone_id, "dummy")

public_zone_id = trimsuffix("${module.vpc.public_zone_id}!", "!")

public_zone_id = module.vpc.public_zone_id != null ? module.vpc.public_zone_id : "dummy"

Proposal

Although the specific value of the zone ID is unknown here, it seems like Terraform ought to be able to tell that there will definitely be some value that will definitely not be null, and thus that the count should be 1, even if the actual value can't be filled in until later. There are a lot of attributes on a lot of resources that are required to be present if the resource creation succeeds, and since Terraform defines null as the absence of a value, checking for != null should only require the presence of a value.

So the proposal is that in the specific case of count = some_resource.some_attribute != null ? X : Y where the resource will be created in the same Terraform invocation, Terraform evaluate the expression based on whether or not the attribute is guaranteed to be present, rather than based on the actual attribute value. Obviously the "guaranteed" part is important; for optional attributes the current behavior would be unavoidable and expected. But for attributes that are always exported and can never possibly be null, it should be possible to evaluate that expression statically at plan time.

References

• The "count" value depends on resource attributes that cannot be determined until apply, but resource attributes are already applied #26078 overlaps with this but isn't quite the same; what I'm asking for here is more narrowly focused than what people were asking for in the discussion there.

[apparentlymart]

Hi @sgrimm! Thanks for sending in this enhancement request.

I think the root problem here is that the Terraform schema model doesn't have any explicit sense of an attribute that is guaranteed to be non-null, and so Terraform conservatively assumes that any attribute could potentially end up being null. To fix this would require giving providers some way to either declare that a particular attribute is never null or to say dynamically in the plan "unknown value but definitely not null". The first seems significantly easier to implement than the second, but neither is trivial because they both involve changes to the provider wire protocol, and thus we must coordinate with the provider SDK team and the provider teams to make the change.

In the meantime, I think you could make this work by having the calling module handle the decision about whether it's null. You didn't show how module.vpc decides whether or not to return a zone id, but let's say for example that it's an enable_public_dns_zone variable on your root module, something like this:

variable "enable_public_dns_zone" {
  type = bool
}

module "vpc" {
  source = "./modules/vpc"

  enabled = var.enable_public_dns_zone
}

module "foo" {
  source = "./modules/foo"

  hostname       = "example"
  public_zone_id = var.enable_public_dns_zone ? module.vpc.public_zone_id : null
}

The key here is that Terraform does know the value for var.enable_public_dns_zone, so we can rely on that to take the module.vpc.public_zone_id value out of the decision entirely when var.enable_public_dns_zone is false, and thus it will be known in the case where it's null.

Another potential answer is to design your modules to deal in lists rather than single values, and then you can use an empty list to represent it not being active and a single-element list to represent it being active. This is consistent with Terraform's usual model of disabling things by having zero of them, and is advantageous because it separates the information about whether it's enabled (the length of the list) from the final value (the value of the first element of the list, which might be unknown). Terraform's language is primarily oriented around repetition of objects based on lists, and so modules designed around that principle tend to gel better with other language features.

[lievertz]

It appears that count depending on variable is fine, but if count depends on a variable through a transformation function like templatefile it is no longer fine and will likewise hit this issue. Perhaps I am wrong, but I think templatefile will always give a non-null result (or raise an error)? I wanted to bring up this situation because there is no provider involved.

My specific situation I hit this with is I have a module that may create an IAM policy if one is provided. I could use a var.json_doc AND a var.create_policy, but the boolean is implied by the presence of the doc, so the interface is (arguably) better without the explicit var.create_policy. Instead I just have count = var.json_iam_doc == null ? 0 : 1. This has been working, but I recently called this module from another module which provides the json_iam_doc variable as the result of a templatefile which itself accepts a string var -- that's where I hit this issue.

I confirmed this behavior on terraform v0.14.3 and v0.14.4.

To summarize -- it seems like the enhancement here also applies to function results which could arguably be known to be non-null. Since provider dependency was raised as an implementation barrier, it seemed relevant to raise that there are a number of relevant conditions that do not rely on the provider, but are functions within terraform itself.

I hope this is useful input -- thanks for all of your work!

[Cajga]

I run into the same. I use terraform 0.14.7. The simplest workaround that I found for the "using freshly created resource ID in count" is the following. (@apparentlymart may I ask you to verify it, just to make sure :))

Put his into the foo module:

locals {
  # note: putting [*] behind a single value will create a list (if it is null then it will be an empty list)
  some_id_list = var.some_id[*]
}

resource "something_which_depends_on_the_fresh_input_id" "this" {
  count = length(local.some_id_list)
.
.
}

variable "some_id" {
  type        = string
  default     = null
}

Then you call the module with the fresh id:

resource "aws_eip" "test" {
}

module "foo" {
  source = "../../"

  some_id = aws_eip.test.id
}

This will run with a single terraform apply.