Add default support to object

[atavakoliyext]

Current Terraform Version

0.12.6

Use-cases

I have a variable that takes a map of objects with many fields. Most of the fields in these objects have defaults that are rarely changed during typical use. I'd prefer that users could just exclude them, rather than set them to null/empty & have the module code take that as an indicator to use a default.

I'd also prefer to continue to use objects instead of map(any), in order to get the type checking benefits that objects provide, and making sure users don't typo a key that doesn't exist, which would silently be accepted in a map.

For a more concrete example, suppose I have a custom module for defining build pipelines in a CI system, and the following variable in my module:

variable "stages" = {
  type = map(object({
    name = string
    steps = list(object({
      name = string
      cmd  = string
    })
    deps = list(string)
    on_succeeded_deps = string
    on_failed_deps = string
    artifacts = object({
      in  = list(string)
      out = list(string)
    })
    env = map(string)
  })
}

This is how it might be used:

module "pipeline" {
  source = "..."

  stages = {
    notify = {
      name = "Notify"
      steps = module.pipeline_defaults.steps.notify
      deps = []
      on_succeeded_deps = null
      on_failed_deps = null
      artifacts = {
        in  = []
        out = []
      }
      env = {}
    }
    build = {
      name = "Build"
      steps = module.pipeline_defaults.steps.build
      deps = ["notify"]
      on_succeeded_deps = "START"
      on_failed_deps = "CANCEL"
      artifacts = {
        in  = []
        out = ["package.tar.gz"]
      }
      env = {}
    }
    test = {
      name = "Test"
      steps = module.pipeline_defaults.steps.test
      deps = ["build"]
      on_succeeded_deps = "START"
      on_failed_deps = "CANCEL"
      artifacts = {
        in  = ["package.tar.gz"]
        out = ["coverage.json"]
      }
      env = {}
    }
    deploy_qa = {
      name = "QA"
      steps = module.pipeline_defaults.steps.deploy
      deps = ["test"]
      on_succeeded_deps = "START"
      on_failed_deps = "CANCEL"
      artifacts = {
        in  = ["package.tar.gz"]
        out = []
      }
      env = {
        SITE = "qa"
      }
    }
    acceptance = {
      name = "Acceptance"
      steps = [/* some custom set of steps not in module.pipeline_defaults */]
      deps = ["deploy_qa"]
      on_succeeded_deps = "START"
      on_failed_deps = "PAUSE"
      artifacts = {
        in  = ["package.tar.gz"]
        out = ["coverage_acceptance.json"]
      }
      env = {}
    }
    prod = {
      name = "Production"
      steps = module.pipeline_defaults.steps.deploy
      deps = ["acceptance"]
      on_succeeded_deps = "PAUSE"
      on_failed_deps = "PAUSE"
      artifacts = {
        in  = ["package.tar.gz"]
        out = []
      }
      env = {
        SITE = "prod"
      }
    }
  }
}

Whereas what I'd really like to do is have this as my variable:

variable "stages" = {
  type = map(object({
    name = string

    steps = {
      type = list(object({
        name = string
        cmd  = string
      })
      default = null // my module will pick a default based on name
    }

    deps = {
      type    = list(string)
      default = []
    }

    on_succeeded_deps = { default = "START" }
    on_failed_deps    = { default = "CANCEL" }

    artifacts = {
      type = object({
        in = {
          type = list(string)
          default = []
        }
        out = {
          type = list(string)
          default = []
        }
      })
      default = { in = [], out = [] }
    }

    env = {
      type    = map(string)
      default = {}
    }
  })
}

And use it like this:

module "pipeline" {
  source = "..."

  stages = {
    notify = {
      name = "Notify"
    }

    build = {
      name      = "Build"
      deps      = ["notify"]
      artifacts = { out = ["package.tar.gz"] }
    }

    test = {
      name      = "Test"
      deps      = ["build"]
      artifacts = {
        in  = ["package.tar.gz"]
        out = ["coverage.json"]
      }
    }

    deploy_qa = {
      name      = "QA"
      deps      = ["test"]
      artifacts = { in = ["package.tar.gz"] }
      env       = { SITE = "QA" }
    }

    acceptance = {
      name           = "Acceptance"
      deps           = ["deploy_qa"]
      on_failed_deps = "PAUSE"
      steps          = [
        /* custom steps */
      ]
      artifacts = {
        in  = ["package.tar.gz"]
        out = ["coverage_acceptance.json"]
      }
    }

    prod = {
      name              = "Production"
      deps              = ["acceptance"]
      on_succeeded_deps = "PAUSE"
      on_failed_deps    = "PAUSE"
      artifacts         = { in = ["package.tar.gz"] }
      env               = { SITE = "prod" }
    }
  }
}

Attempted Solutions

I've tried setting the optional fields to null, and then handling the null cases in my module code with conditionals; however, this results in some difficult to read module code. I've also tried doing the null checks in locals and using those in the rest of the module code. While this keeps the module code simpler, I'd prefer to just be able to use the variables in my module code without the extra layer of abstraction.

Neither of the above attempts solved the issue that users of the module would still have to set these all fields to null/empty values to get their defaults applied. I tried solving this by pulling the most commonly defaulted fields into a single config object that could be set to null, but this required users to still define the whole config object, with mostly null fields, when they only wanted to change one field from the default. I also tried using map(any), but this resulted in the loss of type checking & added risk of key name typos being silently ignored.

Proposal

I propose that object type definitions have the following syntax:

object({
  <KEY> = <TYPE> | { [type = <TYPE>,][ default = ...] }
})

This would allow us to define variables like this:

variable "my_object" {
  type = object({
    foo = string
    bar = { default = "bar" } # type is inferred as string
    baz = {
      type    = list(string)
      default = ["foo", "bar", "baz"]
    })
  })
}

This allows defaults to be specified at any level of a complex type definition (like in my use-case above), and has parity with the existing syntax for how variables specify their types/defaults, but at the object field level.

For primitive types, only default is required, as the type can be inferred from it. For complex types, or if default = null, type is required. default could probably be made optional when type is present, making <KEY> = { type = <TYPE> } behave identically to <KEY> = <TYPE>.

References

• Optional arguments in object variable type definition #19898

The issue above is a feature proposal for marking certain object fields as optional(...). My proposal is similar in that fields with defaults are naturally optional, but is more generic in that the default value can be specified, and my proposal has parity with the existing default syntax for variables, but applied to object fields. There was a comment in that same issue on specifying partial defaults at the variable level, but IMHO this is a bit harder to reason about than specifying defaults closer to where they apply, when you have complex nested structures.

[apparentlymart]

Hi @atavakoliyext! Thanks for sharing this and sorry nobody replied before; I think this request ended up being discussed as part of the broader issue you linked to about optional attributes, so we didn't remember to update this one until now.

We're intending to ship support for optional attributes with optional default values in the forthcoming v1.3.0 release, and the behavior is already included in the latest v1.3.0 alpha release in case you'd like to try it out early. The syntax is a little different than what you suggested here but I believe the functionality is equivalent.

Thanks!

[atavakoliyext]

Thanks @apparentlymart :) at the time I created this issue, the main gap in the optional(...) proposal was a lack of a default value, but that appears to have been long taken care of

It'd be great if the two ways of specifying defaults/optionals were unified at some point in the future, but I'm excited to use the v1.3.0 feature and, if it turns out there's still a strong need for something like what I described in this thread, I'll be happy to open a new ticket

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.