Add -update flag to terraform init

[mwarkentin]

Hello,

We've noticed some strange behaviour around changes that are made in modules which we download from git. The first time we do a atlantis plan, the plan will not reflect the latest changes in that module. If we then discard the plan and re-plan, we see the change picked up.

As an example, I had made a change to switch this lambda function to nodejs8.10, however the plan was still showing nodejs4.3:

+ module.sumologic_cloudwatch.aws_lambda_function.sumo_autosub[1]
      id:                                        <computed>
      arn:                                       <computed>
      environment.#:                             "1"
      environment.0.variables.%:                 "2"
      environment.0.variables.LAMBDA_ARN:        "arn:aws:lambda:us-east-1:****:function:scylla-sumologic-cloudwatch"
      environment.0.variables.LOG_GROUP_PATTERN: "dms-tasks-datatruck"
      function_name:                             "scylla-sumologic-autosub-1"
      handler:                                   "cloudwatch-autosub.handler"
      invoke_arn:                                <computed>
      last_modified:                             <computed>
      memory_size:                               "128"
      publish:                                   "false"
      qualified_arn:                             <computed>
      role:                                      "arn:aws:iam::****:role/****"
      runtime:                                   "nodejs4.3"

After discarding the plan via Atlantis UI (having a atlantis discard comment command might be nice), and then asking atlantis to plan again, the plan looks right:

+ module.sumologic_cloudwatch.aws_lambda_function.sumo_autosub[1]
      id:                                        <computed>
      arn:                                       <computed>
      environment.#:                             "1"
      environment.0.variables.%:                 "2"
      environment.0.variables.LAMBDA_ARN:        "arn:aws:lambda:us-east-1:****:function:scylla-sumologic-cloudwatch"
      environment.0.variables.LOG_GROUP_PATTERN: "dms-tasks-datatruck"
      function_name:                             "scylla-sumologic-autosub-1"
      handler:                                   "cloudwatch-autosub.handler"
      invoke_arn:                                <computed>
      last_modified:                             <computed>
      memory_size:                               "128"
      publish:                                   "false"
      qualified_arn:                             <computed>
      role:                                      "arn:aws:iam::****:role/****"
      runtime:                                   "nodejs8.10"

The module is instantiated like this:

module "sumologic_cloudwatch" {
  source = "git@github.com:****/****.git//modules/sumologic-collector?ref=master"

  environment = "${var.environment}"

  log_group_patterns = [
    "datatruck-${var.environment}",
    "dms-tasks-datatruck",
  ]
}

We haven't noticed this same behaviour w/ local relative modules (../modules/foo). Any suggestions for debugging or information we can provide?

[mwarkentin]

Looks like #41 is similar, not sure if there's been any further thoughts on any changes or if we should just go ahead and implement the workaround provided there.

[lkysow]

@mwarkentin did you try the workaround from the issue you linked:

extra_arguments:
- command_name: init
  arguments:
  - "-upgrade=true"

Just wondering if that's the flag that needs to be added.

[mwarkentin]

Another dev did the work but I believe this is what we ended up with in our config:

workflows:
  default:
    plan:
      steps:
      - init:
          extra_args: ["-upgrade"]
      - plan

[lkysow]

Okay great! Should I close this issue?

[mwarkentin]

I will leave that to you to decide. I did find it a pretty surprising issue and we had to spelunk in the github issues to figure out how to fix it, so calling it out in the docs (unless I just missed it) might be a good minimum fix?

[mwarkentin]

The workaround we did until we figured out the extra arg incantation was extremely annoying:

• atlantis plan
• wait
• delete plan through UI
• atlantis plan
• wait
• atlantis apply

[lkysow]

To be honest I'm not sure of the pros/cons of setting this to true by default. I'm assuming there's a reason it's not true by default already. I'll have to look into it.

[mwarkentin]

There was some explanation in the other issue, but the default really doesn’t work well for a workflow like ours where we track the latest master branch of a module.

yea, it was done so we don't force update modules without people knowing. We currently only run terraform init with the new versions of terraform that in turn downloads all the projects dependencies which includes the modules.

I can see how it would work fine with the use case you mentioned where you point to specific versions (although in that case adding -update wouldn’t seem to have a negative impact either).

[lkysow]

So after talking to the Terraform team, -upgrade defaults to false because they wanted terraform init to be idempotent. In our case, adding -upgrade makes sense though because the benefit of it getting the latest version is worth losing the idempotency.

[lkysow]

I've updated the title to reflect what needs to be done

[Trillon008]

Another way it can be done without losing terraform init idempotency is to implement terraform get command. Make this command defaults to terraform get --update. But this is less trivial work.

[lkysow]

Closed by #495