feat: auto-plan projects when modules change

[iamnoah]

Fixes #920

See testdata for an example repo structure. Any project that uses modules in the same repo should be able to take advantage.

Only files that are not considered part of a project will be considered. With the current auto-plan heuristic, only changed files in /modules/ without a main.tf would trigger a check of the modules dependencies.

Adds --autoplan-modules and --autoplan-modules-from-projects

[jamengual]

What are the repo structure and the module instantiation used?

a more detailed example could help to understand the feature

[iamnoah]

@jamengual updated the description to point to the test data. It's layout agnostic as it parses the module dependencies out of the projects.

[jamengual]

so this only works when the source = is relative to the checked out repo not when the source = cloudposse/vpc/aws which is the terraform registry ?

[iamnoah]

@jamengual correct, it's only detecting changes to local modules.

[iamnoah]

Sorry for the churn, in testing this out, I've squashed some bugs and cleaned things up.

I think the docs now lay out the use case. Since it's a departure from current behavior, it requires a flag to activate, but I think --autoplan-modules does what one would intuitively expect. If the module indexing does something undesired, e.g., take a long time or fail on unusual projects, it can be disabled selectively by changing its pattern.

[iamnoah]

Anything else I need to do to have this considered? A week of using it in our pipelines hasn't revealed any new issues.

[pauloconnor]

This sounds similar to something I have in my atlantis.yaml generation script. I'm curious to know if you can disable autoplanning for individual child modules. For example, I have a child module that is called hundreds of times that I don't want to autoplan for obvious reasons, so I include a header in my main.tf file so my script knows to ignore it.

[iamnoah]

@pauloconnor yup! --autoplan-file-list=**/*.tf...,!modules/really-common-child-module/* would keep changes to that module from being planned.

[pauloconnor]

@iamnoah Damn, that's fantastic! Hopefully this'll get merged in soon.

[iamnoah]

@jamengual rebased and tried to fix linter issues. LMK what can help move this forward.

[nitrocode]

@iamnoah please merge from the default branch to fix the branch test failure

[jamengual]

@iamnoah can you fix the conflicts?

[nitrocode]

@iamnoah please fix conflicts

[nitrocode]

@iamnoah please fix linting error in failed branch pr check

[nitrocode]

@iamnoah apologies but looks like this PR has some conflicts, could you resolve them?

[jamengual]

@iamnoah Thanks for the contribution

[lattwood]

@jamengual what are the chances of getting a release cut with this in it? Looks like the merge happened less than two hours after the release was tagged.

[jamengual]

we do pre-release for new features before they go to the stable branch.

We will do a prerelease soon.