Update and enhance Atlantis Integration

[Andriy Knysh (aknysh)]

what

• Update and enhance Atlantis Integration
• Update and overhaul the Atlantis Integration docs https://pr-328.atmos-docs.ue2.dev.plat.cloudposse.org/integrations/atlantis/
• Add --format and --file flags to atmos describe component command

why

• Allow configuring Atlantis Integration in the settings.atlantis section in the YAML stack configs (instead, or in addition to, configuring it in integrations.atlantis in atmos.yaml)

  Configuring the Atlantis Integration in the settings.atlantis sections in the stack configs has the following advantages:

  • The settings section is a first class section in Atmos (similar to vars). It participates in deep-merging and in the inheritance chain. It can be defined and overridden at any level (organization/namespace, OU/tenant, region/environment, account/stage, base component, component). You can define the base settings at the org, tenant or account level, and then override some settings at the component level, making the whole configuration DRY

  • When executing the atmos atlantis generate repo-config command, you don't need to pass the --config-template and --project-template flags to specify which config and project templates to use. Instead, Atmos will get this information from the settings.atlantis section

  • When executing the atmos describe component <component> -s <stack> command, you will see the configured Atlantis Integration in the outputs. For example:

      atmos_component: test/test-component-override
      atmos_stack: tenant1-ue2-dev
      component: test/test-component
      settings:
        atlantis:
          config_template:
            allowed_regexp_prefixes:
            - dev/
            automerge: false
            delete_source_branch_on_merge: false
            parallel_apply: false
            parallel_plan: true
            version: 3
          config_template_name: config-1
          project_template:
            apply_requirements:
            - approved
            autoplan:
              enabled: true
              when_modified:
              - '**/*.tf'
              - varfiles/$PROJECT_NAME.tfvars.json
            delete_source_branch_on_merge: false
            dir: '{component-path}'
            name: '{tenant}-{environment}-{stage}-{component}'
            terraform_version: v1.3
            workflow: workflow-1
            workspace: '{workspace}'
          project_template_name: project-1
          workflow_templates:
            workflow-1:
              apply:
                steps:
                - run: terraform apply $PLANFILE
              plan:
                steps:
                - run: terraform init
                - run: terraform workspace select $WORKSPACE || terraform workspace new $WORKSPACE
                - run: terraform plan -out $PLANFILE -var-file varfiles/$PROJECT_NAME.tfvars.json
        spacelift:
          protect_from_deletion: true
          stack_destructor_enabled: false
          workspace_enabled: true
      vars:
        enabled: true
        environment: ue2
        namespace: cp
        region: us-east-2
        stage: dev
        tenant: tenant1
      workspace: test-component-override-workspace-override

  • If you configure the Atlantis Integration in the settings.atlantis sections in the stack configs, then the
    command atmos describe affected will be able to use it and output the affected Atlantis projects in the atlantis_project field. For example:

    [
       {
          "component": "infra/vpc",
          "component_type": "terraform",
          "component_path": "components/terraform/infra/vpc",
          "stack": "tenant1-ue2-dev",
          "atlantis_project": "tenant1-ue2-dev-infra-vpc",
          "affected": "component"
       },
       {
          "component": "infra/vpc",
          "component_type": "terraform",
          "component_path": "components/terraform/infra/vpc",
          "stack": "tenant1-ue2-prod",
          "atlantis_project": "tenant1-ue2-prod-infra-vpc",
          "affected": "component"
       }
    ]

• Add --affected-only flag to atmos atlantis generate repo-config command

  If you want to generate the atlantis.yaml file before Atlantis can parse it, you can use the Dynamic Repo Config Generation feature of Atlantis. You can add a run command to pre_workflow_hooks. The repo config will be generated right before Atlantis can parse it.

  repos:
    - id: /.*/
      pre_workflow_hooks:
        - run: "./repo-config-generator.sh"
          description: "Generating configs"

  To help with dynamic repo config generation, the atmos atlantis generate repo-config command accepts the --affected-only flag. If set to true, Atmos will generate Atlantis projects only for the Atmos components changed between two Git commits.

  repos:
    - id: /.*/
      pre_workflow_hooks:
        - run: "atmos atlantis generate repo-config --affected-only=true"
          description: "Generating configs"

  If the --affected-only=true flag is passed, Atmos uses two different Git commits to produce a list of affected Atmos components and stacks and then generate the atlantis.yaml file for the affected Atlantis projects only.

• Update atmos describe affected command to include the atlantis_project attribute in the output of the affected components and stacks. The attribute shows the affected Atlantis projects between two diff Git commits (Note that this works only when Atlantis integration is configured in the settings.atlantis sections in the YAML stack config (see the description above)

  [
     {
        "component": "infra/vpc",
        "component_type": "terraform",
        "component_path": "components/terraform/infra/vpc",
        "stack": "tenant1-ue2-dev",
        "atlantis_project": "tenant1-ue2-dev-infra-vpc",
        "affected": "component"
     },
     {
        "component": "infra/vpc",
        "component_type": "terraform",
        "component_path": "components/terraform/infra/vpc",
        "stack": "tenant1-ue2-prod",
        "atlantis_project": "tenant1-ue2-prod-infra-vpc",
        "affected": "component"
     }
  ]

• Add --format and --file flags to atmos describe component command - useful to have when working with the command in GitHub actions and OPA validation

related

• Closes atmos describe affected does not show the project name if the atlantis integration is enabled #294
• Closes implement atmos atlantis generate repo-config --affected-only #303

[RB (nitrocode)]

Does plan support the env key for arbitrary environment variables?

https://www.runatlantis.io/docs/custom-workflows.html#step

[Andriy Knysh (aknysh)]

i'm not sure I'm following, can you provide an example of that?

(the workflows configs are just copied to the atlantis.yaml file verbatim if you use the repo level workflows)

[RB (nitrocode)]

Oh ok so something like this would work then since atlantis supports? That means that this object is now an any?

    workflow_templates:
      workflow-1:
        plan:
          steps:
          - env:
              name: ENV_NAME
              value: hard-coded-value
          - env:
              name: ENV_NAME_2
              command: 'echo "dynamic-value-$(date)"'

If so, perfect :)

[Andriy Knysh (aknysh)]

atmos does not touch the internals of the workflows definitions. You can def use any ENV vars in there that are set in the process (atmos does not check what's inside the workflows and does not set any ENV vars for it)

[RB (nitrocode)]

I skimmed the code. Looks okay to me. Might help if PePe Amengual (@jamengual) looks it over too. I'll read over the docs today and submit a review

[Andriy Knysh (aknysh)]

I skimmed the code. Looks okay to me. Might help if PePe Amengual (@jamengual) looks it over too. I'll read over the docs today and submit a review

PePe Amengual (@jamengual) approved it a few days ago :)