Allow importing a file with specific overrides upon import without changing import #307

Activity

1. osterman commented on Jan 20, 2023

   

   Erik Osterman (Cloud Posse) (osterman)

   on Jan 20, 2023

   Member

   More actions

   Also, could be implemented using #267
2. osterman commented on Jan 20, 2023

   

   Erik Osterman (Cloud Posse) (osterman)

   on Jan 20, 2023

   Last edited by osterman

   Member

   More actions

   I like this syntax, borrowed/inspired from "Home Assistant"

   This is orgs/acme/plat/dev/us-east-1.yaml, which deploys 2 EKS clusters, blue and green.

   imports:
   - path: catalog/eks/cluster
     input:
       color: blue
   - path: catalog/eks/cluster
     input:
       color: green
   

   This is catalog/eks/cluster.yaml

   inputs:
     color:
       type: string
       values:
       - blue
       - green
       description: The color of the cluster
   
   components:
     terraform:
       eks-{{ inputs.color }}/cluster:
         vars:
           enabled: true
           name: {{ inputs.color }}
   
3. aknysh commented on Jan 20, 2023

   

   Andriy Knysh (aknysh)

   on Jan 20, 2023

   Member

   More actions

   yes, this is a good idea. It can be implemented like the following:

   1. We add imports keyword to have the following schema:

   imports:
   - path: catalog/eks/cluster
     input: {}  # Or context: {}

   We have import, but we don't want to change it for backwards compatibility

   2. We add support for Go templates into all files. If those files are imported, we process the templates with he data from input/context

   👍React with 👍1Reacted by Erik Osterman (Cloud Posse)
4. max-lobur commented on Jan 27, 2023

   

   Max Lobur (max-lobur)

   on Jan 27, 2023

   More actions

   Posting my proposal for reference https://cloudposse.slack.com/archives/CA4TC65HS/p1674832541932279
5. max-lobur commented on Jan 27, 2023

   

   Max Lobur (max-lobur)

   on Jan 27, 2023

   More actions

   "Ansible-style" yaml for atmos

   import:
     - catalog/eks/cluster
     - catalog/eks/karpenter
     - catalog/eks/karpenter-provisioner
     - catalog/eks/efs
   ...
   
   vars:
     region: us-east-2
     region_short: ue2
     tenant: plat
     stage: dev
     stack: "{{ tenant }}-{{ region_short }}"
     description: "Component instance {{ name }} at {{ stack }}"
     
   components:
     terraform:
       eks/reloader:
         vars:
           enabled: true
           name: reloader
           repository: "https://stakater.github.io/stakater-charts"
           chart: "{{ name }}"
           chart_version: "v1.0.1"
           create_namespace: true
           kubernetes_namespace: "{{ name }}"
           tags:
             Team: sre
             Service: "{{ name }}"
             Region: "{{ region }}"
             Tenant: "{{ Tenant }}"
             Stack: "{{ stack }}"
             Description: "{{ description }}"
           values:
             # https://github.com/stakater/Reloader/tree/master/deployments/kubernetes/chart/reloader
             rbac:
               enabled: true
             serviceAccount:
               create: true
               name: "{{ name }}"
             resources:
               limits:
                 cpu: "20m"
                 memory: "128Mi"
               requests:
                 cpu: "10m"
                 memory: "64Mi"
   

   note that vars defined on root level vars map (these are kind of global vars), and component level maps are both available at the component level. Both vars maps are self-defining - the map is rendered using itself:

   1. Deep meerge all vars maps up the tree
   2. vars_map.render(vars_map) - rendering the map using the map itself, so you can reuse vars in other vars
   3. pass rendered map to a component

   Then we basically define component templates in a catalog, and only importing catalog + customizing vars down the tree. And of course you can still override the whole component like currently

   I tried to put all possible relations in the example. Note how description is defined on top level uses name where name doesn’t exist yet. It means whenever you try to instantiate a component without name defined you will get a template error (undefined variable). So you can write templates using some variables that you assume will be defined later, in stack, in region, or down to component specific vars. In this way we can also standardize tags sets etc.
6. Andriy Knysh (aknysh)

   closed this as completedin #312on Jan 31, 2023
7. Erik Osterman (Cloud Posse) (osterman)

   added a commit that references this issue on Oct 2, 2026

   fix(security): remediate 4 more Dependabot alerts in website deps

   ...

   2cc0b8c