This repository is a starting point of housing YAML templates for Azure DevOps (ADO) Pipelines. The format for these templates is generic enough that individuals can take what they need to create their own versions of these templates. This project has been started based on my interaction with folks within the community and hopefully can be leveraged as a learning tool for those looking how to best leverage YAML Templates within their organizations.
All templates, unless they are task or variable templates, will call other templates. There should not be any embedding of tasks within the job or stage levels. This is done to optimize reusability while also maintaining a single point of maintenance, the task for any changes. If changes are required to the task they should be done in a non breaking fashion, if this is not possible then use best judgement on creating a new task.
For getting started would recommend watching my presentation to the Omaha Azure User Group. In it will walk you through the evolution of using one YAML file per pipeline, templates, in the same pipeline, and lastly scaling it to the structure in this repo.
Every file will have a '_' followed by the type of template it is. This is to help easily identify what type of file is being called by the parent templates. '_env' means the job can be used across multiple environments. This helps indicate if this job can scale and help differentiate it from a build job/stage which may only execute once.
Each Azure environment will have it's one explicit variables. In this case it's the service connection related to the specific environment. I have only included 'dev'; however, the thought is multiple environments could be laid out. The assumption is the environment name passed in will match the abbreviation on the file.
This list is somewhat comprehensive starting at the stage level and moving on down. The thought here is a template is fully encapsulated by it's stage. For example if deploying just Bicep Infrastructure a completed template would consist of a stage template decomposing to jobs and the tasks which accomplishes the deployment of a Bicep file. A Bicep and dotNet deployment would be a stage, most likely, calling the same job templates the Bicep deployment template does and then also include the necessary dotNet templates.
I will try to keep this up to date; however, as software development goes sometimes closing the documentation loop is the last part. If something is missing or not clear please open up an issue. There very well could be additional stages that either aren't documented yet or I changed direction but kept the stage out there for future/others to use.
| Template Name | Description | Stage Template File |
|---|---|---|
| Retain ADO Pipelines | This template will apply a manual retention lease to a pipeline | retain_pipeline_stage.yml |
| Build Bicep File | Will publish the bicep infrastructure folders as pipeline artifact as well as run a validate and what-if command against designated environments |
bicep_build_stage.yml |
| Terraform Build | Will install Terraform, init, plan in parallel across environment and regions. Does require Azure Pipelines Terraform Task and the remote backend as configured in the azure_terraform_dev_variables.yml By default Terraform files are expected to be in a folder called 'Infrastructure' and leverage a variable file like dev.eus.variables.tfvars. This is easily changed be changed as needed for an organization by updating the 'TerraformWorkingDirectory' in the variables file and/or the 'additionalParameters' in the stage | terraform_build_stage.yml |
| Terraform Apply | Will install Terraform, init, plan, apply across environment and regions with a stage for each region/environment combination Does require Azure Pipelines Terraform Task and the remote backend as configured in the azure_terraform_dev_variables.yml. By default Terraform files are expected to be in a folder called 'Infrastructure' and leverage a variable file like dev.eus.variables.tfvars. This is easily changed be changed as needed for an organization by updating the 'TerraformWorkingDirectory' in the variables file and/or the 'additionalParameters' in the stage. | terraform_apply_stage.yml |
| Azure Load Testing | Will execute an Azure Load Test file from the pipeline artifact. This will require the Service Connection to have the Test Contributor Role in Azure. This template can be grabed at the stage, job, or task level. A complete walkthrough and accompanied video can be found on my blog An Azure Load Testing CI/CD Template | load_test_stage.yml |
| Azure Load Testing with Deployment Environments | Create an Azure Deployment Environment, Deploy a dotnetcore application into it, run an Azure Load Test, and delete the ADE. Additional information found on Azure Deployment Environments Series | ade_loadtest_stage.yml |
| Azure Data Factory Build | Execute the npm package to produce a single deployable ARM template for Data Factory. Additional information available on Microsoft Blog.. An example is located at adf_pipelines_yaml_ci_cd | adf_build_stage.yml |
| Azure Data Factory Deploy | Deploy the artifact generate from an Azur Data Factory Build and deploy it. Support for Linked ARM templates is avaiable in this as well. Additional information available on Microsoft Blog. An example is located at adf_pipelines_yaml_ci_cd | adf_deploy_stage.yml |
| Azure Synapse Build | Leverage the Synapse Workspace Deployment Marketplace task to generate the ARM template for validation and producing a deployable artifact. Example available at cicd-synapse | synapse_build_stage.yml |
| Azure Synapse Deploy | Leverage the Synapse Workspace Deployment Marketplace task deploy to an existing Synapse Workspace. Example available at cicd-synapse | synapse_build_stage.yml |
| DotNet Build Stage | Run a dotnet build and a dotnet publish command against a project. Ability to include tests as an option |
dotnet_build_stage.yml |
| Bicep DotNet Build Stage | Run a dotnet build and a dotnet publish command against a project. Ability to include tests as an option. Will also include publishing a bicep artifact which will include running a what-if against all desired Azure environments. An example is contained in the repository ToDo_AzureDeploymentEnvironment |
bicep_dotnet_build_stage.yml |
| DotNet SQL Build Stage | Run a dotnet build against a SQL Project. Will also include a list of folders to produce as artifacts for future stage use An example is contained in the repository cicd-adventureWorks |
dotnet_sql_build_stage.yml |
| Dacpac Deploy Stage | Leverages the SqlAzureDacpacDeployment@1 task to take a dacpac file and deploy it against an existing SQL Server and Database. An example is contained in the repository cicd-adventureWorks | dacpac_deploy_stage.yml |
| Terraform Build AWS | Will install Terraform, init, plan in parallel across environment and regions. Does require Azure Pipelines Terraform Task and the remote backend as configured in the aws_terraform_dev2_variables.yml By default Terraform files are expected to be in a folder called 'infrastructure' and leverage a variable file like dev.eus.variables.tfvars. This is easily changed be changed as needed for an organization by updating the 'TerraformWorkingDirectory' in the variables file and/or the 'additionalParameters' in the stage | terraform_build_aws_stage.yml |
| Terraform Apply | Will install Terraform, init, plan, apply across environment and regions with a stage for each region/environment combination Does require Azure Pipelines Terraform Task and the remote backend as configured in the aws_terraform_dev2_variables.yml. By default Terraform files are expected to be in a folder called 'infrastructure' and leverage a variable file like dev.eus.variables.tfvars. This is easily changed be changed as needed for an organization by updating the 'TerraformWorkingDirectory' in the variables file and/or the 'additionalParameters' in the stage. | terraform_apply_aws_stage.yml |