[Feature]: Support multiple parameter files in New-AzResourceGroupDeployment

[Dylan-Prins]

Description of the new feature

Support multiple parameter files in New-AzResourceGroupDeployment like with 'az deployment group create'

Proposed implementation details (optional)

No response

[dingmeng-xue]

Hi @Dylan-Prins , I didn't see multiple parameter files example in az deployment group create. Could you provide one example to us? In addition, what's the reason the parameters are defined in multiple files?

[Dylan-Prins]

You can see it here in example 3: https://docs.microsoft.com/en-us/cli/azure/deployment/sub?view=azure-cli-latest

I use bicep with multiple modules. this means that I have a large main file and would like to split my parameter file according to my modules. Otherwise my parameter files would be to big to maintain

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @satyavel, @apclouds.

Issue Details

---

Description of the new feature

Support multiple parameter files in New-AzResourceGroupDeployment like with 'az deployment group create'

Proposed implementation details (optional)

No response

Author:	Dylan-Prins
Assignees:	-
Labels:	feature-request, Service Attention, ARM - Templates, customer-reported
Milestone:	-

[michael-crawford]

I just hit this issue too, where I'm trying to create equivalent "wrapper" scripts to handle complex deployments, which consist of a baseline repository containing templates and configuration files, which can be overridden by a customization repository which can then augment and/or override the templates and configuration files. This has an additional complication, in that in both the baseline and custom repos, we have a parameters file which is common across all environments and can also have per-environment parameters files.

With the templates, these strictly override, so if we have:

1. Path\To\Baseline\Deployments\simple-template.bicep // Standard version of template, used by all clients
2. Path\To\Custom\ClientB\Deployments\simple-template.bicep // ClientB needs a slight modification of the template
3. Path\To\Custom\ClientB\Deployments\extra-template.bicep // ClientB also needs an extra template not in baseline

For Client A: We do not find any variant of simple-template, so we use the baseline version
For Client B: We do find the custom variant of simple-template, and use this custom version instead.
For Client B: We do find the custom extra-template, not in baseline, but we use that when referenced.
No issue here.

But, with the Parameters files, we want to have these combine, and the behavior we see in the Azure CLI is exactly correct:
Imagine we have these 5 parameters in both the simple and extra template. We might set the defaults to:

• Company
• Location=EastUS
• Environment=Production
• Size=Large
• EnableHA=false

So, I might have these files, with these parameters defined in them.

1. Path\To\Baseline\Conf\simple-template.parameters.json // The baseline shared config
   • Company=Demo // I need to provide a value here, as this doesn't have one in the template itself.
   • Location=WestUS // We want to use WestUS currently for this project. But don't want to change this in the template.
2. Path\To\Baseline\Conf\simple-template.d.parameters.json // The baseline config specific to Dev Env
   • Environment=Development // I want to override the default for this other environment
   • Size=Small // Use small for development
3. Path\To\ClientA\Custom\Conf\simple-template.parameters.json // The ClientA custom shared config
   • Company=ClientA // This should override the baseline for Company
4. Path\To\ClientA\Custom\Conf\simple-template.t.parameters.json // The ClientA custom config specific to Test Env
   • Environment=Testing // Specify this is Testing
   • Size=Medium // Testing uses Medium
5. Path\To\ClientA\Custom\Conf\simple-template.d.parameters.json // The ClientA custom config specific to Dev Env
   • Environment=Development // Specify this is Development
   • Size=Small// Development uses Small
6. Path\To\ClientB\Custom\Conf\simple-template.parameters.json // The ClientB custom shared config
   • Company=ClientB // This should override the baseline for Company
   • EnableHA=true //This client wants to use HA by default in all environments (except ...)
7. Path\To\ClientB\Custom\Conf\simple-template.t.parameters.json // The ClientB custom config specific to Test Env
   • Environment=Testing // Specify this is Testing
   • Size=Medium // Testing uses Medium
8. Path\To\ClientB\Custom\Conf\simple-template.d.parameters.json // The ClientB custom config specific to DevEnv
   • Environment=Development// Specify this is Development
   • Size=Small// Development uses Small
   • EnableHA=false //This client wants to use HA by default in all environments (except in Dev, so turn it back off here)
9. Path\To\ClientB\Custom\Conf\extra-template.parameters.json // The ClientB custom shared config for extra
   • Company=ClientB // This should override the baseline for Company (not set in template, baseline doesn't exist here)
   • Location=CentralUS // This extra thing goes in a different region, for all environments
10. Path\To\ClientB\Custom\Conf\extra-template.t.parameters.json // The ClientB custom config specific to Test Env for extra
    • Environment=Testing // Specify this is Testing
    • Size=Medium // Testing uses Medium

So, when running the deployment using Azure CLI, I can do this, and due to the order in which I specify the template files, parameters are set in earlier files and remain set, unless they are repeated in later files, in which case they are replaced. So, for example, there is no Company default, so I want that set to "Demo" by the baseline, this would not change per environment in baseline, but for client A, I want this set to "ClientA", and that should apply, without me having to repeat it, for all environments which are not the default Production.

# $templatefile is set to the baseline, or if it exists, the custom version instead
# parametersfile_param is "--parameters @"the baseline parameters - IF - it exists, otherwise blank"
# envparametersfile_param is "--parameters @"the baseline per env parameters - IF - it exists, otherwise blank"
# customparametersfile_param is "--parameters @"the custom parameters - IF - it exists, otherwise blank"
# customenvparametersfile_param is "--parameters @"the custom per env parameters - IF - it exists, otherwise blank"
az deployment group create --name $deployment \
                          --resource-group $resource_group \
                          --template-file $templatefile \
                          $parametersfile_param \
                          $envparametersfile_param \
                          $customparametersfile_param \
                          $customenvparametersfile_param

This works exactly as I expect, I don't need to repeat (AND - keep in sync, there could be dozens or hundreds of these files!) parameters which are the same across multiple clients or environments, I only have to override at each environment what is different for that environment, or for each client what is different for that client but cross environment, but can also then override for each client's per-environment needs too. By making all conf files optional, if i don't need to change an earlier default, I don't even need the file at that level.

I wrote this first, and thought both language CLIs worked the same way. I only found out tonight, that this is not the case. So, for the equivalent PowerShell variant, I was trying to do this first:

New-AzResourceGroupDeployment -Name $Deployment `
                             -ResourceGroupName $ResourceGroup `
                             -TemplateFile $TemplateFile `
                             $ParametersFileParam `
                             $EnvParametersFileParam `
                             $CustomParametersFileParam `
                             $CustomEnvParametersFileParam `

Didn't work, told me the -TemplateParameterFile was already defined, suggested I use an Array. I then tried to create an Array of the files, only listing the ones which exist in the order in which I want them to be processed, then overridden where duplicate parameters exist, but that didn't work either.

So, that's why I'm here reporting this, with a detailed use case so you can understand why it's important to support multiple parameter files with override behavior consistently across both Language CLI variants.

By not having this, you're forcing me to waste hours of my time to figure out the logic to read in these parameter files, then combine them, doing the override logic myself into an object hash, so I can then only provide one file. I would rather have this done by you, so both the Azure CLI and PowerShell variants of this same command work in a compatible way.

[Edit] This should work the same way for all scopes, tenant, management-group, subscription, same as I'm describing here. With the Azure CLI, this behavior is consistent across all 4 scopes.

[Alex-wdy]

We have notified the relevant team, and there is no plan yet, we will reply as soon as possible after the team review is completed.

[michael-crawford]

Thanks @Alex-wdy - I hope you guys agree the behavior of the bash/CLI vs PowerShell variants of the same command should work in a compatible way, unless there's a really good reason as to why they should not. I think that the superior multi-parameter-file specification with override behavior in the bash/CLI variant should work in a similar way in the PowerShell variant.

I later discovered that I also can't specify a parameter file AND override just a couple of parameters specifically - again, it's all or nothing. So, even if I have a parameters file, but want to always override a couple of key parameters like Company and Environment, I have to read in the parameters file, create a hash, then add the overrides to the hash and specify it instead. I don't have to do that with the bash/CLI, which makes it MUCH, MUCH more user-friendly and easy to use for this type of combination with overrides use-case.

[DataBach-maker]

I am also urgently looking for this feature. Thank you

[JANA-mrioux]

I just came across this myself and would really make life easier if we were able to pass multiple parameter file to the PowerShell command.

[anwtn]

One workaround I've used is to copy the desired environment-specific file (e.g., params.apps.uat.json or params.apps.prd.json) into a generic file name, such as params.apps.json, during a pipeline step. Similarly, I can do the same for other parameter files, such as params.storage.json. This allows me to use a static path, as required by loadJsonContent, while still effectively managing multiple parameter files across different environments.

Although this isn't a perfect solution, it provides the flexibility needed to work around this limitation with handling multiple parameter files.

Reference: https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/bicep-functions-files#loadjsoncontent