Deletion of bicep file with parameter file - JSON serialization

[mathwro]

Describe the bug

When i'm deleting a bicep file and it's matching json parameter file from our management group structure, the validation steps are failing due to failing to serialize the json of the parameter file.

AzOps/src/functions/Invoke-AzOpsPush.ps1

Line 400 in 3a77d8c

$templateContent = Get-Content $deletion | ConvertFrom-Json -AsHashtable

Steps to reproduce

1. Add bicep file and parameter file to management group structure (Example, network.bicep & network.parameters.json)
2. Merge to main and deploy
3. Now delete both files

Screenshots

[daltondhcp]

Thanks for reporting! Quick note: deleting custom templates only removes them from the repo, not Azure (see Issue #785 for details).

I couldn't replicate the error with a valid parameters.json. For me, it only occurs with an invalid JSON, like missing a closing brace.

Can you run Get-Content network.parameters.json | ConvertFrom-Json -AsHashTable locally successfully? If so, sharing your template and parameter file would help us debug further. Thanks!

[mathwro]

I've tried to replicate the commands done, just locally and i dont get any serialization errors when performed locally with powershell 7.4.

Here is parameter file with a identical structure.

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
  "contentVersion": "1.0.0.3",
  "parameters": {
    "environment": {
      "value": "pt"
    },
    "application": {
      "value": "demo"
    },
    "partition": {
      "value": "swc"
    },
    "identifier": {
      "value": "001"
    },
    "networkConfiguration": {
      "value": {
        "type": "internal",
        "vnetAddressPrefix": [
          "10.0.0.0/21"
        ],
        "peeredVnetIpPrefixes": [
          "10.0.0.0/23"
        ],
        "dhcpOptions": {
          "dnsServers": [
            "1.1.1.1"
          ]
        },
        "subnets": [
          {
            "subnet": "10.0.128.0/22",
            "udr": "udr01",
            "serviceEndpoints": [],
            "delegations": [
              {
                "name": "Microsoft.ContainerService/managedClusters",
                "properties": {
                  "serviceName": "Microsoft.ContainerService/managedClusters"
                }
              }
            ],
            "nsgRules": [
            ]
              
          },
          {
            "subnet": "10.0.0.0/23",
            "udr": "udr01",
            "serviceEndpoints": [],
            "delegations": [
              {
                "name": "Microsoft.ContainerService/managedClusters",
                "properties": {
                  "serviceName": "Microsoft.ContainerService/managedClusters"
                }
              }
            ],
            "nsgRules": []              
          },
          {
            "subnet": "10.192.132.0/24",
            "udr": "udr01",
            "serviceEndpoints": [],
            "delegations": [
              {
                "name": "Microsoft.ContainerService/managedClusters",
                "properties": {
                  "serviceName": "Microsoft.ContainerService/managedClusters"
                }
              }
            ],
            "nsgRules": []
          },
          {
            // "snet-lb",
            "subnet": "10.0.0.0/26",
            "udr": "udr-lndz-to-conn",
            "serviceEndpoints": [],
            "nsgRules": []
          },
          {
            "subnet": "10.0.0.128/27",
            "udr": "udr01",
            "serviceEndpoints": [
              {
                "service": "Microsoft.ContainerRegistry"
              }
            ],
            "nsgRules": [
              {
                "name": "OUT",
                "properties": {
                  "protocol": "Tcp",
                  "sourcePortRange": "*",
                  "destinationPortRange": "443",
                  "sourceAddressPrefix": "10.0.0.128/27", 
                  "destinationAddressPrefix": "10.0.0.0/29",
                  "access": "Allow",
                  "priority": 1101,
                  "direction": "Outbound"
                }
              }
            ]
          },
          {
            "subnet": "10.0.0.64/26",
            "udr": "udr01",
            // "delegations": [],
            "serviceEndpoints": [
              {
              "service": "Microsoft.ContainerRegistry"
              }
            ],
            "nsgRules": [
              {
                "name": "OUT-443",
                "properties": {
                  "protocol": "Tcp",
                  "sourcePortRange": "*",
                  "destinationPortRange": "443",
                  "sourceAddressPrefix": "10.0.0.64/26",
                  "destinationAddressPrefix": "10.0.0.0/29",
                  "access": "Allow",
                  "priority": 1101,
                  "direction": "Outbound"
                }
              }
            ]
          },
          {
            "subnet": "10.192.1.160/27",
            "udr": "udr01",
            "serviceEndpoints": [
              {
                "service": "Microsoft.ContainerRegistry"
              }
            ],
            "nsgRules": [
              {
                "name": "OUT-443F",
                "properties": {
                  "protocol": "Tcp",
                  "sourcePortRange": "*",
                  "destinationPortRange": "443",
                  "sourceAddressPrefix": "10.192.1.160/27",
                  "destinationAddressPrefix": "10.192.0.0/29",
                  "access": "Allow",
                  "priority": 1101,
                  "direction": "Outbound"
                }
              }
            ]
          },
          {
            "subnet": "10.0.0.0/29",
            "udr": "udr01",
            // "delegations": [],
            // "serviceEndpoints": [
            //   {
            //     "service": "Microsoft.KeyVault"
            //   }
            // ],
            "nsgRules": []
          },
          {
            "subnet": "10.0.0.208/29",
            "udr": "udr01",
            // "delegations": [],
            // "serviceEndpoints": [
            //   {
            //     "service": "Microsoft.KeyVault"
            //   }
            // ],
            "nsgRules": []
          },
          {
            // "snet-pe",
            "subnet": "10.0.0.216/29",
            "nsgRules": [],
            "udr": "udr01",
            // "delegations": [],
            "nsgRules": []
          },
          {
            // "snet-pe",
            "subnet": "10.0.0.200/29",
            "nsgRules": [],
            "udr": "udr01",
            // "delegations": [],
            "nsgRules": []
          }
        ]
      }
    },
    "routeTables": {
      "value": [
        {
          "routeTablePrefix": "udr01",
          "routes": [
            {
              "name": "On-Prem",
              "properties": {
                "addressPrefix": "0.0.0.0/0",
                "nextHopType": "VirtualAppliance",
                "nextHopIpAddress": "10.0.2.4"
              }
            }
          ]
        }
      ]
    },
    "hubVnetName": {
      "value": "vnet001"
    },
    "hubVnetRG": {
      "value": "rg-005"
    },
    "hubSubscriptionId": {
      "value": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
    }
  }
}

[daltondhcp]

I am unable to repro with just your parameter file (no errors when deleting), can you please share the bicep deployment template as well?

[mathwro]

A bit delayed but finally got some time to clean it up properly.

I've verified again, and are still not seeing any local errors regardless of what i'm doing.

param networkConfiguration object
param application string
param partition string
param rgLocation string = 'West Europe'
param environment string
param identifier string
param routeTables array
param hubVnetName string
param hubVnetRg string = resourceGroup().name
param hubSubscriptionId string

var subnetNamePrefix = 'snet-${application}-${partition}-'
var networkSecurityGroupNamePrefix = 'nsg-${subnetNamePrefix}'
var intraSubnetRules = [for (item, i) in networkConfiguration.subnets: {
  rules: [
    {
      name: 'AllowCurrentSubnet'
      properties: {
        protocol: '*'
        sourcePortRange: '*'
        sourceAddressPrefix: item.subnet
        destinationAddressPrefix: '*'
        destinationPortRange: '*'
        access: 'Allow'
        priority: 4095
        direction: 'Inbound'
      }
    }
    {
      name: 'DenyInternalVnetInbound'
      properties: {
        protocol: '*'
        sourcePortRange: '*'
        sourceAddressPrefixes: union(networkConfiguration.vnetAddressPrefix, networkConfiguration.peeredVnetIpPrefixes)
        destinationAddressPrefix: '*'
        destinationPortRange: '*'
        access: 'Deny'
        priority: 4096
        direction: 'Inbound'
      }
    }
  ]
}]

resource networkSecurityGroups 'Microsoft.Network/networkSecurityGroups@2022-01-01' = [for (item, i) in networkConfiguration.subnets: {
  name: '${networkSecurityGroupNamePrefix}${padLeft((i + 1), 3, '0')}'
  location: rgLocation
  tags: {
    networkZone: networkConfiguration.type
  }
  properties: {
    securityRules: concat(item.nsgRules, intraSubnetRules[i].rules)
  }
}]

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
  "contentVersion": "1.0.0.3",
  "parameters": {
    "environment": {
      "value": "pt"
    },
    "application": {
      "value": "demo"
    },
    "partition": {
      "value": "swc"
    },
    "identifier": {
      "value": "001"
    },
    "networkConfiguration": {
      "value": {
        "type": "internal",
        "vnetAddressPrefix": [
          "10.0.0.0/21"
        ],
        "peeredVnetIpPrefixes": [
          "10.0.10.0/23"
        ],
        "dhcpOptions": {
          "dnsServers": [
            "1.1.1.1"
          ]
        },
        "subnets": [
          {
            "subnet": "10.0.128.0/22",
            "udr": "udr01",
            "serviceEndpoints": [],
            "delegations": [
              {
                "name": "Microsoft.ContainerService/managedClusters",
                "properties": {
                  "serviceName": "Microsoft.ContainerService/managedClusters"
                }
              }
            ],
            "nsgRules": [
            ]
              
          },
          {
            "subnet": "10.0.0.0/23",
            "udr": "udr01",
            "serviceEndpoints": [],
            "delegations": [
              {
                "name": "Microsoft.ContainerService/managedClusters",
                "properties": {
                  "serviceName": "Microsoft.ContainerService/managedClusters"
                }
              }
            ],
            "nsgRules": []              
          },
          {
            "subnet": "10.192.132.0/24",
            "udr": "udr01",
            "serviceEndpoints": [],
            "delegations": [
              {
                "name": "Microsoft.ContainerService/managedClusters",
                "properties": {
                  "serviceName": "Microsoft.ContainerService/managedClusters"
                }
              }
            ],
            "nsgRules": []
          },
          {
            // "snet-lb",
            "subnet": "10.0.0.0/26",
            "udr": "udr-lndz-to-conn",
            "serviceEndpoints": [],
            "nsgRules": []
          },
          {
            "subnet": "10.0.0.128/27",
            "udr": "udr01",
            "serviceEndpoints": [
              {
                "service": "Microsoft.ContainerRegistry"
              }
            ],
            "nsgRules": [
              {
                "name": "OUT",
                "properties": {
                  "protocol": "Tcp",
                  "sourcePortRange": "*",
                  "destinationPortRange": "443",
                  "sourceAddressPrefix": "10.0.0.128/27", 
                  "destinationAddressPrefix": "10.0.0.0/29",
                  "access": "Allow",
                  "priority": 1101,
                  "direction": "Outbound"
                }
              }
            ]
          },
          {
            "subnet": "10.0.0.64/26",
            "udr": "udr01",
            // "delegations": [],
            "serviceEndpoints": [
              {
              "service": "Microsoft.ContainerRegistry"
              }
            ],
            "nsgRules": [
              {
                "name": "OUT-443",
                "properties": {
                  "protocol": "Tcp",
                  "sourcePortRange": "*",
                  "destinationPortRange": "443",
                  "sourceAddressPrefix": "10.0.0.64/26",
                  "destinationAddressPrefix": "10.0.0.0/29",
                  "access": "Allow",
                  "priority": 1101,
                  "direction": "Outbound"
                }
              }
            ]
          },
          {
            "subnet": "10.192.1.160/27",
            "udr": "udr01",
            "serviceEndpoints": [
              {
                "service": "Microsoft.ContainerRegistry"
              }
            ],
            "nsgRules": [
              {
                "name": "OUT-443F",
                "properties": {
                  "protocol": "Tcp",
                  "sourcePortRange": "*",
                  "destinationPortRange": "443",
                  "sourceAddressPrefix": "10.192.1.160/27",
                  "destinationAddressPrefix": "10.192.0.0/29",
                  "access": "Allow",
                  "priority": 1101,
                  "direction": "Outbound"
                }
              }
            ]
          },
          {
            "subnet": "10.0.0.0/29",
            "udr": "udr01",
            // "delegations": [],
            // "serviceEndpoints": [
            //   {
            //     "service": "Microsoft.KeyVault"
            //   }
            // ],
            "nsgRules": []
          },
          {
            "subnet": "10.0.0.208/29",
            "udr": "udr01",
            // "delegations": [],
            // "serviceEndpoints": [
            //   {
            //     "service": "Microsoft.KeyVault"
            //   }
            // ],
            "nsgRules": []
          },
          {
            // "snet-pe-prometheus",
            "subnet": "10.0.0.216/29",
            "nsgRules": [],
            "udr": "udr01",
            // "delegations": [],
            "nsgRules": []
          },
          {
            // "snet-pe-aksapi",
            "subnet": "10.0.0.200/29",
            "nsgRules": [],
            "udr": "udr01",
            // "delegations": [],
            "nsgRules": []
          }
        ]
      }
    },
    "routeTables": {
      "value": [
        {
          "routeTablePrefix": "udr01",
          "routes": [
            {
              "name": "On-Prem",
              "properties": {
                "addressPrefix": "0.0.0.0/0",
                "nextHopType": "VirtualAppliance",
                "nextHopIpAddress": "10.0.2.4"
              }
            }
          ]
        }
      ]
    },
    "hubVnetName": {
      "value": "vnet001"
    },
    "hubVnetRG": {
      "value": "rg-005"
    },
    "hubSubscriptionId": {
      "value": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
    }
  }
}

[Jefajers]

Hi thanks for supplying the network.bicep template as well.

We are however not able to reproduce any issue during processing of neither the network.bicep or network.parameters.json.

As you mention that you are not able to get any local errors with the supplied templates in this issue.

Are you able to reproduce the error with network.bicep or network.parameters.json files attached to this issue using the module?. If you are then we could setup a call to better understand the circumstances (versions, settings, environment structure, etc) involved to attempt identifying what is happening.

Current module logic is expected to skip the .bicep file entirely during deletion processing and when processing the parameters.json file it would identify its not of a supported resource deletion type and then skip it as well.

With the final result: files are able to be deleted and the module has not performed any action towards Azure in regards to the potential file content.

[mathwro]

Hi, let me follow up with my customer if possible to set up a call for this and get back to you.

From what we are seeing it doesn't seem to skip the parameter file. It does indeed skip the bicep file, but it still attempts to serialize the JSON parameter file before determining any actions for it.

I'm unaware of the full context of what is going on, but i was hoping it might be possible to skip any steps involving the parameter file, if the bicep/arm template it's targeted for were deleted as well

[Jefajers]

Thanks let us know how it goes, and regarding "It does indeed skip the bicep file, but it still attempts to serialize the JSON parameter file before determining any actions for it.". The serialize step is indeed attempted and is required to parse the content inside the file before deciding on whether to act or not.

[mathwro]

We are happy to arrange for a meeting if possible @Jefajers

We have recently seen a similar issue again and would like to have this resolved somehow

[Jefajers]

Hi @mathwro, thanks for the supplied templates I have been able to reproduce the issue and identified a module logic issue which is only used during deletion to overcome the fact that we don't have the files in the actual branch.

The logic in this scenario creates an erroneous bicep file which triggers these conversion errors down the line that we see in this thread. In the up coming AzOps release 2.6.0 this logic as been updated and I am not able to reproduce the issue.