ADF Deployment - Incremental Deployment: (404) Not Found.

[NobleNooblet]

I've been looking into different ways to deploy ADF infrastructure to different environments and I came across this tool which is super cool! I've created a simple setup to test the deployment process and I've picked up an issue when using the "Incremental Deployment (new)" functionality.

I'll try and explain my process as simple as possible.

My setup:
DevOps Repo - let's call it DevOps-master
Data Factory in Dev - let's call it adf-dev (linked to DevOps-master)
Data Factory in Test - let's call it adf-test (which is my target deployment)
I've created a simple release pipeline that triggers when any changes are pushed to DevOps-master and it deploys the changes to adf-test.

Test scenario 1:
Incremental Deployment set to False
Step 1: Create a new pipeline: pipeline1
Step 2: Add an activity setVariable and populate it with dummy data
Step 3: Save changes (which pushes the changes to DevOps-master)

Result:
Release is triggered
Release completed successfully
Confirmed that adf-test contains the changes made in adf-dev

Test scenario 2:
Repeat Test scenario 1 with Incremental Deployment set to True

Result:
Release is triggered
Release failed with error:

2024-07-15T02:56:22.6997385Z Publish options are provided.
2024-07-15T02:56:22.7006802Z STEP: Verifying whether ADF exists...
2024-07-15T02:56:23.6038710Z ##[warning]You're using Az.DataFactory version 1.18.4. The latest version of Az.DataFactory is 1.18.5. Upgrade your Az modules using the following commands:
  Update-Module Az.* -WhatIf    -- Simulate updating your Az modules.
  Update-Module Az.*            -- Update your Az modules.
2024-07-15T02:56:23.6275981Z Azure Data Factory exists.
2024-07-15T02:56:23.6304705Z Loading Deployment State from ADF...
2024-07-15T02:56:24.4575155Z ##[command]Disconnect-AzAccount -Scope Process -ErrorAction Stop
2024-07-15T02:56:24.4931467Z ##[command]Clear-AzContext -Scope Process -ErrorAction Stop
2024-07-15T02:56:24.6031159Z ##[error]The remote server returned an error: (404) Not Found.
2024-07-15T02:56:24.6672637Z ##[section]Finishing: Publish ADF adf-test from JSON files

No changes reflecting in adf-test

I've played around a bit and managed to find a workaround. Here's what I've done.
Navigated to adf-test -> created a global parameter of type Object called adftools_deployment_state with value {} and published the change.

I repeated test scenario 2 and this time it completed successfully, and I can confirm that adftools_deployment_state updated with hash values.

Here's my take on it:
To me it seems that the incremental process fails if you have no global parameter(s) set up. Adding the global parameter adftools_deployment_state manually resolved my issue, and I can make use of the incremental function going forward.

Is this potentially a bug or am I understanding the use of "incremental Deployment" incorrectly?

Thank you for your time

[NowinskiK]

Thanks @NobleNooblet for the very details and clear description!
It must be a bug. You're using the tool correctly.

It looks like it is similar to #395

[NowinskiK]

I created issue #401