[BUG] Possible breaking change in ChainedTokenCredential in Azure 1.13.0 and 1.13.1

[kappa-mu]

Library name and version

Azure.Identity 1.13.1

Describe the bug

Azure.Identity version 1.13.0 and 1.13.1 failing with ChainedTokenCredential

My code is using ChainedTokenCredential with ManagedIdentityCredential for deployed application and VisualStudioCredential for working locally. This has been working fine until 1.12.1 but stopped working in 1.13.0 and 1.13.1
While working locally, it is trying to use the ManagedIdentityCredential first and when it fails, the error is coming out to the surface instead of trying to use the VisualStudioCredential

Working: 1.12.1
Not working: 1.13.0 and 1.13.1

Expected behavior

We should be able to fetch the secret as ChainedTokenCredential should be successful like in version 1.12.1

Actual behavior

Exception:
Azure.Identity.AuthenticationFailedException: 'The ChainedTokenCredential failed due to an unhandled exception: ManagedIdentityCredential authentication failed: [Managed Identity] Authentication unavailable. Either the requested identity has not been assigned to this resource, or other errors could be present. Ensure the identity is correctly assigned and check the inner exception for more details. For more information, visit https://aka.ms/msal-managed-identity.
Status: BadRequest
Content:
{"error":"invalid_request","error_description":"Identity not found"}

Headers:
Server: IMDS/150.870.65.1475
x-ms-request-id: e00d2743-fdbf-4f84-a2f2-763ba9ec69ac
Date: Mon, 11 Nov 2024 06:34:38 GMT

[Managed Identity] Error Code: invalid_request Error Description: Identity not found
See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/managedidentitycredential/troubleshoot'

Reproduction Steps

Create a simple .Net console app.
Add nuget packages:
Azure.Identity
Azure.Core
Azure.Security.KeyVault.Secrets

Add the following code:

// See https://aka.ms/new-console-template for more information
using Azure.Core;
using Azure.Identity;
using Azure.Security.KeyVault.Secrets;

Console.WriteLine("Hello, World!");


// Values from app registration
var clientId = "<<ClientID>>";
var tenantId = "<<TenantID>>";
var clientSecret = GetClientSecretFromKeyvault();


// using Azure.Identity;
var options = new ClientSecretCredentialOptions
{
    AuthorityHost = AzureAuthorityHosts.AzurePublicCloud,
};

// https://learn.microsoft.com/dotnet/api/azure.identity.clientsecretcredential
var clientSecretCredential = new ClientSecretCredential(
    tenantId, clientId, clientSecret, options);

static TokenCredential GetTokenCredential()
{
    //ManagedIdentityCredential for Azure
    //Visual Studio Credential for local development
    return new ChainedTokenCredential(
        new ManagedIdentityCredential(),
        new VisualStudioCredential()
    );
}

static string GetClientSecretFromKeyvault()
{
    // Create a new secret client using the default credential from Azure.Identity using environment variables previously set,
    // including AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, and AZURE_TENANT_ID.
    SecretClient client = GetSecretClient();

    // Retrieve a secret using the secret client.
    var secret = client.GetSecret("HelloSecret");
    return secret.Value.Value;
}

static SecretClient GetSecretClient()
{

    return new SecretClient
        (vaultUri: new Uri("https://<<keyvaultname>>.vault.azure.net/"),
        credential: GetTokenCredential()
    );
}

Environment

Environmet: Local, using visual studio 22 version 17.11.5, using in windows
App Type: .Net 8 Console App

[github-actions]

Thank you for your feedback. Tagging and routing to the team member best able to assist.

[christothes]

Hi @kappa-mu
I'm sorry you've hit this issue. There were some changes to the implementation of ManagedIdentityCredential in version 1.13.0+ that have caused some slight changes in behavior that cause issues like this. When you are in a local environment where some service or proxy responds to the IMDS request to http://169.254.169.254/metadata/identity/oauth2/token (normally nothing responds to this request on a local environment), this gets treated as a failure rather than allowing the request to flow through to the next credential in the chain.

While I we investigate the best way to address this change, here are a couple workarounds you could try:

Use a locally defined environment variable to indicate you are in a local development environment (recommended)

static TokenCredential GetTokenCredential()
{
	if (!string.IsNullOrEmpty(Environment.GetEnvironmentVariable("THIS_IS_MY_DEV_ENV"))
	{
		return new VisualStudioCredential();
	}
	else
	{
		return new ManagedIdentityCredential();
	}
}

The advantage of this option is that you only attempt ManagedIdentityCredential when the code is deployed and never when in the local dev environment. The only disadvantage is that anyone working on this code must define a custom environment variable on your development machine.

Utilize a modified DefaultAzureCredential instead of ManagedIdentityCredential

With the DefaultAzureCredentialOptions , you can exclude the credentials that you don't want and get something close to your original code's behavior.

The advantage to this approach is that it just utilizes the DefaultAzureCredential. The disadvantage is that if you are deployed to a production environment that utilizes the IMDS endpoint, such as VMs, the attempt to connect to the IMDS endpoint is designed to fail fast rather than retrying.

[kappa-mu]

Hi @christothes thank you so much for your suggestions. I can take the first suggestions, although at this moment we are happy to wait for it to get resolved by downgrading the version of Azure.Identity we use. In future, if needed, we will implement the first suggestion which we do not want to implement generally because of the elegance of ChainedTokenCredential.