Skip to content

Credentials in azure-identity should be pickle-able #37451

Closed

Description

Is your feature request related to a problem? Please describe.

The azure-ai-evaluation SDK exposes a credential field in its evaluators (example: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/evaluators/_content_safety/_violence.py#L14) to be used when calling the Azure AI RAI service. Right now, this credential is optional, and at execution time we choose DefaultAzureCredential if the user did not pass anything in. However, we need to make this credential parameter required in the evaluators since choosing a default credential is not safe SDK behavior.

If the evaluator is being run on a single input OR when it is used in the batch evaluate API with no credential being passed in, the customer will not see any errors. However, if the user passes a credential to the evaluator and uses it in the evaluate function, it will not work since evaluate uses multiprocessing to do batch runs of multiple evaluators. Multiprocessing requires the evaluator to be pickle-able, but the credentials in azure-identity are not pickle-able, which breaks this scenario.

Describe the solution you'd like
The credentials in azure-identity should be pickle-able so that they can be used in evaluators when doing batch evaluation using evaluate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Labels

Azure.Identityneeds-author-feedbackWorkflow: More information is needed from author to address the issue.no-recent-activityThere has been no recent activity on this issue.

Type

No type

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions