Skip to content

Commit 784c5e0

Browse files
authored
Add suppport for TestFx to record with user account (#24896)
* Add suppport for TestFx to record with user account * Mark using existing service principal also as not recommended
1 parent b15c6bb commit 784c5e0

File tree

14 files changed

+533
-518
lines changed

14 files changed

+533
-518
lines changed

documentation/testing-docs/using-azure-test-framework.md

Lines changed: 55 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,31 @@
11
# Using Azure PowerShell Test Framework
22

3-
- [Using Microsoft.Rest.ClientRuntime.Azure.TestFramework](#using-microsoftrestclientruntimeazuretestframework)
3+
- [Using Azure PowerShell Test Framework](#using-azure-powershell-test-framework)
44
- [Getting Started](#getting-started)
55
- [Azure PowerShell Test Framework](#azure-powershell-test-framework)
66
- [Setup prior to Record or Playback of tests](#setup-prior-to-record-or-playback-of-tests)
7-
- [Run Command Set-TestFxEnvironment to Build Connection String (Recommended)](#run-command-set-testfxenvironment-to-build-connection-string-recommended)
8-
- [Create New Service Principal](#create-new-service-principal)
9-
- [Use Existing Service Principal](#use-existing-service-principal)
7+
- [Run Command Set-TestFxEnvironment to Build Connection String](#run-command-set-testfxenvironment-to-build-connection-string)
8+
- [Use user account to record test cases (Recommended)](#use-user-account-to-record-test-cases-recommended)
9+
- [Create New Service Principal (Not Recommended)](#create-new-service-principal-not-recommended)
10+
- [Use Existing Service Principal (Not Recommended)](#use-existing-service-principal-not-recommended)
1011
- [Manually Set Environment Variables to Build Connection String](#manually-set-environment-variables-to-build-connection-string)
1112
- [Environment Variables](#environment-variables)
12-
- [Record Test with service principal](#record-test-with-service-principal)
13-
- [Playback Test](#playback-test)
13+
- [Record Tests](#record-tests)
14+
- [Playback Tests](#playback-tests)
1415
- [JSON Config File V.S. Environment Variables](#json-config-file-vs-environment-variables)
1516
- [Record or Playback Tests](#record-or-playback-tests)
1617
- [Change Test Environment settings at run-time](#change-test-environment-settings-at-run-time)
17-
- [Once you set your connection string, you can add or update key/value settings](#once-you-set-your-connection-string-you-can-add-or-update-keyvalue-settings)
18-
- [Note:](#note)
18+
- [Once you set your connection string, you can add or update key/value settings](#once-you-set-your-connection-string-you-can-add-or-update-keyvalue-settings)
19+
- [Note:](#note)
1920
- [Troubleshooting](#troubleshooting)
20-
- [Issue: exceptions in Microsoft.Azure.Test.HttpRecorder](#issue-exceptions-in-microsoftazuretesthttprecorder)
21+
- [Issue: exceptions in Microsoft.Azure.Test.HttpRecorder](#issue-exceptions-in-microsoftazuretesthttprecorder)
2122
- [Supported Environments in Test Framework](#supported-environments-in-test-framework)
22-
- [Default Environments and associated Uri](#default-environments-and-associated-uri)
23-
- [Environment = Prod](#environment--prod)
24-
- [Environment = Dogfood](#environment--dogfood)
25-
- [Environment = Next](#environment--next)
26-
- [Environment = Current](#environment--current)
27-
- [Environment = Custom](#environment--custom)
23+
- [Default Environments and associated Uri](#default-environments-and-associated-uri)
24+
- [Environment = Prod](#environment--prod)
25+
- [Environment = Dogfood](#environment--dogfood)
26+
- [Environment = Next](#environment--next)
27+
- [Environment = Current](#environment--current)
28+
- [Environment = Custom](#environment--custom)
2829

2930
## Getting Started
3031

@@ -42,66 +43,73 @@ The target framework of test is .Net 6, please ensure .Net runtime Microsoft.NET
4243

4344
## Setup prior to Record or Playback of tests
4445

45-
In order to Record/Playback a test, test framework needs to setup a connection string that consists of various key/value pairs that provide necessary information.
46+
To Record/Playback a test case, the test framework must establish a connection string comprising multiple key/value pairs that supply essential information.
4647

4748
You can choose either option to configure the settings:
4849

49-
- Run the [`Set-TestFxEnvironment` cmdlet](#run-command-set-testfxenvironment-to-build-connection-string) (Recommended)
50+
- [Run the Set-TestFxEnvironment cmdlet](#use-user-account-to-record-test-cases)
5051
- [Manually set the environment variables](#manually-set-environment-variables-to-build-connection-string)
5152

52-
### Run Command Set-TestFxEnvironment to Build Connection String (Recommended)
53+
### Run Command Set-TestFxEnvironment to Build Connection String
5354

54-
This cmdlet will allow you to create a credentials file (located in `C:/Users/<currentuser>/.Azure/testcredentials.json`) that will be used to set the connection string when scenario tests are run. This credentials file will be used in all future sessions unless it is deleted.
55+
This cmdlet enables you to generate a credentials file (located in `C:/Users/<currentuser>/.Azure/testcredentials.json`) that sets the connection string during scenario test execution. This credentials file persists across sessions unless manually deleted.
5556

56-
#### Create New Service Principal
57+
#### Use user account to record test cases (Recommended)
5758

58-
Using a service principal is the preferred option for recording tests because it works with both .NET Framework and .NET Core.
59-
In order to create a new service principal, run the following command with an unused service principal display name:
59+
Using a user account is the preferred method for recording test cases as it avoids storing secret in the local file with plain text. You can obtain the object ID of the user account either from the Azure portal or through Azure PowerShell.
60+
61+
```powershell
62+
Set-TestFxEnvironment -UserId <UserId> -SubscriptionId <SubscriptionId> -TenantId <TenantId> -RecorderMode "Record"
63+
```
64+
65+
#### Create New Service Principal (Not Recommended)
66+
67+
Using a service principal is not recommended due to security concerns, but it remains an available option for your use.
68+
To create a new service principal, execute the following command with a brand new service principal display name:
6069

6170
```powershell
6271
Set-TestFxEnvironment -ServicePrincipalDisplayName <DisplayName> -SubscriptionId <SubscriptionId> -TenantId <TenantId> -RecorderMode "Record"
6372
```
6473

65-
This command will first create a new service principal. And then set the `Contributor` role assignment for this service principal based upon the subscription provided. After that, it will place the service principal application id and automatically generated secret into the credentials file.
74+
This command will initially create a new service principal, then assign the `Contributor` role to this service principal according to the specified subscription. Subsequently, it will update the credentials file with the application ID and an automatically generated secret for the service principal.
6675

67-
If the display name of the service principal already exists, it will prompt if you would like to create a new one with the same name.
68-
If the answer is "Y", the new generated application id and the secret will be saved.
76+
If the display name of the service principal already exists, you will be prompted to decide whether to create a new one with the same name. If you choose "Y", the newly generated application ID and secret will be stored.
6977

70-
Alternatively, if you prefer creating a service principal by yourself from Azure portal, follow the [Azure AD guide to create a Application Service Principal](https://learn.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal#create-an-active-directory-application).
78+
Alternatively, if you prefer to create a service principal manually through the Azure portal, you can refer to [Azure AD guide to create a Application Service Principal](https://learn.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal#create-an-active-directory-application).
7179

72-
#### Use Existing Service Principal
80+
#### Use Existing Service Principal (Not Recommended)
7381

74-
If you would like to use an existing service principal, run the following command with an existing service principal application id and secret:
82+
If you wish to utilize an existing service principal, execute the following command with the application ID and secret of the desired service principal:
7583

7684
```powershell
7785
Set-TestFxEnvironment -ServicePrincipalId <ServicePrincipalApplicationId> -ServicePrincipalSecret <ServicePrincipalSecret> -SubscriptionId <SubscriptionId> -TenantId <TenantId> -RecorderMode "Record"
7886
```
7987

80-
For existing service principal, this command will respect your own settings and won't assign the `Contributor` role automatically.
88+
For an existing service principal, this command will respect your current settings and won't assign the `Contributor` role automatically.
89+
8190

82-
8391

8492
### Manually Set Environment Variables to Build Connection String
8593

8694
#### Environment Variables
8795

8896
`TEST_CSM_ORGID_AUTHENTICATION`
8997

90-
* This determines how to connect to Azure. It includes both your authentication and the Azure environment information.
98+
* This variable determines how to connect to Azure. It encompasses both your authentication credentials and the Azure environment information.
9199

92100
`AZURE_TEST_MODE`
93101

94-
* This specifies whether the test framework will `Record` test sessions or `Playback` previously recorded test sessions.
102+
* This variable specifies whether the test framework will `Record` test sessions or `Playback` recorded test sessions.
95103

96104
#### Record Tests
97105

98-
After the service principal is created, you will need to give it access to Azure resources. This can be done with the following PowerShell command. The argument for this command is the application id (See [Service Principal Application ID](https://learn.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal#get-application-id-and-authentication-key))
106+
After creating the service principal, you will need to grant it access to Azure resources. This can be achieved using the following PowerShell command. The argument for this command is the application ID (See [Service Principal Application ID](https://learn.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal#get-application-id-and-authentication-key))
99107

100108
```powershell
101109
New-AzRoleAssignment -ApplicationId <ApplicationId> -Scope "/subscriptions/<SubscriptionId>" -RoleDefinitionName Contributor
102110
```
103111

104-
To use this option, set the following environment variable before starting Visual Studio. The following values are substituted into the below environment variable:
112+
To use this option, set the following environment variables before starting Visual Studio. The following values are substituted into the below environment variables:
105113

106114
`ClientId`
107115

@@ -118,7 +126,7 @@ AZURE_TEST_MODE=Record
118126

119127
#### Playback Tests
120128

121-
The default test mode is `Playback`, so setting up the `AZURE_TEST_MODE` is not required. You can optionally set environment variables:
129+
The default test mode is `Playback`, so there is no need to set up the `AZURE_TEST_MODE` variable. You may optionally set the environment variables:
122130

123131
```
124132
TEST_CSM_ORGID_AUTHENTICATION=Environment=Prod;SubscriptionId=<SubscriptionId>;TenantId=<TenantId>;ServicePrincipal=<ClientId>;ServicePrincipalSecret=<ClientSecret>;HttpRecorderMode=Playback;
@@ -127,27 +135,26 @@ AZURE_TEST_MODE=Playback
127135

128136
## JSON Config File V.S. Environment Variables
129137

130-
Opting for config file is the recommended way to build connection string because any changes you make will take effect immediately without having to restart Visual Studio. However, updating the environment variables is different. It requires rebooting Visual Studio before it can read the updated values. So following is the steps how Test Framework detects the settings.
138+
The recommended approach for building the connection string is using a config file, as any changes made will take effect immediately without needing to restart Visual Studio. However, updating environment variables is a different process and requires a restart of Visual Studio for the updated values to be recognized. Here are the steps the Test Framework follows to detect settings:
131139

132140
- If JSON config file exists
133-
- It will be used to build the connection string. Anything set in the environment variables will be ignored
141+
- The Test Framework will use it to build the connection string, ignoring any settings from environment variables.
134142
- If JSON config file does not exist
135-
- Test framework will first retrieve the environment variable `TEST_CSM_ORGID_AUTHENTICATION` and use its value to build the connection string except for the test mode (Record/Playback).
136-
- Then test framework will try to get the value of the environment variable `AZURE_TEST_MODE`
137-
- If `AZURE_TEST_MODE` is set, its value will be used as the test mode
138-
- Otherwise, the property named `HttpRecorderMode` configured in `TEST_CSM_ORGID_AUTHENTICATION` will be used
139-
- If the property `HttpRecorderMode` is also not set, `Playback` will be applied as the default value
143+
- The Test Framework will first check the TEST_CSM_ORGID_AUTHENTICATION environment variable and use its value to build the connection string.
144+
- If `HttpRecorderMode` is not set, then the Test Framework will check the `AZURE_TEST_MODE` environment variable.
145+
- If `AZURE_TEST_MODE` is set, its value will be used as the test mode.
146+
- If `AZURE_TEST_MODE` is not set, the default test mode will be `Playback`.
140147

141-
If you are not sure the settings on your machine, please run command `Get-TestFxEnvironment`. It will consolidate above steps and display the ultimate result.
148+
If you are unsure about the settings on your machine, you can run the command `Get-TestFxEnvironment` to consolidate the steps mentioned above and display the final result.
142149

143150
## Record or Playback Tests
144151

145-
- [Run the tests](https://github.com/Azure/azure-powershell/blob/main/documentation/development-docs/azure-powershell-developer-guide.md#recordingrunning-tests) and make sure that you got a generated `.json` file that matches the test name under the `SessionRecords` folder in the test project.
152+
- [Run the tests](https://github.com/Azure/azure-powershell/blob/main/documentation/development-docs/azure-powershell-developer-guide.md#recordingrunning-tests) and make sure that you have a generated `.json` file that corresponds to the test name, and it should be located under the `SessionRecords` folder within the test project.
146153
- If you want to switch from Record to Playback or from Playback to Record, consider below steps.
147-
- If you opt for JSON config file, update the value of the property `HttpRecorderMode` in the JSON.
148-
- If you opt for environment variables
149-
- If you have `AZURE_TEST_MODE` set, update the value of this variable
150-
- Otherwise, update the value of the property `HttpRecorderMode` defined in the variable `TEST_CSM_ORGID_AUTHENTICATION`
154+
- If you choose to use a JSON config file, update the value of the property `HttpRecorderMode` in the JSON file.
155+
- If you prefer environment variables
156+
- If you have `HttpRecorderMode` defined in the environment variable `TEST_CSM_ORGID_AUTHENTICATION`, update its value accordingly.
157+
- Otherwise, update the value of `AZURE_TEST_MODE`.
151158

152159
## Change Test Environment settings at run-time
153160

src/Attestation/Attestation.Test/ScenarioTests/AttestationTestRunner.cs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -79,7 +79,7 @@ private static AttestationClient GetAttestationClient(MockContext context)
7979
// When recording, we should have a connection string passed into the code from the environment
8080
if (HttpMockServer.Mode == HttpRecorderMode.Record)
8181
{
82-
accessToken = TestEnvironmentFactory.GetTestEnvironment().GetServicePrincipalAccessToken(new[] { "https://attest.azure.net/.default" });
82+
accessToken = TestEnvironmentFactory.GetTestEnvironment().GetAccessToken(new[] { "https://attest.azure.net/.default" });
8383
}
8484

8585
return new AttestationClient(new AttestationCredentials(accessToken), HttpMockServer.CreateInstance());

0 commit comments

Comments
 (0)