Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable long running tests and use new federated credentials for testing #864

Merged
merged 59 commits into from
May 24, 2024
Merged
Show file tree
Hide file tree
Changes from 52 commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
f8b7ada
WIP: Get token credential from SP
nturinski Apr 26, 2024
666331d
WIP: See is test provider can sign in
nturinski Apr 29, 2024
ab7d2b3
WIP: Remove webConfig, dont import dev into production code
nturinski Apr 29, 2024
6f26be0
WIP: Remove web entry point from package.json
nturinski Apr 29, 2024
b430191
Add await to the logIn
nturinski Apr 29, 2024
e736093
Add listing tests
nturinski Apr 30, 2024
5702f73
Fix up resource listing test
nturinski Apr 30, 2024
818c9d8
Update service connection
nturinski Apr 30, 2024
fb86ea2
Hardcode long running tests
nturinski Apr 30, 2024
70ea454
Add some console.debug
nturinski Apr 30, 2024
565b639
Make token static
nturinski Apr 30, 2024
914981e
Test code
nturinski May 2, 2024
a3f8c03
Add default
nturinski May 2, 2024
88ddc26
Add resource provider to global test
nturinski May 2, 2024
b02c6eb
Use the resource provider instead of trying to calling it directly
nturinski May 2, 2024
1461993
Try using the key vault values
nturinski May 3, 2024
e39f237
Use fetch and see response type
nturinski May 3, 2024
703699a
Dont parse response2
nturinski May 3, 2024
383bf52
wip
nturinski May 3, 2024
d09823b
Add node-fetch as a resource
nturinski May 3, 2024
bfb4815
Make token code use function
nturinski May 3, 2024
3e5a7fe
Use service client from @azure/core-client
nturinski May 6, 2024
dcd9520
Final test using service client
nturinski May 6, 2024
941b5ba
Remove unused context
nturinski May 6, 2024
ae77264
Confirm access token is being made
nturinski May 6, 2024
1e6b7a8
Leverage auth package
nturinski May 7, 2024
12e4fd7
Add create and delete tests for Resource Groups
nturinski May 9, 2024
1dde188
Merge main
nturinski May 9, 2024
511c8e2
Comments
nturinski May 9, 2024
5b043ed
Import settingUtils from bundle
nturinski May 9, 2024
8d072e8
Import LocationListStep from bundle?
nturinski May 9, 2024
8a34174
Reorganize and remove some inports from bundle
nturinski May 9, 2024
b676b40
Add hasPortalUrl
nturinski May 9, 2024
9d443e2
See if there are any build errors
nturinski May 9, 2024
7a0c045
Merge branch 'main' of https://github.com/microsoft/vscode-azureresou…
nturinski May 9, 2024
7870694
use auth refactor
hossam-nasr May 14, 2024
53b89af
update package json
hossam-nasr May 14, 2024
c222174
use newly released auth package
hossam-nasr May 16, 2024
082f9fa
use parameter
hossam-nasr May 16, 2024
96ef64f
Merge branch 'main' into hossamnasr/useNewConstructor
hossam-nasr May 16, 2024
c5f542b
remove installing azure account
hossam-nasr May 16, 2024
5940e78
remove pretest step
hossam-nasr May 16, 2024
6b345a3
remove installing azure account fr fr
hossam-nasr May 16, 2024
53a928d
remove from recommendations
hossam-nasr May 16, 2024
8108fad
reintroduce gulp
hossam-nasr May 16, 2024
717d7e1
nuclear warfare
hossam-nasr May 17, 2024
7619718
add webpack to pretest
hossam-nasr May 17, 2024
aa343d7
logs
hossam-nasr May 17, 2024
c73b107
try logging error
hossam-nasr May 17, 2024
9ba3554
use the new env vars with the underscore
hossam-nasr May 17, 2024
efaf5d9
don't swallow errors
hossam-nasr May 23, 2024
46ae5d4
cleanup
hossam-nasr May 23, 2024
7e62b84
only set to true when doing azure federated credentials
hossam-nasr May 23, 2024
adf99a1
use new branch for quick testing
hossam-nasr May 23, 2024
8c5d849
use main again
hossam-nasr May 23, 2024
2bd383d
refactor subscription provider logic
hossam-nasr May 23, 2024
f33e2dc
remove commented out code
hossam-nasr May 23, 2024
82410be
set in the helper
hossam-nasr May 24, 2024
9dc02eb
make optional
hossam-nasr May 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .azure-pipelines/1esmain.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,5 @@ resources:
# Use those templates
extends:
template: azure-pipelines/1esmain.yml@azExtTemplates
parameters:
useAzureFederatedCredentials: true
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you look at the rest of the file, this pipeline file is only run when scheduled for nightly builds. Normal build pipelines that occur on PRs are done through GitHub Actions and not through this file. This is why useAzureFederatedCredentials is always set to true. For completeness sake, we could also set it to only be true if the build reason is Schedule, but that would make it harder to test manually for any reason

1 change: 0 additions & 1 deletion .vscode/extensions.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
{
"recommendations": [
"dbaeumer.vscode-eslint",
"ms-vscode.azure-account"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no longer need this recommendation

]
}
17 changes: 10 additions & 7 deletions extension.bundle.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
//
// The tests should import '../extension.bundle'. At design-time they live in tests/ and so will pick up this file (extension.bundle.ts).
// At runtime the tests live in dist/tests and will therefore pick up the main webpack bundle at dist/extension.bundle.js.
export { LocationListStep } from '@microsoft/vscode-azext-azureutils';
export * from '@microsoft/vscode-azext-utils';
export * from './api/src/AzExtResourceType';
// export * from './api/src';
Expand All @@ -23,25 +24,27 @@ export * from './api/src/resources/base';
export * from './api/src/resources/workspace';
export * from './api/src/utils/getApi';
export * from './api/src/utils/wrapper';
export * from './src/api/DefaultAzureResourceProvider';
export { convertV1TreeItemId } from './src/api/compatibility/CompatibleAzExtTreeDataProvider';
export * from './src/commands/openInPortal';
export { hasPortalUrl } from './src/commands/openInPortal';
export * from './src/commands/tags/getTagDiagnostics';
export * from './src/commands/viewProperties';
export * from './src/services/AzureResourcesService';
// Export activate/deactivate for main.js
export { createResourceGroup } from './src/commands/createResourceGroup';
export * from './src/commands/deleteResourceGroup/v2/deleteResourceGroupV2';
export { activate, deactivate } from './src/extension';
export * from './src/extensionVariables';
export * from './src/hostapi.v2.internal';
export * from './src/tree/BranchDataItemWrapper';
export * from './src/tree/InvalidItem';
export { ResourceGroupsItem } from './src/tree/ResourceGroupsItem';
export * from './src/tree/azure/AzureResourceItem';
export * from './src/tree/azure/SubscriptionItem';
export { createSubscriptionContext as createSubscriptionContext2 } from './src/tree/azure/VSCodeAuthentication';
export * from './src/tree/azure/grouping/GroupingItem';
export * from './src/tree/azure/grouping/LocationGroupingItem';
export * from './src/tree/azure/grouping/ResourceGroupGroupingItem';
export * from './src/tree/azure/grouping/ResourceTypeGroupingItem';
export * from './src/tree/azure/SubscriptionItem';
export * from './src/tree/BranchDataItemWrapper';
export * from './src/tree/InvalidItem';
export { ResourceGroupsItem } from './src/tree/ResourceGroupsItem';
export * from './src/utils/azureClients';
export * from './src/utils/settingUtils';
export * from './src/utils/wrapFunctionsInTelemetry';
// NOTE: The auto-fix action "source.organizeImports" does weird things with this file, but there doesn't seem to be a way to disable it on a per-file basis so we'll just let it happen
3 changes: 1 addition & 2 deletions gulpfile.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/

import { gulp_installAzureAccount, gulp_webpack } from '@microsoft/vscode-azext-dev';
import { gulp_webpack } from '@microsoft/vscode-azext-dev';
import * as fs from 'fs/promises';
import * as gulp from 'gulp';
import * as path from 'path';
Expand Down Expand Up @@ -50,6 +50,5 @@ async function cleanReadme(): Promise<void> {

exports['webpack-dev'] = gulp.series(prepareForWebpack, () => gulp_webpack('development'));
exports['webpack-prod'] = gulp.series(prepareForWebpack, () => gulp_webpack('production'));
exports.preTest = gulp_installAzureAccount;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We no longer need to install Azure Account before tests

exports.listIcons = listIcons;
exports.cleanReadme = cleanReadme;
1,016 changes: 731 additions & 285 deletions package-lock.json

Large diffs are not rendered by default.

11 changes: 7 additions & 4 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -525,7 +525,8 @@
"%azureResourceGroups.deleteConfirmation.EnterName%",
"%azureResourceGroups.deleteConfirmation.ClickButton%"
],
"default": "EnterName"
"default": "EnterName",
"scope": "machine"
},
"azureResourceGroups.showHiddenTypes": {
"type": "boolean",
Expand All @@ -535,7 +536,8 @@
"azureResourceGroups.groupBy": {
"type": "string",
"description": "%azureResourceGroups.groupBy%",
"default": "resourceType"
"default": "resourceType",
"scope": "machine"
},
"azureResourceGroups.suppressActivityNotifications": {
"type": "boolean",
Expand Down Expand Up @@ -625,8 +627,8 @@
"lint": "eslint --ext .ts .",
"lint-fix": "eslint --ext .ts . --fix",
"listIcons": "gulp listIcons",
"pretest": "gulp preTest",
"test": "node ./out/test/runTest.js",
"pretest": "webpack",
"webpack": "tsc && gulp webpack-dev",
"webpack-prod": "npm run build && gulp webpack-prod",
"webpack-profile": "webpack --profile --json --mode production > webpack-stats.json && echo Use http://webpack.github.io/analyse to analyze the stats",
Expand All @@ -635,6 +637,7 @@
},
"devDependencies": {
"@azure/arm-resources-subscriptions": "^2.1.0",
"@azure/identity": "^4.1.0",
"@microsoft/api-extractor": "^7.33.8",
"@microsoft/eslint-config-azuretools": "^0.2.1",
"@microsoft/vscode-azext-dev": "^2.0.0",
Expand Down Expand Up @@ -668,7 +671,7 @@
"@azure/arm-resources": "^5.2.0",
"@azure/arm-resources-profile-2020-09-01-hybrid": "^2.1.0",
"@azure/ms-rest-js": "^2.7.0",
"@microsoft/vscode-azext-azureauth": "^2.3.0",
"@microsoft/vscode-azext-azureauth": "^2.4.1",
"@microsoft/vscode-azext-azureutils": "^2.0.0",
"@microsoft/vscode-azext-utils": "^2.4.0",
"buffer": "^6.0.3",
Expand Down
28 changes: 27 additions & 1 deletion src/extension.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

'use strict';

import { AzureDevOpsSubscriptionProviderInitializer, createAzureDevOpsSubscriptionProviderFactory } from '@microsoft/vscode-azext-azureauth';
import { registerAzureUtilsExtensionVariables, setupAzureLogger } from '@microsoft/vscode-azext-azureutils';
import { AzExtTreeDataProvider, AzureExtensionApiFactory, IActionContext, callWithTelemetryAndErrorHandling, createApiProvider, createAzExtLogOutputChannel, createExperimentationService, registerUIExtensionVariables } from '@microsoft/vscode-azext-utils';
import { AzureSubscription } from 'api/src';
Expand Down Expand Up @@ -69,7 +70,32 @@ export async function activate(context: vscode.ExtensionContext, perfStats: { lo
activateContext.telemetry.properties.isActivationEvent = 'true';
activateContext.telemetry.measurements.mainFileLoad = (perfStats.loadEndTime - perfStats.loadStartTime) / 1000;

ext.subscriptionProviderFactory = createVSCodeAzureSubscriptionProviderFactory();
// if this for a nightly test, we want to use the test subscription provider
const useAzureFederatedCredentials: boolean = !/^(false|0)?$/i.test(process.env['AzCode_UseAzureFederatedCredentials'] || '')
hossam-nasr marked this conversation as resolved.
Show resolved Hide resolved
if (useAzureFederatedCredentials) {
// when running tests, ensure we throw the errors and they aren't silently swallowed
hossam-nasr marked this conversation as resolved.
Show resolved Hide resolved
activateContext.errorHandling.rethrow = true;
const serviceConnectionId: string | undefined = process.env['AzCode_ServiceConnectionID'];
const domain: string | undefined = process.env['AzCode_ServiceConnectionDomain'];
const clientId: string | undefined = process.env['AzCode_ServiceConnectionClientID'];

if (!serviceConnectionId || !domain || !clientId) {
throw new Error(`Using Azure DevOps federated credentials, but federated service connection is not configured\n
process.env.AzCodeServiceConnectionID: ${serviceConnectionId ? "✅" : "❌"}\n
process.env.AzCodeServiceConnectionDomain: ${domain ? "✅" : "❌"}\n
process.env.AzCodeServiceConnectionClientID: ${clientId ? "✅" : "❌"}\n
`);
}

const initializer: AzureDevOpsSubscriptionProviderInitializer = {
serviceConnectionId,
domain,
clientId,
}
ext.subscriptionProviderFactory = createAzureDevOpsSubscriptionProviderFactory(initializer);
} else {
ext.subscriptionProviderFactory = createVSCodeAzureSubscriptionProviderFactory();
}

ext.tagFS = new TagFileSystem(ext.appResourceTree);
context.subscriptions.push(vscode.workspace.registerFileSystemProvider(TagFileSystem.scheme, ext.tagFS));
Expand Down
6 changes: 1 addition & 5 deletions src/extensionVariables.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
import { AzureSubscriptionProvider } from "@microsoft/vscode-azext-azureauth";
import { AzExtTreeDataProvider, IAzExtLogOutputChannel, IExperimentationServiceAdapter } from "@microsoft/vscode-azext-utils";
import { AzExtResourceType } from "api/src/AzExtResourceType";
import { DiagnosticCollection, Disposable, ExtensionContext, TreeView, UIKind, env } from "vscode";
import { DiagnosticCollection, Disposable, ExtensionContext, TreeView } from "vscode";
import { ActivityLogTreeItem } from "./activityLog/ActivityLogsTreeItem";
import { TagFileSystem } from "./commands/tags/TagFileSystem";
import { AzureResourcesApiInternal } from "./hostapi.v2.internal";
Expand Down Expand Up @@ -47,10 +47,6 @@ export namespace ext {

export let subscriptionProviderFactory: () => Promise<AzureSubscriptionProvider>;

// When debugging thru VS Code as a web environment, the UIKind is Desktop. However, if you sideload it into the browser, you must
// change this to UIKind.Web and then webpack it again
export const isWeb: boolean = env.uiKind === UIKind.Web;

export namespace v2 {
export let api: AzureResourcesApiInternal;
}
Expand Down
4 changes: 2 additions & 2 deletions test/api/mockServiceFactory.ts
Original file line number Diff line number Diff line change
Expand Up @@ -97,13 +97,13 @@ class BasicMockResources extends MockResources {

export const createMockSubscriptionWithFunctions = (): BasicMockResources => {
const mockResources = new BasicMockResources();
ext.testing.overrideAzureServiceFactory = createTestAzureResourcesServiceFactory(mockResources);
ext.testing.overrideAzureServiceFactory = createMockAzureResourcesServiceFactory(mockResources);
const mockAzureSubscriptionProvider = new MockAzureSubscriptionProvider(mockResources);
ext.testing.overrideAzureSubscriptionProvider = () => mockAzureSubscriptionProvider;
return mockResources;
}

export function createTestAzureResourcesServiceFactory(mockResources: MockResources): AzureResourcesServiceFactory {
export function createMockAzureResourcesServiceFactory(mockResources: MockResources): AzureResourcesServiceFactory {
return () => {
return {
async listResources(_context, subscription: AzureSubscription): Promise<GenericResource[]> {
Expand Down
20 changes: 17 additions & 3 deletions test/global.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@

import { TestOutputChannel, TestUserInput } from '@microsoft/vscode-azext-dev';
import * as vscode from 'vscode';
import { ext, registerOnActionStartHandler } from '../extension.bundle';
import { ext, registerOnActionStartHandler, settingUtils } from '../extension.bundle';

export let longRunningTestsEnabled: boolean;

export const userSettings: { key: string, value: unknown }[] = [];
// Runs before all tests
suiteSetup(async function (this: Mocha.Context): Promise<void> {
this.timeout(1 * 60 * 1000);
Expand All @@ -22,6 +22,20 @@ suiteSetup(async function (this: Mocha.Context): Promise<void> {
// Use `TestUserInput` by default so we get an error if an unexpected call to `context.ui` occurs, rather than timing out
context.ui = new TestUserInput(vscode);
});
const groupBySetting = settingUtils.getWorkspaceSetting('groupBy')
userSettings.push({ key: 'groupBy', value: groupBySetting });

const deleteConfirmationSetting = settingUtils.getWorkspaceSetting('deleteConfirmation');
userSettings.push({ key: 'deleteConfirmation', value: deleteConfirmationSetting });

// TODO: Set to true for testing purposes
longRunningTestsEnabled = true; //!/^(false|0)?$/i.test(process.env.ENABLE_LONG_RUNNING_TESTS || '');
hossam-nasr marked this conversation as resolved.
Show resolved Hide resolved
});

longRunningTestsEnabled = !/^(false|0)?$/i.test(process.env.ENABLE_LONG_RUNNING_TESTS || '');
suiteTeardown(async function (this: Mocha.Context): Promise<void> {
for (const setting of userSettings) {
// reset the settings to their original values
console.debug(`Resetting setting '${setting.key}' to '${setting.value}'`);
await settingUtils.updateGlobalSetting(setting.key, setting.value);
}
});
127 changes: 127 additions & 0 deletions test/nightly/crud.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/

import { type Location } from '@azure/arm-resources-subscriptions';
import { createTestActionContext, runWithTestActionContext } from '@microsoft/vscode-azext-dev';
import { AzExtParentTreeItem, createResourceClient, createResourceGroup, deleteResourceGroupV2, ext, IActionContext, LocationListStep, randomUtils, settingUtils, SubscriptionItem } from '../../extension.bundle';
import { longRunningTestsEnabled } from "../global.test";
import assert = require("assert");

let rgName: string;
let locations: Location[];
let testSubscription: SubscriptionItem;

suite('Resource CRUD Operations', function (this: Mocha.Suite): void {
this.timeout(7 * 60 * 1000);

suiteSetup(async function (this: Mocha.Context): Promise<void> {
if (!longRunningTestsEnabled) {
this.skip();
}

ext.testing.overrideAzureServiceFactory = undefined;
ext.testing.overrideAzureSubscriptionProvider = undefined;

const subscriptionTreeItems = await ext.appResourceTree.getChildren() as unknown as SubscriptionItem[];
if (subscriptionTreeItems.length > 0) {
const testContext = await createTestActionContext();
testSubscription = subscriptionTreeItems[0] as SubscriptionItem;
const context = { ...testContext, ...testSubscription.subscription };
locations = await LocationListStep.getLocations(context);
}

rgName = randomUtils.getRandomHexString(12);
});

test('Create Resource Group (single)', async () => {
await runWithTestActionContext('createResourceGroup', async context => {
const testInputs: (string | RegExp)[] = [rgName, locations[0].displayName!];
await context.ui.runWithInputs(testInputs, async () => {
await createResourceGroup(context, testSubscription);
});

assert.ok(await resourceGroupExists(context, rgName));
});
});

test('Create Resource Groups (all locations)', async () => {
await Promise.all(locations.map(async l => {
await runWithTestActionContext('createResourceGroup', async context => {
const testInputs: (string | RegExp)[] = [`${rgName}-${l.name}`, l.displayName!];
await context.ui.runWithInputs(testInputs, async () => {
await createResourceGroup(context, testSubscription);
});

assert.ok(await resourceGroupExists(context, `${rgName}-${l.name}`));
});
}));
});

test('Get Resources', async () => {
const subscriptionTreeItems = await ext.appResourceTree.getChildren();
assert.ok(subscriptionTreeItems.length > 0);
for (const subscription of subscriptionTreeItems) {
const groupTreeItems = await ext.appResourceTree.getChildren(subscription as AzExtParentTreeItem);
await Promise.all(groupTreeItems.map(async g => {
const children = await ext.appResourceTree.getChildren(g as AzExtParentTreeItem);
console.log(children);
}));
}

assert.ok(true);
});

test('Delete Resource (EnterName) - Fails when invalid', async () => {
await settingUtils.updateGlobalSetting('deleteConfirmation', 'EnterName');
await runWithTestActionContext('Delete Resource', async context => {
await context.ui.runWithInputs([rgName, 'rgName'], async () => {
try {
await deleteResourceGroupV2(context);
} catch (_) {
console.debug('Expected error: ', _);
// expected to fail here
}
assert.ok(await resourceGroupExists(context, rgName));
});
});
});

test('Delete Resource (EnterName)', async () => {
await settingUtils.updateGlobalSetting('deleteConfirmation', 'EnterName');
await runWithTestActionContext('Delete Resource', async context => {
await context.ui.runWithInputs([rgName, rgName], async () => {
await deleteResourceGroupV2(context);
});

// ext.azureResourceProvider.onDidChangeResource!(async e => {
// console.log(e);
// assert.ok(!(await resourceGroupExists(context, rgName)));
// })
});

});

test('Delete Resource (ClickButton)', async () => {
await settingUtils.updateGlobalSetting('deleteConfirmation', 'ClickButton');
const deleteArray: string[] = Array(locations.length).fill('Delete');
await runWithTestActionContext('Delete Resource', async context => {
await context.ui.runWithInputs([new RegExp(`${rgName}-`), ...deleteArray], async () => {
await deleteResourceGroupV2(context);
});
});

assert.ok(true);
});
});

async function resourceGroupExists(context: IActionContext, rgName: string): Promise<boolean> {
const client = await createResourceClient([context, testSubscription.subscription]);
try {
await client.resourceGroups.get(rgName);
return true;
} catch (_) {
return false;
}
}
19 changes: 19 additions & 0 deletions test/nightly/global.nightly.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See LICENSE.md in the project root for license information.
*--------------------------------------------------------------------------------------------*/

import * as vscode from 'vscode';
import { longRunningTestsEnabled } from '../global.test';

export const resourceGroupsToDelete: string[] = [];

// Runs before all nightly tests
suiteSetup(async function (this: Mocha.Context): Promise<void> {
// TODO: Use other environment variables to determine if the tests should be run
if (longRunningTestsEnabled) {
this.timeout(2 * 60 * 1000);
await vscode.commands.executeCommand('azureResourceGroups.logIn');
}
});

Loading
Loading