Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[App config] Perf tests #15763

Merged
20 commits merged into from
Jun 25, 2021
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3,188 changes: 1,713 additions & 1,475 deletions common/config/rush/pnpm-lock.yaml

Large diffs are not rendered by default.

5 changes: 5 additions & 0 deletions rush.json
Original file line number Diff line number Diff line change
Expand Up @@ -784,6 +784,11 @@
"packageName": "@azure-tests/perf-core-rest-pipeline",
"projectFolder": "sdk/core/perf-tests/core-rest-pipeline",
"versionPolicyName": "test"
},
{
"packageName": "@azure-tests/perf-app-configuration",
"projectFolder": "sdk/appconfiguration/perf-tests/app-configuration",
"versionPolicyName": "test"
}
]
}
9 changes: 9 additions & 0 deletions sdk/appconfiguration/perf-tests/app-configuration/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
### Guide

1. Build the app-config perf tests package `rush build -t perf-app-configuration`.
HarshaNalluru marked this conversation as resolved.
Show resolved Hide resolved
2. Copy the `sample.env` file and name it as `.env`.
3. Create a storage account and populate the `.env` file with `APPCONFIG_CONNECTION_STRING` variable.
HarshaNalluru marked this conversation as resolved.
Show resolved Hide resolved
4. Run the tests as follows

- list settings
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's special about these settings?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

list configuration settings? what do you mean?

- `npm run perf-test:node -- ListSettingsTest --warmup 2 --duration 7 --iterations 2 --parallel 2`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this service rate-limited?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is throttling similar to Form recognizer or metrics advisor.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the operation is not a LRO, can you experiment with the number for the parallel param if the limit is not too bad?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The limit is 20,000 requests per hour, it is quite simple to hit the limit.
We'll soon get the proxy-tool so that we don't have to hit the live service all the time.

45 changes: 45 additions & 0 deletions sdk/appconfiguration/perf-tests/app-configuration/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
{
"name": "@azure-tests/perf-app-configuration",
"version": "1.0.0",
"description": "",
"main": "",
"keywords": [],
"author": "",
"license": "ISC",
"devDependencies": {
HarshaNalluru marked this conversation as resolved.
Show resolved Hide resolved
"@azure/app-configuration": "^1.2.0-beta.2",
"@azure/core-http": "^1.2.0",
"@azure/test-utils-perfstress": "^1.0.0",
"@types/uuid": "^8.0.0",
"@types/node": "^8.0.0",
"dotenv": "^8.2.0",
"uuid": "^8.3.0",
"tslib": "^2.2.0",
"ts-node": "^9.0.0",
"typescript": "~4.2.0"
},
"private": true,
"scripts": {
"perf-test:node": "ts-node test/index.spec.ts",
"audit": "node ../../../common/scripts/rush-audit.js && rimraf node_modules package-lock.json && npm i --package-lock-only 2>&1 && npm audit",
"build": "tsc -p .",
"build:samples": "echo skipped",
"build:test": "echo skipped",
"check-format": "prettier --list-different --config ../../../.prettierrc.json --ignore-path ../../../.prettierignore \"src/**/*.ts\" \"test/**/*.ts\" \"*.{js,json}\"",
"clean": "rimraf dist dist-esm test-dist typings *.tgz *.log",
"format": "prettier --write --config ../../../.prettierrc.json --ignore-path ../../../.prettierignore \"src/**/*.ts\" \"test/**/*.ts\" \"*.{js,json}\"",
"integration-test:browser": "echo skipped",
"integration-test:node": "echo skipped",
"integration-test": "echo skipped",
"lint:fix": "eslint package.json src test --ext .ts --fix --fix-type [problem,suggestion]",
"lint": "eslint package.json src test --ext .ts -f html -o app-config-perf-test-lintReport.html || exit 0",
"pack": "npm pack 2>&1",
"prebuild": "npm run clean",
"unit-test:browser": "echo skipped",
"unit-test:node": "echo skipped",
"unit-test": "echo skipped",
"test:browser": "echo skipped",
"test:node": "echo skipped",
"test": "echo skipped"
}
}
16 changes: 16 additions & 0 deletions sdk/appconfiguration/perf-tests/app-configuration/sample.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# This is the connection string for your app-configuration resource
# You can get this from the Azure portal.
APPCONFIG_CONNECTION_STRING=<app-configuration connection string goes here>

# The following is only required for the secretReference sample
# The name of the key vault to use in the samples.
# Create a Key Vault in the Azure Portal and enter its URI (e.g. https://mytest.vault.azure.net/) here.
KEYVAULT_URI=<key-vault-uri>
HarshaNalluru marked this conversation as resolved.
Show resolved Hide resolved

# Used to authenticate using Azure AD as a service principal for role-based authentication.
#
# See the documentation for `EnvironmentCredential` at the following link:
# https://docs.microsoft.com/javascript/api/@azure/identity/environmentcredential
AZURE_TENANT_ID=<AD tenant id or name>
AZURE_CLIENT_ID=<ID of the user/service principal to authenticate as>
AZURE_CLIENT_SECRET=<client secret used to authenticate to Azure AD>
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
import { PerfStressTest, getEnvVar } from "@azure/test-utils-perfstress";
import { AppConfigurationClient } from "@azure/app-configuration";

export abstract class AppConfigTest<TOptions> extends PerfStressTest<TOptions> {
client: AppConfigurationClient;

constructor() {
super();
const connectionString = getEnvVar("APPCONFIG_CONNECTION_STRING");
HarshaNalluru marked this conversation as resolved.
Show resolved Hide resolved
this.client = new AppConfigurationClient(connectionString);
}

public async globalSetup() {}

public async globalCleanup() {}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import { PerfStressProgram, selectPerfStressTest } from "@azure/test-utils-perfstress";

// Expects the .env file at the same level
import * as dotenv from "dotenv";
import { ListSettingsTest } from "./listSettings.spec";
dotenv.config();
HarshaNalluru marked this conversation as resolved.
Show resolved Hide resolved

console.log("=== Starting the perfStress test ===");

const perfStressProgram = new PerfStressProgram(selectPerfStressTest([ListSettingsTest]));

perfStressProgram.run();
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.

import { generateUuid } from "@azure/core-http";
import { PerfStressOptionDictionary, executeParallel } from "@azure/test-utils-perfstress";
import { AppConfigTest } from "./appConfigBase.spec";

interface ListTestOptions {
count: number;
HarshaNalluru marked this conversation as resolved.
Show resolved Hide resolved
}

export class ListSettingsTest extends AppConfigTest<ListTestOptions> {
static prefix = generateUuid();
public options: PerfStressOptionDictionary<ListTestOptions> = {
count: {
required: true,
description: "Number of settings to be listed",
longName: "count",
defaultValue: 10
}
};

public async globalSetup() {
await super.globalSetup();
await executeParallel(
async (_count: number, _index: number) => {
await this.client.addConfigurationSetting({
key: ListSettingsTest.prefix + generateUuid(),
value: "random"
});
},
this.parsedOptions.count.value!,
32
);
}

async runAsync(): Promise<void> {
for await (const response of this.client
.listConfigurationSettings({ keyFilter: ListSettingsTest.prefix + "*" })
.byPage()) {
for (const _ of response.items) {
HarshaNalluru marked this conversation as resolved.
Show resolved Hide resolved
}
}
}

public async globalCleanup() {
const keys: string[] = [];
for await (const response of this.client
.listConfigurationSettings({ keyFilter: ListSettingsTest.prefix + "*" })
.byPage()) {
for (const setting of response.items) {
keys.push(setting.key);
}
}
await executeParallel(
async (count: number, _: number) => {
await this.client.deleteConfigurationSetting({ key: keys[count] });
},
this.parsedOptions.count.value!,
32
);
await super.globalCleanup();
}
}
16 changes: 16 additions & 0 deletions sdk/appconfiguration/perf-tests/app-configuration/tsconfig.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"extends": "../../../../tsconfig.package",
"compilerOptions": {
"module": "CommonJS",
"declarationDir": "./typings/latest",
"lib": [
"ES6",
"ESNext.AsyncIterable"
],
"noEmit": true
},
"compileOnSave": true,
"include": [
"./test/**/*.ts"
]
}