Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(test runner): allow to pass arbitrary location to test.step #32504

Merged
merged 18 commits into from
Sep 17, 2024
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions docs/src/test-api/class-test.md
Original file line number Diff line number Diff line change
Expand Up @@ -1579,6 +1579,46 @@ test('test', async ({ page }) => {
});
```

**Custom Step Location**
dgozman marked this conversation as resolved.
Show resolved Hide resolved

Allows the specification of a custom location for a test step. This is particularly useful when `test.step` is used within helper functions or other layers of abstraction, enabling more accurate pinpointing of the step's origin in the source code.

```js
import { test, expect } from '@playwright/test';
import { getStepLocation } from './helpers';

test('user login process with custom step location', async ({ page }) => {
const location = getStepLocation();
await test.step(`Navigate to login page`, async () => {
await page.goto('https://example.com/login');
}, { location });

await test.step(`Fill in login credentials and submit`, async () => {
await page.fill('input#username', 'testuser');
await page.fill('input#password', 'testpass');
await page.click('button[type="submit"]');
}, { location });

await test.step(`Check if the profile is visible post-login`, async () => {
const profileVisible = await page.isVisible('.profile');
expect(profileVisible).toBeTruthy();
}, { location });
});

function getStepLocation() {
const error = new Error();
const stackInfo = error.stack.split('\n')[2]; // Adjust based on environment
const match = /at (.+):(\d+):(\d+)/.exec(stackInfo);
return {
file: match[1],
line: parseInt(match[2], 10),
column: parseInt(match[3], 10)
};
}
```

This revised code demonstrates how to track and report the location of each step in a user login process, making debugging and tracing much easier in complex tests. The getStepLocation function is crafted to fetch the exact location where the test step is defined, which can be dynamically adjusted to fit different JavaScript environments.

**Decorator**

You can use TypeScript method decorators to turn a method into a step.
Expand Down
4 changes: 2 additions & 2 deletions packages/playwright/src/common/testType.ts
Original file line number Diff line number Diff line change
Expand Up @@ -259,11 +259,11 @@ export class TestTypeImpl {
suite._use.push({ fixtures, location });
}

async _step<T>(title: string, body: () => Promise<T>, options: { box?: boolean } = {}): Promise<T> {
async _step<T>(title: string, body: () => Promise<T>, options: {box?: boolean, location?: Location } = {}): Promise<T> {
const testInfo = currentTestInfo();
if (!testInfo)
throw new Error(`test.step() can only be called from a test`);
const step = testInfo._addStep({ category: 'test.step', title, box: options.box });
const step = testInfo._addStep({ category: 'test.step', title, location: options.location, box: options.box });
return await zones.run('stepZone', step, async () => {
try {
const result = await body();
Expand Down
47 changes: 46 additions & 1 deletion packages/playwright/types/test.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4584,6 +4584,51 @@ export interface TestType<TestArgs extends KeyValue, WorkerArgs extends KeyValue
* });
* ```
*
* **Custom Step Location**
*
* Allows the specification of a custom location for a test step. This is particularly useful when `test.step` is used
* within helper functions or other layers of abstraction, enabling more accurate pinpointing of the step's origin in
* the source code.
*
* ```js
* import { test, expect } from '@playwright/test';
* import { getStepLocation } from './helpers';
*
* test('user login process with custom step location', async ({ page }) => {
* const location = getStepLocation();
* await test.step(`Navigate to login page`, async () => {
* await page.goto('https://example.com/login');
* }, { location });
*
* await test.step(`Fill in login credentials and submit`, async () => {
* await page.fill('input#username', 'testuser');
* await page.fill('input#password', 'testpass');
* await page.click('button[type="submit"]');
* }, { location });
*
* await test.step(`Check if the profile is visible post-login`, async () => {
* const profileVisible = await page.isVisible('.profile');
* expect(profileVisible).toBeTruthy();
* }, { location });
* });
*
* function getStepLocation() {
* const error = new Error();
* const stackInfo = error.stack.split('\n')[2]; // Adjust based on environment
* const match = /at (.+):(\d+):(\d+)/.exec(stackInfo);
* return {
* file: match[1],
* line: parseInt(match[2], 10),
* column: parseInt(match[3], 10)
* };
* }
* ```
*
* This revised code demonstrates how to track and report the location of each step in a user login process, making
* debugging and tracing much easier in complex tests. The getStepLocation function is crafted to fetch the exact
* location where the test step is defined, which can be dynamically adjusted to fit different JavaScript
* environments.
*
* **Decorator**
*
* You can use TypeScript method decorators to turn a method into a step. Each call to the decorated method will show
Expand Down Expand Up @@ -4703,7 +4748,7 @@ export interface TestType<TestArgs extends KeyValue, WorkerArgs extends KeyValue
* @param body Step body.
* @param options
*/
step<T>(title: string, body: () => T | Promise<T>, options?: { box?: boolean }): Promise<T>;
step<T>(title: string, body: () => T | Promise<T>, options?: { box?: boolean, location?: Location }): Promise<T>;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please note that Location here corresponds to the global lib-dom Location, as opposed to the Playwright's Location interface.

I'd suggest moving the definition of Playwright's Location from testReporter.d.ts to playwright/types/test.d.ts since it's now part of the public API.

@dgozman - what's your view?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at it along with the issues you mentioned, I think it's right to change like you said.
The maintainers fixed the bug according to your issue. Thank you for the review. 👍🏻👍🏻

/**
* `expect` function can be used to create test assertions. Read more about [test assertions](https://playwright.dev/docs/test-assertions).
*
Expand Down
93 changes: 93 additions & 0 deletions tests/playwright-test/test-step.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1239,3 +1239,96 @@ fixture | fixture: page
fixture | fixture: context
`);
});

test('test with custom location', async ({ runInlineTest }) => {
const result = await runInlineTest({
'reporter.ts': stepIndentReporter,
'helper.ts': `
import { test } from '@playwright/test';

export function getStepLocation() {
return { file: 'custom-file.ts', line: 99, column: 1 };
}

export async function customStep(test, title, action) {
const location = getStepLocation();
return await test.step(title, action, { location });
}
`,
'playwright.config.ts': `
module.exports = {
reporter: './reporter',
};
`,
'a.test.ts': `
import { test } from '@playwright/test';
import { customStep } from './helper';

test('test with custom location', async ({ page }) => {
await customStep(test, 'User logs in', async () => {
await page.goto('http://localhost:1234/login');
await page.fill('input#username', 'testuser');
await page.fill('input#password', 'testpass');
await page.click('button[type="submit"]');
await page.waitForSelector('.profile');
const profileVisible = await page.isVisible('.profile');
expect(profileVisible).toBeTruthy();
});
});
`
}, { reporter: '', workers: 1 });

expect(result.exitCode).toBe(0);
expect(result.outputLines).toEqual([
'hook |Before Hooks',
'test.step |User logs in @ custom-file.ts:99',
'test.step |↪ success: Profile is visible',
'hook |After Hooks'
]);
});

test('nested step test with custom locations', async ({ runInlineTest }) => {
const result = await runInlineTest({
'reporter.ts': stepIndentReporter,
'helper.ts': `
import { test } from '@playwright/test';

export function getStepLocation(line) {
return { file: 'nested-steps.ts', line: line, column: 1 };
}

export async function customStep(test, title, line, action) {
const location = getStepLocation(line);
return await test.step(title, action, { location });
}
`,
'playwright.config.ts': `
module.exports = {
reporter: './reporter',
};
`,
'a.test.ts': `
import { test } from '@playwright/test';
import { customStep } from './helper';

test('nested step test with custom locations', async ({ page }) => {
await customStep(test, 'Outer step', 10, async () => {
await page.goto('http://localhost:1234');
await customStep(test, 'Inner step', 20, async () => {
const buttonVisible = await page.isVisible('button#submit');
expect(buttonVisible).toBeTruthy();
});
});
});
`
}, { reporter: '', workers: 1 });

expect(result.exitCode).toBe(0);
expect(result.outputLines).toEqual([
'hook |Before Hooks',
'test.step |Outer step @ nested-steps.ts:10',
'test.step |Inner step @ nested-steps.ts:20',
'test.step |↪ success: Button is visible',
'hook |After Hooks'
]);
});
2 changes: 1 addition & 1 deletion utils/generate_types/overrides-test.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ export interface TestType<TestArgs extends KeyValue, WorkerArgs extends KeyValue
afterAll(inner: (args: TestArgs & WorkerArgs, testInfo: TestInfo) => Promise<any> | any): void;
afterAll(title: string, inner: (args: TestArgs & WorkerArgs, testInfo: TestInfo) => Promise<any> | any): void;
use(fixtures: Fixtures<{}, {}, TestArgs, WorkerArgs>): void;
step<T>(title: string, body: () => T | Promise<T>, options?: { box?: boolean }): Promise<T>;
step<T>(title: string, body: () => T | Promise<T>, options?: { box?: boolean, location?: Location }): Promise<T>;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In addition to this, you have to update documentation at docs/src/test-api/class-test.md.

expect: Expect<{}>;
extend<T extends KeyValue, W extends KeyValue = {}>(fixtures: Fixtures<T, W, TestArgs, WorkerArgs>): TestType<TestArgs & T, WorkerArgs & W>;
info(): TestInfo;
Expand Down
Loading