Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: run flaky tests on GitHub action #4902

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from
Draft

fix: run flaky tests on GitHub action #4902

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
c65d0b3
fix: run flaky tests on GitHub action
bonyuta0204 Jan 8, 2024
97760e9
fix: typo in test
bonyuta0204 Jan 16, 2024
6a4bae1
Merge remote-tracking branch 'upstream/main' into fix/flaky-tests
bonyuta0204 Jan 17, 2024
c3134f4
wip: add error detail
bonyuta0204 Jan 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions packages/vitest/src/node/error.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
}

export async function printError(error: unknown, project: WorkspaceProject | undefined, options: PrintErrorOptions) {
console.log('[printError] error:', error)

Check failure on line 26 in packages/vitest/src/node/error.ts

View workflow job for this annotation

GitHub Actions / lint 

Unexpected console statement
const { showCodeFrame = true, fullStack = false, type } = options
const logger = options.logger
let e = error as ErrorWithDiff
Expand Down
4 changes: 0 additions & 4 deletions test/watch/test/file-watching.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,10 +40,6 @@ afterEach(() => {
cleanups.splice(0).forEach(cleanup => cleanup())
})

// TODO: Fix flakiness and enable on CI
if (process.env.GITHUB_ACTIONS)
test.only('skip tests on CI', () => {})

test('editing source file triggers re-run', async () => {
Copy link
Member

@sheremet-va sheremet-va Jan 17, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the flakiness in files that rely on watch mode might be due to the watcher incorrectly emitting ready event too soon. You can try incorporating the utility I wrote for Vite tests from here: https://github.com/sheremet-va/vite/blob/4df5ac29087f448a40f80707dec78d197e74c6d2/playground/hmr-ssr/__tests__/hmr.spec.ts#L960

We already check for a ready event with an env variable for tests (maybe it can be improved with the utility here):

vitest/packages/vite-node/src/hmr/emitter.ts

Line 54 in 8877e22

if (process.env.VITE_TEST_WATCHER_DEBUG) {

vitest/packages/vitest/src/node/plugins/index.ts

Line 181 in 8877e22

if (options.watch && process.env.VITE_TEST_WATCHER_DEBUG) {

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your advice!
I will check it out!

const vitest = await runVitestCli(...cliArgs)

Expand Down
8 changes: 2 additions & 6 deletions test/watch/test/stdin.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,6 @@ afterEach(() => {
cleanups.splice(0).forEach(fn => fn())
})

// TODO: Fix flakiness and enable on CI
if (process.env.GITHUB_ACTIONS)
test.only('skip tests on CI', () => {})

test('quit watch mode', async () => {
const vitest = await runVitestCli(...cliArgs)

Expand All @@ -47,7 +43,7 @@ test('filter by filename', async () => {

vitest.write('math')

await vitest.waitForStdout('Pattern matches 1 results')
await vitest.waitForStdout('Pattern matches 1 result')
await vitest.waitForStdout('› math.test.ts')

vitest.write('\n')
Expand All @@ -64,7 +60,7 @@ test('filter by test name', async () => {
await vitest.waitForStdout('Input test name pattern')

vitest.write('sum')
await vitest.waitForStdout('Pattern matches 1 results')
await vitest.waitForStdout('Pattern matches 1 result')
await vitest.waitForStdout('› sum')

vitest.write('\n')
Expand Down
Loading