Skip to content

fix(tests): enhance reliability of homepage navigation test#278

Merged
yamcodes merged 1 commit intomainfrom
fix-broken-e2e-tests
Nov 4, 2025
Merged

fix(tests): enhance reliability of homepage navigation test#278
yamcodes merged 1 commit intomainfrom
fix-broken-e2e-tests

Conversation

@yamcodes
Copy link
Owner

@yamcodes yamcodes commented Nov 4, 2025

Updated the homepage to quickstart navigation test to wait for the "Set sail" button to be visible instead of relying on the "networkidle" state. Added a timeout for the navigation verification to improve test reliability.

Summary by CodeRabbit

  • Tests
    • Improved test reliability for documentation navigation by optimizing wait conditions and timeout handling.

Updated the homepage to quickstart navigation test to wait for the "Set sail" button to be visible instead of relying on the "networkidle" state. Added a timeout for the navigation verification to improve test reliability.
@changeset-bot
Copy link

changeset-bot bot commented Nov 4, 2025

⚠️ No Changeset found

Latest commit: d784590

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@vercel
Copy link

vercel bot commented Nov 4, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
arkenv Ready Ready Preview Comment Nov 4, 2025 6:48pm

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Nov 4, 2025

Walkthrough

The test file docs-navigation.test.ts was updated to improve reliability by replacing implicit network idle waits with explicit visibility and URL navigation waits, ensuring consistent test behavior across different browser environments.

Changes

Cohort / File(s) Summary
Test Reliability Improvements
tooling/playwright-www/tests/docs-navigation.test.ts
Replaced initial network idle wait with explicit visibility wait for "Set sail" button; added explicit 30-second waitForURL for quickstart page navigation after button click

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~5 minutes

  • Verify that the new explicit waits (visibility wait and waitForURL) are appropriate timeout values
  • Confirm the button selector a[href='/docs/quickstart'] is correct and stable
  • Check that subsequent assertions still validate the expected behavior

Suggested labels

www, tests

Poem

🐰 Hopping through tests with clearer sight,
No more guessing when things are right,
Explicit waits make browsers agree,
Now every click flows steadily!

Pre-merge checks and finishing touches

✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: enhancing reliability of the homepage navigation test through improved wait conditions.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch fix-broken-e2e-tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions bot added the tests This issue or PR is about adding, removing or changing tests label Nov 4, 2025
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
tooling/playwright-www/tests/docs-navigation.test.ts (2)

69-69: Consider removing redundant URL assertion.

After waitForURL successfully resolves on Line 68, this assertion will always pass. While being explicit isn't harmful, you could streamline the test by removing this line since the URL match is already verified.

Apply this diff if you prefer a more concise test:

 		// Wait for navigation to complete with timeout
 		await page.waitForURL("**/docs/quickstart", { timeout: 30000 });
-		await expect(page).toHaveURL("/docs/quickstart");
 		await expect(page.locator("body")).toBeVisible();

58-71: Consider applying this pattern to other navigation tests.

Since networkidle proved unreliable for this test (especially on webkit), you might want to audit other tests in this file that still use waitForLoadState("networkidle") (lines 15, 33, 48, 78, 89, 95, 107, 130, 140) to see if they could benefit from similar explicit visibility/navigation checks.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f73d2bd and d784590.

📒 Files selected for processing (1)
  • tooling/playwright-www/tests/docs-navigation.test.ts (1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.test.ts

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

**/*.test.ts: Place tests alongside source files and use the .test.ts suffix (e.g., create-env.test.ts, types.test.ts, errors.test.ts, utils.test.ts)
Use Vitest's describe/it structure in tests
Test both success and failure cases for environment validation and utilities
Mock process.env to cover different environment scenarios in tests
In tests, save and restore process.env in setup/teardown (beforeEach/afterEach) to avoid leakage between cases
Verify both runtime behavior and TypeScript types in tests

Files:

  • tooling/playwright-www/tests/docs-navigation.test.ts
🧠 Learnings (1)
📓 Common learnings
Learnt from: CR
Repo: yamcodes/arkenv PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-09-12T06:23:45.463Z
Learning: Applies to **/*.test.ts : Verify both runtime behavior and TypeScript types in tests
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
  • GitHub Check: test-e2e (e2e)
  • GitHub Check: test-e2e (a11y)
🔇 Additional comments (1)
tooling/playwright-www/tests/docs-navigation.test.ts (1)

60-68: LGTM! Good improvement to test reliability.

Replacing the networkidle wait with an explicit visibility check for the button and adding waitForURL for navigation verification is a solid approach. This makes the test more deterministic and browser-agnostic, particularly for webkit as noted in the comment.

@yamcodes yamcodes merged commit 1870c09 into main Nov 4, 2025
15 checks passed
@yamcodes yamcodes deleted the fix-broken-e2e-tests branch November 4, 2025 18:56
@coderabbitai coderabbitai bot mentioned this pull request Dec 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tests This issue or PR is about adding, removing or changing tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant