Skip to content

docs: document test execution order and $.tester access - #3192

Open
pdenert wants to merge 1 commit into
masterfrom
docs/test-execution-order
Open

docs: document test execution order and $.tester access#3192
pdenert wants to merge 1 commit into
masterfrom
docs/test-execution-order

Conversation

@pdenert

@pdenert pdenert commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

Summary

Closes #2137 and #405.

  • Test execution order (Add info about test execution order to the docs #2137): new page under Documentation → Other explaining the deterministic order Patrol runs tests in — files are discovered recursively and sorted alphabetically by full path (verified in TestFinder.findAllTests, which sorts with a.path.compareTo(b.path)), and tests within a file run in declaration order like package:flutter_test. Also covers selecting a single file and --exclude.
  • $.tester (tap() throwing WaitUntilVisibleTimeoutException #405): short tip in Tips and tricks showing that PatrolIntegrationTester wraps Flutter's WidgetTester and any WidgetTester API is reachable via $.tester.

Verification

Docs-only change. Validated meta.json and matched the MDX structure (frontmatter, <Info> callout) against existing pages in the same section. Registered the new page in documentation/other/meta.json.

🤖 Generated with Claude Code

Add a 'Test execution order' page explaining that test files run in
alphabetical path order and tests within a file run in declaration
order, and add a tip showing how to reach the underlying WidgetTester
via $.tester.

Closes #2137
Closes #405

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 23, 2026 16:46
@github-actions github-actions Bot added the docs Concerns doc comments, READMEs, or docs label Jul 23, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Documents Patrol test ordering and access to the underlying Flutter tester.

Changes:

  • Adds test execution-order guidance.
  • Documents $.tester access.
  • Registers the new documentation page.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
tips-and-tricks.mdx Adds $.tester guidance.
test-execution-order.mdx Documents test ordering and selection.
meta.json Registers the new page.
Comments suppressed due to low confidence (1)

docs/documentation/other/test-execution-order.mdx:78

  • This guarantee is false for web runs: patrol test --web-workers N explicitly configures Playwright with multiple workers, and the CLI treats values greater than 1 as a sharded run (packages/patrol_cli/lib/src/web/web_test_backend.dart:463-469). In that mode tests execute concurrently and cannot rely on the file/declaration order described here. Please scope the sequential guarantee to native platforms (with Xcode parallel execution disabled as required by the setup docs) and call out the web-workers exception.
  Patrol does not run tests in parallel or in a randomized order. The order
  described above is deterministic, so tests that depend on running after one
  another (though we recommend keeping tests independent) will behave
  consistently across machines and CI.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +11 to +12
Patrol finds every file ending with `_test.dart` (or your custom
[`test_file_suffix`](/documentation/write-your-first-test)) under the test
@github-actions

Copy link
Copy Markdown

🚀 Preview Deployment Ready!

Preview URL: https://pr-3192-patrol-docs.vercel.app


Latest Deployment

  • Commit: b3b72e9
  • Branch: docs/test-execution-order
  • Triggered by: @pdenert
  • Deployed at: 2026-07-23 16:50:43 UTC

This preview URL is stable and will be updated with each new commit to this PR.


### Order within a single test file

Within a file, tests run **in the order they are declared** — top to bottom —

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not true for iOS look: #2944 and #3020

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Concerns doc comments, READMEs, or docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add info about test execution order to the docs

4 participants