Skip to content

Skip tests during PGO benchmark profiling#231

Merged
jeremy merged 2 commits intomainfrom
benchfail
Mar 10, 2026
Merged

Skip tests during PGO benchmark profiling#231
jeremy merged 2 commits intomainfrom
benchfail

Conversation

@jeremy
Copy link
Copy Markdown
Member

@jeremy jeremy commented Mar 10, 2026

Summary

  • Add -run='^$' to go test invocations in the release workflow and scripts/collect-profile.sh so only benchmarks execute during PGO profile collection
  • Fixes failures in packages like cli and commands that have no benchmarks but whose tests fail in CI (no TTY, missing fixtures)

Test plan

  • Verified go test -bench=. -benchtime=1x -run='^$' ./internal/commands/ passes (no tests run)
  • Verified go test -bench=. -benchtime=1x -run='^$' ./internal/cli/ passes (no tests run)
  • Verified go test -bench=. -benchtime=1x -run='^$' ./internal/dateparse/ runs benchmarks correctly

Summary by cubic

Run only benchmarks during PGO profiling and benchmark CI by adding -run='^$' to go test in the release workflow, the test.yml benchmarks job, and scripts/collect-profile.sh. This avoids CI failures in packages like cli and commands that have no benchmarks but whose tests need a TTY/fixtures.

Written for commit 4a007c3. Summary will update on new commits.

Add -run='^$' to go test invocations in the release workflow and
collect-profile.sh so only benchmarks execute. Without this, packages
like cli and commands fail their tests in CI (no TTY, missing fixtures)
even though they have no benchmarks, causing the PGO profile step to
report failures.
@jeremy jeremy requested a review from a team as a code owner March 10, 2026 10:40
Copilot AI review requested due to automatic review settings March 10, 2026 10:40
@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 10, 2026

Sensitive Change Detection (shadow mode)

This PR modifies control-plane files:

  • .github/workflows/release.yml
  • .github/workflows/test.yml

Shadow mode — this check is informational only. When activated, changes to these paths will require approval from a maintainer.

@github-actions github-actions bot added ci CI/CD workflows bug Something isn't working labels Mar 10, 2026
Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes failures during PGO (Profile-Guided Optimization) benchmark profile collection caused by packages (like cli and commands) whose regular tests fail in CI environments without a TTY or required fixtures. The fix adds -run='^$' to go test invocations, which is the standard Go idiom for skipping all test functions while still executing benchmarks.

Changes:

  • Add -run='^$' to the go test invocation in scripts/collect-profile.sh to skip test execution during local PGO profile collection
  • Add -run='^$' to the go test invocation in .github/workflows/release.yml's "Collect PGO profile" step for the same reason

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
scripts/collect-profile.sh Adds -run='^$' so only benchmarks run when collecting PGO profiles locally
.github/workflows/release.yml Adds -run='^$' so only benchmarks run in the release pipeline's PGO profile collection step

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/release.yml
Same fix as release.yml — add -run='^$' so only benchmarks execute,
not tests that may fail without a TTY.
@jeremy jeremy merged commit a1c35f4 into main Mar 10, 2026
25 checks passed
@jeremy jeremy deleted the benchfail branch March 10, 2026 10:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ci CI/CD workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants