Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable "fail fast" feature in "Test Go" workflow #2014

Merged
merged 1 commit into from
Dec 19, 2022
Merged

Disable "fail fast" feature in "Test Go" workflow #2014

merged 1 commit into from
Dec 19, 2022

Conversation

per1234
Copy link
Contributor

@per1234 per1234 commented Dec 16, 2022

Please check if the PR fulfills these requirements

See how to contribute

  • The PR has no duplicates (please search among the Pull Requests
    before creating one)
  • The PR follows
    our contributing guidelines
  • [N/A] Tests for the changes have been added (for bug fixes / features)
  • [N/A] Docs have been added / updated (for bug fixes / features)
  • [N/A] UPGRADING.md has been updated with a migration guide (for breaking changes)

What kind of change does this PR introduce?

Infrastructure enhancement

What is the current behavior?

For efficiency, the "Test Go" GitHub Actions workflow splits the test suite into multiple jobs which run in parallel. This is done via a "job matrix".

By default GitHub Actions uses a "fail fast" behavior for job matrixes, where all in progress or pending jobs of the matrix are immediately canceled upon failure of any job.

The benefit of this "fail fast" behavior is efficiency in the case where a failure is universal and those other jobs were certain to eventually fail. However, in other cases a failure is either specific to the configuration of a single job (e.g., a Windows-specific bug), or a spurious result of a transient network outage. In the latter case, canceling the other jobs is very harmful. Running the jobs for the other operating systems would highlight the nature of an OS-specific failure for the contributor. Canceling other jobs due to a transient failure means all those jobs must be reran instead of only the specific job that suffered the failure.

What is the new behavior?

The workflow is hereby configured to disable the "fail fast" behavior. All the matrix jobs will now run even if one of them has failed.

Does this PR introduce a breaking change, and is titled accordingly?

No breaking change.

Other information

The workflow was previously configured for this behavior by #1914


Reference:

https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstrategyfail-fast

For efficiency, the "Test Go" GitHub Actions workflow splits the test suite into multiple jobs which run in parallel.
This is done via a "job matrix".

By default GitHub Actions uses a "fail fast" behavior for job matrixes, where all in progress or pending jobs of the
matrix are immediately canceled upon failure of any job.

The benefit of this "fail fast" behavior is efficiency in the case where a failure is universal and those other jobs
were certain to eventually fail. However, in other cases a failure is either specific to the configuration of a single
job (e.g., a Windows-specific bug), or a spurious result of a transient network outage. In the latter case, canceling
the other jobs is very harmful. Running the jobs for the other operating systems would highlight the nature of an
OS-specific failure for the contributor. Canceling other jobs due to a transient failure means all those jobs must be
reran instead of only the specific job that suffered the failure.

The workflow is hereby configured to disable the "fail fast" behavior. All the matrix jobs will now run even if one of
them has failed.
@per1234 per1234 added type: enhancement Proposed improvement topic: infrastructure Related to project infrastructure labels Dec 16, 2022
@per1234 per1234 self-assigned this Dec 16, 2022
Copy link
Contributor

@MatteoPologruto MatteoPologruto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Per!

@per1234 per1234 merged commit de90579 into arduino:master Dec 19, 2022
@per1234 per1234 deleted the disable-fail-fast branch December 19, 2022 08:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: infrastructure Related to project infrastructure type: enhancement Proposed improvement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants