Emit a specific exit code when no tests match inputs to swift test.
#536
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This PR modifies the entry point function used by Swift Package Manager so that when no tests match the inputs (or, in the case of `swift test list`, when there are no Swift Testing tests in the test target), Swift Package Manager can detect it and respond appropriately. Because exit codes exist in a flat namespace, and because on POSIX-y systems only the low 8 bits of an exit code are reliable, our options here are limited. I have selected the `EX_UNAVAILABLE` code from sysexits.h to represent this state. On Windows, all 32 bits of the exit code are propagated, but there is no `EX_UNAVAILABLE` value, so I've used the Win32 `ERROR_NOT_FOUND` error code instead. My assumption is that neither of these codes is likely to _actually_ be passed to `exit()` by test code (because why would they be?) A separate PR is required in Swift Package Manager to correctly handle this exit code and emit the `noMatchingTests` diagnostic that's currently emitted only for XCTest.
grynspan
added
enhancement
grynspan
requested review from
stmontgomery,
dennisweissmann,
briancroom,
SeanROlszewski and
suzannaratcliff
as code owners
|
@swift-ci please test |
|
@swift-ci please test |
|
@swift-ci please test |
|
@swift-ci please test |
grynspan
added a commit
to swiftlang/swift-package-manager
that referenced
this pull request
Jul 14, 2024
This PR detects when a Swift Testing run returns `EXIT_NO_TESTS_FOUND` and treats it as a successful test run. This change handles the change from swiftlang/swift-testing#536. Separately, the refactor in #7766 stops Swift Package Manager from reporting `"No matching test cases were run"` when XCTest has no tests matching passed `--filter` arguments. A third PR after these two have been merged will restore that functionality by collating results from both XCTest and Swift Testing. This change partially resolves rdar://131704587.
|
@swift-ci please test |
dennisweissmann
approved these changes
Jul 15, 2024
grynspan
added a commit
to swiftlang/swift-package-manager
that referenced
this pull request
Jul 15, 2024
This PR detects when a Swift Testing run returns `EXIT_NO_TESTS_FOUND` and treats it as a successful test run. This change handles the change from swiftlang/swift-testing#536. Separately, the refactor in #7766 stops Swift Package Manager from reporting `"No matching test cases were run"` when XCTest has no tests matching passed `--filter` arguments. A third PR after these two have been merged will restore that functionality by collating results from both XCTest and Swift Testing. > [!NOTE] > Test coverage can be added after Swift Testing is in the Swift toolchain. This change partially resolves rdar://131704587.
grynspan
added a commit
to swiftlang/swift-package-manager
that referenced
this pull request
Jul 16, 2024
This PR detects when a Swift Testing run returns `EXIT_NO_TESTS_FOUND` and treats it as a successful test run. This change handles the change from swiftlang/swift-testing#536. Separately, the refactor in #7766 stops Swift Package Manager from reporting `"No matching test cases were run"` when XCTest has no tests matching passed `--filter` arguments. A third PR after these two have been merged will restore that functionality by collating results from both XCTest and Swift Testing. > [!NOTE] > Test coverage can be added after Swift Testing is in the Swift toolchain. This change partially resolves rdar://131704587.
grynspan
added a commit
to swiftlang/swift-package-manager
that referenced
this pull request
Jul 18, 2024
This PR detects when a Swift Testing run returns `EXIT_NO_TESTS_FOUND` and treats it as a successful test run. This change handles the change from swiftlang/swift-testing#536. Separately, the refactor in #7766 stops Swift Package Manager from reporting `"No matching test cases were run"` when XCTest has no tests matching passed `--filter` arguments. A third PR after these two have been merged will restore that functionality by collating results from both XCTest and Swift Testing. > [!NOTE] > Test coverage can be added after Swift Testing is in the Swift toolchain. This change partially resolves rdar://131704587.
2 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR modifies the entry point function used by Swift Package Manager so that when no tests match the inputs (or, in the case of
swift test list, when there are no Swift Testing tests in the test target), Swift Package Manager can detect it and respond appropriately.Because exit codes exist in a flat namespace, and because on POSIX-y systems only the low 8 bits of an exit code are reliable, our options here are limited. I have selected the
EX_UNAVAILABLEcode from sysexits.h to represent this state. On Windows, all 32 bits of the exit code are propagated, but there is noEX_UNAVAILABLEvalue, so I've used the Win32ERROR_NOT_FOUNDerror code instead. My assumption is that neither of these codes is likely to actually be passed toexit()by test code (because why would they be?)A separate PR is required in Swift Package Manager to correctly handle this exit code and emit the
noMatchingTestsdiagnostic that's currently emitted only for XCTest.Resolves rdar://131704539.
Checklist: