[Modify Feature]: Have pipelines accept non-zero exit codes below 1000(?) as passing instead of failing. #157749
Labels
Area-Validation-Pipeline
Issues related to the manifest validation pipeline.
Issue-Feature
Complex enough to require an in depth planning process and actual budgeted, scheduled work.
Description of the new feature/enhancement
During validation, the pipelines run every executable added to the filesystem during the install process, then close each and record the exit code. If none give a zero exit code, I believe, then the PR gets the
Validation-Executable-Error
label. Then a moderator is supposed to install the package into a sandbox or VM and verify.It's something of a known issue with our pipelines that they always error with normal CLI application design. This is because the pipeline runs the CLI application without any arguments, and expects a zero exit code. But most CLI applications are designed to return a non-zero exit code when run without arguments. Frequently, this number is single-digit, so 10 could be a better value than 1000. But I've seen applications with exit codes as high as 127 that ran without issue in a VM.
Proposed technical implementation details (optional)
If a lower value than 1000 would be better, that would be great. I might not have thought through all edge cases, and I'm not an expert on exit codes, so please be gentle.
I've generally been approving PRs with exit codes below 1000 without verifying, and haven't heard of any issues from users. So I would like to have this added into the pipelines, to facilitate better CLI application acceptance into the repo.
This might interoperate well with the suggestion for an
ApplicationType
ofGUI
,CLI
, etc, where these policies could be applied solely to the CLI applications.The text was updated successfully, but these errors were encountered: