Explain that workflows that submit sarif probably shouldn't fail

[jsoref]

Code of Conduct

• I have read and agree to the GitHub Docs project's Code of Conduct

What article on docs.github.com is affected?

https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning

What part(s) of the article would you like to see updated?

Uploading code scanning data to GitHub

Says:

GitHub can display code analysis data generated externally by a third-party tool. You can upload code analysis data with the upload-sarif action. For more information, see Uploading a SARIF file to GitHub.

It doesn't say anything about exit codes for such workflows.

(It doesn't link to the upload-sarif action, which may be for the best as using that will delay workflows by 6 seconds. -- The action is intentionally not listed in the GitHub Marketplace unlike, e.g. checkout.)

Additional information

Normally if you want to prevent a pull request from being merged, you'd have your workflow "fail" triggering an ❌.

But, if you do that for a workflow that submits sarifs (at least using some of the apis, especially the github/codeql-action/upload-sarif), then you'll get:

And the status link goes to:

[Sharra-writes]

@jsoref Thanks for opening this issue! I'll get it triaged for review. Since you've opened a handful of issues this time, I'm going to see if any of them seem to be related enough to group together when I ask for an SME review, since that might help expedite things. If you have thoughts on how closely related they are, I would very much welcome the input.

[github-actions]

Thanks for opening an issue! We've triaged this issue for technical review by a subject matter expert 👀

[jsoref]

So, these are related in that they're all sarif endpoints, but they're also quite different:

• Only the top 5,000 results will be included, prioritized by severity. does not describe actual deployed behavior #38085 this appears to be a bug in the implementation, so I'd expect a SME to talk to engineering and change the internal w/o needing a doc change. Note that for ::error::/::warning::/::notice:: the limits are at least sort of per category (I could find the doc references, although from memory the behavior is also quite odd). It's a tension between two approaches.
• Explain that workflows that submit sarif probably shouldn't fail #38062 this is a design thing -- the SME would have to know how this is supposed to work -- I'm really grasping at straws
• [REST] Document /code-scanning/analysis #38037 this would require an SME to decide to provide the information required to write a doc (I could provide something based on my implementation, but that isn't a great idea).

[Sharra-writes]

I appreciate the insight! I'll get to work on those SME review requests.