Enable a controlled switchover between CodeQL releases

[henrymercer]

This PR adds functionality that we will use to enable a controlled switchover between CodeQL releases. Specifically, we would like to ensure that each repository running CodeQL transitions from the previous CodeQL release to the new CodeQL release once. We want to avoid repositories flip-flopping between CodeQL releases during the rollout of a new CodeQL release, specifically during the Actions runner image update process, as this creates alert churn.

To do this, we decide the default version of CodeQL on Dotcom using feature flags rather than by using what's in the toolcache. Specifically, we look at all the default_codeql_version_x_enabled feature flags and pick the latest enabled version.

One niggle with this approach is that if the CLI version we need isn't in the toolcache, there isn't yet a nice way to find the CodeQL Bundle release corresponding to that CLI version. In the medium term, we'll consider tagging each CodeQL bundle release with a CLI version number, for instance codeql-bundle-v2.12.0. In the meantime, we add a special asset to the release cli-version-2.12.0.txt specifying the version number. The contents of this asset don't matter, since we don't want to have to download anything.

GHES behaviour is unchanged: we account for the new toolcache format, but we continue to use the CodeQL bundle release specified within the Action (in defaults.json) by default. We also continue to allow CodeQL bundles that have been baked into Actions runner images to override the version in defaults.json.

Commit-by-commit-review recommended.

TODO before merging

• Add a changelog note for the impact on the layout of CodeQL within the Actions runner images.
• Adapt the functionality that enables using the toolcache for manually specified bundle URLs to the new runner image structure. This will require a search through toolcache versions or a bundle tag name to CodeQL CLI version lookup.
• Add some more unit tests for setting up CodeQL. Also require an owner and a repo in the GitHub Releases API mocking as a regression test.

Merge / deployment checklist

• Confirm this change is backwards compatible with existing workflows. — see changelog notes
• Confirm the readme has been updated if necessary.
• Confirm the changelog has been updated if necessary.

[henrymercer]

I've added some things we need to do before merging this to the PR description, but I think this is ready for an initial look.

[angelapwen]

Looked through commit-by-commit and everything makes sense to me. Just a few comments for now, will take another 👀 tomorrow!

[aeisenberg]

One niggle with this approach is that if the CLI version we need isn't in the toolcache, there isn't yet a nice way to find the CodeQL Bundle release corresponding to that CLI version. In the medium term, we'll consider tagging each CodeQL bundle release with a CLI version number, for instance codeql-bundle-v2.12.0. In the meantime, we add a special asset to the release cli-version-2.12.0.txt specifying the version number. The contents of this asset don't matter, since we don't want to have to download anything.

Could we add a json file in this repo that has mappings from bundle release number to version number or URL? This would be helpful in general for our users since there's no easy way for them to figure out this mapping right now without downloading and running codeql version.

This could go in defaults.json, which we're already updating for each release.

[angelapwen]

👍 on new additions. CI failure is due to known Swift autobuild hang and I've rerun.

[henrymercer]

Could we add a json file in this repo that has mappings from bundle release number to version number or URL? This would be helpful in general for our users since there's no easy way for them to figure out this mapping right now without downloading and running codeql version.

This could go in defaults.json, which we're already updating for each release.

I have a slight preference for keeping the GitHub Releases the source of truth:

• This aligns better with our medium term goal of changing the version numbering system of bundle releases to include the CLI version number. Once we've done that a mapping in defaults.json would be redundant.
• We can automatically add the CLI version marker file to the bundle release as part of the publishing process, saving a bit of manual work.

[aeisenberg]

Given the complexity of this change and the fact that the actions changelog is not very visible to users, are you considering adding this to a changelog post?

[henrymercer]

Given the complexity of this change and the fact that the actions changelog is not very visible to users, are you considering adding this to a changelog post?

I think a changelog post would be a good idea. Will discuss with Alona once we've agreed on a good Action changelog note.

[angelapwen]

✅ from me on the code changes, but I believe there is still ongoing discussion about the inclusion of the section for advanced users in the changelog note.

[angelapwen]

Approving but looks like merge conflicts need to be resolved 😄

[henrymercer]

Hmm since we already have merge conflicts here, it might make sense to get #1492 in first and then rebase on that, as I expect the TypeScript update will probably create some more conflicts.

[henrymercer]

TypeScript PR is failing on Windows, so let's do this PR first instead.

[angelapwen]

Looks like the PR checks are failing with parsing SARIF on Windows but it should be unrelated to this PR 🤔