Sample Release Checklist to improve Release Automation

[smlambert]

Creating this Work in Progress checklist based on the release doc to capture what main activities happen during a release, and to annotate which activities can and should be automated in some fashion. This should serve to streamline release activities and help release champions complete all the required tasks.

One Week Prior to Release:

• Disable nightly testing to free up resources and ensure no competing jobs during release week
• Run a trial release pipeline to ensure less surprises on release day

Release Week Checklist:

• Add website banner (automate* via github workflow - champion creates a website issue (likely based on a template), on creation of a website issue with particular subject and contents (enforced by a template), a workflow is triggered which creates a PR for the website, similar to this example PR)
• Launch build pipelines for each version being released (as per release doc) once release tags are available via launch page in Jenkins. Typically been separating hotspot and openj9 pipelines in release week. Capture pertinent info in a table per implementation.
• Summarize test results. Find each launched build pipeline in TRSS to view a summary of test results. Create a summary of test failures per pipeline to add to the Promote release issue in the TSC repo which can be revamped to include a (shorter version of this checklist and ideally contains a succinct table as per example below, you can use trss-parser. (automate* via github workflow or Jenkins job trigger to add a comment/table to TSC release issue)
• Triage each build and test failure (using Triage guidelines) and determine blocking or non-blocking.
• Fix blocking failures if they exist.
• Get TSC 'ready to publish' approval, once no blocking failures exist.
• Publish the release (run the restricted access release tool job on Jenkins)
• Verify binaries published successfully to github releases repo and website (automate*, this could also be an automated test)
• Update support page (automate* github workflow to create a PR to update support.handlebars)
• Update release notes (automate* - github workflow to create update for release notes page)
• Run homebrew-cask_updater (via Jenkins homebrew-cask_updater job link) once binaries published successfully (this can be automated / triggered by a test for published artifacts)
• Publicize the release via Slack #release channel and Twitter (can be partially automated)

Hotspot

Job name / ID	scmReference/tag	Other Args if not defaults	Summarized Results	Published
jdk8 Pipeline ID
jdk11 Pipeline ID
jdk15 Pipeline ID

OpenJ9

Job name / ID	scmReference/tag	Other Args if not defaults	Summarized Results	Published
jdk8 Pipeline ID
jdk11 Pipeline ID
jdk15 Pipeline ID

[smlambert]

This issue relates to #158 especially on the section of fully defining/documenting input arguments to build and publish steps in the release workflow and as part of realizing a fully verifiable build process.

[smlambert]

Still need to understand and know what happens for the docker jobs and add that to the checklist.

[tellison]

Consider adding in a task, one week prior to release, to

• Update list of CA certificates if required.

[smlambert]

re: #178 (comment) this is not in the release doc (which is what I am basing the checklist off of). Should add it there and have the checklist reflect it.