Automate PR and Release Tasks

[tmarkley]

Right now the process for labeling and backporting PRs to ensure changes make it into a particular release is a mostly manual process. Backport PRs are created automatically (if there are no conflicts) but they are still manually reviewed and merged. Generating the release notes is also done manually.

I think we need to discuss opportunities for automating the following:

• Enable release notes drafter to create the release notes as changes are merged to particular branches
  • Attempted in [CI] Add release notes drafter #1504 and https://github.com/tmarkley/OpenSearch-Dashboards/pull/4/files
  • Logistical note: changes that go into a patch release should get added to the notes for the next minor release if one is scheduled for that major version. For example, if a bug fix goes into main and is targeted for a v2.0.1 patch release before v2.1.0 has been released, the backport to 2.x targets v.2.0.1 and v2.1.0.
• Every PR should have at least one version label (e.g. v2.1.0) as well as one category label (e.g. bug, new feature, breaking, security, docs, etc.)
  • All categories can be found in the v2.0.0 release notes: https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/release-notes/opensearch-dashboards.release-notes-2.0.0.md
    • breaking, security, enhancement/new feature, bug, infrastructure, docs, maintenance, refactor, tests
  • This can be based on other labels. For example, if the PR is labeled with bug label, the PR should target the next patch release AND the next minor release. If the PR is labeled with new feature, or enhancement and there isn't a breaking label, it should target the next minor release. If the PR is labeled with breaking, it should target the next major version release.
  • The PR version label(s) must always match the version labels on the issue it is resolving.
  • If integrated with release notes drafter, minor non-user facing changes (e.g. modifying a GitHub action) can use a label such as skip release notes
  • Can be done with PR Labeler or Probot Auto Labeler: https://github.com/release-drafter/release-drafter#categorize-pull-requests
• Based on the version(s) a PR targets, the backport x.x label(s) should be automatically added.
• Backport PRs should have a backport label (to make searching and filtering easier) as well as the corresponding version label(s) from the original PR and issue.
• Once a backport PR passes all GitHub CI workflow checks (i.e. build and verify, functional tests, bwc tests, and release artifacts), it should be automatically merged.
• For PRs that only change documentation (e.g. modifying the README, release notes, etc.), CI checks should be skipped. This could be done with a skip ci label.

[joshuarrrr]

Enable release notes drafter

One of the things I've wondered about when making my first few commits is whether some more standardized commit formatting (and enforcement with git hooks) might make the release notes drafting easier, such as specifying the category of change with consistent tags ([Bug], [UX], [Security]), etc. While I'm hesitant to add any more friction to the contribution process, it seems like something that might also help make the semver/backporting decisions more automated.

Once a backport PR passes all GitHub CI workflow checks (i.e. build and verify, functional tests, bwc tests, and release artifacts), it should be automatically merged.

💯 🚀

[tmarkley]

Enable release notes drafter

One of the things I've wondered about when making my first few commits is whether some more standardized commit formatting (and enforcement) might make the release notes drafting easier, such as specifying the category of change with consistent tags ([Bug], [UX], [Security]), etc. While I'm hesitant to add any more friction to the contribution process, it seems like something that might also help make the semver/backporting decisions more automated.

Yes great point Josh! I do think that would help with the category automation if we grab the category from the commit. This could also be done by using any categories from the issue that the PR is resolving, but having a standardized format for commits also helps with readability and in some cases discoverability/searchability too.

[ananzh]

Every PR should have at least one version label (e.g. v2.1.0) as well as one category label (e.g. bug, new feature, breaking, security, docs, etc.)

Regarding this, I think we could create another label need version for PRs that are not decided or missed the target version. Use cases could be 1) uncertainty on the version 2) missing target version deadline due to PR reviews or other reasons 3) changing target version. When we launch a new version, we could review all the PRs with both the label of the target version and the label of need version.