Add the release runbook to RELEASING.md

CHANGELOG.md

@@ -73,6 +73,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 
 ### 📝 Documentation
 
+- Add the release runbook to RELEASING.md ([#2533](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2533))
 - [MD] Add design documents of multiple data source feature [#2538](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2538)
 - [MD] Tweak multiple data source design doc [#2724](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2724)
 

RELEASING.md

@@ -1,3 +1,76 @@
 ## Releasing
 
-This project follows [OpenSearch project branching, labelling, and releasing](https://github.com/opensearch-project/.github/blob/main/RELEASING.md).
+This project follows [OpenSearch project branching, labelling, and releasing](https://github.com/opensearch-project/.github/blob/main/RELEASING.md).
+
+## Runbook
+
+### Overview
+
+The OpenSearch project releases versioned distributions of OpenSearch, OpenSearch Dashboards, and the OpenSearch plugins. This runbook details the steps involved in performing major, minor, and patch version releases for the OpenSearch Dashboards project; these steps need to be completed by the release managers (RM) assigned to each release. The RM is also responsible for updating the release status on the release tracking issues maintained on GitHub ([example](https://github.com/opensearch-project/OpenSearch-Dashboards/issues/2230)).
+
+\*Important Dates: https://opensearch.org/releases.html
+
+### Release Phase 1 - Preparation
+
+For major and minor releases, the OpenSearch build repository [maintainers](https://github.com/opensearch-project/opensearch-build/blob/main/MAINTAINERS.md) will create a release issue in the OpenSearch Dashboards repository ([example](https://github.com/opensearch-project/OpenSearch-Dashboards/issues/2230)) which links to the overall issue in the OpenSearch build repository ([example](https://github.com/opensearch-project/opensearch-build/issues/2447)). For patch releases, they will only create the overall issue in the OpenSearch build repository ([example](https://github.com/opensearch-project/opensearch-build/issues/2650)).
+
+The OpenSearch Dashboards release issue will be assigned to the RM who should prepare for the release by reviewing all listed tasks. They should also compare the current release issue to the issue of the previous release version to ensure that all new processes have been captured.
+
+The RM should review the [public roadmap](https://github.com/orgs/opensearch-project/projects/1) and confirm the release scope with other OpenSearch Dashboards [maintainers](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/MAINTAINERS.md) as well as the feature owners. Since release labels are intended to highlight the features and fixes meant for the upcoming release, the RM should verify that all issues and pull requests are labelled accordingly. For example, if current release version was v2.3.0, all features not ready for the release should be labeled as v2.4.0 or later by discussing with the feature owners. The RM should also check all PRs for the current release version to confirm that they are merged into the `main` branch and their backported PRs are merged with all CI passing.
+
+#### How to validate that merged commits have been properly backported?
+
+1. For any PRs merged to main, make sure it has backport labels ([example - Add v2.3.0 release notes](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2318))
+2. Backport PR is generated automatically by [opensearch-trigger-bot](https://github.com/apps/opensearch-trigger-bot)
+   ([example - [Backport 2.3] Add v2.3.0 release notes](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2333))
+3. If the backport PR is not generated by opensearch-trigger-bot after 24 hours, create it manually following this [instruction](https://github.com/opensearch-project/.github/blob/main/RELEASING.md#backporting)
+4. Ensure that all CI passed and it has two approvals. Then merge the PR.
+
+#### Prepare BWC data and update BWC versions
+
+Backward Compatibility Tests (BWC) are cypress tests that identify any obvious compatibility bugs with previous versions. The RM should generate test data and test locally following instructions [here](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/TESTING.md#backwards-compatibility-tests) and cut PR to include both generated data and version upgrade for automated build. (See example [PR](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2393/files))
+
+#### Cut release branch for major / minor release
+
+For major / minor release, the RM should cut the release branch from the parent branch, [following OpenSearch project branching](https://github.com/opensearch-project/.github/blob/main/RELEASING.md#opensearch-branching)
+
+### Release Phase 2 - Pre-Release
+
+The release process for OpenSearch is centralized. Jenkins workflows are in place to regularly find differences in the OpenSearch and OpenSearch Dashboards components and create new snapshots for those that have been updated. The RM should update the release branch version in the distribution manifest (see example [PR](https://github.com/opensearch-project/opensearch-build/pull/2586/files)) and increment the parent branch version (see example [PR](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2295/files)).
+
+#### Write release notes
+
+OpenSearch Dashboards maintains a [CHANGELOG.md](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/CHANGELOG.md) and verifies it as part of the PR checklist step. For the time being, the RM should create release notes PR with the label `doc`, referring to the `CHANGELOG.md` (see example [PR](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2318))
+
+### Release Phase 3 - Release testing
+
+#### Verify integration and BWC test results
+
+The automated integration test and BWC test are executed concurrently with the release artifacts build. The RM should examine the test results and assist in triaging the broken test case.
+
+Example build:
+
+x64: https://build.ci.opensearch.org/job/integ-test-opensearch-dashboards/995/
+
+arm64: https://build.ci.opensearch.org/job/integ-test-opensearch-dashboards/996/
+
+Example test results:
+https://opensearch-project.github.io/opensearch-dashboards-functional-test/site/?version=2.3.0&build_number=4104&test_number=996
+
+Note: change `arch` to match the operating system CPU architecture for which the build is being produced.
+
+#### Sanity test with tarball and docker image
+
+Once the release candidate artifacts are built, the RM should configure the OpenSearch cluster with OpenSearch Dashboards according to the [instructions in the OpenSearch build repo](https://github.com/opensearch-project/opensearch-build/issues/2447#issuecomment-1241406594) and produce sanity tests to identify broken functionalities caused by new features / code changes. If you find any, please file bug reports and assist in triaging the bugfix.
+
+### Release Phase 4 - Release Announcement
+
+Release artifacts and announcements will be available on https://opensearch.org/releases.html. Any website documentation changes will require a PR on the [OpenSearch documentation-website repo.](https://github.com/opensearch-project/documentation-website)
+
+### Release Phase 5 - Post-Release
+
+After a release is announced, OpenSearch build repository maintainers will trigger a job that creates a tag in each repository based on the commit hash and branch that was included in the release; the release tag could take a few hours to show up on GitHub.
+
+The RM should update the [release page](https://github.com/opensearch-project/OpenSearch-Dashboards/releases/) with the latest download URL and release notes after the release tag is created.
+
+If needed, the RM could conduct a retrospective review of the release, and publish their findings regarding any missed steps and process improvements.