broadinstitute · rexwangcc · Nov 3, 2020 · Nov 1, 2020 · Nov 1, 2020 · Nov 1, 2020
diff --git a/README.md b/README.md
diff --git a/docs/md/dev-process.md b/docs/md/dev-process.md
@@ -68,13 +68,7 @@ make pull requests,
 ask for reviews
 and merge back to `master` on Github.
 
-Currently we always deploy the latest master
-to the development environment after merge,
-but in the future we might need
-to cut off releases on master
-and deploy the released versions
-to the server only.
-It's not decided yet.
+For the release process, please refer to the [release guide](/docs/md/dev-release.md)
 
 1. Clone the repo
     ```

diff --git a/docs/md/dev-release.md b/docs/md/dev-release.md
@@ -0,0 +1,22 @@
+# Release Process
+
+Currently developers follow a simple process
+in order to release a new version of WFL:
+
+1. Developers create a standalone "release PR",
+which should be exclusively used by releasing WFL.
+2. In the PR, developers need to semanticaly bump
+the version string in the `/version` file at the root
+of the repo.
+3. In the PR, developers need to run `./ops/cli.py release`
+to generate the changelog for this release. (`-d` flag can
+be used to do a dry run without writing to the `CHANGELOG.md`
+file)
+4. Once the release PR is reviewed by merged to `master`
+branch, developers can go to Github Action
+[Release Latest Version](https://github.com/broadinstitute/wfl/actions?query=workflow%3A%22Release+Latest+Version%22)
+and manually tirgger a run with the `Run workflow` button to build and push the tagged docker images of WFL to
+[DockerHub](https://hub.docker.com/repository/docker/broadinstitute/workflow-launcher-api).
+
+!!! tip
+    Note it can take up to 30 minutes for the Github Action to finish the build and push.
diff --git a/docs/md/README.md → docs/md/index.md b/docs/md/README.md → docs/md/index.md
@@ -1,13 +1,12 @@
 # Welcome to WorkFlow Launcher
 
-
-[To the Build Board](https://internal.broadinstitute.org/~jwarren/green-hornet-status.html)
+[![Build Board](https://img.shields.io/badge/-The%20Build%20Boad%20(VPN)-blue)](https://internal.broadinstitute.org/~jwarren/green-hornet-status.html)
+![Docsite Build](https://github.com/broadinstitute/wfl/workflows/Publish%20docs%20via%20GitHub%20Pages/badge.svg?branch=master)
+![Build on Master](https://github.com/broadinstitute/wfl/workflows/Tests%20on%20Pull%20Requests%20and%20Master/badge.svg?branch=master)
+[![GitHub tag (latest by date)](https://img.shields.io/github/v/tag/broadinstitute/wfl?label=Latest%20Release)](https://github.com/broadinstitute/wfl/blob/master/CHANGELOG.md)
 
 ## Overview
 
-![Docsite Build](https://github.com/broadinstitute/wfl/workflows/Publish%20docs%20via%20GitHub%20Pages/badge.svg?branch=master)
-![ Build on Master ](https://github.com/broadinstitute/wfl/workflows/Tests%20on%20Pull%20Requests%20and%20Master/badge.svg?branch=master)
-
 [WorkFlow Launcher (WFL)](https://github.com/broadinstitute/wfl.git)
 is a workload manager.
 
@@ -30,8 +29,8 @@ on Kubernetes clusters.
 ???+ tip
     This is the Quickstart section, which should cover the most frequent
     uses cases that interact with WFL. For more detailed information, please
-    check other sections such as the [development guide](/docs/md/dev.md)
-    or [modules design principles](/docs/md/module.md).
+    check other sections such as the [development guide](/docs/md/dev-process.md)
+    or [modules design principles](/docs/md/modules-general.md).
 
 ### Build
 
@@ -155,14 +154,10 @@ before they can be run, namely, they require a `wfl` server running locally:
 ### Deploy
 
 Currently, we mainly deploy WFL to `broad-gotc-dev` and `broad-gotc-prod` projects.
-When it's time to deploy WFL, developers need to bump the version string
-in the `version` file at the root of repo, which could be done either in
-a standalone PR or along with a feature PR. After having done that, the Github Action
-[Release Latest Version](https://github.com/broadinstitute/wfl/actions?query=workflow%3A%22Release+Latest+Version%22)
-will get triggered to build and push the tagged docker images of WFL to
-[DockerHub](https://hub.docker.com/repository/docker/broadinstitute/workflow-launcher-api).
-
-From here, the developers who have broad VPN connected can go to the
+When it's time to deploy WFL, for most of the time developers need to
+release a new version following the steps in [Release Guide](/docs/md/dev-release.md)
+
+After which, the developers who have broad VPN connected can go to the
 [Jenkins Page](https://gotc-jenkins.dsp-techops.broadinstitute.org/job/deploy-wfl/build?delay=0sec)
 to deploy applicable versions of WFL to various available cloud projects.
 
@@ -184,7 +179,7 @@ Learn more about the deployment details in
 
 ## Implementation
 
-For frontend details, check [Frontend Section](/docs/md/frontend.md)
+For frontend details, check [Frontend Section](/docs/md/dev-frontend.md)
 
 ### Top-level files
 

diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml
@@ -19,9 +19,10 @@ extra:
       link: 'https://github.com/broadinstitute/wfl'
 
 nav:
-  - Get Started: README.md
+  - Get Started: index.md
   - Development Guide:
     - Development Process and Tips: dev-process.md
+    - Release Process: dev-release.md
     - Local Development SandBox: dev-sandbox.md
     - Logging: dev-logging.md
     - Deployment: dev-deployment.md

diff --git a/ops/cli.py b/ops/cli.py
@@ -7,6 +7,7 @@
 import argparse
 import json
 import os
+import re
 import shutil
 import subprocess
 from dataclasses import dataclass
@@ -38,6 +39,7 @@ class WflInstanceConfig:
     rendered_values_file: str = None
     vault_token_path: str = None
     wfl_root_folder: str = f"{os.path.dirname(os.path.realpath(__file__))}/.."
+    current_changelog: str = None
 
 
 def check_env_instance_present(config: WflInstanceConfig) -> None:
@@ -279,7 +281,51 @@ def check_git_tag(config: WflInstanceConfig) -> None:
         exit(1)
 
 
+def _markdownify_commit_msg(commit: str) -> str:
+    "Turn a single commit message to markdown style."
+    regex = re.compile("\#[0-9][0-9][0-9]")
+    num_pr = regex.search(commit)[0]
+    marked_commit = regex.sub(f"[\g<0>](https://github.com/broadinstitute/wfl/pull/{num_pr[1:]})", commit)
+    marked_commit = f'- {marked_commit}'
+    return marked_commit
+
+
+def get_git_commits_since_last_tag(config: WflInstanceConfig) -> None:
+    "Read commit messages since last tag, store to config and print."
+    command = 'git log --pretty=format:"%s" $(git describe --tags --abbrev=0 origin/master^)..origin/master'
+    info("=>  Reading commit messages from git log")
+    lines = shell(command).split("\n")
+    info("=>  Markdown-ify log messages")
+    current_changelog = "\n".join([_markdownify_commit_msg(line) for line in lines])
+    config.current_changelog = current_changelog
+    info("=>  Current changelog crafted")
+    info(current_changelog)
+
+
+def write_changelog(config: WflInstanceConfig) -> None:
+    "Append current changelog info to the changelog file at start position."
+    content = f"# Release {config.version}\n" + config.current_changelog
+    changelog_location = f"{config.wfl_root_folder}/CHANGELOG.md"
+
+    info("=>  Loading changelog from file at `./CHANGELOG.md`")
+    shell(f"touch {changelog_location}")
+    with open(changelog_location, "r") as fp:
+        existing = fp.read().strip()
+
+    with open(changelog_location, "w") as fp:
+        fp.write(content)
+        fp.write("\n\n")
+        fp.write(existing)
+    success(f"Changelog is successfully written to {changelog_location}")
+
+
 command_mapping: Dict[str, List[Callable[[WflInstanceConfig], None]]] = {
+    "release": [
+        read_version,
+        get_git_commits_since_last_tag,
+        exit_if_dry_run,
+        write_changelog
+    ],
     "info": [
         check_env_instance_present,
         read_version,

diff --git a/readme.md b/readme.md
@@ -0,0 +1,21 @@
+# WorkFlow Launcher
+
+[![Build Board](https://img.shields.io/badge/-The%20Build%20Boad%20(VPN)-blue)](https://internal.broadinstitute.org/~jwarren/green-hornet-status.html)
+![Docsite Build](https://github.com/broadinstitute/wfl/workflows/Publish%20docs%20via%20GitHub%20Pages/badge.svg?branch=master)
+![Build on Master](https://github.com/broadinstitute/wfl/workflows/Tests%20on%20Pull%20Requests%20and%20Master/badge.svg?branch=master)
+[![GitHub tag (latest by date)](https://img.shields.io/github/v/tag/broadinstitute/wfl?label=Latest%20Release)](https://github.com/broadinstitute/wfl/blob/master/CHANGELOG.md)
+
+
+**For a complete introduction to WorkFlow Launcher, please visit its [documentation website](https://broadinstitute.github.io/wfl)!**
+
+## Overview
+
+[WorkFlow Launcher (WFL)](https://github.com/broadinstitute/wfl.git)
+is a workload manager.
+
+For example, a workload could be a set of Whole Genome samples to be reprocessed in a
+given project/bucket, the workflow is the processing of an individual sample
+in that workload running WGS reprocessing; a workload could also be a queue of
+incoming notifications that describe all of the required inputs to launch Arrays
+scientific pipelines in Cromwell.
+