chore: multiple release branches without merge conflicts (aws#11307)

As we prepare for 2.0, we need to release the CDK concurrently in multiple version lines (1.x and 2.0.0-alpha.x).

In order to avoid merge conflicts of `lerna.json` and `CHANGELOG.md` between the v1 and v2 branches, we extracted the version number from `lerna.json` to `version.vNNN.json` and changelog to `CHANGELOG.vNNN.json` (1.0 is still CHANGELOG.md because it is tracked externally).

A new file called `release.json` has been introduced and includes *static* information about which version line this branch serves. This allows us to avoid merge conflicts caused by
version bumps between release branches.

This change also cleans up some of the scripts related to versioning and bumps. The main bump script is now implemented in `scripts/bump.js` and interacts with `standard-version` as a library instead of through the CLI. To that end, the `.versionrc.json` file was also removed.

See CONTRIBUTING for more details about how this works.


----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*

.gitignore

@@ -46,3 +46,4 @@ yarn-error.log
 # Cloud9
 .c9
 
+/.versionrc.json

CONTRIBUTING.md

@@ -779,24 +779,59 @@ CDK](https://github.com/aws/aws-cdk/issues/3398) we will either remove the
 legacy behavior or flip the logic for all these features and then
 reset the `FEATURE_FLAGS` map for the next cycle.
 
-### Versioning
-
-All `package.json` files in this repo use a stable marker version of `0.0.0`.
-This means that when you declare dependencies, you should always use `0.0.0`.
-This makes it easier for us to bump a new version (the `bump.sh` script will
-just update the central version and create a CHANGELOG entry) and also reduces
-the chance of merge conflicts after a new version is released.
-
-Additional scripts that take part in the versioning mechanism:
-
-- `scripts/get-version.js` can be used to obtain the actual version of the repo.
-  You can use either from JavaScript code by `require('./scripts/get-version')`
-  or from a shell script `node -p "require('./scripts/get-version')"`.
-- `scripts/get-version-marker.js` returns `0.0.0` and used to DRY the version
-  marker.
-- `scripts/align-version.sh` and `scripts/align-version.js` are used to align
-  all package.json files in the repo to the official version. This script is
-  invoked in CI builds and should not be used inside a development environment.
+### Versioning and Release
+
+The `release.json` file at the root of the repo determines which release line
+this branch belongs to.
+
+```js
+{
+  "majorVersion": 1 | 2,
+  "releaseType": "stable" | "alpha" | "rc"
+}
+```
+
+To reduce merge conflicts in automatic merges between version branches, the
+current version number is stored under `version.vNN.json` (where `NN` is
+`majorVersion`) and changelogs are stored under `CHANGELOG.NN.md` (for
+historical reasons, the changelog for 1.x is under `CHANGELOG.md`).  When we
+fork to a new release branch (e.g. `v2-main`), we will update `release.json` in
+this branch to reflect the new version line, and this information will be used
+to determine how releases are cut.
+
+The actual `version` field in all `package.json` files should always be `0.0.0`.
+This means that local development builds will use version `0.0.0` instead of the
+official version from the version file.
+
+#### `./bump.sh`
+
+This script uses [standard-version] to update the version in `version.vNN.json`
+to the next version. By default it will perform a **minor** bump, but `./bump.sh
+patch` can be used to perform a patch release if that's needed.
+
+This script will also update the relevant changelog file.
+
+[standard-version]: https://github.com/conventional-changelog/standard-version
+
+#### `scripts/resolve-version.js`
+
+The script evaluates evaluates the configuration in `release.json` and exports an
+object like this:
+
+```js
+{
+  version: '2.0.0-alpha.1',          // the current version
+  versionFile: 'version.v2.json',    // the version file
+  changelogFile: 'CHANGELOG.v2.md',  // changelog file name
+  prerelease: 'alpha',               // prerelease tag (undefined for stable)
+  marker: '0.0.0'                    // version marker in package.json files
+}
+```
+
+#### scripts/align-version.sh
+
+In official builds, the `scripts/align-version.sh` is used to update all
+`package.json` files based on the version from `version.vNN.json`.
 
 ## Troubleshooting
 

bump.sh

@@ -13,21 +13,4 @@
 #
 # --------------------------------------------------------------------------------------------------
 set -euo pipefail
-version=${1:-minor}
-
-echo "Starting ${version} version bump"
-
-# /bin/bash ./install.sh
-
-# Generate CHANGELOG and create a commit (see .versionrc.json)
-npx standard-version --release-as ${version}
-
-# I am sorry.
-#
-# I've gone diving through the code of `conventional-changelog` to see if there
-# was a way to configure the string and ultimately I decided that a 'sed' was the simpler
-# way to go.
-sed -i.tmp -e 's/BREAKING CHANGES$/BREAKING CHANGES TO EXPERIMENTAL FEATURES/' CHANGELOG.md
-rm CHANGELOG.md.tmp
-git add CHANGELOG.md
-git commit --amend --no-edit
+./scripts/bump.js ${1:-minor}

lerna.json

@@ -8,8 +8,9 @@
     "packages/@aws-cdk-containers/*",
     "packages/@monocdk-experiment/*",
     "packages/@aws-cdk/*/lambda-packages/*",
-    "tools/*"
+    "tools/*",
+    "scripts/script-tests"
   ],
   "rejectCycles": "true",
-  "version": "1.71.0"
+  "version": "0.0.0"
 }

pack.sh

@@ -7,6 +7,11 @@ export PATH=$PWD/node_modules/.bin:$PATH
 export NODE_OPTIONS="--max-old-space-size=4096 ${NODE_OPTIONS:-}"
 root=$PWD
 
+# Get version and changelog file name (these require that .versionrc.json would have been generated)
+version=$(node -p "require('./scripts/resolve-version').version")
+changelog_file=$(node -p "require('./scripts/resolve-version').changelogFile")
+marker=$(node -p "require('./scripts/resolve-version').marker")
+
 PACMAK=${PACMAK:-jsii-pacmak}
 ROSETTA=${ROSETTA:-jsii-rosetta}
 TMPDIR=${TMPDIR:-$(dirname $(mktemp -u))}
@@ -57,15 +62,6 @@ done
 # Remove a JSII aggregate POM that may have snuk past
 rm -rf dist/java/software/amazon/jsii
 
-# Get version
-version="$(node -p "require('./scripts/get-version')")"
-
-# Ensure we don't publish anything beyond 1.x for now
-if [[ ! "${version}" == "1."* ]]; then
-  echo "ERROR: accidentally releasing a major version? Expecting repo version to start with '1.' but got '${version}'"
-  exit 1
-fi
-
 # Get commit from CodePipeline (or git, if we are in CodeBuild)
 # If CODEBUILD_RESOLVED_SOURCE_VERSION is not defined (i.e. local
 # build or CodePipeline build), use the HEAD commit hash).
@@ -83,12 +79,11 @@ cat > ${distdir}/build.json <<HERE
 HERE
 
 # copy CHANGELOG.md to dist/ for github releases
-cp CHANGELOG.md ${distdir}/
+cp ${changelog_file} ${distdir}/CHANGELOG.md
 
 # defensive: make sure our artifacts don't use the version marker (this means
 # that "pack" will always fails when building in a dev environment)
 # when we get to 10.0.0, we can fix this...
-marker=$(node -p "require('./scripts/get-version-marker')")
 if find dist/ | grep "${marker}"; then
   echo "ERROR: build artifacts use the version marker '${marker}' instead of a real version."
   echo "This is expected for builds in a development environment but should not happen in CI builds!"

package.json

@@ -45,7 +45,8 @@
       "packages/@aws-cdk-containers/*",
       "packages/@monocdk-experiment/*",
       "packages/@aws-cdk/*/lambda-packages/*",
-      "tools/*"
+      "tools/*",
+      "scripts/script-tests"
     ],
     "nohoist": [
       "**/jszip",

packages/aws-cdk/CONTRIBUTING.md

@@ -116,7 +116,7 @@ Note that these tests can only be executed using the `run-against-dist` wrapper.
 
 ##### Implementation
 
-The implemention of the regression suites is not trivial to reason about and follow. Even though the code includes inline comments, we break down the exact details to better serve us in maintaining it and regaining context.
+The implementation of the regression suites is not trivial to reason about and follow. Even though the code includes inline comments, we break down the exact details to better serve us in maintaining it and regaining context.
 
 Before diving into it, we establish a few key concepts:
 
@@ -126,7 +126,7 @@ Before diving into it, we establish a few key concepts:
 - `FRAMEWORK_VERSION` - This is the version of the framework we are testing. It varries between the two variation of the regression suites.
 Its value can either be that of `CANDIDATE_VERSION` (for testing against the latest framework code), or `PREVIOUS_VERSION` (for testing against the previously published version of the framework code).
 
-Following are the steps invovled in running these tests:
+Following are the steps involved in running these tests:
 
 1. Run [`./bump-candidate.sh`](../../bump-candidate.sh) to differentiate between the local version and the published version. For example, if the version in `lerna.json` is `1.67.0`, this script will result in a version `1.67.0-rc.0`. This is needed so that we can launch a verdaccio instance serving local tarballs without worrying about conflicts with the public npm uplink. This will help us avoid version quirks that might happen during the *post-release-pre-merge-back* time window.
 
@@ -143,7 +143,7 @@ Following are the steps invovled in running these tests:
     - [Install the CLI](./test/integ/run-against-dist#L30) using the `CANDIDATE_VERSION` version `CANDIDATE_VERSION` env variable.
     - Execute the given script.
 
-6. Both cli regression test scripts run the same [`run_regression_against_framework_version`](./test/integ/test-cli-regression.bash#L22) function. This function accepts which framework version should the regression run against, it can be either `CANDIDATE_VERSION` or `PREVIOUS_VERSION`. Note that the argument is not the actual value of the versio, but instead is just an [indirection indentifier](./test/integ/test-cli-regression.bash#L81). The function will:
+6. Both cli regression test scripts run the same [`run_regression_against_framework_version`](./test/integ/test-cli-regression.bash#L22) function. This function accepts which framework version should the regression run against, it can be either `CANDIDATE_VERSION` or `PREVIOUS_VERSION`. Note that the argument is not the actual value of the version, but instead is just an [indirection indentifier](./test/integ/test-cli-regression.bash#L81). The function will:
 
     - Calculate the actual value of the previous version based on the candidate version. (fetches from github)
     - Download the previous version tarball from npm and extract the integration tests.

release.json

@@ -0,0 +1,4 @@
+{
+  "majorVersion": 1,
+  "releaseType": "stable"
+}

scripts/align-version.js

@@ -4,8 +4,9 @@
 //
 const fs = require('fs');
 
-const marker = require('./get-version-marker');
-const repoVersion = require('./get-version');
+const ver = require('./resolve-version');
+const marker = ver.marker;
+const repoVersion = ver.version;
 
 for (const file of process.argv.splice(2)) {
   const pkg = JSON.parse(fs.readFileSync(file).toString());

scripts/align-version.sh

@@ -13,7 +13,7 @@ files="./package.json $(npx lerna ls -p -a | xargs -n1 -I@ echo @/package.json)"
 ${scriptdir}/align-version.js ${files}
 
 # validation
-marker=$(node -p "require('./scripts/get-version-marker').replace(/\./g, '\\\.')")
+marker=$(node -p "require('./scripts/resolve-version').marker.replace(/\./g, '\\\.')")
 
 # Get a list of all package.json files. None of them shouldn contain 0.0.0 anymore.
 # Exclude a couple of specific ones that we don't care about.

scripts/bump-candidate.sh

@@ -6,8 +6,8 @@
 # a verdaccio instance that serves local tarballs, and if those tarballs have the same version as
 # already published modules, it messes things up.
 #
-# It does so by using a pre-release rc tag, making it so that locally packed versions will always be
-# suffixed with '-rc', distinguishing it from publisehd modules.
+# It does so by using a pre-release "rc" tag, making it so that locally packed versions will always be
+# suffixed with '-rc', distinguishing it from published modules.
 #
 # If you need to run integration tests locally against the distribution tarballs, you should run this
 # script locally as well before building and packing the repository.
@@ -16,8 +16,5 @@
 #
 # --------------------------------------------------------------------------------------------------
 set -euo pipefail
-version=${1:-minor}
-
-echo "Starting candidate ${version} version bump"
-
-npx standard-version --release-as ${version} --prerelease=rc --skip.commit --skip.changelog
+scriptdir=$(cd $(dirname $0) && pwd)
+BUMP_CANDIDATE=true ${scriptdir}/bump.js ${1:-minor}

scripts/bump.js

@@ -0,0 +1,49 @@
+#!/usr/bin/env node
+const standardVersion = require('standard-version');
+const fs = require('fs');
+const path = require('path');
+const ver = require('./resolve-version');
+const repoRoot = path.join(__dirname, '..');
+
+const releaseAs = process.argv[2] || 'minor';
+const forTesting = process.env.BUMP_CANDIDATE || false;
+
+async function main() {
+  if (releaseAs !== 'minor' && releaseAs !== 'patch') {
+    throw new error(`invalid bump type "${releaseAs}". only "minor" (the default) and "patch" are allowed. major version bumps require *slightly* more intention`);
+  }
+
+  console.error(`Starting ${releaseAs} version bump`);
+  console.error('Current version information:', JSON.stringify(ver, undefined, 2));
+
+  const changelogPath = path.join(repoRoot, ver.changelogFile);
+  const opts = {
+    releaseAs: releaseAs,
+    skip: { tag: true },
+    packageFiles: [ { filename: ver.versionFile, type: 'json' } ],
+    bumpFiles: [ { filename: ver.versionFile, type: 'json' } ],
+    infile: ver.changelogFile,
+    prerelease: ver.prerelease,
+    scripts: {
+      postchangelog: `${path.join(__dirname, 'changelog-experimental-fix.sh')} ${changelogPath}`
+    }
+  };
+
+  if (forTesting) {
+    opts.skip.commit = true;
+    opts.changelog = true;
+
+    // if we are on a "stable" branch, add a pre-release tag ("rc") to the
+    // version number as a safety in case this version will accidentally be
+    // published.
+    opts.prerelease = ver.prerelease || 'rc'
+    console.error(`BUMP_CANDIDATE is set, so bumping version for testing (with the "${opts.prerelease}" prerelease tag)`);
+  }
+
+  return standardVersion(opts);
+}
+
+main().catch(err => {
+  console.error(err.stack);
+  process.exit(1);
+});

scripts/changelog-experimental-fix.sh

@@ -0,0 +1,11 @@
+#!/bin/bash
+set -euo pipefail
+changelog="${1:-}"
+
+if [ -z "${changelog}" ]; then
+  echo "Usage: $0 CHANGELOG.md"
+  exit 1
+fi
+
+sed -i.tmp -e 's/BREAKING CHANGES$/BREAKING CHANGES TO EXPERIMENTAL FEATURES/' ${changelog}
+rm ${changelog}.tmp