How to create a Heimdall2 release

Warning

Anyone facilitating a release is expected to have basic fluency with the GIT as a DevOps tool used for source code management

Creating a Release

Follow the steps listed here to conduct a release for the SAF Heimdall application.

1. Make sure your local Heimdall repo is up-to-date, install dependencies, and check latest commits.

1. Ensure you have the latest version of the Heimdall repository's master branch.
   • Use the command git pull origin master within the root directory.
2. Do not delete or modify the yarn.lock file, it should be identical to the one pulled from master.
   • If you are using Windows, then you must use WSL.
   • If using a VPN or proxy software, you might need to disable it or figure out a solution to provide any needed certs.
3. Run yarn install --registry https://registry.npmjs.org. Note that this may take a while and could require you to fix broken dependencies.
   • It is recommended to delete the node_modules folder if it exists before installing dependencies in order to prevent conflicts.
   • We're using the --registry flag because using the regular yarn registry caused OpenSSL errors.
   • If you receive an openssl error, try referencing guidance in this closed issue
   • Make sure that the yarn install completes successfully.
   • If the cypress command fails (exit code 1) turn cypress checking off by setting the CYPRESS_INSTALL_BINARY variable on the terminal that the yarn command is being executed:
     • Windows: set CYPRESS_INSTALL_BINARY=0
     • Linux / Mac: export CYPRESS_INSTALL_BINARY=0
4. Run yarn frontend build. This will build the latest version of the Tailwind CSS for inclusion within Heimdall / the HTML export. Make sure to commit the changes! (changes are commit directly to master)
   • use git status within the application's root directory to check for changes. If there are changes, use git add <file1 file2 file3> for all files that have changed, then git commit -sm "<message>" (Note: lerna will push changes in step 3)
5. Check the repository to identify if there has been a commit in the frontend since the last release. If there were none, make a new commit in frontend.
   • How to Check: To check if a commit to the frontend has been done since the last release go to the releases page, check the Draft release commits to determine if any of them were for the frontend of the application.
   • Explanation: At the moment, a commit in the Heimdall frontend directory is required so that Heimdall-lite has a version bump, allowing the Heimdall-lite version number to match the overall repository version number. If the version numbers do not match, a warning message reading 'Heimdall is out of date' will display on Heimdall-lite.
   • use git status within the application's frontend directory (heimdall2/apps/frontend/) to check for changes. If there are changes, use the same steps as stated in the previous step.
   • What to Change: If there is no new commit in the Heimdall frontend, make a change to a file in a file within apps/frontend. For example, you could add a real or trivial change to a markdown file, .gitignore file, etc.
   • Note: You can verify this step when running the lerna command in step 3, where you will see what will be released. If the frontend (@mitre/heimdall-lite) is not one of those items, return to the instructions above (step 1.4).

2. Update the necessary files in the root directory of the Heimdall2 repository

• VERSION (with the new version number)

  • The new version number will follow semantic versioning rules by incrementing one of three numbers of the format X.Y.Z (Major.Minor.Patch).
    • If the new release only contains bug fixes to existing features (or dependency updates from dependabot), then increment the patch number (Z) by 1.
    • If the new release contains new features that are backwards-compatible, then increment the minor version (Y).
    • If the new release contains new features that significantly alter the core functionality of the application that aren't backwards compatible, then increment the major version (X).
    • Note: Major version increments are a significant decision - please discuss internally before considering changing the major version.

• CHANGELOG (Copy and paste from the draft release markdown)

  • Ensure that the text for the new release follows the same format as the text contained by the previous releases in this file.
  • Note: There might not be a draft release until a human-origin commit has been merged (i.e. the dependabot PRs being merged will not trigger the draft release workflow in some cases)

• Commit the changes directly to master (Note: lerna will push changes in step 3)

3. Create a release tag using lerna

• Run npx lerna version v2.x.x within the application root directory. Use the same version number that you used from the VERSION and CHANGELOG file changes.
• You will be asked - ? Are you sure you want to create these versions? type y for Yes
  • The results from the lerna command should like something like below. If @mitre/heimdall-lite is not shown, go back to step 1.

lerna notice cli v5.6.2
lerna info current version 2.8.2
lerna info Looking for changed packages since v2.8.2

Changes:
 - heimdall-server: 2.8.1 => 2.8.3 (private)
 - @mitre/heimdall-lite: 2.8.2 => 2.8.3
 - @heimdall/cypress-tests: 2.8.1 => 2.8.3 (private)

• To check if the step finished successfully go to the Releases page and ensure that a Tag with the same name you created is there (example v2.x.x).
• Consider prioritizing the actions required for the release by canceling the Dependabot PRs. (SEE BELOW). Heimdall uses Dependabot to keep dependencies up-to-date. Dependabot usually has a large number of PRs open, all of which will try to update when a push is made to the master branch.

4. Verify that the containerized versions of the application and their respective workflows function properly

1. Verify the containers push successfully to Docker Hub by looking at the workflow status of Pushing Heimdall Lite to Docker Hub and Push Heimdall Server to Docker Hub. They should be triggered by the commit generated by Lerna.

   Note: This may require clearing cache or running incognito

NOTE: If this is your first time running Heimdall via the docker image, consider referencing the README.

2. Generate the docker secrets by executing the script ./setup-docker-env.sh
   • The script creates a .env (same directory that the script is executed) with all of the necessary key-pair values to run the docker container.
   • When running this script, input 1d for the JWT_EXPIRE_TIME, y for enabling API keys, and 127.0.0.1 for FQDN/hostname/IP.
   • Remove any existing database content: rm -rf data/ on Windows use rmdir /s /q data
3. Pull the image for testing.
   • To pull the latest version of Heimdall from DockerHub:
     • Make sure the action that updates DockerHub has finished running before you try to pull this image.
     • Edit your docker-compose.yml and change the server image: mitre/heimdall2:release-latest to image: mitre/heimdall2:latest. Note that the latest tag for Heimdall in DockerHub reflects the current state of the master branch, while release-latest reflects the last tagged release.
     • Run docker compose pull to ensure you have the latest container images specified in your docker-compose.yml;
   • If you want to build the image locally (not recommended) or you are using an Apple silicon chip:
     • Edit your docker-compose.yml and change image: mitre/heimdall2:release-latest to build: .
     • Run docker compose build.
     • NOTE: Building this on a company network may not work given that building this image requires access to the internet and the image will not have company specific certificates.
4. Ensure the container starts successfully.
   • Run docker compose up.
   • Access Heimdall in your browser at 127.0.0.1 Note: You will get a self-signed certificate warning from your browser if you did not provide your own; this is not a problem for testing purposes.
   • Afterwards, you can kill your running containers with CTRL+C and then docker compose down. You can remove all of these containers with docker system prune.

5. Validate and promote the Heroku deployment

1. It can take some time to automatically push out a new deployment. Verify that mitre-heimdall-staging has been deployed by checking the version number. If it has been automatically deployed, move to step 5.2. If it has not updated, you can manually deploy it with the instructions below (5.1).

   • To manually deploy on Heroku:
   • Login to Heroku:
   • Click the mitre-saf workspace if it is not already selected
   • Click on drop down menu then "Deploy a branch..."
   • Make sure that master is selected
   • You may be prompted by GitHub to login
   • Click "Deploy"

2. Verify the staging environment before promoting to production by validating the functionality below. If there are issues, address them with the team prior to moving forward with the release.

   • Uploading an Evaluation and Profile
   • Compare View
   • Filtering
   • Database Storage
   • OAuth Login
   • Results Table / Marking Results as Viewed
   • Exports
   • Tags (note: you can change tags from the Load modal and from inside the results view itself in the top left of the screen; try both)
   • Groups
   • API keys

3. Promote mitre-heimdall-staging to production in Heroku

   • Note: If you receive an error rather than the picture below, you may not have the proper authorization. In this case, reach out to the team to request proper authorization.
   • The production deployment should now say "Deploy at:" with the current time, and clicking "Open App" should take you to the version of Heimdall you just promoted.

6. Update and publish the release.

• Associate the release with the tag created by Lerna.

• Click on the edit (pencil) icon of the draft release

• Update the content on the Write panel as follows:

  • Remove "## New Features" and "## Bug Fixes" header if they exist
  • Add the header "## What's New" in the description if there were any content under the previously removed headers
  • Keep the dependabot PRs updates under the header "## Dependency Updates"

• Update the tag by using the dropdown menu on the draft release page. At first, it will be populated with the string untagged followed by a long hash. Click the dropdown, delete the existing tag string, and pick the correct version from the list that should appear.

• Click Publish Release
  • Note: You may need to cancel extraneous workflows again. (SEE BELOW)

7. Validate that the workflows associated with the release succeeded.

• Heimdall Lite and Heimdall Server should both have successfully been pushed to the appropriate repositories (Dockerhub, Ironbank, hopefully Github Container Registry soon)
  • To check if the sophos pipeline succeeded you can check Ironbank to see if the merge request for the deployment appears at Iron Bank Repository
• The Deploy to Github Pages workflow completing triggers the 'pages build and deployment' workflow, which is what actually updates heimdall-lite.mitre.org (see #8)
• The Github Packages/NPM workflow may fail. This is expected because the overall workflow attempts to publish individual components (Heimdall Lite, InSpecJS, HDF Converters) and the overall workflow will fail if one of the components has not been updated in this release. Click on the workflow that ran to confirm the components that were updated this release were actually published.

8. Validate heimdall-lite.mitre.org works

• Go to heimdall-lite.mitre.org.
• Hard refresh the website to get the latest (you may have to wait several minutes for the Heroku promotion to be in effect).
• Confirm that there is no warning message about the version on the screen.

9. Recommended: Update and Release the SAF CLI

• Wait for the npm workflows to finish before releasing the SAF-CLI release. Monitor Push-To-NPM Action to validate completion.

  Note: Any components of the monorepo that do not have a new version (ex: InSpecJS, HDF Converters) may appropriately fail to push to npm because they do not have an updated version.

• Follow the SAF CLI release instructions

  Note: This action may fail if HDF-Converters and/or InspectJS does not have a version update. Refer to the results of the lerna command previously ran

---

Workflow Map to Release Step

Workflow Name	Workflow File	Action	Triggered	Release Step
Push Heimdall Lite to Docker Hub on every merge to master and tag as latest	push-lite-to-docker.yml	Push Heimdall Lite to Docker Hub on every merge to master and tag as latest	on push to master	Step 4
Push Heimdall Server to Docker Hub on every merge to master and tag as latest	push-server-to-docker.yml	Push Heimdall Server to Docker Hub on every merge to master and tag as latest	on push - master	Step 4
Push Heimdall Lite to docker on every release and tag as release-latest and version	release-lite-to-docker.yml	Push Heimdall Lite to docker on every release and tag as release-latest and version	on release - publish	Step 6
Push Heimdall Server to Docker Hub on every release and tag as release-latest and version	release-server-to-docker.yml	Push Heimdall Server to Docker Hub on every release and tag as release-latest and version	on release - publish	Step 6
Push Heimdall-Lite to NPM and GitHub Packages and Push InspecJs to NPM	push-to-npm.yml	Push Heimdall-Lite to NPM and GitHub Packages and Push InspecJs to NPM	on release - publish	Step 7
Deploy to Github Pages	gh-pages.yml	Generate the GitHub Pages and deployment - updates Heimdall Lite	on release - publish	Step 7

---

Cancelling workflow runs

Dependabot's PRs are currently set to have their CI/CD workflows triggered on every merge into master. Use the following script to cancel all of the dependabot PR based workflows.

Script based on this StackOverflow answer: https://stackoverflow.com/a/60766849/645647

token=YOUR_GITHUB_PERSONAL_ACCESS_TOKEN # needs the 'repo' scope
repo=your_user/your_repo # mitre/heimdall2

# get all the workflow run ids -> https://github.com/mitre/heimdall2/actions/runs/{VERY_LONG_ID_NUMBER}
# might need to add in parameters to increase page size and/or a forloop to iterate over pages OR use the 'gh' tool which seems like it returns all results at once instead of paginating them
# might need to adjust the filter if it's not precisely cancelling any and all of dependabot's PRs' workflows
ids=$(curl -s -H "Authorization: token $token" \
     https://api.github.com/repos/$repo/actions/runs?per_page=100 | \
     jq '.workflow_runs[] | select(([.status] | inside(["in_progress", "queued"])) and (.head_branch | startswith("dependabot/"))) | .id')
# create a list of ids that the for-loop iterates over
list=("${(@f)ids}")
# can't batch cancel workflows so have to iterate
for i in $list; do curl \
     -H "Authorization: token $token" \
     -X POST "https://api.github.com/repos/$repo/actions/runs/$i/cancel"; done

Ironbank Pipeline Troubleshooting [WIP]

We are working on getting steps here that explains how the pipeline works and how to fix when it breaks.