GitHub Actions - Reusable Workflows Library

GitHub Actions reusable workflows library to be reused between GitHub repos.

See Documentation for how to call these workflows directly from your own GitHub Actions workflow.

Examples

In your GitHub repo, import these workflows by adding small yaml files to the .github/workflows/ directory.

• Lint YAML
• Lint JSON
• Lint XML
• Lint Bash / Shell Scripts
• Lint Python
  • PyLint
  • Flake8
• Lint README / Markdown documentation
• Lint GitHub CODEOWNERS
• Security - Scan for Secrets & Issues
  • Semgrep Local
  • Semgrep Cloud
  • Trivy Filesystem Scan
  • Trivy Docker Image Scan
  • Grype Filesystem Scan
• Analyze your Terraform code security & best practices
  • tfsec
  • tflint
  • Checkov
• Terraform Plan & Apply
• Lint Ansible Playbooks
• Lint Packer HCL
• Lint Redhat Kickstart
• Lint Debian Preseed
• Lint Ubuntu AutoInstaller Cloud Init
• Lint Jenkinsfiles
• Lint Groovy
• Lint Javascript
• Docker Build and push to DockerHub
• Docker Build and push to AWS ECR
• Docker Build and push to multiple registries
• Kubernetes - Pluto - Check for Outdated APIs
• Kubernetes - Polaris - Security & Best Practices Check
• Check for Broken URL Links
• Auto-Merge Production hotfixes back to Staging
• Mirror Repos to GitLab for DR Backups
• AWS CodeArtifact - Publish a Python Package
• Permissions
  • Security Alerts
• Linting Auto-fixers
• Creating or Commenting on Pull Requests
• Merging Pull Requests
• Production

Lint YAML

Finds all YAML in your repo and lints it.

Copy this into .github/workflows/yaml.yaml:

on:
  push:
    paths:
      - '**/*.yml'
      - '**/*.yaml'
jobs:
  check_yaml:
    uses: HariSekhon/GitHub-Actions/.github/workflows/yaml.yaml@master

Lint JSON

Finds all JSON in your repo and lints it.

Copy this into .github/workflows/json.yaml:

on:
  push:
    paths:
      - '**/*.json'
jobs:
  check_json:
    uses: HariSekhon/GitHub-Actions/.github/workflows/json.yaml@master

Lint XML

Finds all XML in your repo and lints it.

Copy this into .github/workflows/xml.yaml:

on:
  push:
    paths:
      - '**/*.xml'
jobs:
  check_xml:
    uses: HariSekhon/GitHub-Actions/.github/workflows/xml.yaml@master

Lint Bash / Shell Scripts

Finds all *.sh scripts in your repo and lints them.

Copy this into .github/workflows/shellcheck.yaml:

on:
  push:
    paths:
      - '**/*.sh'
jobs:
  shellcheck:
    uses: HariSekhon/GitHub-Actions/.github/workflows/shellcheck.yaml@master

Lint Python

PyLint

Finds all *.py code in your repo and lints it.

Copy this into .github/workflows/pylint.yaml:

on:
  push:
    paths:
      - '**/*.py'
jobs:
  pylint:
    uses: HariSekhon/GitHub-Actions/.github/workflows/pylint.yaml@master
    with:
      python-version: '3.10'

Flake8

Finds all *.py code in your repo and lints it.

Copy this into .github/workflows/flake8.yaml:

on:
  push:
    paths:
      - '**/*.py'
jobs:
  flake8:
    uses: HariSekhon/GitHub-Actions/.github/workflows/flake8.yaml@master
    with:
      python-version: '3.10'

Lint README / Markdown documentation

Finds all markdown files in your repo and lints them.

Copy this into .github/workflows/markdown.yaml:

on:
  push:
    paths:
      - '**/*.md'
jobs:
  check_markdown:
    uses: HariSekhon/GitHub-Actions/.github/workflows/markdown.yaml@master

Lint GitHub CODEOWNERS

Lints the GitHub CODEOWNERS / .github/CODEOWNERS files.

Copy this into .github/workflows/codeowners.yaml:

on:
  push:
    paths:
      - CODEOWNERS
      - .github/CODEOWNERS
jobs:
  check_codeowners:
    uses: HariSekhon/GitHub-Actions/.github/workflows/codeowners.yaml@master

Security - Scan for Secrets & Issues

Semgrep Local

Create .github/workflows/semgrep.yaml containing:

on: [push]
jobs:
  semgrep:
    uses: HariSekhon/GitHub-Actions/.github/workflows/semgrep.yaml@master

Alerts for the above badge appear under the GitHub repo's Security tab -> Code scanning alerts.

The badge will go red if there are any alerts.

Semgrep Cloud

Create .github/workflows/semgrep-cloud.yaml containing:

on: [push]
jobs:
  semgrep:
    uses: HariSekhon/GitHub-Actions/.github/workflows/semgrep-cloud.yaml@master
    secrets:
      SEMGREP_APP_TOKEN: ${{ secrets.SEMGREP_APP_TOKEN }}

Alerts for the above badge appears in the Semgrep dashboard at:

https://semgrep.dev

The badge will go red only if failing to run and publish to Semgrep Cloud, whether there are any alerts of not. You must check the dashboard.

Trivy Filesystem Scan

Alerts for the above badge appear under the GitHub repo's Security tab -> Code scanning alerts.

on:
  push:
    paths-ignore:
      - '**/*.md'
jobs:
  trivy:
    uses: HariSekhon/GitHub-Actions/.github/workflows/trivy.yaml@master

Trivy Docker Image Scan

Alerts for the above badge appear under the GitHub repo's Security tab -> Code scanning alerts.

on:
  push:
    paths-ignore:
      - '**/*.md'
jobs:
  trivy:
    uses: HariSekhon/GitHub-Actions/.github/workflows/trivy_image.yaml@master
    with:
      docker_image: harisekhon/bash-tools
      severity: ''

Grype Filesystem Scan

Alerts for the above badge appear under the GitHub repo's Security tab -> Code scanning alerts.

on: [push]
jobs:
  grype:
    uses: HariSekhon/GitHub-Actions/.github/workflows/grype.yaml@master

Analyze your Terraform code security & best practices

tfsec

Alerts appear under Security -> Code scanning alerts.

Create .github/workflows/tfsec.yaml containing:

on:
  push:
    paths:
      - '**/*.tf'
      - '**/*.tfvars'
      - '**/*.hcl'
jobs:
  tfsec:
    uses: HariSekhon/GitHub-Actions/.github/workflows/tfsec.yaml@master

tflint

Create .github/workflows/tflint.yaml containing:

on:
  push:
    paths:
      - '**/*.tf'
      - '**/*.tfvars'
      - '**/*.hcl'
jobs:
  tfsec:
    uses: HariSekhon/GitHub-Actions/.github/workflows/tflint.yaml@master

Checkov

Alerts appear under Security -> Code scanning alerts.

Checkov can scan more than just Terraform code, so you probably don't want to restrict to only the Terraform file extensions paths filter used in the above tfsec workflow.

Create .github/workflows/checkov.yaml containing:

on: [push]
jobs:
  checkov:
    uses: HariSekhon/GitHub-Actions/.github/workflows/checkov.yaml@master

Terraform Plan & Apply

Plans - updates Pull Requests with the results of validation, format check and full Change Plan outputs

Apply - applies when merged to default branch, eg. master or main

on: [push, pull_request]
jobs:
  terraform:
    uses: HariSekhon/GitHub-Actions/.github/workflows/terraform.yaml@master
    with:
      dir: path/to/terraform/code
      args: -var-file=some-other.tfvars
    secrets:
      ...

For more sophisticated examples including approvals, secrets, branch and path selection etc. see my Terraform repo's templates for terraform-plan.yaml and terraform-apply.yaml

Lint Ansible Playbooks

Finds all Ansible playbook.y*ml in your repo and lints them.

Copy this into .github/workflows/ansible-playbook-syntax.yaml:

on:
  push:
    paths:
      - '**/playbook.yml'
      - '**/playbook.yaml'
jobs:
  check_ansible_playbook_syntax:
    uses: HariSekhon/GitHub-Actions/.github/workflows/ansible-playbook-syntax.yaml@master

Lint Packer HCL

Finds all *.pkr.hcl Packer code in your repo and lints them.

Copy this into .github/workflows/packer.yaml:

on:
  push:
    paths:
      - '**/*.pkr.hcl'
jobs:
  check_packer_hcl:
    uses: HariSekhon/GitHub-Actions/.github/workflows/packer.yaml@master

Lint Redhat Kickstart

Lints Redhat Kickstart automated installer files.

Copy this into .github/workflows/kickstart.yaml:

on:
  push:
    paths:
      - '**/anaconda-ks.cfg'
      - '**/ks.cfg'
      - '**/kickstart.cfg'
jobs:
  check_kickstart:
    uses: HariSekhon/GitHub-Actions/.github/workflows/kickstart.yaml@master
    with:
      files: installers/anaconda-ks.cfg

Lint Debian Preseed

Lints Debian Preseed automated installer files.

Copy this into .github/workflows/preseed.yaml:

on:
  push:
    paths:
      - '**/preseed.cfg'
jobs:
  check_preseed:
    uses: HariSekhon/GitHub-Actions/.github/workflows/preseed.yaml@master
    with:
      files: installers/preseed.cfg

Lint Ubuntu AutoInstaller Cloud Init

Lints Ubuntu AutoInstaller Cloud Init automated installer files.

Copy this into .github/workflows/autoinstall-user-data.yaml:

on:
  push:
    paths:
      - '**/autoinstall-user-data'
jobs:
  check_cloudinit:
    uses: HariSekhon/GitHub-Actions/.github/workflows/autoinstall-user-data.yaml@master
    with:
      files: installers/autoinstall-user-data

Lint Jenkinsfiles

Finds all files named Jenkinsfile in the repo and lints them using a live Jenkins in docker.

Create .github/workflows/jenkinsfile.yaml:

on:
  push:
    paths:
      - '**/*Jenkinsfile*'
jobs:
  jenkinsfile:
    uses: HariSekhon/GitHub-Actions/.github/workflows/jenkinsfile.yaml@master

Lint Groovy

Finds all Groovy files named *.groovy in the repo and lints them using groovyc.

This is a basic check but good for a Jenkins Groovy Shared Library.

Create .github/workflows/groovyc.yaml:

on:
  push:
    paths:
      - '**/*.groovy'
jobs:
  check_groovyc:
    uses: HariSekhon/GitHub-Actions/.github/workflows/groovyc.yaml@master

Lint Javascript

Finds all Javascript files named *.js in the repo and lints them using eslint.

Create .github/workflows/eslint.yaml:

on:
  push:
    paths:
      - '**/*.js'
      - package.json
      - package-lock.json
jobs:
  check_eslint:
    uses: HariSekhon/GitHub-Actions/.github/workflows/eslint.yaml@master

Docker Build and push to DockerHub

Create .github/workflows/dockerhub_build.yaml:

on: [push]
jobs:
  docker_build:
    uses: HariSekhon/GitHub-Actions/.github/workflows/dockerhub_build.yaml@master
    with:
      repo: harisekhon/bash-tools  # DockerHub user/repo
      tags: latest ubuntu          # builds, tags as harisekhon/bash-tools:latest and harisekhon/bash-tools:ubuntu and pushes to DockerHub
      context: devops-bash-tools-ubuntu  # path to dir containing the source and Dockerfile
      #max-cache: true                   # if you need multi-stage caching (uses a separate cache image)
    secrets:
      DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
      DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}

Docker Build and push to AWS ECR

Create .github/workflows/docker_build_aws_ecr.yaml:

on: [push]
jobs:
  docker_build:
    uses: HariSekhon/GitHub-Actions/.github/workflows/docker_build_aws_ecr.yaml@master
    with:
      repo: MY_ECR_REPO  # without the 'xxx.dkr.ecr.<region>.amazonaws.com' prefix
    secrets:
      AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
      AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      AWS_DEFAULT_REGION: ${{ secrets.AWS_DEFAULT_REGION }}

This auto-adds tags:

• latest
• Git branch or tag
• Git SHA
• Epoch in seconds
• Date
• Date & Timestamp

Makes heavy use of all several possible caches including branch/tag specific caches to speed up builds / re-builds and avoid cache invalidation between environments.

Supports multi-stage build caching using GHCR for intermediate layer caching since AWS ECR doesn't support this at time of writing, simply by adding:

    with:
      max-cache: true

Docker Build and push to multiple registries

Supports building + pushing to any combination of the following, just add the relevant secrets, see docker_build.yaml for details:

• ACR - Azure Container Registry
• ECR - AWS Elastic Container Registry
• GCR - Google Container Registry
• GAR - Google Artifact Registry
• GHCR - GitHub Container Registry
• GitLab Registry
• Quay.io Registry
• DockerHub

Create .github/workflows/docker_build.yaml:

on: [push]
jobs:
  docker_build:
    uses: HariSekhon/GitHub-Actions/.github/workflows/docker_build.yaml@master
    with:
      repo_tags: |
        harisekhon/bash-tools:latest
        ghcr.io/harisekhon/bash-tools:latest
      context: devops-bash-tools-ubuntu  # path to dir containing the source and Dockerfile
    # GHCR uses the local github.token, for other registries, add secrets, see docker_build.yaml for details
    secrets:
      DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
      DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}

Kubernetes - Pluto - Check for Outdated APIs

Checks all Kubernetes YAML files for outdated API objects using Pluto.

Create .github/workflows/pluto.yaml:

on:
  push:
    paths:
      - '**/*.yaml'
jobs:
  pluto:
    uses: HariSekhon/GitHub-Actions/.github/workflows/pluto.yaml@master

Kubernetes - Polaris - Security & Best Practices Check

Checks all Kubernetes YAML files for security issues and best practices.

Polaris currently fails on very advanced patches such as found in my Kubernetes-configs repo.

Create .github/workflows/polaris.yaml:

on:
  push:
    paths:
      - '**/*.yaml'
jobs:
  polaris:
    uses: HariSekhon/GitHub-Actions/.github/workflows/polaris.yaml@master

Check for Broken URL Links

Create .github/workflows/url_links.yaml:

on: [push]
jobs:
  url_links:
    uses: HariSekhon/GitHub-Actions/.github/workflows/url_links.yaml@master
    with:

      # custom ignore inaccessible / internal / partially constructed links or those containing variables
      # this is a multi-line string, one URL or partial ERE regex match per line
      url_links_ignored: |
        https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize%2Fv

      # ignore URLs without dots as these are usually internal inaccessible local addresses such as http://krb5server rather than public accessible links
      #ignore_urls_without_dots: 'true'  # any value enables this

Auto-Merge Production hotfixes back to Staging

Merges via a Pull Request for full auditing.

Create .github/workflows/merge_production_to_staging.yaml:

on: [push]
jobs:
  merge:
    if: github.ref_name == 'production'
    name: Merge Production Branch to Staging Branch (hotfix backports)
    uses: HariSekhon/GitHub-Actions/.github/workflows/merge-branch.yaml@master
    with:
      head: production  # from - optional - if omitted defaults to the trigger branch, which is always 'production' due to the if condition above
      base: staging     # to

Mirror Repos to GitLab for DR Backups

Mirrors all/given GitHub repos to GitLab - including all branches and tags, and GitHub repo description.

There are similar workflows in this repo to mirror GitHub repos to AWS CodeCommit or GCP Source Repos too.

on:
  # allow to run manually with one or more repos
  workflow_dispatch:
    inputs:
      repos:
        description: The GitHub repos to mirror to GitLab, space separated (empty defaults to all repos accessible)
        type: string
        default: ""
        required: false

  schedule:
    # mirror to GitLab hourly
    - cron: '0 * * * *'

jobs:
  gitlab_mirror:
    name: GitLab Mirror
    if: github.event.repository.fork == false && github.ref_type == 'branch' && github.ref_name == github.event.repository.default_branch
    uses: HariSekhon/GitHub-Actions/.github/workflows/gitlab-mirror.yaml@master
    with:
      #organization: my-org  # optional: mirror your company's repos instead of your personal repos
      repos: ${{ github.event.inputs.repos }}  # if triggering manually, mirror those given repos, if blank, mirror all of them
    secrets:
      GH_TOKEN: ${{ secrets.GH_TOKEN }}
      GITLAB_TOKEN: ${{ secrets.GITLAB_TOKEN }}

AWS CodeArtifact - Publish a Python Package

on:
  tags:
    - v*

jobs:
  aws_codeartifact_python_publish:
    uses: HariSekhon/GitHub-Actions/.github/workflows/codeartifact_python_publish.yaml@master
    with:
      domain: mycompany     # your AWS CodeArtifact service domain name
      repo: mycompany-core  # your CodeArtifact repo name
      #command: make publish_package  # default. Can be any command using CODEARTIFACT_AUTH_TOKEN and CODEARTIFACT_REPO_URL
    secrets:
      AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
      AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      AWS_DEFAULT_REGION: ${{ secrets.AWS_DEFAULT_REGION }}

Permissions

These workflows are locked down to the minimal required permissions as per the best practice principal of least privilege, usually just contents: read, but some require extra permissions to create Pull Requests or write Security Alerts to the GitHub Security tab.

If you've locked down your GitHub Organizations permissions to default to contents: read (which I recommend), then you may want to copy the permissions key out of the workflow to your calling workflow to grant them the needed permissions.

Security Alerts

permissions:
  actions: read
  contents: read
  security-events: write

These 3 permissions are needed for workflows that report to GitHub Security tab, including:

• checkov.yaml
• kics.yaml
• semgrep.yaml
• tfsec.yaml
• trivy.yaml

Linting Auto-fixers

For workflows that lint-and-fix code, such as terraform-fmt-write.yaml, you'll need to grant:

permissions:
  contents: write       # if called by on: push
  pull-requests: write  # if called by on: pull_request

Creating or Commenting on Pull Requests

For workflows that create or comment on PRs, such as terraform.yaml and tfsec-pr-commenter.yaml you'll need to grant:

permissions:
  contents: read
  pull-requests: write

Merging Pull Requests

For workflows that merge PRs, such as merge-branch.yaml you'll need to grant:

permissions:
  contents: write
  pull-requests: write

Production

As per GitHub Actions Security Best Practices, you should consider fixing your @<ref> to an exact immutable hashref.

Alternatively, you may want to fork this repo to have full control over all updates.

You can create environment branches in your forked repo to stage updates across dev/staging/production and enable the fork-sync github actions workflow in your fork to keep the master branch sync'd every few hours, and the fork-update-pr github actions workflow to raise GitHub Pull Requests for your environment branches to audit, authorize & control updates.