Merge pull request #1163 from ucb-bar/conda

Use conda + Update initial setup docs

.github/CI_README.md

@@ -22,7 +22,7 @@ For example:
 This specifies that the `prepare-chipyard-cores` job needs the both the `make-keys` and the `setup-complete` steps to
 be completed before it can run.
 
-Chipyard runs its CI using a docker image created from `dockerfiles/Dockerfile`.
+Chipyard runs its CI using a docker image created from `dockerfiles/Dockerfile` and on Berkeley's compute infrastructure.
 See its [README](../dockerfiles/README.md) for more details.
 
 Finally, within each job's `steps:` section, the steps are run sequentially and state persists throughout a job.
@@ -71,9 +71,7 @@ Our own composite actions are defined in the `.github/actions/<ActionName>/actio
 This directory contains most the collateral for the Chipyard CI to work.
 The following is included in `.github/scripts/: directory
 
-    `build-toolchains.sh`        # build either riscv-tools or esp-tools
-    `create-hash.sh`             # create hashes of riscv-tools/esp-tools to use as hash keys
-    `remote-do-rtl-build.sh`            # use verilator to build a sim executable (remotely)
+    `remote-do-rtl-build.sh`     # use verilator to build a sim executable (remotely)
     `defaults.sh`                # default variables used
     `check-commit.sh`            # check that submodule commits are valid
     `build-extra-tests.sh`       # build default chipyard tests located in tests/
@@ -101,21 +99,9 @@ To get the CI to work correctly you need to create the following GH Repository S
 
 | Secret | Value |
 | -------| ------------- |
-| BUILDSERVER | the hostname of the remote build server (likely be a millennium machine) |
-| BUILDUSER | the login to use on the build server |
 | BUILDDIR | the directory to use on the build server |
-| SERVERKEY | a private key to access the build server |
 
-The main workflow also constructs and places in the environment a SERVER and a work directyory on that server env using the above secrets.
-The SERVER is constructed like this:
-```bash
-SERVER = ${{ secrets.BUILDUSER }}@${{ secrets.BUILDSERVER }}
-```
-
-Additionally, you need to add under the "PERMISSIONS" "SSH Permissions" section a private key that is on the build server that you are using.
-After adding a private key, it will show a fingerprint that should be added under the jobs that need to be run.
-
-Note: On the remote server you need to have the `*.pub` key file added to the `authorized_keys` file.
+Additionally, you need to install conda on the build servers that exist.
 
 Notes on CIRCLE CI
 ------------------

.github/ISSUE_TEMPLATE/bug-report.yml

@@ -31,7 +31,7 @@ body:
       description: OS setup for reproducibility
       placeholder: OS information
       value: |
-        Ex: Output of `uname -a` and `lsb_release -a`
+        Ex: Output of `uname -a` + `lsb_release -a` + `printenv` + `conda list`
     validations:
       required: true
 

.github/PULL_REQUEST_TEMPLATE.md

@@ -37,6 +37,7 @@ Provide a brief description of the PR immediately below this comment, if the tit
 - [ ] Did you state the type-of-change/impact?
 - [ ] Did you delete any extraneous prints/debugging code?
 - [ ] Did you mark the PR with a `changelog:` label?
+- [ ] (If applicable) Did you update the conda `.conda-lock.yml` file if you updated the conda requirements file?
 - [ ] (If applicable) Did you add documentation for the feature?
 - [ ] (If applicable) Did you add a test demonstrating the PR?
 <!-- Do this if this PR is a bugfix that should be applied to the latest release -->

.github/actions/cleanup-conda/action.yml

@@ -0,0 +1,28 @@
+name: cleanup-conda
+description: 'Remove extra conda environments'
+
+runs:
+  using: "composite"
+  steps:
+    - name: Remove extra conda environments
+      run: |
+        CONDA_REMOVE_NAMES=$(conda env list | awk '{print $1}' | tail -n +3 | grep "${{ env.conda-env-name-no-time }}" || true)
+        if [ -z "$CONDA_REMOVE_NAMES" ]; then
+          echo "No matching conda environments for ${{ env.conda-env-name-no-time }}. Skip removal."
+        else
+          echo "Removing $CONDA_REMOVE_NAMES conda environments."
+          for env in $CONDA_REMOVE_NAMES; do
+            conda env remove -n $env
+          done
+        fi
+        conda env list | awk '{print $1}' | tail -n +4 | while read envname; do
+          ENV_DATE=$(echo $envname | sed "s/cy-[[:digit:]]\+-\(.*\)-\(riscv\|esp\)-tools/\1/")
+          NUM_DIFF=$(( ( $(date +%s) - $(date --date="$ENV_DATE" +%s) )/(60*60*24) ))
+          if (( $NUM_DIFF > 7 )); then
+            echo "Removing $envname since it is $NUM_DIFF days old."
+            conda env remove -n $envname
+          else
+            echo "Skipping removal of $envname since it is $NUM_DIFF days old."
+          fi
+        done
+      shell: bash -leo pipefail {0}

.github/actions/create-conda-env/action.yml

@@ -0,0 +1,33 @@
+name: create-conda-env
+description: 'Create conda environments if they dont exist'
+inputs:
+  install-collateral:
+    description: 'Install Spike/Libgloss/etc'
+    required: false
+    default: true
+
+runs:
+  using: "composite"
+  steps:
+    - name: Create conda environments
+      run: |
+        if conda env list | grep -q "envs/${{ env.conda-env-name-no-time }}"; then
+          echo "Using pre-existing conda environments with prefix ${{ env.conda-env-name-no-time }}"
+        else
+          echo "Creating a conda environment for each toolchain with the toolchain installed"
+          conda activate base
+          conda-lock install -n ${{ env.conda-env-name-no-time }}-$(date --date "${{ env.workflow-timestamp }}" +%Y%m%d)-riscv-tools ./conda-requirements-riscv-tools-linux-64.conda-lock.yml
+          conda-lock install -n ${{ env.conda-env-name-no-time }}-$(date --date "${{ env.workflow-timestamp }}" +%Y%m%d)-esp-tools ./conda-requirements-esp-tools-linux-64.conda-lock.yml
+          conda deactivate
+
+          if [[ "${{ inputs.install-collateral }}" == 'true' ]]; then
+            echo "Add extra toolchain collateral to RISC-V install area"
+            conda activate ${{ env.conda-env-name-no-time }}-$(date --date "${{ env.workflow-timestamp }}" +%Y%m%d)-riscv-tools
+            ./scripts/build-toolchain-extra.sh riscv-tools
+            conda deactivate
+            conda activate ${{ env.conda-env-name-no-time }}-$(date --date "${{ env.workflow-timestamp }}" +%Y%m%d)-esp-tools
+            ./scripts/build-toolchain-extra.sh esp-tools
+            conda deactivate
+          fi
+        fi
+      shell: bash -leo pipefail {0}

.github/actions/git-workaround/action.yml

@@ -0,0 +1,15 @@
+name: git-workaround
+description: 'Workaround https://github.com/actions/checkout/issues/766'
+
+runs:
+  using: "composite"
+  steps:
+    - name: Workaround
+      run: |
+        if git config --global -l | grep -q "safe.directory=$GITHUB_WORKSPACE"; then
+            echo "Skip adding safe directory"
+        else
+            echo "Add $GITHUB_WORKSPACE to global git config"
+            git config --global --add safe.directory "$GITHUB_WORKSPACE"
+        fi
+      shell: bash -leo pipefail {0}

.github/actions/prepare-rtl/action.yml

@@ -13,28 +13,30 @@ inputs:
     description: type of build
     required: false
     default: "sim"
+  toolchain:
+    description: toolchain to use
+    required: false
+    default: "riscv-tools"
 
 runs:
   using: "composite"
   steps:
-    - name: Build RISC-V toolchains
-      uses: ./.github/actions/toolchain-build
-
-    - uses: actions/cache@v2
+    - uses: actions/cache@v3
       id: rtl-build-id
       with:
         path: |
           sims/verilator
           sims/firesim/sim
           generators/gemmini/software/gemmini-rocc-tests
-        key: ${{ inputs.group-key }}-${{ github.ref }}-${{ github.sha }}
+        key: ${{ inputs.group-key }}-${{ github.sha }}
 
     - name: Run RTL build if not cached
       run: |
+        conda activate ${{ env.conda-env-name-no-time }}-$(date --date "${{ env.workflow-timestamp }}" +%Y%m%d)-${{ inputs.toolchain }}
         if [[ "${{ steps.rtl-build-id.outputs.cache-hit }}" != 'true' ]]; then
-          echo "Cache miss on ${{ inputs.group-key }}-${{ github.ref }}-${{ github.sha }}"
+          echo "Cache miss on ${{ inputs.group-key }}-${{ github.sha }}"
           ./.github/scripts/${{ inputs.build-script }} ${{ inputs.group-key }} ${{ inputs.build-type }}
         else
           echo "Cache hit do not rebuild RTL for ${{ inputs.group-key }}"
         fi
-      shell: bash
+      shell: bash -leo pipefail {0}

.github/actions/run-tests/action.yml

@@ -12,20 +12,29 @@ inputs:
     description: rtl build script to use
     required: false
     default: "run-tests.sh"
+  toolchain:
+    description: toolchain to use
+    required: false
+    default: "riscv-tools"
 
 runs:
   using: "composite"
   steps:
     - name: Init submodules (since only the RTL is cached)
-      run: ./scripts/init-submodules-no-riscv-tools.sh --skip-validate
-      shell: bash
+      run: |
+        conda activate ${{ env.conda-env-name-no-time }}-$(date --date "${{ env.workflow-timestamp }}" +%Y%m%d)-${{ inputs.toolchain }}
+        ./scripts/init-submodules-no-riscv-tools.sh --skip-validate
+      shell: bash -leo pipefail {0}
 
     # Note: You shouldn't need the other inputs since it shouldn't build RTL from scratch
     - name: Build RTL
       uses: ./.github/actions/prepare-rtl
       with:
         group-key: ${{ inputs.group-key }}
+        toolchain: ${{ inputs.toolchain }}
 
     - name: Run RTL tests
-      run: ./.github/scripts/${{ inputs.run-script }} ${{ inputs.project-key }}
-      shell: bash
+      run: |
+        conda activate ${{ env.conda-env-name-no-time }}-$(date --date "${{ env.workflow-timestamp }}" +%Y%m%d)-${{ inputs.toolchain }}
+        ./.github/scripts/${{ inputs.run-script }} ${{ inputs.project-key }}
+      shell: bash -leo pipefail {0}

.github/scripts/build-extra-tests.sh

@@ -7,9 +7,5 @@ set -ex
 SCRIPT_DIR="$( cd "$( dirname "$0" )" && pwd )"
 source $SCRIPT_DIR/defaults.sh
 
-export RISCV="$GITHUB_WORKSPACE/riscv-tools-install"
-export LD_LIBRARY_PATH="$RISCV/lib"
-export PATH="$RISCV/bin:$PATH"
-
 make -C $LOCAL_CHIPYARD_DIR/tests clean
 make -C $LOCAL_CHIPYARD_DIR/tests

.github/scripts/check-commit.sh

@@ -52,13 +52,22 @@ dir="generators"
 branches=("master" "main" "dev")
 search
 
-submodules=("riscv-gnu-toolchain" "riscv-isa-sim" "riscv-pk" "riscv-tests")
+submodules=("esp-tools-feedstock")
+dir="toolchains/esp-tools"
+branches=("main")
+search
+
+submodules=("riscv-isa-sim" "riscv-pk" "riscv-tests")
 dir="toolchains/esp-tools"
 branches=("master")
 search
 
+submodules=("riscv-tools-feedstock")
+dir="toolchains/riscv-tools"
+branches=("main")
+search
 
-submodules=("riscv-gnu-toolchain" "riscv-isa-sim" "riscv-pk" "riscv-tests")
+submodules=("riscv-isa-sim" "riscv-pk" "riscv-tests")
 dir="toolchains/riscv-tools"
 branches=("master")
 search
@@ -69,7 +78,7 @@ dir="toolchains/riscv-tools"
 branches=("riscv")
 search
 
-submodules=("qemu" "libgloss")
+submodules=("libgloss")
 dir="toolchains"
 branches=("master")
 search
@@ -84,11 +93,6 @@ dir="tools"
 branches=("master" "dev")
 search
 
-submodules=("dromajo-src")
-dir="tools/dromajo"
-branches=("master")
-search
-
 submodules=("firesim")
 dir="sims"
 branches=("master" "main" "dev" "1.13.x")

.github/scripts/defaults.sh

@@ -5,32 +5,21 @@ CI_MAKE_NPROC=8
 # chosen based on a 24c system shared with 1 other project
 REMOTE_MAKE_NPROC=4
 
-# verilator version
-VERILATOR_VERSION=v4.034
-
-HOME=$GITHUB_WORKSPACE
-
 # remote variables
 # CI_DIR is defined externally based on the GH repository secret BUILDDIR
 
 REMOTE_PREFIX=$CI_DIR/${GITHUB_REPOSITORY#*/}-${GITHUB_REF_NAME//\//-}
 REMOTE_WORK_DIR=$GITHUB_WORKSPACE
-REMOTE_RISCV_DIR=$GITHUB_WORKSPACE/riscv-tools-install
-REMOTE_ESP_DIR=$GITHUB_WORKSPACE/esp-tools-install
 REMOTE_CHIPYARD_DIR=$GITHUB_WORKSPACE
 REMOTE_SIM_DIR=$REMOTE_CHIPYARD_DIR/sims/verilator
 REMOTE_FIRESIM_DIR=$REMOTE_CHIPYARD_DIR/sims/firesim/sim
 REMOTE_FPGA_DIR=$REMOTE_CHIPYARD_DIR/fpga
 REMOTE_JAVA_OPTS="-Xmx10G -Xss8M"
 # Disable the supershell to greatly improve the readability of SBT output when captured by Circle CI
 REMOTE_SBT_OPTS="-Dsbt.ivy.home=$REMOTE_WORK_DIR/.ivy2 -Dsbt.supershell=false -Dsbt.global.base=$REMOTE_WORK_DIR/.sbt -Dsbt.boot.directory=$REMOTE_WORK_DIR/.sbt/boot"
-REMOTE_VERILATOR_DIR=$REMOTE_PREFIX-$GITHUB_SHA-verilator-install
 
 # local variables (aka within the docker container)
-LOCAL_CHECKOUT_DIR=$HOME/project
-LOCAL_RISCV_DIR=$HOME/riscv-tools-install
-LOCAL_ESP_DIR=$HOME/esp-tools-install
-LOCAL_CHIPYARD_DIR=$HOME
+LOCAL_CHIPYARD_DIR=$GITHUB_WORKSPACE
 LOCAL_SIM_DIR=$LOCAL_CHIPYARD_DIR/sims/verilator
 LOCAL_FIRESIM_DIR=$LOCAL_CHIPYARD_DIR/sims/firesim/sim