comments and docs for regression and tracking info

.github/workflows/process.yml

@@ -61,7 +61,8 @@ jobs:
   regression-test:
     runs-on: ubuntu-latest
     needs: [tracking]
-    # depend on make and tracking job but dont skip if tracking was skipped
+    # depend on make and tracking job but don't skip if tracking was skipped
+    # run PROCESS on the regression test input files and compare results to the tracked files
     if: always()
     continue-on-error: true
     strategy:
@@ -80,6 +81,8 @@ jobs:
 
   run-tracking-inputs:
     runs-on: ubuntu-latest
+    # run PROCESS on the regression test input files and archive output MFILEs
+    # only runs upon merge to main
     if: github.ref == 'refs/heads/main'
     steps:
       - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
@@ -114,10 +117,12 @@ jobs:
         run: hatch run lint:fmt
 
   tracking:
+    # update tracker to include the MFILEs created by the run-tracking-inputs job
     concurrency:
       group: tracking-jobs
       cancel-in-progress: false
     runs-on: ubuntu-latest
+    # depends on run-tracking-inputs; only runs upon merge to main
     needs: run-tracking-inputs
     if: github.ref == 'refs/heads/main'
     env:
@@ -168,6 +173,7 @@ jobs:
       group: docs-jobs
       cancel-in-progress: false
     runs-on: ubuntu-latest
+    # depends on tracking; only runs upon merge to main
     needs: tracking
     if: github.ref == 'refs/heads/main'
     steps:

documentation/source/development/ci-guide.md

@@ -1,14 +1,19 @@
 
 # GitHub Actions
 
-Our GitHub actions Continuous Integration (CI) pipeline serves to ensure each branch and pull request conforms to our testing and style requirements. Due to the requirement of all stages on the built PROCESS artefacts and Docker image, all of our CI can currently be found in one workflow file: `process.yml`. A summary of each job within this workflow can be found below:
+Our GitHub actions Continuous Integration (CI) pipeline serves to ensure each branch and pull request conforms to our testing and style requirements. All of our CI can currently be found in one workflow file: `process.yml`. A summary of each job within this workflow can be found below:
 
 | Name | Functionality |
 | ---- | ------------- |
 | unit-test | Installs PROCESS and runs the unit tests. The job will fail if any of the unit tests fail. |
 | integration-test | Installs PROCESS and runs the integration tests. The job will fail if any of the integration tests fail. |
-| regression-test | Installs PROCESS and runs the regression tests with a 0% and 5% tolerance, respectively. The job will fail if any of the regression tests fail. |
+| regression-test | Installs PROCESS and runs the regression tests with a 0.2% and 5% tolerance, respectively. The job will fail if any of the regression tests fail. The job uses tracked MFILEs for regression test comparisons. |
 | run-tracking-inputs | Installs PROCESS and runs the regression test input files, archiving the output MFILEs. Only runs on the **main** branch. |
 | tracking | Collects MFILEs for input files of interest and creates a dashboard of changes in key values over time (one datapoint for each commit on main). Only runs on the **main** branch. |
 | pre-commit-quality-check | ensures the pushed code meets our standards as defined in `.pre-commit-config.yaml`. |
-| docs | Builds and deploys the documentation onto GitHub pages. |
+| docs | Builds and deploys the documentation onto GitHub pages. |
+
+!!! Info "Regression Job"
+    When creating a PR, the tracked MFILEs from the point at which you branched off main will be used during the regression test job. This job may fail on your PR. If so, these changes to the output will need to be reviewed. If the changes are accepted (i.e. they come from a necessary change to a model), upon merge to main the 
+    `run-tracking-inputs` job will run the regression input files to generate the associated MFILEs to be tracked by the 
+    `tracking job`, and these will now reflect the changes your PR has introduced. Therefore when the regression job runs upon merge of your code into main, it will use these updated tracked MFILEs, and the regression tests will pass.

documentation/source/development/testing.md

@@ -24,14 +24,20 @@ filename, then call a method to create a plot file, then check that the plot was
 
 ### Regression tests
 
-Regression tests detect changes in the entire program's behaviour by checking that for a given 
+Regression tests detect changes in the entire program's behaviour by checking that for a given
 input, it produces exactly the same output as before those changes. It detects changes in the 
-program's output. Therefore if your code changes the program's output, it will fail the regression 
-test. In this case that output difference will need to be reviewed and if accepted, the expected (or 
-reference) will be updated.
+program's output. Therefore, if your code changes the program's output, it will fail the regression 
+test. In this case, that output difference will need to be reviewed and, if accepted,
+the expected (or reference) output will be updated to reflect these changes when your code is merged into main.
 
 Regression tests compare the output of PROCESS locally when running an input file to a reference output stored in a 
-[repository](https://github.com/timothy-nunn/process-tracking-data). The test suite will download the reference output for the commit where the your current branch branched off of main. This means **each branch is accountable for only its changes since it branched off of main**. 
+[repository](https://github.com/timothy-nunn/process-tracking-data). The test suite will download the reference output for the commit where the current branch branched off main. This means **each branch is accountable for only its changes since it branched off main**. 
+
+
+!!! Info "CI System"
+    Our tests are run automatically using the Continous Integration (CI) system on GitHub.
+    More information about how the jobs run on our CI and update the tracked files can be found [here](https://ukaea.github.io/PROCESS/development/ci-guide/).
+
 
 ## pytest
 
@@ -154,7 +160,7 @@ something has changed, in this case by >5%.
 ### Running the CI on a branch
 
 When those local changes are committed and pushed, the CI system for the branch runs. This runs 5% 
-and 0% tolerance regression jobs, which are allowed to fail. This shows the author and reviewer 
+and 0.2% tolerance regression jobs, which are allowed to fail. This shows the author and reviewer 
 what the changes to the regression results are as a result of the code changes on that branch.
 
 ## Drawbacks to this approach