Merge branch 'master' into issue/LSSTDESC#128

.github/workflows/pythonpackage.yml

@@ -0,0 +1,49 @@
+name: Python package
+
+on: [push, pull_request]
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+    strategy:
+      max-parallel: 4
+      matrix:
+        python-version: [3.7]
+
+    steps:
+    - uses: actions/checkout@v2
+    - name: Cache Python packages
+      uses: actions/cache@v1
+      with:
+        path: ~/.cache/pip
+        key: ${{ runner.os }}-py${{ matrix.python-version }}-pip-${{ hashFiles('setup.py') }}
+        restore-keys: |
+          ${{ runner.os }}-py${{ matrix.python-version }}-pip-
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v1
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Install gfortran  # required by camb
+      run: |
+        sudo apt-get update -qq
+        sudo apt-get install gfortran -y
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install .[full]
+    - name: Lint with flake8
+      run: |
+        pip install flake8
+        # stop the build if there are Python syntax errors or undefined names
+        flake8 descqa descqagen descqaweb descqarun tests --count --select=E9,F63,F7,F82 --show-source --statistics
+        # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
+        flake8 descqa descqagen descqaweb descqarun tests --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
+    - name: Lint with pylint
+      run: |
+        pip install pylint
+        pylint --disable=all --enable=F,E,unreachable,duplicate-key,unnecessary-semicolon,global-variable-not-assigned,unused-variable,binary-op-exception,bad-format-string,anomalous-backslash-in-string,bad-open-mode --extension-pkg-whitelist=numpy descqa descqagen descqaweb descqarun
+    - name: Test with pytest
+      run: |
+        pip install pytest
+        pytest

CONTRIBUTING.md

@@ -1,5 +1,7 @@
 # How to contribute to DESCQA?
 
+## Our GitHub workflow
+
 ### Step 1: Fork the DESCQA GitHub repo and clone a local copy on NERSC
 
 _Note_: You can skip this step and start with Step 2, if you already did it once.
@@ -72,6 +74,51 @@ First, push your changes to GitHub
 Then go to https://github.com/LSSTDESC/descqa/ to create a pull request.
 
 
+## Integration test
+
+We use [GitHub Actions](https://help.github.com/en/actions/automating-your-workflow-with-github-actions)
+to run integration tests on pull requests (PRs).
+Once you submit a PR, you will see whether your PR passes the integration test.
+
+Because it's difficult to directly test most of the DESCQA functionalities
+(as they need to be run on NERSC), we heavily rely on linters for integration
+tests. In particular we use flake8 and pylint to check any potential syntax or
+logic errors. PRs need to pass these tests before they can be merged.
+
+You can see our integration test workflow [here](.github/workflows/pythonpackage.yml).
+
+### How to fix the errors?
+
+Check the output of GitHub Actions and it'll tell you what's wrong.
+You can then fix the code accordingly.
+
+Note that linters are not perfect, sometimes they report false positive.
+Other times the code runs fine but has [anti-patterns](https://en.wikipedia.org/wiki/Anti-pattern)
+which trigger the linters.
+You should change your code to follow best practices and make linter happy.
+In cases where you are certain that the linter made a false positive report,
+you can add the follow comment in your code to disable specific errors.
+You should add these exceptions at where the error takes place.
+
+```python
+# pylint: disable=<pylint code>  # noqa: <flake8 code>
+```
+
+### Build failed due to dependencies
+
+If your test depends on additional python packages or other libraries,
+first you should ask yourself if these additional dependencies are _really_ needed.
+Even if we manage to include these dependencies in the integration test, we still
+need to add them to our NERSC environment, which may be difficult.
+
+If you need to add additional python packages as dependencies, they should be added to
+`setup.py`, under `extras_require` > `full`. The integration test will automatically
+install all the packages specified there.
+
+If you need to add non-python libraries as dependencies, please do make sure you
+_really_ need them. Discuss with other DESCQA developers.
+There's not simple intructions for how to add non-python libraries.
+
 
 ## How to run DESCQA?
 
@@ -126,5 +173,5 @@ Then you can simply run `./run_master.sh`; however, there are many useful option
 
 As the master script is running, all the error messages will be printed out in real time if you have set `-v`. You can also go to the web interface to check you result:
 
-https://portal.nersc.gov/project/lsst/descqa/v2/?run=all
+https://portal.nersc.gov/cfs/lsst/descqa/v2/?run=all
 

README.md

@@ -7,20 +7,20 @@
 
 This repository contains the DESCQA framework that validates simulated galaxy catalogs. For more information about this framework, please check out the [DESCQA paper](https://arxiv.org/abs/1709.09665).
 
-A [web interface](https://portal.nersc.gov/project/lsst/descqa/) hosted on NERSC displays recent validation results from the DESCQA framework.
+A [web interface](https://portal.nersc.gov/cfs/lsst/descqa/) hosted on NERSC displays recent validation results from the DESCQA framework.
 
 **! Important !** Starting from DESCQA v2 (current version), we have separated the configurations and readers of catalogs from DESCQA and moved them to a standalone repo, the [GCRCatalogs](https://github.com/LSSTDESC/gcr-catalogs) repo. We have also changed much of the validation tests. If you are looking for the catalogs and tests in DESCQA v1 (as presented in the [companion paper](https://arxiv.org/abs/1709.09665)), please see the [v1 subdiectory](v1).
 
 
 ## Quick start guide
 
-1. First of all, try accessing the catalogs! You can find information about the catalogs in [this Confluence page](https://confluence.slac.stanford.edu/x/Z0uKDQ) and [this presentation](https://docs.google.com/presentation/d/1W5lZrQci9J4jaTdLWUIwkPKtq1lbDT3SzTh-YgIkl6k/edit?usp=sharing) ([video](https://youtu.be/4k9Yj6aI1uc)). However, the easiest way to start is to go to https://jupyter-dev.nersc.gov and login with your NERSC account, and follow [this tutorial notebook](https://github.com/LSSTDESC/gcr-catalogs/blob/master/examples/GCRCatalogs%20Demo.ipynb) ([download link](https://raw.githubusercontent.com/LSSTDESC/gcr-catalogs/master/examples/GCRCatalogs%20Demo.ipynb), you can then upload the notebook through the jupyter interface). 
+1. First of all, try accessing the catalogs! You can find information about the catalogs in [this Confluence page](https://confluence.slac.stanford.edu/x/Z0uKDQ) and [this presentation](https://docs.google.com/presentation/d/1W5lZrQci9J4jaTdLWUIwkPKtq1lbDT3SzTh-YgIkl6k/edit?usp=sharing) ([video](https://youtu.be/4k9Yj6aI1uc)). However, the easiest way to start is to go to https://jupyter.nersc.gov and login with your NERSC account, and follow [this tutorial notebook](https://github.com/LSSTDESC/gcr-catalogs/blob/master/examples/GCRCatalogs%20Demo.ipynb) ([download link](https://raw.githubusercontent.com/LSSTDESC/gcr-catalogs/master/examples/GCRCatalogs%20Demo.ipynb), you can then upload the notebook through the jupyter interface).
 
-2. Once you can access the catalogs, try to make some plots about things you are interested in. You can find [some ideas that have been proposed](https://github.com/LSSTDESC/descqa/labels/validation%20test), but you are more than welcome to come up with new ones! 
+2. Once you can access the catalogs, try to make some plots about things you are interested in. You can find [some ideas that have been proposed](https://github.com/LSSTDESC/descqa/labels/validation%20test), but you are more than welcome to come up with new ones!
 
 3. Now that you are able to make some plots, think about how to "validate" the catalogs (i.e., are there any observation/theory data that can be plotted on the same figure for comparison? How to decide whether a catalog is satisfactory?)
 
-4. Now we can integrate your work into the [DESCQA web interface](https://portal.nersc.gov/project/lsst/descqa/v2/)! This step is slightly more involved, but you can follow [the instruction here](CONTRIBUTING.md).
+4. Now we can integrate your work into the [DESCQA web interface](https://portal.nersc.gov/cfs/lsst/descqa/v2/)! This step is slightly more involved, but you can follow [the instruction here](CONTRIBUTING.md).
 
 
 
@@ -39,7 +39,7 @@ A [web interface](https://portal.nersc.gov/project/lsst/descqa/) hosted on NERSC
 
 ## Dependencies
 
-See `install_requires` and `extras_require` in [setup.py](setup.py) for the full installation dependencies. 
+See `install_requires` and `extras_require` in [setup.py](setup.py) for the full installation dependencies.
 
 Note, to *run* DESCQA (v2), [GCRCatalogs](https://github.com/LSSTDESC/gcr-catalogs) is also required.