-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #55 from rl-institut/feature-21/provide-example-ci…
…-and-code-test-implementation Provide example CI and code test implementation
- Loading branch information
Showing
13 changed files
with
594 additions
and
20 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
# SPDX-FileCopyrightText: 2023 Jonas Huber <@jh-RLI> | ||
# SPDX-FileCopyrightText: © Reiner Lemoine Institut | ||
# | ||
# SPDX-License-Identifier: MIT | ||
|
||
# tox environments that run on both pull requests and merge to main | ||
name: automated-testing | ||
|
||
on: | ||
push: | ||
branches: [production, develop] | ||
pull_request: | ||
branches: [production, develop] | ||
|
||
jobs: | ||
test: | ||
strategy: | ||
# fail-fast: false | ||
matrix: | ||
platform: [ubuntu-latest, macos-latest, windows-latest] | ||
python-version: [3.7, 3.9] | ||
# you can separate the tox-envs in different .yml files | ||
# see version 0.10.1 | ||
# https://github.com/joaomcteixeira/python-project-skeleton/releases/tag/v0.10.1 | ||
# below the list of tox envs that you wish to include in the matrix | ||
tox-envs: [lint, test] | ||
# workaround to ignore 'pr' when merging to main | ||
# see: https://github.community/t/how-to-conditionally-include-exclude-items-in-matrix-eg-based-on-branch/16853/6 | ||
isMerge: | ||
- ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }} | ||
exclude: | ||
- isMerge: true | ||
tox-envs: pr | ||
|
||
runs-on: ${{ matrix.platform }} | ||
|
||
steps: | ||
- uses: actions/checkout@v2 | ||
|
||
- name: Set up Python ${{ matrix.python-version }} | ||
uses: actions/setup-python@v2 | ||
with: | ||
python-version: ${{ matrix.python-version }} | ||
|
||
- name: Install dependencies | ||
run: | | ||
python -m pip install --upgrade pip setuptools wheel | ||
pip install tox | ||
- name: unit tests | ||
run: tox -e ${{ matrix.tox-envs }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,98 @@ | ||
# Continuous Integration | ||
|
||
Continuous Integration (CI) is a good practice in software development that | ||
involves automated testing of code, ensuring repository quality, reporting, and | ||
automate deployments. <br> | ||
The process involves the utilization of multiple technologies. | ||
Once the setup is complete, it's possible to create new test and | ||
also to interpret the output of success or error messages as response from the | ||
test-execution. A brief summary of this process is also provided for reference. | ||
|
||
## Automation | ||
|
||
The package [tox](https://github.com/tox-dev/tox) automates the process of testing | ||
Python projects. It provides support for creating and managing test environments | ||
across cross different environments, Python versions, and dependencies. <br> | ||
It ensures that the code is reliable, functional, and compatible on different | ||
computers leading to higher code quality and more robust software. | ||
Combined with GitHub Actions, it allows you to automate a wide range of tasks. | ||
|
||
### Key features and functionalities | ||
|
||
For each specified Python version, tox creates isolated virtual environments. | ||
This is important because different projects might have different dependencies | ||
or require specific Python versions and run on different platforms like Windows, Linux, or MacOS. | ||
Isolation prevents conflicts between packages used by different projects. | ||
|
||
1. **Virtual Environments**: Tox creates isolated virtual environments for each specified Python version. This is crucial because different projects might have different dependencies or require specific Python versions. Isolation prevents conflicts between packages used by different projects. | ||
2. **Automated Testing**: Tox automates the entire testing process. It can be configured to run various types of tests, such as unit tests, integration tests, and even style checks. This ensures that the project is thoroughly tested in different environments without manual intervention. | ||
3. **Cross-Version and Cross-Platform Testing**: Tox enables you to test your code against multiple versions of Python, including Python 2 and Python 3. Additionally, it supports testing on various operating systems, allowing you to identify and address compatibility issues specific to different platforms. | ||
4. **Dependency Management**: Tox can help manage project dependencies by allowing you to specify different sets of dependencies for different testing environments. This helps ensure that your code works with different combinations of dependencies. | ||
5. **Configuration via tox.ini**: Tox uses a configuration file named `tox.ini` to define testing environments, dependencies, and testing commands. This file also allows you to set up various testing environments, customize test commands, and control other testing-related behaviors. | ||
6. **Consistency and Reproducibility**: Tox aids in maintaining consistency and reproducibility in your testing process. By setting up a standard testing workflow and environments, you reduce the risk of inconsistent results due to differences in developers' local setups. | ||
7. **Continuous Integration (CI)**: Tox is often used in conjunction with CI systems like Jenkins, Travis CI, CircleCI, and GitHub Actions. By incorporating Tox into your CI pipeline, you can ensure that your code is automatically tested in multiple environments whenever changes are pushed to the repository. | ||
8. **Plugin System**: Tox has a plugin system that allows for extensibility. This means you can add custom functionality to the testing process, such as additional test runners, code analysis tools, and more. | ||
|
||
## Install | ||
|
||
Install the required package [`tox`](https://tox.wiki/en/stable/installation.html) in a python environment. <br> | ||
💻 `pip install tox` install tox <br> | ||
|
||
## Setup | ||
|
||
Configuring `tox` may involve some complexity. <br> | ||
See the official [tox user guide](https://tox.wiki/en/stable/user_guide.html) for further information. | ||
|
||
All test stages are setup in the configuration file `tox.ini`. | ||
This file configures all test environments as well as the tools and configuration | ||
used to run tests and checks. | ||
|
||
## Testing | ||
|
||
### pytest | ||
|
||
For smaller projects or a more straightforward approach, | ||
testing can also be effectively accomplished using [pytest](https://github.com/pytest-dev/pytest/). | ||
This allows for efficient testing without the additional complexity of setting | ||
up and managing multiple environments. | ||
|
||
### black | ||
|
||
Checking code syntax with `black` | ||
|
||
### flake8 | ||
|
||
Verifying `docstrings` with `flake8` | ||
|
||
### isort | ||
|
||
Sorting imports with `isort` | ||
|
||
### Django tests | ||
|
||
Testing Django projects with [Django tests](https://docs.djangoproject.com/en/3.2/topics/testing/). | ||
|
||
## GitHub Actions | ||
|
||
GitHub offers an integrated CI called `GitHub Actions`, | ||
which enables the definition of test workflows and events. | ||
These workflows can be executed, for instance, | ||
with every pull request or new commit on a specific branch. | ||
To set up workflows for your repository, follow the [Guides for GitHub Actions](https://docs.github.com/en/actions/guides). | ||
|
||
### Automated Testing | ||
|
||
All tests are defined within the workflow file `.github/workflows/automated-testing.yml`. | ||
The workflow defines the tasks for a action, and a runner executes these tasks on a | ||
virtual machine provided by GitHub. | ||
|
||
The workflow is triggered whenever there's a pull request or a commit to the `develop` or `production` branches. | ||
The workflow prepares the Python environment and installs relevant dependencies before executing the `tox` automation suite. | ||
If any errors are detected, the workflow fails and generates a detailed report for developers to review. | ||
|
||
### Building and deploying the documentation | ||
|
||
Another use case that involves the GitHub-Actions is the automated deployment of the documentation. | ||
While all minor versions are manually deployed using [mike](https://rl-institut.github.io/super-repo/latest/development/documentation/#mike). | ||
GitHub-Actions workflow builds and deploys the `develop` branch. | ||
This ensures that the documentation is automatically updated if a new feature or update is available. |
24 changes: 24 additions & 0 deletions
24
docs/development/continuous-integration/setup_github_actions.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
# Configure Continuous Integration using GitHub Actions | ||
|
||
## What is included in the tests? | ||
|
||
Our continuous integration (CI) process executes a series of tests, including | ||
running `pytest` and performing linting and code quality checks. | ||
|
||
Here's an overview of the tasks involved in our example: | ||
|
||
- Running `pytest` | ||
- Checking code syntax with `black` | ||
- Verifying `docstrings` with `flake8` | ||
- Sorting imports with `isort` | ||
|
||
All these tools are executed as part of a job defined within a GitHub Actions workflow file. | ||
The workflow defines the tasks for a job, and a runner executes these tasks on a | ||
virtual machine provided by GitHub. | ||
|
||
!!! Info "Setup GitHub Actions" | ||
To set up GitHub Actions workflows for your repository, follow the [official GitHub guide](https://docs.github.com/en/actions/guides). | ||
|
||
Our CI workflow is triggered whenever there's a pull request or a commit to the `develop` or `production` branches. | ||
The workflow prepares the Python environment and installs relevant dependencies before executing the `tox` automation suite. | ||
If any errors are detected, the workflow fails and generates a detailed report for developers to review. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,65 @@ | ||
# Configure Tox test-automation | ||
|
||
To streamline our test environment setup, we use `tox`, a powerful tool that | ||
automates testing across different environments. | ||
|
||
!!! Note | ||
[Tox](https://tox.wiki/en/stable/) is a powerful tool that simplifies | ||
and automates the process of testing Python projects across different environments. | ||
It ensures that your code is reliable, functional, and compatible with various | ||
Python versions and dependencies, ultimately leading to higher code quality | ||
and more robust software. Combined with a runner like GitHub Actions, | ||
it allows you to automate a wide range of tasks. | ||
|
||
??? Info "Key features and functionalities" | ||
|
||
Tox creates isolated virtual environments for each specified Python version. This is crucial because different projects might have different dependencies or require specific Python versions. Isolation prevents conflicts between packages used by different projects. | ||
|
||
1. **Virtual Environments**: Tox creates isolated virtual environments for each specified Python version. This is crucial because different projects might have different dependencies or require specific Python versions. Isolation prevents conflicts between packages used by different projects. | ||
|
||
2. **Automated Testing**: Tox automates the entire testing process. It can be configured to run various types of tests, such as unit tests, integration tests, and even style checks. This ensures that the project is thoroughly tested in different environments without manual intervention. | ||
|
||
3. **Cross-Version and Cross-Platform Testing**: Tox enables you to test your code against multiple versions of Python, including Python 2 and Python 3. Additionally, it supports testing on various operating systems, allowing you to identify and address compatibility issues specific to different platforms. | ||
|
||
4. **Dependency Management**: Tox can help manage project dependencies by allowing you to specify different sets of dependencies for different testing environments. This helps ensure that your code works with different combinations of dependencies. | ||
|
||
5. **Configuration via tox.ini**: Tox uses a configuration file named `tox.ini` to define testing environments, dependencies, and testing commands. This file also allows you to set up various testing environments, customize test commands, and control other testing-related behaviors. | ||
|
||
6. **Consistency and Reproducibility**: Tox aids in maintaining consistency and reproducibility in your testing process. By setting up a standard testing workflow and environments, you reduce the risk of inconsistent results due to differences in developers' local setups. | ||
|
||
7. **Continuous Integration (CI) Integration**: Tox is often used in conjunction with CI systems like Jenkins, Travis CI, CircleCI, and GitHub Actions. By incorporating Tox into your CI pipeline, you can ensure that your code is automatically tested in multiple environments whenever changes are pushed to the repository. | ||
|
||
8. **Plugin System**: Tox has a plugin system that allows for extensibility. This means you can add custom functionality to the testing process, such as additional test runners, code analysis tools, and more. | ||
|
||
## What is included in the tests? | ||
|
||
Here's an overview of the testing-tasks involved: | ||
|
||
- Running `pytest` | ||
- Checking code syntax with `black` | ||
- Verifying `docstrings` with `flake8` | ||
- Sorting imports with `isort` | ||
|
||
!!! Info "Run test on multiple platforms" | ||
Our Tox environments are executed on all major platforms including windows, linux and macos. | ||
|
||
## Installation | ||
|
||
Begin by installing [`tox`](https://tox.wiki/en/stable/installation.html): | ||
|
||
```bash | ||
pip install tox | ||
``` | ||
|
||
!!! Note | ||
Setup and activate a [python environment](https://docs.python.org/3/library/venv.html) first. | ||
|
||
## Setup | ||
|
||
Configuring `tox` may involve some complexity. | ||
Start by defining what tests you want to run and what dependencies | ||
(such as packages or external services) are required for the tests. | ||
Organize these test tasks into logical steps. | ||
|
||
!!! Info "Tox User Guide" | ||
The official [tox User Guide](https://tox.wiki/en/stable/user_guide.html) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
# Usage | ||
|
||
## Add new code test | ||
|
||
To add a new test for any new functionality you have added or you plan to add you should add a test. | ||
The test is imply added to the tests/ | ||
directory. | ||
|
||
See our examples .... | ||
|
||
## Debugging | ||
|
||
### Local Debugging | ||
|
||
You can run your automation suite locally by typing the following command in your console. This is useful for verifying the outcome of your automation pipeline before pushing your code: | ||
|
||
```bash | ||
tox | ||
``` | ||
|
||
Running this command will generate a comprehensive report, which may require some effort to debug. | ||
|
||
### Remote Debugging on GitHub | ||
|
||
For remote debugging, you can monitor the status of your automation jobs in two ways: | ||
|
||
1. Create a pull request and view the CI status integration there. | ||
2. Visit the [Actions tab of your repository](https://github.com/rl-institut/super-repo/actions) to see all available job executions. | ||
|
||
These options allow you to closely monitor the progress and outcomes of your CI workflow, ensuring the reliability and quality of your codebase. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.