Skip to content

Commit

Permalink
Merge pull request #55 from rl-institut/feature-21/provide-example-ci…
Browse files Browse the repository at this point in the history
…-and-code-test-implementation

Provide example CI and code test implementation
  • Loading branch information
Ludee authored Aug 24, 2023
2 parents 0238692 + 17c45ea commit 6ad23b3
Show file tree
Hide file tree
Showing 13 changed files with 594 additions and 20 deletions.
51 changes: 51 additions & 0 deletions .github/workflows/automated-testing.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# SPDX-FileCopyrightText: 2023 Jonas Huber <@jh-RLI>
# SPDX-FileCopyrightText: © Reiner Lemoine Institut
#
# SPDX-License-Identifier: MIT

# tox environments that run on both pull requests and merge to main
name: automated-testing

on:
push:
branches: [production, develop]
pull_request:
branches: [production, develop]

jobs:
test:
strategy:
# fail-fast: false
matrix:
platform: [ubuntu-latest, macos-latest, windows-latest]
python-version: [3.7, 3.9]
# you can separate the tox-envs in different .yml files
# see version 0.10.1
# https://github.com/joaomcteixeira/python-project-skeleton/releases/tag/v0.10.1
# below the list of tox envs that you wish to include in the matrix
tox-envs: [lint, test]
# workaround to ignore 'pr' when merging to main
# see: https://github.community/t/how-to-conditionally-include-exclude-items-in-matrix-eg-based-on-branch/16853/6
isMerge:
- ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
exclude:
- isMerge: true
tox-envs: pr

runs-on: ${{ matrix.platform }}

steps:
- uses: actions/checkout@v2

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
python -m pip install --upgrade pip setuptools wheel
pip install tox
- name: unit tests
run: tox -e ${{ matrix.tox-envs }}
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Add pre-commit and setup pre-commit-hooks [(#56)](https://github.com/rl-institut/super-repo/pull/56)
- Add a community chat on Element [(#59)](https://github.com/rl-institut/super-repo/pull/59)
- Add documentation on the regular community meetings [(#60)](https://github.com/rl-institut/super-repo/pull/60)
- Add CI-based automated testing and linting on commit and pull requests [(#55)](https://github.com/rl-institut/super-repo/pull/55)

### Changed

Expand Down
2 changes: 1 addition & 1 deletion README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ Introduction
============
| A template repo to test and document elements and features for research software.
| It implements the collection of repository elements collected in this `Etherpad <https://etherpad.wikimedia.org/p/super-repo>`_.
| The goal is to simplify and standardise the creation of software in GitHub repositories.
| The goal is to simplify and standardize the creation of software in GitHub repositories.
Documentation
=============
Expand Down
Original file line number Diff line number Diff line change
@@ -1,15 +1,36 @@
# Test-Driven Development

The Test-Driven Development can be used for any code project. The following example shows how to implement the testing technique in a Djnago app.
The Test-Driven Development can be used for any code project.

When adding new functionality or features, write tests for them first before implementing them. This ensures that your changes don't affect existing functionality and that all functions work as expected.
When adding new functionality or features, write tests for them first before implementing the new functionality. This ensures that your changes don't affect existing functionality and that all functions work as expected.

You can run test in django by running:
`python manage.py test`
## Example: Implement a test case and functionality to test a django app

1 First, make sure you have an Django app that you want to test.
The following example shows how to implement the testing technique in a Django app.

2 Next, write a test for a function or feature you want to implement. Let's say you want to implement a view that returns a list of objects from the database. You also need a model to represent the database table object. A possible test could look like this:
!!! Info
You can run all test in django by running:

```bash
python manage.py test
```

To tun specific test cases you have to provide the app name followed by the name of the test case.
In the example below the command would look like this:

```bash
python manage.py test app.MyModelViewTestCase
```

### 1. Step

First, make sure you have an Django app that you want to test.

### 2. Step

Next, write a test for a function or feature you want to implement. Let's say you want to implement a view that returns a list of objects from the database. You also need a model to represent the database table object. A possible test could look like this:

app/tests/my_test.py

```python
from django.test import TestCase
Expand All @@ -26,9 +47,17 @@ class MyModelViewTestCase(TestCase):

```

3 Run the test to make sure it fails because the view has not been implemented yet.
### 3. Step

4 Implement the model and view to make the test pass.
Run the test to make sure it fails because the view has not been implemented yet.

```bash
python manage.py test
```

### 4. Step

Implement the model and view to make the test pass.

models.py

Expand All @@ -55,9 +84,17 @@ class MyModelListView(ListView):

```

5 Run the test again to make sure it passes.
### 5. Step

Run the test again to make sure it passes.

```bash
python manage.py test
```

6 Write additional tests to make sure the view works as expected.
### 6. Step

Write additional tests to make sure the view works as expected.

```python
class MyModelViewTestCase(TestCase):
Expand All @@ -77,4 +114,10 @@ class MyModelViewTestCase(TestCase):

```

7 Run all the tests to make sure they all pass.
### 7. Step

Run all the tests to make sure they all pass.

```bash
python manage.py test
```
98 changes: 98 additions & 0 deletions docs/development/continuous-integration/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# Continuous Integration

Continuous Integration (CI) is a good practice in software development that
involves automated testing of code, ensuring repository quality, reporting, and
automate deployments. <br>
The process involves the utilization of multiple technologies.
Once the setup is complete, it's possible to create new test and
also to interpret the output of success or error messages as response from the
test-execution. A brief summary of this process is also provided for reference.

## Automation

The package [tox](https://github.com/tox-dev/tox) automates the process of testing
Python projects. It provides support for creating and managing test environments
across cross different environments, Python versions, and dependencies. <br>
It ensures that the code is reliable, functional, and compatible on different
computers leading to higher code quality and more robust software.
Combined with GitHub Actions, it allows you to automate a wide range of tasks.

### Key features and functionalities

For each specified Python version, tox creates isolated virtual environments.
This is important because different projects might have different dependencies
or require specific Python versions and run on different platforms like Windows, Linux, or MacOS.
Isolation prevents conflicts between packages used by different projects.

1. **Virtual Environments**: Tox creates isolated virtual environments for each specified Python version. This is crucial because different projects might have different dependencies or require specific Python versions. Isolation prevents conflicts between packages used by different projects.
2. **Automated Testing**: Tox automates the entire testing process. It can be configured to run various types of tests, such as unit tests, integration tests, and even style checks. This ensures that the project is thoroughly tested in different environments without manual intervention.
3. **Cross-Version and Cross-Platform Testing**: Tox enables you to test your code against multiple versions of Python, including Python 2 and Python 3. Additionally, it supports testing on various operating systems, allowing you to identify and address compatibility issues specific to different platforms.
4. **Dependency Management**: Tox can help manage project dependencies by allowing you to specify different sets of dependencies for different testing environments. This helps ensure that your code works with different combinations of dependencies.
5. **Configuration via tox.ini**: Tox uses a configuration file named `tox.ini` to define testing environments, dependencies, and testing commands. This file also allows you to set up various testing environments, customize test commands, and control other testing-related behaviors.
6. **Consistency and Reproducibility**: Tox aids in maintaining consistency and reproducibility in your testing process. By setting up a standard testing workflow and environments, you reduce the risk of inconsistent results due to differences in developers' local setups.
7. **Continuous Integration (CI)**: Tox is often used in conjunction with CI systems like Jenkins, Travis CI, CircleCI, and GitHub Actions. By incorporating Tox into your CI pipeline, you can ensure that your code is automatically tested in multiple environments whenever changes are pushed to the repository.
8. **Plugin System**: Tox has a plugin system that allows for extensibility. This means you can add custom functionality to the testing process, such as additional test runners, code analysis tools, and more.

## Install

Install the required package [`tox`](https://tox.wiki/en/stable/installation.html) in a python environment. <br>
💻 `pip install tox` install tox <br>

## Setup

Configuring `tox` may involve some complexity. <br>
See the official [tox user guide](https://tox.wiki/en/stable/user_guide.html) for further information.

All test stages are setup in the configuration file `tox.ini`.
This file configures all test environments as well as the tools and configuration
used to run tests and checks.

## Testing

### pytest

For smaller projects or a more straightforward approach,
testing can also be effectively accomplished using [pytest](https://github.com/pytest-dev/pytest/).
This allows for efficient testing without the additional complexity of setting
up and managing multiple environments.

### black

Checking code syntax with `black`

### flake8

Verifying `docstrings` with `flake8`

### isort

Sorting imports with `isort`

### Django tests

Testing Django projects with [Django tests](https://docs.djangoproject.com/en/3.2/topics/testing/).

## GitHub Actions

GitHub offers an integrated CI called `GitHub Actions`,
which enables the definition of test workflows and events.
These workflows can be executed, for instance,
with every pull request or new commit on a specific branch.
To set up workflows for your repository, follow the [Guides for GitHub Actions](https://docs.github.com/en/actions/guides).

### Automated Testing

All tests are defined within the workflow file `.github/workflows/automated-testing.yml`.
The workflow defines the tasks for a action, and a runner executes these tasks on a
virtual machine provided by GitHub.

The workflow is triggered whenever there's a pull request or a commit to the `develop` or `production` branches.
The workflow prepares the Python environment and installs relevant dependencies before executing the `tox` automation suite.
If any errors are detected, the workflow fails and generates a detailed report for developers to review.

### Building and deploying the documentation

Another use case that involves the GitHub-Actions is the automated deployment of the documentation.
While all minor versions are manually deployed using [mike](https://rl-institut.github.io/super-repo/latest/development/documentation/#mike).
GitHub-Actions workflow builds and deploys the `develop` branch.
This ensures that the documentation is automatically updated if a new feature or update is available.
24 changes: 24 additions & 0 deletions docs/development/continuous-integration/setup_github_actions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Configure Continuous Integration using GitHub Actions

## What is included in the tests?

Our continuous integration (CI) process executes a series of tests, including
running `pytest` and performing linting and code quality checks.

Here's an overview of the tasks involved in our example:

- Running `pytest`
- Checking code syntax with `black`
- Verifying `docstrings` with `flake8`
- Sorting imports with `isort`

All these tools are executed as part of a job defined within a GitHub Actions workflow file.
The workflow defines the tasks for a job, and a runner executes these tasks on a
virtual machine provided by GitHub.

!!! Info "Setup GitHub Actions"
To set up GitHub Actions workflows for your repository, follow the [official GitHub guide](https://docs.github.com/en/actions/guides).

Our CI workflow is triggered whenever there's a pull request or a commit to the `develop` or `production` branches.
The workflow prepares the Python environment and installs relevant dependencies before executing the `tox` automation suite.
If any errors are detected, the workflow fails and generates a detailed report for developers to review.
65 changes: 65 additions & 0 deletions docs/development/continuous-integration/setup_tox.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Configure Tox test-automation

To streamline our test environment setup, we use `tox`, a powerful tool that
automates testing across different environments.

!!! Note
[Tox](https://tox.wiki/en/stable/) is a powerful tool that simplifies
and automates the process of testing Python projects across different environments.
It ensures that your code is reliable, functional, and compatible with various
Python versions and dependencies, ultimately leading to higher code quality
and more robust software. Combined with a runner like GitHub Actions,
it allows you to automate a wide range of tasks.

??? Info "Key features and functionalities"

Tox creates isolated virtual environments for each specified Python version. This is crucial because different projects might have different dependencies or require specific Python versions. Isolation prevents conflicts between packages used by different projects.

1. **Virtual Environments**: Tox creates isolated virtual environments for each specified Python version. This is crucial because different projects might have different dependencies or require specific Python versions. Isolation prevents conflicts between packages used by different projects.

2. **Automated Testing**: Tox automates the entire testing process. It can be configured to run various types of tests, such as unit tests, integration tests, and even style checks. This ensures that the project is thoroughly tested in different environments without manual intervention.

3. **Cross-Version and Cross-Platform Testing**: Tox enables you to test your code against multiple versions of Python, including Python 2 and Python 3. Additionally, it supports testing on various operating systems, allowing you to identify and address compatibility issues specific to different platforms.

4. **Dependency Management**: Tox can help manage project dependencies by allowing you to specify different sets of dependencies for different testing environments. This helps ensure that your code works with different combinations of dependencies.

5. **Configuration via tox.ini**: Tox uses a configuration file named `tox.ini` to define testing environments, dependencies, and testing commands. This file also allows you to set up various testing environments, customize test commands, and control other testing-related behaviors.

6. **Consistency and Reproducibility**: Tox aids in maintaining consistency and reproducibility in your testing process. By setting up a standard testing workflow and environments, you reduce the risk of inconsistent results due to differences in developers' local setups.

7. **Continuous Integration (CI) Integration**: Tox is often used in conjunction with CI systems like Jenkins, Travis CI, CircleCI, and GitHub Actions. By incorporating Tox into your CI pipeline, you can ensure that your code is automatically tested in multiple environments whenever changes are pushed to the repository.

8. **Plugin System**: Tox has a plugin system that allows for extensibility. This means you can add custom functionality to the testing process, such as additional test runners, code analysis tools, and more.

## What is included in the tests?

Here's an overview of the testing-tasks involved:

- Running `pytest`
- Checking code syntax with `black`
- Verifying `docstrings` with `flake8`
- Sorting imports with `isort`

!!! Info "Run test on multiple platforms"
Our Tox environments are executed on all major platforms including windows, linux and macos.

## Installation

Begin by installing [`tox`](https://tox.wiki/en/stable/installation.html):

```bash
pip install tox
```

!!! Note
Setup and activate a [python environment](https://docs.python.org/3/library/venv.html) first.

## Setup

Configuring `tox` may involve some complexity.
Start by defining what tests you want to run and what dependencies
(such as packages or external services) are required for the tests.
Organize these test tasks into logical steps.

!!! Info "Tox User Guide"
The official [tox User Guide](https://tox.wiki/en/stable/user_guide.html)
30 changes: 30 additions & 0 deletions docs/development/continuous-integration/usage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Usage

## Add new code test

To add a new test for any new functionality you have added or you plan to add you should add a test.
The test is imply added to the tests/
directory.

See our examples ....

## Debugging

### Local Debugging

You can run your automation suite locally by typing the following command in your console. This is useful for verifying the outcome of your automation pipeline before pushing your code:

```bash
tox
```

Running this command will generate a comprehensive report, which may require some effort to debug.

### Remote Debugging on GitHub

For remote debugging, you can monitor the status of your automation jobs in two ways:

1. Create a pull request and view the CI status integration there.
2. Visit the [Actions tab of your repository](https://github.com/rl-institut/super-repo/actions) to see all available job executions.

These options allow you to closely monitor the progress and outcomes of your CI workflow, ensuring the reliability and quality of your codebase.
6 changes: 3 additions & 3 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -73,9 +73,9 @@ nav:
- development/git/index.md
- Gitignore: development/git/gitignore.md
- Pre-commit: development/git/pre-commit.md
- Best-Practice:
- development/best-practice/index.md
- Test-Driven-Development: development/best-practice/test-driven-developement-workflow.md
- Continuous Integration:
- development/continuous-integration/index.md
- Test-Driven-Development: development/best-practice/test-driven-development-workflow.md
- Documentation:
- development/documentation/index.md
- Release:
Expand Down
Loading

0 comments on commit 6ad23b3

Please sign in to comment.