Skip to content

Commit

Permalink
improving docs
Browse files Browse the repository at this point in the history
  • Loading branch information
mistercrunch committed Jun 28, 2024
1 parent c0501e3 commit 45fe467
Show file tree
Hide file tree
Showing 2 changed files with 80 additions and 20 deletions.
98 changes: 79 additions & 19 deletions docs/docs/contributing/development.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -441,50 +441,110 @@ If using the eslint extension with vscode, put the following in your workspace `
]
```
## Testing
### Python Testing
## GitHub Actions and `act`
For automation and CI/CD, Superset makes extensive use of GitHub Actions (GHA). You
can find all of the workflows and other assets under the `.github/` folder. This includes:
- running the backend unit test suites (`tests/`)
- running the frontend test suites (`superset-frontend/src/**.*.test.*`)
- running our Cypress end-to-end tests (`superset-frontend/cypress-base/`)
- linting the codebase, including all Python, Typescript and Javascript, yaml and beyond
- checking for all sorts of other rules conventions
When you open a pull request (PR), the appropriate GitHub Actions (GHA) workflows will
automatically run depending on the changes in your branch. It's perfectly reasonable
(and required!) to rely on this automation. However, the downside is that it's mostly an
all-or-nothing approach and doesn't provide much control to target specific tests or
iterate quickly.
All python tests are carried out in [tox](https://tox.readthedocs.io/en/latest/index.html)
a standardized testing framework.
All python tests can be run with any of the tox [environments](https://tox.readthedocs.io/en/latest/example/basic.html#a-simple-tox-ini-default-environments), via,
At times, it may be more convenient to run GHA workflows locally. For that purpose
we use [act](https://github.com/nektos/act), a tool that allows you to run GitHub Actions (GHA)
workflows locally. It simulates the GitHub Actions environment, enabling developers to
test and debug workflows on their local machines before pushing changes to the repository. More
on how to use it in the next section.
:::note
In both GHA and `act`, we can run a more complex matrix for our tests, executing against different
database engines (PostgreSQL, MySQL, SQLite) and different versions of Python.
This enables us to ensure compatibility and stability across various environments.
:::
### Lower-Level Control
Note that for all GHAs, it's also possible to go lower level and the content of any workflow
more directly on a local environment of your own making.
If you prefer this approach for the control it gives you, it's possible to leverage the Docker Compose setup
as a backend to run integration tests. As to how to run specific workflows or individual steps
from said workflows, you can simply refer to the workflows themselves and reverse-engineer them
as they act as a comprehensive recipe for lower level execution.
### Using `act`
First, install `act` -> https://nektosact.com/
To list the workflows, simply:
```bash
tox -e <environment>
act --list
```
For example,
To run a specific workflow:
```bash
tox -e py38
act --job {workflow_name} --secret GITHUB_TOKEN=$GITHUB_TOKEN --event pull_request --container-architecture linux/amd64
```
Alternatively, you can run all tests in a single file via,
In the example above, notice that:
- we target a specific workflow, using `--job`
- we pass a secret using `--secret`, as many jobs require read access (public) to the repo
- we simulate a `pull_request` event using `--event`, here we could simulate a
`push` event or something else
- we specify `--container-architecture`, which tends to emulate GHA more reliably
:::note
`act` is a rich tool that offers all sorts of features, allowing you to simulate different
events (pull_request, push, ...), semantics around passing secrets where required and much
more. For more information, refer to [act's documentation](https://nektosact.com/)
:::
---
## Testing
### Python Testing
#### Unit Tests
For unit tests located in `tests/unit_tests/`, it's usually easy to simply run the script locally using:
```bash
tox -e <environment> -- tests/test_file.py
pytest tests/unit_tests/*
```
or for a specific test via,
#### Integration Tests
For more complex pytest-defined integration tests (not to be confused with our end-to-end Cypress tests), many tests will require having a working test environment. Some tests require a database, Celery, and potentially other services or libraries installed.
### Running Tests with `act`
To run integration tests locally using `act`, ensure you have followed the setup instructions from the [GitHub Actions and `act`](#github-actions-and-act) section. You can run specific workflows or jobs that include integration tests. For example:
```bash
tox -e <environment> -- tests/test_file.py::TestClassName::test_method_name
act --job test-python-38 --secret GITHUB_TOKEN=$GITHUB_TOKEN --event pull_request --container-architecture linux/amd64
```
Note that the test environment uses a temporary directory for defining the
SQLite databases which will be cleared each time before the group of test
commands are invoked.
#### Running locally using a test script
There is also a utility script included in the Superset codebase to run python integration tests. The [readme can be
found here](https://github.com/apache/superset/tree/master/scripts/tests)
There is also a utility script included in the Superset codebase to run Python integration tests. The [readme can be found here](https://github.com/apache/superset/tree/master/scripts/tests).
To run all integration tests for example, run this script from the root directory:
To run all integration tests, for example, run this script from the root directory:
```bash
scripts/tests/run.sh
```
You can run unit tests found in './tests/unit_tests' for example with pytest. It is a simple way to run an isolated test that doesn't need any database setup
You can run unit tests found in `./tests/unit_tests` with pytest. It is a simple way to run an isolated test that doesn't need any database setup:
```bash
pytest ./link_to_test.py
Expand Down
2 changes: 1 addition & 1 deletion requirements/development.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
# pip-compile-multi
#
-r base.txt
-e file:///Users/max/code/superset
-e file:.
# via
# -r requirements/base.in
# -r requirements/development.in
Expand Down

0 comments on commit 45fe467

Please sign in to comment.