You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Linter helps and advises us about code quality by running sanity checks and displaying warnings and errors about code smells. Also, potentially it helps to prevent bugs in a project.
1
+
Linter helps and advises us about code quality by running sanity checks and displaying warnings and errors about code
2
+
smells. Also, potentially it helps to prevent bugs in a project.
2
3
3
4
As we are in FreshBooks using GitHub, so we would like to use it as much as possible.
4
5
5
6
Recently I was involved in configuring linters as a part of CI/CD in GitHub actions.
6
7
7
8
## Linters configuration
8
9
9
-
I would like to share how to configure it for the python project. I prepared a full [
I would like to share how to configure it for the python project. I prepared a full [github actions python configuration demo repository](https://github.com/iamtodor/github-actions-python-demo).
11
11
12
-
We use flakeheaven as flake8 wrapper, which is very easy to configure in one single `pyproject.toml` configuration file. The whole `pyproject.toml` configuration file could be found in a [repo](https://github.com/iamtodor/github-actions-python-configuration-demo/blob/main/pyproject.toml).
12
+
We use flakeheaven as flake8 wrapper, which is very easy to configure in one single `pyproject.toml` configuration file.
13
+
The whole `pyproject.toml` configuration file could be found in
14
+
a [repo](https://github.com/iamtodor/github-actions-python-configuration-demo/blob/main/pyproject.toml).
Disclaimer: author assumes you are familiar with above-mentioned linters, tools, and checks. I would say the config file is self-explainable, so I will not stop here for a while. Just a few notes about tiny tweaks.
18
+
Disclaimer: author assumes you are familiar with above-mentioned linters, tools, and checks. I would say the config file
19
+
is self-explainable, so I will not stop here for a while. Just a few notes about tiny tweaks.
17
20
18
21
A few checks that we don't want to see complain about:
19
22
20
23
### Documentation
21
24
22
25
We are ok if not every module will be documented. We are ok if not every function or method will be documented.
Our linter requirements live in a separate file, and we don't aim to mix it with our main production requirements. Hence, linter could complain about import libraries as linter env does not have production libraries, quite obvious. So we need to disable this check. We assume that the developer who writes the code and imports the libs is responsible for the tests. So if the test does not pass it means that it's something with import or a code itself. Import checks it's not something we would like to put as a linter job.
31
+
Our linter requirements live in a separate file, and we don't aim to mix it with our main production requirements.
32
+
Hence, linter could complain about import libraries as linter env does not have production libraries, quite obvious. So
33
+
we need to disable this check. We assume that the developer who writes the code and imports the libs is responsible for
34
+
the tests. So if the test does not pass it means that it's something with import or a code itself. Import checks it's
35
+
not something we would like to put as a linter job.
Then we want to have each task be specified in a new line, hence we need to disable `W503` from pycodestyle: Disable line break before binary operator.
50
+
dags/dummy.py
51
+
17: 9 W503 line break before binary operator [pycodestyle]
52
+
>> dummy_operator_2
53
+
^
54
+
18: 9 W503 line break before binary operator [pycodestyle]
55
+
>> dummy_operator_3
56
+
^
57
+
19: 9 W503 line break before binary operator [pycodestyle]
Disclaimer: author assumes you are familiar with [GitHub actions](https://github.com/features/actions)
85
+
**Disclaimer**: author assumes you are familiar with [GitHub actions](https://github.com/features/actions).
51
86
52
87
We configure GitHub Workflow to be triggered on every PR against main (master) branch.
53
88
@@ -58,26 +93,51 @@ Here are the linters and checks we are going to use:
58
93
-[black](https://github.com/psf/black)
59
94
-[isort](https://github.com/PyCQA/isort)
60
95
61
-
The whole `py_linter.yml` config could be found in a [repo](https://github.com/iamtodor/github-actions-python-demo/blob/main/.github/workflows/py_linter.yml). We will walk thru it step by step.
96
+
The whole `py_linter.yml` config could be found in
97
+
a [repo](https://github.com/iamtodor/github-actions-python-demo/blob/main/.github/workflows/py_linter.yml). I will walk you thru it step by step.
We are interested in running linter only when PR has `.py` files. For instance, when we update `README.md` there is no sense to run python linter.
66
102
67
-
103
+
68
104
69
105
We are interested in running a linter only against the modified files. Let's say, we take a look at the provided repo, if I update `dags/dummy.py` I don't want to waste a time and resources running linter against `main.py`. For this purpose we use [Paths Filter GitHub Action](https://github.com/dorny/paths-filter), that is very flexible.
If we have in one PR modified `.py` and any other files such as `.toml`, we don't want to run linter against not `.py`, so we use where we configured filtering only for `.py` files no matter its location: root, tests, src, etc.
109
+
If we have in one PR modified `.py` and any other files such as `.toml`, we don't want to run linter against not `.py`, so we use where we configured filtering only for `.py` files no matter its location: root, tests, src, etc.
74
110
75
111
Changed file can have the following statuses `added`, `modified`, or `deleted`. There is no reason to run a linter against deleted file as your workflow would simply fail, because there is no more that particular changed file in repo. So we need to configure what changes we consider to trigger linter.
I define the variable where I can find the output (the only `.py` files) from the previous filter. This variable would contain modified `.py` files, that I can further pass to a `flakeheaven`, `black`, and `isort`. By default, the output is disabled, and Paths Changes Filter allows to customize it: you can list the files in `.csv`, `.json` or in a `shell` mode. Linters accept files separated simply by space, so our choise here is `shell` mode.
115
+
I define the variable where I can find the output (the only `.py` files) from the previous filter. This variable would contain modified `.py` files, that I can further pass to a `flakeheaven`, `black`, and `isort`. By default, the output is disabled, and Paths Changes Filter allows to customize it: you can list the files in `.csv`, `.json` or in a `shell` mode. Linters accept files separated simply by space, so our choice here is `shell` mode.
Before we run linter on changed files we run a check if there is an actual changes in `.py` files, if there are any `.py` files from the previous step than we can use.
126
+
127
+
128
+
129
+
Next, using the before-mentioned output variable we can safety pass the content from this `steps.filter.outputs.py_scripts_filter_files` variable to linter.
0 commit comments