Skip to content

Commit 4e984e9

Browse files
willbartonwillbarton
committed
Add documentation for tracking changes to Django and Wagtail
In this change I have attempted to document the process we've undertaken to upgrade Django and Wagtail. I've moved the supported-versions matrix from the Python standard doc and makes it more concrete to apply to Django and Wagtail and cf.gov and its satellites and libraries. I've documented the approach we have been taking to feature detection and version testing, and the actual approach to keeping satellites, libraries, and cf.gov itself up-to-date. It's a little bit ARID with the changes to the Python standards doc and tox, but that's intentional.
1 parent f7d9614 commit 4e984e9

File tree

2 files changed

+145
-26
lines changed

2 files changed

+145
-26
lines changed

‎guides/tracking-django-wagtail.md‎

Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
# Tracking Django and Wagtail
2+
3+
- [Philosophy](#philosophy)
4+
- [Considerations when supporting multiple versions](#considerations-when-supporting-multiple-versions)
5+
- [Consult the release notes](#consult-the-release-notes)
6+
- [Use feature detection](#use-feature-detection)
7+
- [Imports](#imports)
8+
- [Version detection](#version-detection)
9+
- [Matrix testing](#matrix-testing)
10+
- [Upgrade process](#upgrade-process)
11+
- [Ensure libraries and satellite apps support required versions](#ensure-libraries-and-satellite-apps-support-required-versions)
12+
- [Ensure consumerfinance.gov supports required versions](#ensure-consumerfinancegov-supports-required-versions)
13+
- [Upgrading consumerfinance.gov](#upgrading-consumerfinancegov)
14+
15+
16+
17+
## Philosophy
18+
19+
We have two general types of Python projects:
20+
21+
- The running [consumerfinance.gov](https://github.com/cfpb/cfgov-refresh) website
22+
- Satellite applications of consumerfinance.gov that live in their own repository, e.g., our satellite apps of consumerfinance.gov, like [ccdb5-ui](https://github.com/cfpb/ccdb5-ui)
23+
- Libraries that provide some functionality independent of consumerfinance.gov, e.g., [wagtail-sharing](https://github.com/cfpb/wagtail-sharing)
24+
25+
We use a rubric like the one below to determine which versions of Python and major dependencies should be supported.
26+
27+
|| Library | consumerfinance.gov | Satellite application |
28+
|--|--|--|--|
29+
|Oldest Python 3 supported by Django LTS | ✔️| | |
30+
|Current production Python | | ✔️ | ✔️|
31+
|Latest Python | ✔️ | ✔️ | ✔️|
32+
|Latest Django LTS version | ✔️ | ✔️ | ✔️|
33+
|Latest Django version | ✔️ | ✔️ | ✔️|
34+
|Latest Wagtail LTS version | ✔️ | ✔️ | ✔️|
35+
|Latest Wagtail version | ✔️ | ✔️ | ✔️|
36+
37+
consumerfinance.gov and its satellite apps should support the version of Python 3 we currently run in production and the latest version. At the time of writing that is version 3.6 and 3.8.
38+
39+
Our libraries should support the oldest version of Python 3 that the library's major dependencies supports, and the latest version of Python. For example, [Django 2.2 supports Python 3.5 and up](https://docs.djangoproject.com/en/2.2/releases/2.2/#python-compatibility), so our Django and Wagtail libraries that support Django 2.2 should do the same.
40+
41+
Both consumerfinance.gov and its satellite apps, and our libraries should support the LTS (long-term support) version of Django and Wagtail that we currently run in production, and the latest version (LTS or not) of Django and Wagtail.
42+
43+
## Considerations when supporting multiple versions
44+
45+
### Consult the release notes
46+
47+
Django and Wagtail are generally very good about listing upgrading considerations, deprecations, and breaking changes in their release notes. The release notes should drive the changes that we make.
48+
49+
### Use feature detection
50+
51+
When dealing with support for multiple versions of Python or major dependencies, we prefer to [use feature detection instead of version detection](https://docs.python.org/3/howto/pyporting.html#use-feature-detection-instead-of-version-detection) for incompatibilities/breaking changes that may exist between versions. Feature detection `try`/`except` blocks should always attempt to use **new** code first and then fall back on old code.
52+
53+
### Imports
54+
55+
To detect features on import, or import paths have changed between versions, we prefer to use `try`/`except` to import the **new** version and fall back on the old version:
56+
57+
```python
58+
try:
59+
# Django >= 2.0
60+
from django.urls import include
61+
except ImportError:
62+
# Django < 2.0
63+
from django.conf.urls import include
64+
```
65+
66+
### Version detection
67+
68+
Only when [feature-detection](https://docs.python.org/3/howto/pyporting.html#use-feature-detection-instead-of-version-detection) is not sufficient to support multiple versions of a dependency do we fall back on version-checking. This can happen when APIs change but the naming conventions remain the same, and exception-handling is insufficient or too greedy to detect the difference between an API change and another reason an execption may be raised.
69+
70+
When using version detection, we prefer `if`/`else` conditionals with the **new** API used in the `if`, falling back on the old API in the `else`. For example:
71+
72+
```python
73+
import django
74+
75+
if django.VERSION >= (2, 0, 0):
76+
is_anonymous = bool(request.user.is_anonymous)
77+
else:
78+
is_anonymous = bool(request.user.is_anonymous())
79+
```
80+
81+
When performing a version check, the hard-coded version should always be **inclusive** of the first version to support the new API and should specify the entire semantic version.
82+
83+
Both Django and Wagtail provide versions as tuples of (major version, minor version, patch version, sub-version, sub-version release). For example:
84+
85+
```python
86+
>>> import django
87+
>>> django.VERSION
88+
(2, 2, 13, 'final', 0)
89+
```
90+
91+
Version-detection of Django and Wagtail should specify `(major, minor, patch)`.
92+
93+
### Matrix testing
94+
95+
We use [tox](https://tox.readthedocs.io/en/latest/) for matrix testing against all the versions listed above, and we provide a [sample tox configuration](../tox.ini) that tests against both our current production the latest versions Python and Django suitable as a basis for satellite apps and libraries.
96+
97+
## Upgrade process
98+
99+
The upgrade process for consumerfinance.gov is fairly straight-forward, and reasonably similar for all three categories of projects we maintain.
100+
101+
### Ensure libraries and satellite apps support required versions
102+
103+
1. Add support for the new version(s) to the `tox` matrix
104+
2. Consult release notes of the version(s) being added and make necessary code changes
105+
3. Fix any tests that break as a result
106+
4. Ensure that running functionality works as expected
107+
5. Clean up support for any previous versions that can be dropped
108+
6. Create a new release
109+
110+
New releases of libraries should be automatically uploaded to PyPI by our GitHub Actions.
111+
112+
New releases of satellite apps should be automatically added to their GitHub release page by our GitHub Actions.
113+
114+
### Ensure consumerfinance.gov supports required versions
115+
116+
consumerfinance.gov's [tox.ini]() matrixes on "current" vs "future" versions rather than on specific version numbers, and unlike our satellite apps and libraries pins its dependencies to specific versions. This can create incompatibilities that we need to deal with.
117+
118+
1. Add support for the new version(s) to the `tox` `future-config` matrix configuration.
119+
120+
Include new pins of any required libraries. For example, if a future Wagtail requires an update to BeautifulSoup that is incompatible with our current Wagtail's dependency on BeautifulSoup, pin it in the tox `future-config` `deps`:
121+
122+
```ini
123+
[future-config]
124+
…
125+
deps=
126+
Django>=2.2,<3.0
127+
wagtail>=2.10,<2.11
128+
beautifulsoup4>=4.8.2
129+
```
130+
131+
2. Pin any new releases of satellite apps or libraries in the [requirements](https://github.com/cfpb/cfgov-refresh/blob/master/requirements/libraries.txt) file.
132+
3. Consult release notes of the version(s) being added and make necessary code changes
133+
4. Fix any tests that break as a result
134+
5. Ensure that running functionality works as expected
135+
6. Clean up support for any previous versions that can be dropped
136+
137+
### Upgrading consumerfinance.gov
138+
139+
When we have a new LTS version of Django, or when there's a new release of Wagtail to deploy, the process of upgrading consumerfinance.gov should be simple, because we already have been tracking support for those versions.
140+
141+
1. Pin [Django](https://github.com/cfpb/cfgov-refresh/blob/master/requirements/django.txt) and [Wagtail](https://github.com/cfpb/cfgov-refresh/blob/master/requirements/wagtail.txt) in their respective requirements files.
142+
2. Update the pins of any libraries that we pinned in the tox `future-config` `deps` in the [libraries](https://github.com/cfpb/cfgov-refresh/blob/master/requirements/libraries.txt) requirements file.

‎standards/python.md‎

Lines changed: 3 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
- [Style](#style)
44
- [Linting](#linting)
55
- [Imports](#imports)
6-
- [Dependency support/testing matrix](#dependency-supporttesting-matrix)
6+
- [Dependency support and matrix testing](#dependency-support-and-matrix-testing)
77
- [Considerations](#considerations)
88
- [Specifying dependencies in `requirements.txt` or `setup.py`](#specifying-dependencies-in-requirementstxt-or-setuppy)
99

@@ -42,32 +42,9 @@ We use [isort](https://github.com/timothycrosley/isort) to lint imports to compl
4242

4343
The isort configuration can go into a `tox.ini` file under an `[isort]` header.
4444

45-
## Dependency support/testing matrix
45+
## Dependency support and matrix testing
4646

47-
We have two general types of Python projects:
48-
49-
- Applications that have a running instance, e.g., [cfgov-refresh](https://github.com/cfpb/cfgov-refresh)
50-
- Libraries that provide some functionality independent of applications
51-
52-
We use a rubric like the one below to determine which versions of Python and major dependencies should be supported. Assuming the project has Django and Wagtail as its major dependencies, our recommendation would be this:
53-
54-
|| Library | Application |
55-
|--|--|--|
56-
|Oldest Python 3 supported by Django LTS | ✔️| |
57-
|Current production Python | | ✔️ |
58-
|Latest Python | ✔️ | ✔️ |
59-
|Latest Django LTS version | ✔️ | ✔️ |
60-
|Latest Django version | ✔️ | ✔️ |
61-
|Latest Wagtail LTS version | ✔️ | ✔️ |
62-
|Latest Wagtail version | ✔️ | ✔️ |
63-
64-
Our applications should support the version of Python 3 we currently run in production and the latest version. For example, [cfgov-refresh](https://github.com/cfpb/cfgov-refresh) should support version 3.6 and 3.8 at time of writing.
65-
66-
Our libraries should support the oldest version of Python 3 that the library's major dependencies supports, and the latest version of Python. For example, [Django 2.2 supports Python 3.5 and up](https://docs.djangoproject.com/en/2.2/releases/2.2/#python-compatibility), so our Django and Wagtail libraries that support Django 2.2 should do the same.
67-
68-
"Major dependencies" are the frameworks upon which our applications and libraries are built. Examples are Django and Wagtail. Both our applications and our libraries should support the LTS (long-term support) version of that dependency we currently run in production, and the latest version (LTS or not) of that dependency.
69-
70-
### Considerations
47+
### Matrix testing
7148

7249
We recommend using [tox](https://tox.readthedocs.io/en/latest/) for matrix testing against all the versions listed above, and we provide a [sample tox configuration](../tox.ini) that tests against both our current production the latest versions Python and Django.
7350

0 commit comments

Comments
 (0)