python-connector-base-image: upgrade to python 3.9.19 + update setuptools and pip #38859
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
|
This stack of pull requests is managed by Graphite. Learn more about stacking. Join @alafanechere and the rest of your teammates on |
alafanechere
force-pushed
the
augustin/06-03-python-connector-base-image_upgrade_to_python_3.9.19_update_setuptools_and_pip
branch
from
2d93da1
to
44e9e2f
Compare
| RUN ln -snf /usr/share/zoneinfo/Etc/UTC /etc/localtime | ||
| RUN pip install --upgrade pip==23.2.1 | ||
| RUN pip install --upgrade pip==24.0 setuptools==70.0.0 |
There was a problem hiding this comment.
I don't like that we have the versions hardcoded here in code. But, not blocking, definitely works for us well now.
There was a problem hiding this comment.
We hardcode versions for build reproducibility. If we don't do that a rebuild without any change might end up with different versions... Which can be surprising and lead to unexpected side effets.
| ```dockerfile | ||
| FROM docker.io/python:3.9.18-slim-bookworm@sha256:44b7f161ed03f85e96d423b9916cdc8cb0509fb970fd643bdbc9896d49e1cad0 | ||
| FROM docker.io/python:3.9.19-slim-bookworm@sha256:b92e6f45b58d9cafacc38563e946f8d249d850db862cbbd8befcf7f49eef8209 |
There was a problem hiding this comment.
What are our options for changing to python:3.9.19-alpine?
Debian moves a bit slower and is a bit bigger. Pretty big difference in vulnerability size
sh# grype python:3.9.19-slim-bookworm
✔ Vulnerability DB [no update available]
✔ Pulled image
✔ Loaded image python:3.9.19-slim-bookworm
✔ Parsed image sha256:182cc99a2af6b2eb20b8da7e1a0d3661424b8fa7a0f0fbdac147e5e4ca8a3005
✔ Cataloged contents 34f145fff6b64d6199becdf2f8d67d2810b54df9d610a14f0c1989ab39d0c01e
├── ✔ Packages [118 packages]
├── ✔ File digests [2,954 files]
├── ✔ File metadata [2,954 locations]
└── ✔ Executables [809 executables]
✔ Scanned for vulnerabilities [118 vulnerability matches]
├── by severity: 1 critical, 7 high, 25 medium, 3 low, 55 negligible (27 unknown)
└── by status: 2 fixed, 116 not-fixed, 0 ignored
vs
sh# grype python:3.9.19-alpine
✔ Vulnerability DB [no update available]
✔ Pulled image
✔ Loaded image python:3.9.19-alpine
✔ Parsed image sha256:a3003c79447cb9c6aa127216f7dd3fe0c746723003effce38674cff406bcee25
✔ Cataloged contents d3a44294453f9c846b8a1d36120f54690ee52d77c376b8b7d8d4b6850408415e
├── ✔ Packages [47 packages]
├── ✔ File digests [659 files]
├── ✔ File metadata [659 locations]
└── ✔ Executables [141 executables]
✔ Scanned for vulnerabilities [11 vulnerability matches]
├── by severity: 0 critical, 2 high, 8 medium, 0 low, 0 negligible (1 unknown)
└── by status: 2 fixed, 9 not-fixed, 0 ignored
There was a problem hiding this comment.
@wennergr we originally went for bookworm as some python package like Pandas require system dependencies which are not on alpine.
It's feasible to add them to an alpine base but not straightforward...
https://gist.github.com/orenitamar/f29fb15db3b0d13178c1c4dd611adce2
So we picked bookworm for simplicity and compatibility...
I suggest to cut this new version (1.2.1) with bookworm, and then cut a new one (2.0.0) with alpine.
This would be a major version as it might not be usable by some connectors.
Our batch update connector flow will lead to a best effort thing:
- All connectors incompatible with alpine will fail their CI build so they won't get updated.
alafanechere
deleted the
augustin/06-03-python-connector-base-image_upgrade_to_python_3.9.19_update_setuptools_and_pip
branch
|
Looks good to me! |
What
Relates to https://github.com/airbytehq/airbyte-internal-issues/issues/7166
We want to mitigate some vulnerabilities found on our base image.
Cut a new base image version for python connectors which:
The published image is available here.
It's adoption by connectors will be done in batch via the run of the
upgrade_base_imageairbyte-ci command.