Skip to content

Conversation

@Jarvl
Copy link
Contributor

@Jarvl Jarvl commented Mar 5, 2025

This PR bumps pgbouncer_exporter to version v0.18.0 and decrements pgbouncer to 1.23.1.

The latest version 0.18.0 of pgbouncer_exporter only provides support up to pgbouncer 1.23.1, as pgbouncer 1.24.0 introduces new stats that cause errors within pgbouncer_exporter. Currently, the latest images published for the helm chart are pgbouncer 1.24.0 airflow-pgbouncer-2025.01.10-1.24.0 and pgbouncer_exporter v0.17.0 airflow-pgbouncer-exporter-2024.06.18-0.17.0, which are incompatible with each other as outlined in #47295.

Until pgbouncer_exporter is updated to support these new metrics, the helm chart should use compatible versions of the libraries.

closes: #47295

New pgbouncer image: airflow-pgbouncer-2025.03.05-1.23.1
New pgbouncer_exporter image: airflow-pgbouncer-exporter-2025.03.05-0.18.0

This PR required changes to the pgbouncer build script and Dockerfile to support tag names that do not consistently correspond with the version name.

Note: pgbouncer_exporter v0.18.0 also increments the go version used to 1.23, see: jbub/pgbouncer_exporter@479710b. The expected go version is bumped here to 1.23.7 here to support this change.


- Upgrades pgbouncer_exporter to 0.18.0
- Bumps Airflow pgbouncer_exporter version to current date (2025-03-05)
- Bumps expected Go version to 1.23.7 per pgbouncer_exporter release notes
@boring-cyborg boring-cyborg bot added the area:helm-chart Airflow Helm Chart label Mar 5, 2025
@boring-cyborg
Copy link

boring-cyborg bot commented Mar 5, 2025

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

@Jarvl
Copy link
Contributor Author

Jarvl commented Mar 6, 2025

@potiuk tagging you since you pushed the image for the last version bump to pgbouncer_exporter in #40303

@potiuk
Copy link
Member

potiuk commented Mar 6, 2025

Yep. I am just on it :)

@potiuk
Copy link
Member

potiuk commented Mar 6, 2025

ok. Pushed the image. apache/airflow:airflow-pgbouncer-exporter-2025.03.05-0.18.0 - can you please modfy helm-chart and change the defaults?

@Jarvl
Copy link
Contributor Author

Jarvl commented Mar 6, 2025

ok. Pushed the image. apache/airflow:airflow-pgbouncer-exporter-2025.03.05-0.18.0 - can you please modfy helm-chart and change the defaults?

To clarify - do you mean as part of this PR or a separate PR?

@potiuk
Copy link
Member

potiuk commented Mar 6, 2025

As part of this PR is best :)

@Jarvl Jarvl changed the title Upgrade pgbouncer_exporter version to 0.18.0 Upgrade helm chart pgbouncer_exporter version to 0.18.0 Mar 6, 2025
@potiuk
Copy link
Member

potiuk commented Mar 6, 2025

Let's see if it passes the tests :)

@potiuk
Copy link
Member

potiuk commented Mar 6, 2025

added missing EOL

@Jarvl
Copy link
Contributor Author

Jarvl commented Mar 6, 2025

Just tried the new image alongside pgbouncer 1.24, still getting the same error.

It looks like v0.18.0 of pgbouncer_exporter only provides support up to pgbouncer 1.23, so what's in my PR description currently is inaccurate. This will not resolve the errors noted in #47295.

I created an issue in the upstream library to address: jbub/pgbouncer_exporter#34

How do we want to proceed? It's not going to be as simple as performing this upgrade to provide support for pgbouncer 1.24 since the upstream needs updated with support.

One option could be to proceed with this PR, then decrement the pgbouncer version in the helm chart to 1.23 so there are no longer incompatibilities. Or at least build an image for pgbouncer 1.23. Then once pgbouncer_exporter has adequate support we can bump that to a newer version in the helm chart.

Edit: To clarify, there is not an image built for pgbouncer 1.23 (it was incremented from 1.22 -> 1.24 in #45542).

@Jarvl
Copy link
Contributor Author

Jarvl commented Mar 6, 2025

Just tried the new image alongside pgbouncer 1.24, still getting the same error.

It looks like v0.18.0 of pgbouncer_exporter only provides support up to pgbouncer 1.23, so what's in my PR description currently is inaccurate. This will not resolve the errors noted in #47295.

I created an issue in the upstream library to address: jbub/pgbouncer_exporter#34

How do we want to proceed? It's not going to be as simple as performing this upgrade to provide support for pgbouncer 1.24 since the upstream needs updated with support.

One option could be to proceed with this PR, then decrement the pgbouncer version in the helm chart to 1.23 so there are no longer incompatibilities. Or at least build an image for pgbouncer 1.23. Then once pgbouncer_exporter has adequate support we can bump that to a newer version in the helm chart.

Edit: To clarify, there is not an image built for pgbouncer 1.23 (it was incremented from 1.22 -> 1.24 in #45542).

I updated my PR description to reflect what I noted here.

@potiuk
Copy link
Member

potiuk commented Mar 6, 2025

Let's decrease it to 1.23 in this PR

@potiuk
Copy link
Member

potiuk commented Mar 6, 2025

With some comments left and link to the issue you opened -> then I can build and push the image - initially it will fail when you make the change but then I can push the image and restart failing jobs to pick up the new image

@Jarvl
Copy link
Contributor Author

Jarvl commented Mar 6, 2025

Let's decrease it to 1.23 in this PR

How comfortable are we with doing this and potentially introducing a breaking change into the helm chart? pgbouncer 1.24 introduces new features and configs so decrementing the version in the helm chart could be an unexpected breaking change for people using the defaults.

@potiuk
Copy link
Member

potiuk commented Mar 6, 2025

I think we never released chart with 1.24

@potiuk
Copy link
Member

potiuk commented Mar 6, 2025

We have not released chart for a looooong time.

@Jarvl
Copy link
Contributor Author

Jarvl commented Mar 6, 2025

I think we never released chart with 1.24

Ah good point, I should have checked the release notes!

@Jarvl Jarvl requested a review from ashb as a code owner March 6, 2025 20:31
@Jarvl
Copy link
Contributor Author

Jarvl commented Mar 6, 2025

Note that the pgbouncer Dockerfile needed changes to support the tag name used upstream so those changes are included in 7457aa6.

For example:
https://github.com/pgbouncer/pgbouncer/releases/tag/pgbouncer_1_23_1-fixed

Didn't test that the build works locally though, still working on getting my dev environment set up.

@Jarvl Jarvl changed the title Upgrade helm chart pgbouncer_exporter version to 0.18.0 Update pgbouncer to 1.23.1 and pgbouncer_exporter to 0.18.0 in helm chart Mar 6, 2025
@Jarvl
Copy link
Contributor Author

Jarvl commented Mar 6, 2025

Note that the pgbouncer Dockerfile needed changes to support the tag name used upstream so those changes are included in 7457aa6.

For example: https://github.com/pgbouncer/pgbouncer/releases/tag/pgbouncer_1_23_1-fixed

Didn't test that the build works locally though, still working on getting my dev environment set up.

I validated locally that the build_and_push.sh script works as intended 👍

@Jarvl Jarvl requested a review from potiuk March 6, 2025 21:16
@potiuk
Copy link
Member

potiuk commented Mar 6, 2025

Built and pushed, approved workflows

@Jarvl
Copy link
Contributor Author

Jarvl commented Mar 7, 2025

@potiuk Who would be the best person to ping for a review on this PR?

@potiuk
Copy link
Member

potiuk commented Mar 7, 2025

Me :)

@potiuk potiuk merged commit 506ab62 into apache:main Mar 7, 2025
59 checks passed
@boring-cyborg
Copy link

boring-cyborg bot commented Mar 7, 2025

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

@Jarvl
Copy link
Contributor Author

Jarvl commented Mar 7, 2025

Me :)

Thank you so much for your help and attention!

@Jarvl Jarvl deleted the patch-1 branch March 7, 2025 18:44
nailo2c pushed a commit to nailo2c/airflow that referenced this pull request Apr 4, 2025
…hart (apache#47416)

* Upgrade pgbouncer_exporter to 0.18.0

- Upgrades pgbouncer_exporter to 0.18.0
- Bumps Airflow pgbouncer_exporter version to current date (2025-03-05)
- Bumps expected Go version to 1.23.7 per pgbouncer_exporter release notes

* Update helm chart default pgbouncer_exporter image to 0.18.0

* Add newline to 47416.misc.rst

* Update chart/newsfragments/47416.misc.rst

* fixup! Update chart/newsfragments/47416.misc.rst

* Bump pgbouncer to version 1.23.1

* Change to significant news fragment

* Update newsfragment content

---------

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:helm-chart Airflow Helm Chart

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pgbouncer exporter doesn't support metrics exposed by updated pgbouncer.

3 participants