Future of Pendulum - Python 3.12 and beyond ?

[potiuk]

Hello @sdispater / @Secrus / others,

I have an honest question about the future of pendulum. We have recently started discussion on Airflow devlist "[DISCUSS] Future of Pendulum in Airflow" https://lists.apache.org/thread/vpwxg8r93nrhnth3q63b8polt7xyvkwm - where we seem to not able to make a good decision without having some good answers so I decided (on my own) to be super candid and honest and ask you very direct questions.

Context:

We've been using Pendulum for years (and many people in Python ecosystem have done so). Apache Airflow heavily relies on Pendulum and we developed a bit of a dependency on it - sometimes we use datetime, sometimes pendulum and those two do not necessarily work well together, but generally we are very happy and thankful for how Pandulum helped us to navigate the (surprisingly difficult) timezone and datetime world in Python.

And while we understand that Pendulum does not have a lot of bugs to fix, a lot of traffic and a lot of need to be further developed (with more/better timezone support in recent versiosn of Python), it's pretty understandable, that Pendulum will not be as active as it once was.

Problems:

However with Python 3.12 it poses quite a bit of challenge to us - because it effectively prevents Airflow (and likely many other projects to add Python 3.12 support - distutils has been removed as we know and it holds any users of Pendulum for upgrading to 3.12 - moreover a number of those projects are suffering because having pendulum as dependency causes problems for their users. Even today I had to remove pendulum as part of dependency from our our development tool in airflow - because more and more people have Python 3.12 set as default on their developer workstations: apache/airflow#35786

Luckily I could do it but this causes a major headache for many maintainers who have to respond and solve issues of their users who are attempting to install and use whatever libraries and tooling on Python 3.12 - and wake up to a cryptic message that leaves them a bit in the dark initially.

Current status

Now I understand the work is on-going slowly - #600 (comment) to release 3.0.0 (slower than anticipated and with some bumps). And even simple PRs like this #765 that are necessary are not yet merged after many weeks of waiting - and not even tried in alpha or beta version of 3.0.0. That is a bit worrying - taking into account that so many people rely on it.

Not a complaint

And it's not really a complaint personally to you @sdispater - we are really greatful for all the work done so far. And if you are the only sole maintainer of pendulum for years and no-one there is there to help you and you have your day job, and it does not pay your bills, it's perfectly understandable this happens.

I've been through some of that myself, keeping yourself running on a long-term project where everyone complains but you get little help and you are largely alone can be difficult to handle, things like burnout, or simply lack of time might come into play. So please do not try it as a complain, as a maintainer (yes lucky to be paid for a lot of the work I do for Airflow but still) I understand that this might not be something you are looking forward to.

Questions

But the point is - It just raises a question by many - can we rely on both - 3.12 support and future maintenance of Pendulum? Or maybe the project is effectively abandoned and we should seriously look for alternatives? We've been discussing it for a looong time to come up with alternatives, it will be costly and a bit risky for backwards compatibilities, but if we know that the future is not really bright (or just basically not too gray) - we could make better decisions in Airflow - where to go and what to do. Things like 3.12 problems and other dependencies like that might happen in the near future and it might be that we will have to again deal with similar issues even if the current Python 3.12 issues are solved.

So my question - can we count on Pendulum being maintained in a (near) future?

Offering help

Or maybe what you really need is help ? Maybe you need some more people who you would invite and who would share the burden on reviewing/approving PRs, approving workflows, merging changes and preparing new releases ? There are a number of very experienced maintainers in Airflow and since we are relying on Pendulum a lot - and they have really experiended/good Python engineers. And maybe - if you would be open - and maybe accept some of our more active help and even driving fixing some problems and releasing them. If you do not have enough time/energy/drive - maybe a few pairs of eyes and hand (with enough of time /need and drive) would be a good idea ?

You can take a look at some history in GitHub of some people in Airflow and I can ask them if they would be willing to help - and accept those who you see are really experienced and vested into both projects. And then we can possibly help to drive and timely release the project from now on - up until the time we will maybe simply get it through the time where Python 3.10 will reach EOL and we can start relying fully on tzinfo and what is stdlib ? (though this seems like not really good enough for a number of cases).

Summary

And again - it's not a complaint or nothing that you should feel bad about if the statement is "well - it's not likely to be maintained" - this is very clear messsage.

I've been super candid with all that, and I would really love if we can have some candid answers, that could help us to make good decisions (in Airlfow - but probably many other maintainers out there are asking themselves very similar questions).

Sincerely,

Jarek

[brucearctor]

Is there a move where pendulum gets moved/contributed into github.com/apache ... and is maintained by airflow/others? And, have @sdispater as a committer/PMC?

TBD whether we'd want apache pendulum? Or, apache airflow: Pendulum?

Pendulum is great -- but is now a strong dependency for many people, projects, organizations. I imagine there is a way to help with governance, bugfixes, etc in a structure that resonates.

Let's figure out how to help!

[potiuk]

That's an interesting idea.

Yes - Apache Sofware Fundation has an incubator PMC https://incubator.apache.org/ and it would be possible to donate the project to the ASF this way - to create eventually a new PMC. "apache-pendulum" - and it can even come with a bigger group of people to start with (PPMC - Podling PMC) - projects have to go through incubation period in order to become top level projects. It would need ASF member as mentor (happy to be one in case there is an interest). But of course ASF is one of the many ways similar move could be done.

[potiuk]

Hello @sdispater / @Secrus - any comments here ?

I think lack of reaction from maintainers (of any kind) for such a request - very explicitly - and I hope empathetihically - stating the problems it causes for others is kinda worst thing that can happen if it will continue.

We can neither move this or that direction or even help if we do not see some kind of future expectation - really hard to do anything when all you hear is .. silence.

[Secrus]

Hi @potiuk. Pendulum is primarily @sdispater's project, so you would have to wait for his opinions on that matter. I only help with fixes and issue/PR triage, all the project vision and decisions come from the author.

[josephmancuso]

does anyone know what needs to be done to fix this? i'm willing to sink my resources and the resources of the Masonite community to get pendulum 3.12 support. @sdispater . but i need to be brought up to speed on the issues. a checklist of what needs to be done or something