TravisCI price model change

[icemac]

TravisCI is introducing a new price model today.
If I understand the new model correctly we get 10,000 credits per month and a single linux build minute costs 10 credits. So we have 1,000 build minutes per month for the whole zopefoundation organization (?).

I fear this is not enough.

A simple calculation:
We have 392 public repositories minus 62 archived ones = 330 repositories which might run on TravisCI.
Let's assume 300 repositories have a monthly cron job running 5 jobs (aka 5 different Python versions).
If each job takes only 1 minute we only need 1,500 minutes for the cron jobs (!).
(I know many repositories are run weekly an only a few jobs can run in 1 minute.)

It is possible to apply for more credits which is decided by the TravisCI support on a per request basis.
Maybe we need a new solution to run our CI jobs –

• or is my calculation simply wrong
• or do we have another chance to reduce the needed build minutes so 10,000 credits could be enough
• or is my understanding wrong and each contributor or repository has 10,000 credits?

Source: https://blog.travis-ci.com/2020-11-02-travis-ci-new-billing#building-on-a-public-repositories-only

[icemac]

We switched to Travis-CI.com so we have to add up org an com.

https://travis-ci.org/github/zopefoundation?tab=insights: about 4,700 build minutes.
https://travis-ci.com/github/zopefoundation?tab=insights: about 4,300 build minutes.

This are about 9,000 minutes aka at least 90,000 credits.

This includes two script runs changing many repositories via PRs, so they were run thrice (2 times for the PR and one time for the merge) but we still seem to use way more build time than are given from now on.

[The-Compiler]

I've already mentioned this on Twitter, but let me elaborate here a bit:

If I understand the new model correctly we get 10,000 credits per month

I don't see anything in the announcement saying you get them per month - on the contrary, I'm pretty sure you get them once:

For those of you still kicking the tires on the Travis CI product, we are switching the plan from 100 builds on private repositories to our trial (free) plan with a 10K credit allotment. This will allow you to build on whichever environment you’d like to test out. Trial plans expire when you use all of our credits or after a year.

and

For those of you who have been building on public repositories (on travis-ci.com, with no paid subscription), we will upgrade you to our trial (free) plan with a 10K credit allotment (which allows around 1000 minutes in a Linux environment).

as well as:

When your credit allotment runs out - we’d love for you to consider which of our plans will meet your needs.

So, unless I'm missing something, you'd need to haggle with Travis CI staff to get more build-minutes - again, nothing in there says you'll get them every month, quite the contrary: should your run out of credits again you can repeat the process to request more

From my experience with the Travis CI support so far (though, granted, a while back) and given that this will affect tens if not hundreds of thousands of projects, I don't think this will work out well.

[icemac]

I looked at https://travis-ci.com/organizations/zopefoundation/plan there is a little question mark next to the free plan its hover text reads as: „Free Plan is trial plan. The credits will be not replenished. Switch to Free Plan if you wish to cancel your subscription.“

It seems that @The-Compiler is right: we seem to get the 10,000 credits only once.
Either we have to

• contact TravisCI support to ask for more each time we run out of credits,
• we have to pay for the service (The „5 Concurrent plan“ looks like what we currently use, it costs $249/month!)
• or we have to switch to another CI system (Github actions provides 2,000 minutes/month in the free plan and it seems that public repos are free, see https://github.com/pricing)

[dataflake]

Wringing hands and making assumptions is not really helpful. I have followed the procedure outlined on their blog article announcing the change and lodged a support request asking how to proceed.

[dataflake]

Quick update: I have received a reply that sounded like boilerplate when a single project applies for an ongoing free subscription. They list several requirements which we all meet. I wrote back a lengthy email explaining that I am speaking for an entire GitHub organization and provided information about the history of Zope, the ecosystem and the foundations. Let's see what happens.

[mgedmin]

Let's have a plan B. Is anyone using GitHub actions already in a way they like?

I think currently https://github.com/zopefoundation/z3c.jbot/blob/master/.github/workflows/test.yml is the only project in the zopefoundation namespace that uses GitHub actions. I'm not in love with the excessive boilerplate, but I think we can learn to live with it.

Does anyone have better examples to imitate?

[dataflake]

I'm all for using GH actions, just never found enough time to get into it. Maybe there are repos in the plone GH organizations that have better usage examples?

[jamadden]

AppVeyor may be another option. We already use it when we need to test on Windows; it also supports macOS and Linux builds.

A positive is that they seem to add new Python versions relatively quickly; a negative is that they only do so with full release versions (not betas or RCs, at least as far as I've seen on Windows).

Another positive is that they don't limit the build minutes in a given month, as GitHub actions does. However, they do restrict the free plan to one concurrent build, which could add substantial delays. I don't know what levels of concurrency GitHub actions supports?

[dataflake]

I cannot stand Appveyor. It is so slow that it's basically unusable.

[hugovk]

I have stats on how quickly CIs added Python 3.8 and 3.9!

From release to support:

2019: Python 3.8

🥇 21 hours: Travis CI
🥈 20 days: GitHub Actions
🥉 24 days: Azure Pipelines
4️⃣ 25 days: AppVeyor

2020: Python 3.9

🥇 17 hours: GitHub Actions ⬆️
🥈 20 days: Travis CI ⬇️
🥉 21 days: AppVeyor ⬆️
4️⃣ 22 days: Azure Pipelines ⬇️

[dataflake]

I have left another comment in the issue I had created at Travis CI and told them they are free to close it and that we are moving to GitHub Actions instead. I also told them that I fully understand when business models change, but this silent treatment for 4 months is extremely unprofessional.

From my Travis CI dashboard I also looked at some other GitHub organizations that I am a member of. Both Plone and the Collective suffer from the same issue, they're still on the limited free plan and exhausted their minutes a long time ago.

@icemac I suggest uninstalling the Travis CI "application" at the zopefoundation organization root level hoping that it will become more obvious for everyone that Travis CI won't work anymore.

[jensens]

Well, yes, we need to move all collective repos to GHA - what a waste of time.

I like the idea of removing TravisCI on orga root level, it is dead. I will bring this up for Plone and Collective as well.

[jugmac00]

A couple of days ago, I created a PR for the cpython project, and I was extremely surprised as they still use Travis (partly).

This was the first open source repository I saw with a working Travis CI in months.

Maybe you need a big lobby...

Anyway, I am glad it is over.

[dataflake]

This was the first open source repository I saw with a working Travis CI in months.

Maybe you need a big lobby

I think you hit it on the head. We're small fish, they really don't give a damn. In my comment I also expressed my surprise that they think this unprofessional treatment is OK, as if prospective or current paying clients won't notice any of it and draw their own conclusion. If I were a business owner on one of their paid plans I'd immediately think "business risk - unreliable/unprofessional vendor".

[hugovk]

Likewise I've not had any replies for months for multiple requests for my account and several orgs, and have also moved everything important over to GitHub Actions and disabled all the rest (they'll be moved if needed).

If I see an open source repo using Travis CI, I'm much less likely to contribute because I can't test changes on my fork before creating a PR. 🤷

[mauritsvanrees]

Note that repositories that use travis-ci.ORG actually still work. For example this week I worked on fixing tox and Travis for plone.api and that goes fine. But of course this is only a matter of time, as on top it says:

Please be aware travis-ci.org will be shutting down in several weeks, with all accounts migrating to travis-ci.com. Please stay tuned here for more information.

It is March now and I am surprised it is still running.
Moving to GHA is definitely the plan.

[icemac]

@dataflake Let's wait with the removal of the TravisCI GitHub app until we have fully migrated to GHA. I do no know what will happen on TravisCI's site if we remove the app. It would be bad if they would delete the history of the builds. It could be useful for some packages to be able to see if the tests were previously successful over there.

[dataflake]

It won't really matter for us anymore, but today I received a reply to my original trouble ticket at Travis CI telling me they have added 25k credits. They are asking to contact support again if credits are running low, so it seems to be a one-time credit.

[icemac]

Nice to hear, there are still some dozens of packages to be migrated to GHA, so it could help in between.

[icemac]

We settled on GHA. So I am closing this issue.