Travis CI doesn't report its status or doesn't run on Python pull requests

[vstinner]

Example: python/cpython#20807 has "continuous-integration/travis-ci Expected — Waiting for status to be reported [Required]" but no link to a Travis CI job. Since the job is required, I cannot merge this PR.

• https://www.traviscistatus.com/ seems fine
• https://www.githubstatus.com/ seems fine as well

https://travis-ci.org/github/python/cpython/builds shows that the latest build ran 11 hours ago.

The current https://travis-ci.org/github/python/cpython is running for 4 minutes: tests on "Pull Request #20768 bpo-40939: Remove the old parser".

... but while I was writing this issue, python/cpython#20807 got a green "Travis CI": https://travis-ci.org/github/python/cpython/builds/697264594 passed (but the two coverage jobs are still running). So now I'm confused.

Maybe the fix is just to wait a bit :-) I closed/reopened python/cpython#20807 23 minutes ago to force to re-run Travis CI.

---

I also noticed Travis CI preventing to merge a PR beause the "coverage (C)" job was still running, whereas the other required Travis CI jobs passed successfully. Example: python/cpython#20767 shows that Travis CI is still running, whereas https://travis-ci.org/github/python/cpython/builds/696949107 completed but "Test code coverage (C)" job failed with "The job exceeded the maximum time limit for jobs, and has been terminated.". The last line is "0:46:18 load avg: 5.32 [254/426/1] test_profile passed -- running: test_tokenize (8 min 55 sec), test_multiprocessing_spawn (2 min 36 sec), test_statistics (1 min 53 sec)". Longer than 46 minutes to run the test suite is quite long :-(

Maybe we should just remove the coverage jobs on Travis CI, unless someone actually uses them? See https://bugs.python.org/issue40237

[vstinner]

Another example: python/cpython#20797 got a new commit longer than 30 minutes ago, but Travis CI still says " continuous-integration/travis-ci Expected — Waiting for status to be reported" with no link to a build.

... but again, a few minutes later, all become green and I was able to merge the PR.

I don't know. It's just that I'm used to very reactive Travis CI. Maybe GitHub or Travis CI are just a little bit slower these days :-)

[Mariatta]

TravisCI is third party app that we don't manage. Don't think there is much we can do here. Perhaps best to file the issue with TravisCI.

[brettcannon]

So what is the purpose of this issue? Is it to ask if we should move the coverage report off of Travis? Otherwise the title isn't helpful since we don't control Travis nor have any connections there so we can't do anything about it misbehaving short of (yet again) not making it required.

[vstinner]

The purpose of this issue is to collect the issues that I saw with Travis CI. But sorry, the status is unclear. Sometimes, it seems like Travis CI is broken, but a few minutes later, it's back to life.

[vstinner]

In clear, right now, I don't ask to do anything :-)

[vstinner]

Ok, here is a more concrete proposal: "Don't run Python and C coverage jobs of Travis CI on pull requests" https://bugs.python.org/issue40993 Extract:

Not only it's a waste of resources, but it seems like it's not longer possible to merge a PR as soon as Travis CI required jobs pass: a PR can only be merged when all Travis CI jobs complete. Problem: while a full test suite run takes around 20 min, coverage jobs take longer than 40 minutes.

[vstinner]

Screenshot of the "status not reported" issue:

The Travis CI has no "details" link.

[vstinner]

Screenshot of the "status not reported" issue:

GitHub doesn't give the link to the Travis CI build, whereas the build was already created:
https://travis-ci.org/github/python/cpython/builds/698964418

Maybe there is just a long delay on notifications from Travis CI to GitHub sometimes.

[zware]

I just ran into a similar issue on python/buildmaster-config#204; Travis ran and passed, but never reported back to GitHub.

[brettcannon]

That's probably on Travis' side as the GitHub API for status checks takes the URL in as part of the status check, so either GitHub is dropping an API call on the floor (which is not something I have really come across), or Travis has a bug where they are not sending an update.

[vstinner]

Recent failure: python/cpython#21594 (comment)

[vstinner]

Does someone know how to report issues to Travis CI?

[brettcannon]

@vstinner nope. I believe we lost our connection there after their big round of layoffs after being acquired.