Move coverage jobs to CircleCI

[teohhanhui]

Q	A
Bug fix?	no
New feature?	no
BC breaks?	no
Deprecations?	no
Tests pass?	yes
Fixed tickets	N/A
License	MIT
Doc PR	N/A

I propose to use CircleCI for the test coverage jobs. Also includes some minor CI / test-related fixes.

Todo:

• Use GNU Parallel (only for PHPUnit tests, as Behat tests modify the database...)
• Investigate why ApiPlatformExtensionTest takes a very long time
• Remove coverage jobs from Travis CI

[dunglas]

We can try, but it annoys me to have 3 different CI systems (AppVeyor, Travis and Circle), it means more maintenance.
Isn't it possible to switch entirely from Travis to Circle?

Also, Circle is not very used in the PHP community. It means a harder learning curve for contributors.

[teohhanhui]

I don't think we could switch entirely to CircleCI without slowing down the build process, because CircleCI only allows 4 containers at a time. But it's possible if we manage to speed up the build further. :)

Yes, there's a bit of learning curve, but it's the same for Travis CI. Many people might be using it, but it's a different matter to actually understand it. If anything, CircleCI is easier because it's mostly just running in a customized Docker image. Travis CI has more magic.

[teohhanhui]

You can see the builds of my fork on CircleCI here: https://circleci.com/gh/teohhanhui/workflows/api-platform-core/tree/ci%2Fparallel-jobs

[teohhanhui]

I agree with this article, we could split our builds on Travis CI and CircleCI to take full advantage of the free resources provided by both.

https://www.codementor.io/adiroiban/circle-ci-2-0-for-open-source-projects-hsi1rpufs

[dunglas]

I've added Circle CI to the org, can you push force?

[teohhanhui]

I have to note that the upload-coverage job only passed because I have the COVERALLS_REPO_TOKEN in the CircleCI project for my fork, and it indeed uploaded the coverage report for the forked repository.

So we'll need to disable the coveralls integration for pull requests unless we find a better way.

[dunglas]

It's a blocker to me 😞: the main benefit of Coveralls is to be able to quickly check that the code coverage of a PR is good enough.

[teohhanhui]

We could switch to codecov:

Repo tokens are not required for public repos tested on Travis-Org, CircleCI or AppVeyor.

[dunglas]

Ok for me, but please also keep coveralls for the main branches so we don't loose the history.

[teohhanhui]

@dunglas Let me know once you grant access to Codecov.

And install the integration to the API Platform organization: https://github.com/integration/codecov

[teohhanhui]

What do you think about also using CircleCI for lint jobs? It should simplify our Travis CI config further... And it'll likely allow the Travis CI build to complete faster as there is one less job in the queue.

[dunglas]

Good idea!

[teohhanhui]

But I think we should not expand the scope of this PR further before we verify that things work as expected. Lol...

[codecov-io]

Codecov Report

❗ No coverage uploaded for pull request base (2.2@6ae3583). Click here to learn what that means.
The diff coverage is n/a.

@@          Coverage Diff           @@
##             2.2    #1888   +/-   ##
======================================
  Coverage       ?   96.44%           
  Complexity     ?     2600           
======================================
  Files          ?      191           
  Lines          ?     6526           
  Branches       ?        0           
======================================
  Hits           ?     6294           
  Misses         ?      232           
  Partials       ?        0

Impacted Files	Coverage Δ	Complexity Δ
src/Test/DoctrineOrmFilterTestCase.php	100% <ø> (ø)	10 <0> (?)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6ae3583...a4825cc. Read the comment docs.

[teohhanhui]

This should be good to merge. 😃

[soyuka]

cool stuff thanks @teohhanhui

[bendavies]

you can still easily parallelize behat. you just have to

1. create N databases upfront.
2. export a token for the current parallel slot parallel -j10 'export TEST_TOKEN={%}; vendor/bin/behat ... {}
3. Register a doctrine connection factory which reads the value of TEST_TOKEN and connects to the correct database.

[teohhanhui]

@bendavies:

create N databases upfront.

😱 🙀

It's fast enough as it is. So I think it's okay for now? 😜

[bendavies]

yep, just saying it's possible :)