-
-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Experiment with modularizing Travis jobs #2737
Conversation
Resolves typelevel#2570 This sets up the Cats Travis build to use Travis's stages feature and modularizes the Travis jobs. This probably introduces a little bit of extra overhead since each new job needs to spin up a virtual machine and pull the cache. However, I think that this extra overhead is offset by a number of factors (see further below). We are now only publishing code coverage reports for the default Scala version (currently 2.12.x). I don't think that publishing multiple coverage reports for the same commit even has well-defined results, and I suspect that we didn't really mean to do that. Since running the tests with code coverage is probably the most time-consuming thing that the tests do, this is a significant time saver. Each individual job has a smaller run duration (usually < 20 min). This means that if an individual job does fail and we have to restart it, we are waiting for a smaller amount of time than previously. In theory at least, there's more ability for parallelization. In testing on my fork, it looks like Travis tends to run 3 to 5 jobs in parallel. So since we already had that many jobs before, this may not be much of a win. Modularizing the build allowed for a few small installation optimizations (only installing jekyll for the docs job, only installing node.js for the JS builds, etc). Lastly, I'm hoping that this sets the stage to move toward releasing via [sbt-ci-release](https://github.com/olafurpg/sbt-ci-release) instead of requiring a maintainer to awkwardly publish from their personal computer. This new Travis configuration allows us to run a bunch of tests/validation in parallel and then if all goes well, move on to a stage that is responsible for releasing. Since each job has its own time limit, we can probably avoid previous problems of releasing from Travis causing builds to take too long. Note that Travis stages have awkward interactions with the build matrix, so I'm now using YAML anchors (essentially templates) to reduce duplication rather than the build matrix. This allows for finer-grained control that I think works out pretty nicely.
This looks really cool! Thanks @ceedubs for taking on this. |
Codecov Report
@@ Coverage Diff @@
## master #2737 +/- ##
=========================================
+ Coverage 95.14% 95.2% +0.06%
=========================================
Files 365 363 -2
Lines 6816 6763 -53
Branches 302 291 -11
=========================================
- Hits 6485 6439 -46
+ Misses 331 324 -7
Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Really nice. Thanks so much!
Resolves #2570
This sets up the Cats Travis build to use Travis's stages feature and
modularizes the Travis jobs. This probably introduces a little bit of
extra overhead since each new job needs to spin up a virtual machine and
pull the cache. However, I think that this extra overhead is offset by a
number of factors (see further below). And in experimenting with builds on
my fork, the build time looks to have gone from ~43 min to ~33 min with this
change. See here for an example.
Benefits
We are now only publishing code coverage reports for the default Scala
version (currently 2.12.x). I don't think that publishing multiple
coverage reports for the same commit even has well-defined results, and
I suspect that we didn't really mean to do that. Since running the tests
with code coverage is probably the most time-consuming thing that the
tests do, this is a significant time saver.
Each individual job has a smaller run duration (usually < 20 min). This
means that if an individual job does fail and we have to restart it, we
are waiting for a smaller amount of time than previously.
In theory at least, there's more ability for parallelization. In testing
on my fork, it looks like Travis tends to run 3 to 5 jobs in parallel.
So since we already had that many jobs before, this may not be much of a
win.
Modularizing the build allowed for a few small installation
optimizations (only installing jekyll for the docs job, only installing
node.js for the JS builds, etc).
Lastly, I'm hoping that this sets the stage to move toward releasing via
sbt-ci-release instead of
requiring a maintainer to awkwardly publish from their personal
computer. This new Travis configuration allows us to run a bunch of
tests/validation in parallel and then if all goes well, move on to a
stage that is responsible for releasing. Since each job has its own time
limit, we can probably avoid previous problems of releasing from Travis
causing builds to take too long.
Implementation Notes
Note that Travis stages have awkward interactions with the build matrix,
so I'm now using YAML anchors (essentially templates) to reduce
duplication rather than the build matrix. This allows for finer-grained
control that I think works out pretty nicely.