Experiment with modularizing Travis jobs#2737
Merged
kailuowang merged 1 commit intotypelevel:masterfrom Feb 27, 2019
Merged
Conversation
Resolves typelevel#2570 This sets up the Cats Travis build to use Travis's stages feature and modularizes the Travis jobs. This probably introduces a little bit of extra overhead since each new job needs to spin up a virtual machine and pull the cache. However, I think that this extra overhead is offset by a number of factors (see further below). We are now only publishing code coverage reports for the default Scala version (currently 2.12.x). I don't think that publishing multiple coverage reports for the same commit even has well-defined results, and I suspect that we didn't really mean to do that. Since running the tests with code coverage is probably the most time-consuming thing that the tests do, this is a significant time saver. Each individual job has a smaller run duration (usually < 20 min). This means that if an individual job does fail and we have to restart it, we are waiting for a smaller amount of time than previously. In theory at least, there's more ability for parallelization. In testing on my fork, it looks like Travis tends to run 3 to 5 jobs in parallel. So since we already had that many jobs before, this may not be much of a win. Modularizing the build allowed for a few small installation optimizations (only installing jekyll for the docs job, only installing node.js for the JS builds, etc). Lastly, I'm hoping that this sets the stage to move toward releasing via [sbt-ci-release](https://github.com/olafurpg/sbt-ci-release) instead of requiring a maintainer to awkwardly publish from their personal computer. This new Travis configuration allows us to run a bunch of tests/validation in parallel and then if all goes well, move on to a stage that is responsible for releasing. Since each job has its own time limit, we can probably avoid previous problems of releasing from Travis causing builds to take too long. Note that Travis stages have awkward interactions with the build matrix, so I'm now using YAML anchors (essentially templates) to reduce duplication rather than the build matrix. This allows for finer-grained control that I think works out pretty nicely.
Contributor
|
This looks really cool! Thanks @ceedubs for taking on this. |
Codecov Report
@@ Coverage Diff @@
## master #2737 +/- ##
=========================================
+ Coverage 95.14% 95.2% +0.06%
=========================================
Files 365 363 -2
Lines 6816 6763 -53
Branches 302 291 -11
=========================================
- Hits 6485 6439 -46
+ Misses 331 324 -7
Continue to review full report at Codecov.
|
kailuowang
approved these changes
Feb 27, 2019
Contributor
kailuowang
left a comment
There was a problem hiding this comment.
Really nice. Thanks so much!
LukaJCB
approved these changes
Feb 27, 2019
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #2570
This sets up the Cats Travis build to use Travis's stages feature and
modularizes the Travis jobs. This probably introduces a little bit of
extra overhead since each new job needs to spin up a virtual machine and
pull the cache. However, I think that this extra overhead is offset by a
number of factors (see further below). And in experimenting with builds on
my fork, the build time looks to have gone from ~43 min to ~33 min with this
change. See here for an example.
Benefits
We are now only publishing code coverage reports for the default Scala
version (currently 2.12.x). I don't think that publishing multiple
coverage reports for the same commit even has well-defined results, and
I suspect that we didn't really mean to do that. Since running the tests
with code coverage is probably the most time-consuming thing that the
tests do, this is a significant time saver.
Each individual job has a smaller run duration (usually < 20 min). This
means that if an individual job does fail and we have to restart it, we
are waiting for a smaller amount of time than previously.
In theory at least, there's more ability for parallelization. In testing
on my fork, it looks like Travis tends to run 3 to 5 jobs in parallel.
So since we already had that many jobs before, this may not be much of a
win.
Modularizing the build allowed for a few small installation
optimizations (only installing jekyll for the docs job, only installing
node.js for the JS builds, etc).
Lastly, I'm hoping that this sets the stage to move toward releasing via
sbt-ci-release instead of
requiring a maintainer to awkwardly publish from their personal
computer. This new Travis configuration allows us to run a bunch of
tests/validation in parallel and then if all goes well, move on to a
stage that is responsible for releasing. Since each job has its own time
limit, we can probably avoid previous problems of releasing from Travis
causing builds to take too long.
Implementation Notes
Note that Travis stages have awkward interactions with the build matrix,
so I'm now using YAML anchors (essentially templates) to reduce
duplication rather than the build matrix. This allows for finer-grained
control that I think works out pretty nicely.