Update Travis Configuration

[maxmeyer]

Summary

This is a follow up to #470. It cleans up the travis configuration.

Details

• Remove jruby-variants
• Add Mac OS builds

Motivation and Context

See issue #470.

How Has This Been Tested?

Travis Build

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Refactoring (cleanup of codebase withouth changing any existing functionality)
• Update documentation

Checklist:

• I've added tests for my code
• My change requires a change to the documentation.
• I have updated the documentation accordingly.

[olleolleolle]

There's a small typo sterr -> stderr in the comment block for JRuby's option variables. REALLY good to have those explanations here, as comments. They teach.

[mvz]

This would also need to be fixed. I'll open a ticket so we don't forget.

[mvz]

Done: #477

[mvz]

The --server option increases startup time, which is very relevant for Aruba's cucumber tests since they start up Aruba a lot. Let's try --client and see if that makes any of the JRuby targets finish in time.

[mvz]

I've done some experiments locally, and the following combination of environment variables greatly improves test time:

export JRUBY_OPTS="--client -Xcompile.invokedynamic=false"
export JAVA_OPTS='-Djruby.compile.mode=OFF -XX:+TieredCompilation -XX:TieredStopAtLevel=1'

I've tried this with just bundle exec cucumber features/01_getting_started_with_aruba/run_commands.feature, and it improved clock runtime from 1m11s to 34s, while reducing system load.

I found these options at https://gist.github.com/lucasallan/2f9b5c945280d226916c and https://github.com/jruby/jruby/wiki/Improving-startup-time.

[mvz]

Can we move setting of these variables to a global section to avoid duplication?

[maxmeyer]

Yes. That makes sense. Thanks a lot for your feedback. I will try to push a new version tomorrow evening taking into account your findings.

[junaruga]

Seeing the result of Travis, ruby-head (osx) environment in Travis server has problem right now.
You may be able to remove the ruby-head (osx) if you do not like to get the error every time.
https://travis-ci.org/cucumber/aruba/jobs/259520443

[mvz]

The java variants still don't finish in time 😞

[maxmeyer]

I pushed two commits taking the tip from @junaruga into account by reducing the number of jobs per build:

• jruby only
• jruby + mri ruby

[maxmeyer]

https://travis-ci.org/cucumber/aruba/jobs/260478099 is with jruby only, but it doesn't matter how many parallel tests we're running. jruby also complains about a missing --debug option

[maxmeyer]

According to https://github.com/jruby/jruby/wiki/Improving-startup-time#use-the---dev-flag

Those flags are set by --dev

• client mode where applicable (generally older 32-bit JVMs)
• TieredCompilation and TieredStopAtLevel=1, equivalent to client mode on newer Hotspot-based JVMs
• compile.mode=OFF to disable JRuby's JVM bytecode compiler
• jruby.compile.invokedynamic=false to disable the slow-to-warmup invokedynamic features of JRuby

[mvz]

Those flags are set by --dev

That makes things a lot simpler! 👍

[maxmeyer]

Times for build:

00:00 min - start build
06:00 min - setup project to run tests
07:00 min - run licence finder
07:30 min - run rspec
08:30 min - run cucumber
[TBC]

[mvz]

I'd like to make the following suggestions to get things moving and to avoid dealing with endless backward-compatibility problems:

• Drop support for old rubies now, before the 1.0.0 release. We can discuss what is a good definition of 'old'.
• Move the JRuby variants to the allow_failures section.

[maxmeyer]

Currently we're not facing any backward compatible problems with any of the MRI rubies, correct? "Only" jruby is a "troublemaker". I ran the suite on my laptop with jruby and it was "slow" as well. So I think let's move the jrubies to the `allowed_failures` section - as you @mvz proposed - and try to tackle this after we fixed all other problems / polished the release. Am 2. September 2017 20:35:24 MESZ schrieb Matijs van Zuijlen <notifications@github.com>:

…

I'd like to make the following suggestions to get things moving and to avoid dealing with endless backward-compatibility problems: * Drop support for old rubies now, before the 1.0.0 release. We can discuss what is a good definition of 'old'. * Move the JRuby variants to the `allow_failures` section. -- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: #476 (comment)

[mvz]

Currently we're not facing any backward compatible problems with any of the MRI rubies, correct?

I should have explained more. Yes, there are no problems right now, but if we do not drop official support for those old rubies right now, Aruba could be stuck with them for the entire 1.x cycle, meaning we cannot use modern Ruby syntax even if we want to, and/or we have to keep version checks lying around. Finally, debugging any issues on those older Rubies may require actually installing them which will become harder and harder.

A counter-argument is the desire for RSpec to use Aruba 1.0 voiced in #363 (comment).

[mvz]

Moving JRuby to allowed_failures is definitely more urgent :-).

[maxmeyer]

@mvz you're right. personally I think we could release 2.0.0 right after 1.0.0 which drops support for those rubies. Not sure how to handle support best.

[mvz]

Well, right after may be a little too soon, if we agree that releasing 2.0.0 in 2018 is fine then I'm fine with keeping support for Ruby 1.9.3 in 1.x.

[mvz]

@maxmeyer are you going to update this PR or would you like me to do it?

[maxmeyer]

Feel free to fix it!

[mvz]

I'm going to open a new PR for this to rebase and clean up the branch.

[maxmeyer]

I found some time hacking on aruba. So I rebased the PR based on your work and squashed everything into a single commit. Feel free to merge if you're happy with this PR.