Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Clean up and speed up cucumber test suite #544

Merged
merged 25 commits into from
Mar 7, 2018
Merged

Clean up and speed up cucumber test suite #544

merged 25 commits into from
Mar 7, 2018

Conversation

mvz
Copy link
Contributor

@mvz mvz commented Mar 2, 2018 •
edited
Loading

Summary

Remove slow and needless scenarios from our suite, and speed up some other slow scenarios.

Details

  • Remove scenarios related to developing Aruba. These should be covered in he README.md and CONTRIBUTING.md files.
  • Remove a scenario that demonstrates that a particular feature is not available in JRuby.
  • Speed up scenarios that wait for stuff to happen. These can be sped up by lowering the timeouts.
  • Fail-fast when attempting to run interactively with the in-process runner.
  • Allow non-integer values for timeouts in steps that set them.

Motivation and Context

Our test suite takes a long time on CRuby, and an insanely long time on JRuby.

How Has This Been Tested?

Some specs were added, plus Travis.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Refactoring (cleanup of codebase withouth changing any existing functionality)
  • Update documentation

Checklist:

  • I've added tests for my code
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.

@mvz
Copy link
Contributor Author

mvz commented Mar 2, 2018

I'm going to go through the entire suite using cucumber --format=usage and weed out all the slow steps.

@mvz mvz mentioned this pull request Mar 5, 2018
Merged
mvz added 10 commits March 6, 2018 19:31
@mvz
Remove obsolete step definitions
19233d8
@mvz
Remove non-feature
18305e8 
Something that doesn't work is a bug, not a feature.
@mvz
Use smaller values to demonstrate timeout
b0d3049
@mvz
Remove confusing dev steps
4f7048a 
The step names mentioning 'default executable' and 'default feature' are
unclear and therefore confusing. Better to write out the needed
executable and feature in place.
@mvz
Remove development instruction features
5a77168 
These instructions should not be part of the test suite but go
in CONTRIBUTING.md instead.
@mvz
Improve contribution guidelines
c5f6f4f 
- Fix some typos and wording issues.
- Add developer instructions that were in the feature files before.
@mvz
Lower waiting time in send_signal_to_command.feature
acf2bef
@mvz
Lower waiting time in check_for_exit_statuses.feature
4ccddce
@mvz
Move specs for Aruba::Api::Command to their proper location
42edf01
@mvz
Refuse to launch interactively with the :in_process launcher
baa6cfa
@mvz mvz changed the title [WIP] Clean up and speed up cucumber test suite Clean up and speed up cucumber test suite Mar 6, 2018
@mvz
Clean up feature background
d8aafed 
The background section contained a step that was only relevant to some
of the scenarios. Distribute it where relevant instead.
@olleolleolle
Copy link
Contributor

All the commit messages are clear and distinct. Makes archaeology easier and reviewing simpler.

This attention to detail is a thing which keeps me around this project.

Thank you so much for making this improvement.

olleolleolle
olleolleolle approved these changes Mar 7, 2018
View reviewed changes
Copy link
Contributor

@olleolleolle olleolleolle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Travis has passed, I'd call this ready to go.

@mvz mvz merged commit e68d401 into master Mar 7, 2018
@mvz mvz deleted the cleanup-speedup branch March 7, 2018 11:06
@mvz
Copy link
Contributor Author

mvz commented Mar 7, 2018

🎉

This should really help with getting the features passing on JRuby, since they now actually seem to finish in time!

@mvz mvz mentioned this pull request May 20, 2019
Merged
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@olleolleolle olleolleolle olleolleolle approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mvz @olleolleolle