Flaky spec: Booth Display single booth for any number of polls

[bertocq]

Warning!

This spec probably only fails between 00:00 and 02:00 am Madrid time

What

Tests that fail randomly are called "flakies", this one seems to be one:

Randomized seed: 19087

Failure:

    2) Booth Display single booth for any number of polls
     Failure/Error: expect(page).to have_content 'Choose your booth'
       expected to find text "Choose your booth" in "Validate document Total recounts and results Language: English Español Français Nederlands Português Valencià How to help Menu Decide Madrid Polling Polling officers Notifications You don't have new notifications My activity My account Sign out Admin menu Validate document Total recounts and results × You don't have officing shifts today Poll officing Here you can validate user documents and store voting results"
     # ./spec/features/officing/booth_spec.rb:82:in `block (2 levels) in <top (required)>'

How

• Explain why the test is flaky, or under which conditions/scenario it fails randomly
• Explain why your PR fixes it
• Create a backport PR to consul/consul when the fixing PR is approved

Tips for flaky hunting

Random values issues

If the problem comes from randomly generated values, running multiple times a single spec could help you reproduce the failure by running at your command line:

for run in {1..10}
do
  bin/rspec ./spec/features/budgets/investments_spec.rb:256
done

You can also try running a single spec in Travis:
Add option :focus to the spec and push your branch to Github, for example:

scenario 'Show', :focus do

But remember to remove that :focus changes afterwards when submitting your PR changes!

Test order issues

Running specs in the order they failed may discover that the problem is that a previous test sets an state in the test environment that makes our flaky fail/pass. Tests should be independent from the rest.

After executing rspec you can see the seed used, add it as an option to rspec, for example:
bin/rspec --seed 55638 (check Randomized seed value at beginning of issue)

Other things to watch for

• Time related issues (current time, two time or date comparisons with miliseconds/time when its not needed)
• https://semaphoreci.com/community/tutorials/how-to-deal-with-and-eliminate-flaky-tests

[bertocq]

Happened at https://travis-ci.org/AyuntamientoMadrid/consul/jobs/338257557

[javierm]

Closed via consul#2712 and #1625.