Regression: exit status not waiting for run interactively process to complete

[mroth]

Summary

In version 0.6.2, a run interactively step followed by a the exit status should be step would complete, allowing the results of the process to be examined (as well as ensuring any side effects were complete before moving on to additional steps).

Upon updating to latest stable 0.14.2, this no longer happens, causing commands to run out of order, and all the workarounds I can think of seem fairly hacky and undesirable. It appears run interactively is now functioning as a "background" process by default?

I believe this may be a regression/bug?, since that doesn't appear to be the goal of the behavior from sample Features.

Expected Behavior

There should be a way to let an interactive process complete before checking it's exit status, or another mechanism to avoid "out of order" problems.

Current Behavior

I've annotated the actual Scenario Outline in my test cases here to illustrate what is happening, see comments inline:

Scenario Outline: Wrapped `git reset` can handle files with spaces properly

  Given I am in a git repository
  Given an empty file named "file with spaces.txt"
  ###AAA
  Given I successfully run `git add "file with spaces.txt"`

  ###BBB
  Given I run `<shell>` interactively
    And I type `eval "$(scmpuff init -ws)"`
    And I type "scmpuff_status"
    And I type "git reset 1"
    And I type "echo 'DEBUG: PHASE BBB'"
    And I type "exit"
  Then the exit status should be 0
  ### ^^^ this is now checking exit status of AAA, not BBB!

  # Then the output should contain "BBB"
  ### ^^^ this command however, would force aruba to wait until BBB completes (hacky option #1)

  # Then I stop the command started last
  ### ^^^ (hacky option #2) also seems to ensure BBB completes, but throws a deprecation error

  ###CCC
  ### currently, this phase is failing, because it is taking place *BEFORE* BBB completes
  When I run `scmpuff status`
  Then the stdout from "scmpuff status" should contain:
    """
    untracked:  [1] file with spaces.txt
    """
  Examples:
    | shell |
    | bash  |
    | zsh   |

Possible Solution

I've indicated two possible workarounds in the above code sample comments, but neither seems like it's the intended solution (and one throws a deprecation warning).

Context & Motivation

In our tests, we need to do some stuff in an interactive shell (as our program modifies the shell environment for end-users as it's primary use case), and follow that up by additional (non-interactive) tests afterwards to check on the state of things.

Your Environment

• Version used: 0.14.2 and 0.6.2 (pre-regression)
• Operating System and version: tested on both macOS and Linux (travis-ci)
• Link to your project: https://github.com/mroth/scmpuff

[maxmeyer]

@mrod Thanks for reporting this. This is weird and should not happen.

1. We use an event bus in newer version of aruba to track command starting/stopping
   

   aruba/lib/aruba/setup.rb

   Line 33 in b23f6da

   def events

2. A stopped command should be reported back to the command monitor

   aruba/lib/aruba/setup.rb

   Line 66 in b23f6da

   runtime.command_monitor.last_command_stopped = event.entity

3. In the following step we call #last_command_stopped
   https://github.com/cucumber/aruba/blob/master/lib/aruba/cucumber/command.rb#L273

4. This method should make sure all command started should be stopped (via CommandMonitor)

   aruba/lib/aruba/command.rb

   Line 52 in b23f6da

   event_bus.notify Events::CommandStopped.new(self)

Questions

• What is the value of

  aruba/lib/aruba/platforms/command_monitor.rb

  Line 56 in b23f6da

  @last_command_stopped

  you in your case?
• What happens if the event for stopping a command is send
  

  aruba/lib/aruba/setup.rb

  Line 66 in b23f6da

  runtime.command_monitor.last_command_stopped = event.entity

Help

• Can you please answer the questions for your usecase?

[mroth]

Thanks @maxmeyer, starting on debugging but might need a little help.

I've currently modified the step to be like the following:

  @wip
  Scenario Outline: Wrapped `git reset` can handle files with spaces properly

    Given I am in a git repository
    Given an empty file named "file with spaces.txt"
    ###AAA
    Given I successfully run `git add "file with spaces.txt"`

    ###BBB
    Given I run `<shell>` interactively
      And I type `eval "$(scmpuff init -ws)"`
      And I type "scmpuff_status"
      And I type "git reset 1"
      And I type "echo 'DEBUG: PHASE BBB'"
      And I type "exit"
   ### BELOW IS NEW
    Then the exit status should be 0
    Then I print the value of last command stopped
    Then I manually sent a stop event to runtime monitor
    Then I print the value of last command stopped
    Then the exit status should be 0
   ### ABOVE IS NEW
    ### ^^^ this is checking exit status of AAA, not BBB!

    # Then the output should contain "BBB"
    ### ^^^ this command however, would force aruba to wait until BBB completes (super hacky option #1)

    # Then I stop the command started last
    ### ^^^ (hacky option #2) also seems to ensure BBB completes, but throws a deprecation error

    ###CCC
    ### currently, this phase is failing, because it is taking place *BEFORE* BBB completes
    When I run `scmpuff status`
    Then the stdout from "scmpuff status" should contain:
      """
      untracked:  [1] file with spaces.txt
      """
    Examples:
      | shell |
      | bash  |

With new step definitions:

#
# for debugging cucumber/aruba#432
#
Then(/^I print the value of last command stopped$/) do
  puts last_command_stopped
end

Then(/^I manually sent a stop event to runtime monitor$/) do
  # TODO: not working
  event_bus.notify Events::CommandStopped.new(self)
end

On the first debug evocation, last_command_stopped returns #<Aruba::Processes::SpawnProcess:70196393371320 commandline="git add "file with spaces.txt"": output="">, which seems consistent with the bug in that the interactive command has not been stopped.

However, I'm having trouble figuring out how to properly send the "stop" event to the proper place within a step definition for debugging -- could you please help with the correct way to define that? Thanks!

[maxmeyer]

Thanks. I will get to this soon - hopefully.

[mvz]

I think what happens here is this:

• The non-interactive command finishes and sets @last_command_stopped.
• The interactive command gets started and is told to finish by the fake user, but Aruba doesn't know about this.
• The exit status should be 0 step calls #last_command_stopped. This method sees that @last_command_stopped has a value and therefore does not wait for the interactive command to finish.

I have checked version 0.6.2, and the implementation of exit status should be 0 seems similar. That means it probably looks at the exit status for the first command there as well (note that that command also has exit status 0). It is therefore unclear to me why the scenario works with version 0.6.2.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in a week if no further activity occurs.

[stale]

This issue has been automatically closed because of inactivity. You can support the Cucumber core team on opencollective.

[mvz]

I think this issue still exists & I'm going to take a look at it.

[mvz]

I've looked again at the 0.6.2 implementation, and it stops all processes unless it has deliberately stopped a process before. In the case of this scenario, this means it stops all processes.

[mvz]

@mroth I'm afraid you will have to add some step that waits for the interactive command to finish. This would be either I stop the command started last or "I stop the command "<shell>".

[mvz]

Documentation about this should probably be improved, since the interaction between steps is not very clear in this case.

A step definition like I let the command started last finish might also be nice, since it may better describe the intent in a case like this.