Re-organize and improve output check scenarios

- Move scenarios for checking success + output to a separate feature - Unify order and implementation of scenarios for checking output, stderr and stdout
cucumber · Jun 21, 2021 · 3a9766a · 3a9766a
1 parent 51fb700
commit 3a9766a
Show file tree

Hide file tree

Showing 4 changed files with 231 additions and 246 deletions.
diff --git a/features/03_testing_frameworks/cucumber/steps/command/check_for_success_and_output.feature b/features/03_testing_frameworks/cucumber/steps/command/check_for_success_and_output.feature
@@ -0,0 +1,96 @@
+Feature: Checking success and output together
+
+  In order to specify expected output and success
+  As a developer using Cucumber
+  I want to use the "it should pass with" step family
+
+  Background:
+    Given I use a fixture named "cli-app"
+
+  Scenario: Match passing exit status and partial output
+    Given a file named "features/output.feature" with:
+    """cucumber
+    Feature: Run command
+      Scenario: Run command
+        When I run `echo hello world`
+        Then it should pass with:
+        \"\"\"
+        hello
+        \"\"\"
+    """
+    When I run `cucumber`
+    Then the features should all pass
+
+  Scenario: Match passing exit status and exact output
+    Given a file named "features/output.feature" with:
+    """cucumber
+    Feature: Run command
+      Scenario: Run command
+        When I run `ruby -e 'puts "hello\nworld"; exit 0'`
+        Then it should pass with exactly:
+        \"\"\"
+        hello
+        world
+        \"\"\"
+    """
+    When I run `cucumber`
+    Then the features should all pass
+
+  Scenario: Match failing exit status and partial output
+    Given a file named "features/output.feature" with:
+    """cucumber
+    Feature: Run command
+      Scenario: Run command
+        When I run `ruby -e 'puts "hello\nworld"; exit 1'`
+        Then it should fail with:
+        \"\"\"
+        hello
+        \"\"\"
+    """
+    When I run `cucumber`
+    Then the features should all pass
+
+  Scenario: Match failing exit status and exact output
+    Given a file named "features/output.feature" with:
+    """cucumber
+    Feature: Run command
+      Scenario: Run command
+        When I run `ruby -e 'puts "hello\nworld"; exit 1'`
+        Then it should fail with exactly:
+        \"\"\"
+        hello
+        world
+        \"\"\"
+    """
+    When I run `cucumber`
+    Then the features should all pass
+
+  Scenario: Match failing exit status and output with regex
+    Given a file named "features/output.feature" with:
+    """cucumber
+    Feature: Run command
+      Scenario: Run command
+        When I run `ruby -e 'puts "hello\nworld"; exit 1'`
+        Then it should fail with regex:
+        \"\"\"
+        hello\s*world
+        \"\"\"
+    """
+    When I run `cucumber`
+    Then the features should all pass
+
+  Scenario: Detect output from all processes
+    Given a file named "features/output.feature" with:
+    """cucumber
+    Feature: Run command
+      Scenario: Run command
+        When I run `echo hello`
+        And I run `echo good-bye`
+        Then the stdout should contain exactly:
+        \"\"\"
+        hello
+        good-bye
+        \"\"\"
+    """
+    When I run `cucumber`
+    Then the features should all pass
diff --git a/features/03_testing_frameworks/cucumber/steps/command/check_output_of_command.feature b/features/03_testing_frameworks/cucumber/steps/command/check_output_of_command.feature
@@ -14,35 +14,11 @@ Feature: All output of commands which were executed
       Scenario: Run command
         When I run `echo hello world`
         Then the output should contain "hello world"
+        And the output should not contain "good-bye"
     """
     When I run `cucumber`
     Then the features should all pass
 
-  Scenario: Detect absence of inline subset of output
-    Given a file named "features/output.feature" with:
-    """cucumber
-    Feature: Run command
-      Scenario: Run command
-        When I run `echo hello world`
-        Then the output should not contain "good-bye"
-    """
-    When I run `cucumber`
-    Then the features should all pass
-
-  Scenario: Failed detection of inline subset of output
-    Given a file named "features/output.feature" with:
-    """cucumber
-    Feature: Run command
-      Scenario: Run command
-        When I run `echo hello world`
-        Then the output should contain "goodbye world"
-    """
-    When I run `cucumber`
-    Then the features should not all pass with:
-    """
-    expected "hello world" to string includes: "goodbye world"
-    """
-
   Scenario: Detect subset of output with multiline string
     Given a file named "features/output.feature" with:
     """cucumber
@@ -53,57 +29,73 @@ Feature: All output of commands which were executed
         \"\"\"
         hello
         \"\"\"
+        And the output should not contain:
+        \"\"\"
+        good-bye
+        \"\"\"
     """
     When I run `cucumber`
     Then the features should all pass
 
-  Scenario: Detect absence subset of output with multiline string
+  Scenario: Detect exact multiline output
     Given a file named "features/output.feature" with:
     """cucumber
     Feature: Run command
       Scenario: Run command
-        When I run `ruby -e 'puts "hello\nworld"'`
-        Then the output should not contain:
+        When I run `ruby -e 'puts "hello\nworld"; warn "good-bye"'`
+        Then the output should contain exactly:
         \"\"\"
+        hello
+        world
         good-bye
         \"\"\"
     """
     When I run `cucumber`
     Then the features should all pass
 
-  Scenario: Detect exact multiline output
-    Given a file named "features/output.feature" with:
-    """cucumber
+  Scenario: Detect combined output from normal and interactive processes
+    Given an executable named "bin/aruba-test-cli" with:
+    """
+    #!/usr/bin/env ruby
+
+    while input = gets do
+      break if "" == input
+      puts input
+    end
+    """
+    And a file named "features/output.feature" with:
+    """
     Feature: Run command
       Scenario: Run command
-        When I run `ruby -e 'puts "hello\nworld"'`
-        Then the output should contain exactly:
+        When I run `echo hello`
+        When I run `aruba-test-cli` interactively
+        And I type "good-bye"
+        And I type ""
+        Then the stdout should contain exactly:
         \"\"\"
         hello
-        world
+        good-bye
         \"\"\"
     """
     When I run `cucumber`
     Then the features should all pass
 
-  Scenario: Failed detection of exact multi-line output
+  Scenario: Detect output from named source
     Given a file named "features/output.feature" with:
-    """cucumber
+    """
     Feature: Run command
       Scenario: Run command
-        When I run `ruby -e 'puts "goodbye\nworld"'`
-        Then the output should contain exactly:
+        When I run `echo hello`
+        And I run `echo good-bye`
+        Then the output from "echo hello" should contain "hello"
+        And the output from "echo good-bye" should contain exactly "good-bye"
+        And the output from "echo hello" should contain exactly:
         \"\"\"
         hello
-        world
         \"\"\"
     """
     When I run `cucumber`
-    Then the features should not all pass with:
-    """
-          expected "goodbye\nworld" to output string is eq: "hello\nworld"
-          Diff:
-    """
+    Then the features should all pass
 
   Scenario: Detect exact output with ANSI output
     Given a file named "features/output.feature" with:
@@ -191,121 +183,6 @@ Feature: All output of commands which were executed
     When I run `cucumber`
     Then the features should all pass
 
-  Scenario: Match passing exit status and partial output
-    Given a file named "features/output.feature" with:
-    """cucumber
-    Feature: Run command
-      Scenario: Run command
-        When I run `echo hello world`
-        Then it should pass with:
-        \"\"\"
-        hello
-        \"\"\"
-    """
-    When I run `cucumber`
-    Then the features should all pass
-
-  Scenario: Match passing exit status and exact output
-    Given a file named "features/output.feature" with:
-    """cucumber
-    Feature: Run command
-      Scenario: Run command
-        When I run `ruby -e 'puts "hello\nworld"; exit 0'`
-        Then it should pass with exactly:
-        \"\"\"
-        hello
-        world
-        \"\"\"
-    """
-    When I run `cucumber`
-    Then the features should all pass
-
-  Scenario: Match failing exit status and partial output
-    Given a file named "features/output.feature" with:
-    """cucumber
-    Feature: Run command
-      Scenario: Run command
-        When I run `ruby -e 'puts "hello\nworld"; exit 1'`
-        Then it should fail with:
-        \"\"\"
-        hello
-        \"\"\"
-    """
-    When I run `cucumber`
-    Then the features should all pass
-
-  Scenario: Match failing exit status and exact output
-    Given a file named "features/output.feature" with:
-    """cucumber
-    Feature: Run command
-      Scenario: Run command
-        When I run `ruby -e 'puts "hello\nworld"; exit 1'`
-        Then it should fail with exactly:
-        \"\"\"
-        hello
-        world
-        \"\"\"
-    """
-    When I run `cucumber`
-    Then the features should all pass
-
-  Scenario: Match failing exit status and output with regex
-    Given a file named "features/output.feature" with:
-    """cucumber
-    Feature: Run command
-      Scenario: Run command
-        When I run `ruby -e 'puts "hello\nworld"; exit 1'`
-        Then it should fail with regex:
-        \"\"\"
-        hello\s*world
-        \"\"\"
-    """
-    When I run `cucumber`
-    Then the features should all pass
-
-  Scenario: Detect output from all processes
-    Given a file named "features/output.feature" with:
-    """cucumber
-    Feature: Run command
-      Scenario: Run command
-        When I run `echo hello`
-        And I run `echo good-bye`
-        Then the stdout should contain exactly:
-        \"\"\"
-        hello
-        good-bye
-        \"\"\"
-    """
-    When I run `cucumber`
-    Then the features should all pass
-
-  Scenario: Detect combined output from normal and interactive processes
-    Given an executable named "bin/aruba-test-cli" with:
-    """
-    #!/usr/bin/env ruby
-
-    while input = gets do
-      break if "" == input
-      puts input
-    end
-    """
-    And a file named "features/output.feature" with:
-    """
-    Feature: Run command
-      Scenario: Run command
-        When I run `echo hello`
-        When I run `aruba-test-cli` interactively
-        And I type "good-bye"
-        And I type ""
-        Then the stdout should contain exactly:
-        \"\"\"
-        hello
-        good-bye
-        \"\"\"
-    """
-    When I run `cucumber`
-    Then the features should all pass
-
   Scenario: Check size of output
     Given a file named "features/flushing.feature" with:
     """cucumber
@@ -316,16 +193,3 @@ Feature: All output of commands which were executed
     """
     When I run `cucumber`
     Then the features should all pass
-
-  Scenario: Detect output from named source
-    Given a file named "features/output.feature" with:
-    """
-    Feature: Run command
-      Scenario: Run command
-        When I run `echo hello`
-        And I run `echo good-bye`
-        Then the output from "echo hello" should contain exactly "hello"
-        And the output from "echo good-bye" should contain exactly "good-bye"
-    """
-    When I run `cucumber`
-    Then the features should all pass