Merge branch '3.x' of github.com:codeceptjs/CodeceptJS into 3.x

Author: DavertMik

.github/PULL_REQUEST_TEMPLATE.md

@@ -21,12 +21,13 @@ Applicable plugins:
 - [ ] autoLogin
 - [ ] customLocator
 - [ ] pauseOnFail
-- [ ] puppeteerCoverage
+- [ ] coverage
 - [ ] retryFailedStep
 - [ ] screenshotOnFail
 - [ ] selenoid
 - [ ] stepByStepReport
 - [ ] wdio
+- [ ] subtitles
 
 ## Type of change
 

docs/configuration.md

@@ -11,7 +11,7 @@ After running `codeceptjs init` it should be saved in test root.
 
 Here is an overview of available options with their defaults:
 
-* **tests**: `"./*_test.js"` - pattern to locate tests. Allows to enter [glob pattern](https://github.com/isaacs/node-glob).
+* **tests**: `"./*_test.js"` - pattern to locate tests. Allows to enter [glob pattern](https://github.com/isaacs/node-glob), Can either be a pattern to locate tests or an array of patterns to locate tests / test file names.
 * **grep**: - pattern to filter tests by name
 * **include**: `{}` - actors and page objects to be registered in DI container and included in tests. Accepts objects and module `require` paths
 * **timeout**: `10000` - default tests timeout
@@ -47,7 +47,20 @@ exports.config = {
   require: ["ts-node/register", "should"]
 }
 ```
-
+For array of test pattern
+```js
+exports.config = {
+  tests: ['./*_test.js','./sampleTest.js'],
+  timeout: 10000,
+  output: '',
+  helpers: {},
+  include: {},
+  bootstrap: false,
+  mocha: {},
+  // require modules
+  require: ["ts-node/register", "should"]
+}
+```
 ## Dynamic Configuration
 
  By default `codecept.json` is used for configuration. You can override its values in runtime by using `--override` or `-o` option in command line, passing valid JSON as a value:

docs/helpers/Appium.md

@@ -1667,11 +1667,9 @@ I.saveElementScreenshot(`#submit`,'debug.png');
 
 Saves a screenshot to ouput folder (set in codecept.json or codecept.conf.js).
 Filename is relative to output folder.
-Optionally resize the window to the full available page `scrollHeight` and `scrollWidth` to capture the entire page by passing `true` in as the second argument.
 
 ```js
 I.saveScreenshot('debug.png');
-I.saveScreenshot('debug.png', true) //resizes to available scrollHeight and scrollWidth before taking screenshot
 ```
 
 #### Parameters

docs/playwright.md

@@ -456,6 +456,63 @@ For instance, this is how you can read a trace for a failed test from an example
 npx playwright show-trace /home/davert/projects/codeceptjs/examples/output/trace/open.zip
 ```
 
+## Capturing code coverage
+
+Code coverage can be captured, by enabling the `coverage` plugin in `codecept.config.js`.
+
+```js
+{
+  plugins: {
+    coverage: {
+      enabled: true
+    }
+  }
+}
+```
+
+Once all the tests are completed, `codecept` will create and store coverage in `output/coverage` folder, as shown below.
+
+![](https://user-images.githubusercontent.com/16587779/131362352-30ee9c51-705f-4098-b665-53035ea9275f.png)
+
+### Converting `playwright` coverage to `istanbul` coverage
+
+To convert coverage generated from `playwright` to `istanbul` coverage, you first need to install
+- [`v8-to-istanbul`](https://www.npmjs.com/package/v8-to-istanbul)
+
+Once installed, convert the coverage to a format which `istanbul` can recognize, by writing a script as shown below.
+
+```js
+const v8toIstanbul = require('v8-to-istanbul');
+// read all the coverage file from output/coverage folder
+const coverage = require('./output/coverage/Visit_Home_1630335005.coverage.json');
+const fs = require('fs/promises');
+
+(async () => {
+    for (const entry of coverage) {
+        // Used to get file name
+        const file = entry.url.match(/(?:http(s)*:\/\/.*\/)(?<file>.*)/);
+        const converter = new v8toIstanbul(file.groups.file, 0, {
+            source: entry.source
+        });
+
+        await converter.load();
+        converter.applyCoverage(entry.functions);
+
+        // Store converted coverage file which can later be used to generate report
+        await fs.writeFile('./coverage/final.json', JSON.stringify(converter.toIstanbul(), null, 2));
+    }
+})();
+```
+
+Once the istanbul compatible coverage is generated, use [`nyc`](https://www.npmjs.com/package/nyc) to generate your coverage report in your desired format.
+
+```
+npx nyc report --reporter text -t coverage
+```
+
+The above command will generate a text report like shown below.
+
+![](https://user-images.githubusercontent.com/16587779/131363170-b03b4398-5e9a-4142-bc32-764a5f4a5e11.png)
 ## Extending Helper
 
 To create custom `I.*` commands using Playwright API you need to create a custom helper.

docs/plugins.md

@@ -409,6 +409,29 @@ Scenario('project update test', async (I) => {
 
 -   `config`  
 
+## coverage
+
+Dumps code coverage from Playwright/Puppeteer after every test.
+
+#### Configuration
+
+```js
+plugins: {
+   playwrightCoverage: {
+     enabled: true
+   }
+}
+```
+
+Possible config options:
+
+-   `coverageDir`: directory to dump coverage files
+-   `uniqueFileName`: generate a unique filename by adding uuid
+
+### Parameters
+
+-   `config`  
+
 ## customLocator
 
 Creates a [custom locator][3] by using special attributes in HTML.
@@ -441,7 +464,7 @@ Using `data-test` attribute with `$` prefix:
 // in codecept.conf.js
 plugins: {
  customLocator: {
-   enabled: true
+   enabled: true,
    attribute: 'data-test'
  }
 }
@@ -460,7 +483,7 @@ Using `data-qa` attribute with `=` prefix:
 // in codecept.conf.js
 plugins: {
  customLocator: {
-   enabled: true
+   enabled: true,
    prefix: '=',
    attribute: 'data-qa'
  }
@@ -536,45 +559,6 @@ Enable it manually on each run via `-p` option:
 
     npx codeceptjs run -p pauseOnFail
 
-## puppeteerCoverage
-
-Dumps puppeteers code coverage after every test.
-
-#### Configuration
-
-Configuration can either be taken from a corresponding helper (deprecated) or a from plugin config (recommended).
-
-```js
-plugins: {
-   puppeteerCoverage: {
-     enabled: true
-   }
-}
-```
-
-Possible config options:
-
--   `coverageDir`: directory to dump coverage files
--   `uniqueFileName`: generate a unique filename by adding uuid
-
-    First of all, your mileage may vary!
-
-    To work, you need the client javascript code to be NOT uglified. They need to be built in "development" mode.
-    And the end of your tests, you'll get a directory full of coverage per test run.  Now what?
-    You'll need to convert the coverage code to something istanbul can read.  Good news is someone wrote the code
-    for you (see puppeteer-to-istanbul link below).  Then using istanbul you need to combine the converted
-    coverage and create a report.  Good luck!
-
-    Links:
-
--   [https://github.com/GoogleChrome/puppeteer/blob/v1.12.2/docs/api.md#class-coverage][7]
--   [https://github.com/istanbuljs/puppeteer-to-istanbul][8]
--   [https://github.com/gotwarlost/istanbul][9]
-
-### Parameters
-
--   `config`  
-
 ## retryFailedStep
 
 Retries each failed step in a test.
@@ -673,14 +657,14 @@ Possible config options:
 
 ## selenoid
 
-[Selenoid][10] plugin automatically starts browsers and video recording.
+[Selenoid][7] plugin automatically starts browsers and video recording.
 Works with WebDriver helper.
 
 ### Prerequisite
 
 This plugin **requires Docker** to be installed.
 
-> If you have issues starting Selenoid with this plugin consider using the official [Configuration Manager][11] tool from Selenoid
+> If you have issues starting Selenoid with this plugin consider using the official [Configuration Manager][8] tool from Selenoid
 
 ### Usage
 
@@ -709,7 +693,7 @@ plugins: {
   }
 ```
 
-When `autoCreate` is enabled it will pull the [latest Selenoid from DockerHub][12] and start Selenoid automatically.
+When `autoCreate` is enabled it will pull the [latest Selenoid from DockerHub][9] and start Selenoid automatically.
 It will also create `browsers.json` file required by Selenoid.
 
 In automatic mode the latest version of browser will be used for tests. It is recommended to specify exact version of each browser inside `browsers.json` file.
@@ -721,10 +705,10 @@ In automatic mode the latest version of browser will be used for tests. It is re
 While this plugin can create containers for you for better control it is recommended to create and launch containers manually.
 This is especially useful for Continous Integration server as you can configure scaling for Selenoid containers.
 
-> Use [Selenoid Configuration Manager][11] to create and start containers semi-automatically.
+> Use [Selenoid Configuration Manager][8] to create and start containers semi-automatically.
 
 1.  Create `browsers.json` file in the same directory `codecept.conf.js` is located
-    [Refer to Selenoid documentation][13] to know more about browsers.json.
+    [Refer to Selenoid documentation][10] to know more about browsers.json.
 
 _Sample browsers.json_
 
@@ -749,7 +733,7 @@ _Sample browsers.json_
 
 2.  Create Selenoid container
 
-Run the following command to create a container. To know more [refer here][14]
+Run the following command to create a container. To know more [refer here][11]
 
 ```bash
 docker create                                    \
@@ -782,15 +766,15 @@ When `allure` plugin is enabled a video is attached to report automatically.
 | enableVideo      | Enable video recording and use `video` folder of output (default: false)       |
 | enableLog        | Enable log recording and use `logs` folder of output (default: false)          |
 | deletePassed     | Delete video and logs of passed tests (default : true)                         |
-| additionalParams | example: `additionalParams: '--env TEST=test'` [Refer here][15] to know more   |
+| additionalParams | example: `additionalParams: '--env TEST=test'` [Refer here][12] to know more   |
 
 ### Parameters
 
 -   `config`  
 
 ## stepByStepReport
 
-![step-by-step-report][16]
+![step-by-step-report][13]
 
 Generates step by step report for a test.
 After each step in a test a screenshot is created. After test executed screenshots are combined into slideshow.
@@ -823,6 +807,20 @@ Possible config options:
 
 -   `config` **any** 
 
+## subtitles
+
+Automatically captures steps as subtitle, and saves it as an artifact when a video is found for a failed test
+
+#### Configuration
+
+```js
+plugins: {
+ subtitles: {
+   enabled: true
+ }
+}
+```
+
 ## tryTo
 
 Adds global `tryTo` function inside of which all failed steps won't fail a test but will return true/false.
@@ -897,7 +895,7 @@ This plugin allows to run webdriverio services like:
 -   browserstack
 -   appium
 
-A complete list of all available services can be found on [webdriverio website][17].
+A complete list of all available services can be found on [webdriverio website][14].
 
 #### Setup
 
@@ -909,7 +907,7 @@ See examples below:
 
 #### Selenium Standalone Service
 
-Install `@wdio/selenium-standalone-service` package, as [described here][18].
+Install `@wdio/selenium-standalone-service` package, as [described here][15].
 It is important to make sure it is compatible with current webdriverio version.
 
 Enable `wdio` plugin in plugins list and add `selenium-standalone` service:
@@ -928,7 +926,7 @@ Please note, this service can be used with Protractor helper as well!
 
 #### Sauce Service
 
-Install `@wdio/sauce-service` package, as [described here][19].
+Install `@wdio/sauce-service` package, as [described here][16].
 It is important to make sure it is compatible with current webdriverio version.
 
 Enable `wdio` plugin in plugins list and add `sauce` service:
@@ -970,28 +968,22 @@ In the same manner additional services from webdriverio can be installed, enable
 
 [6]: /basics/#pause
 
-[7]: https://github.com/GoogleChrome/puppeteer/blob/v1.12.2/docs/api.md#class-coverage
-
-[8]: https://github.com/istanbuljs/puppeteer-to-istanbul
-
-[9]: https://github.com/gotwarlost/istanbul
-
-[10]: https://aerokube.com/selenoid/
+[7]: https://aerokube.com/selenoid/
 
-[11]: https://aerokube.com/cm/latest/
+[8]: https://aerokube.com/cm/latest/
 
-[12]: https://hub.docker.com/u/selenoid
+[9]: https://hub.docker.com/u/selenoid
 
-[13]: https://aerokube.com/selenoid/latest/#_prepare_configuration
+[10]: https://aerokube.com/selenoid/latest/#_prepare_configuration
 
-[14]: https://aerokube.com/selenoid/latest/#_option_2_start_selenoid_container
+[11]: https://aerokube.com/selenoid/latest/#_option_2_start_selenoid_container
 
-[15]: https://docs.docker.com/engine/reference/commandline/create/
+[12]: https://docs.docker.com/engine/reference/commandline/create/
 
-[16]: https://codecept.io/img/codeceptjs-slideshow.gif
+[13]: https://codecept.io/img/codeceptjs-slideshow.gif
 
-[17]: https://webdriver.io
+[14]: https://webdriver.io
 
-[18]: https://webdriver.io/docs/selenium-standalone-service.html
+[15]: https://webdriver.io/docs/selenium-standalone-service.html
 
-[19]: https://webdriver.io/docs/sauce-service.html
+[16]: https://webdriver.io/docs/sauce-service.html

examples/coverage_test.js

@@ -0,0 +1,7 @@
+Feature('Local server');
+
+Scenario('Render buttons', ({ I }) => {
+  I.amOnPage('http://127.0.0.1:8080/');
+  I.see('Hi');
+  I.forceClick('Hi');
+});