Visual Testing in General, Guide Version 1

docs/visual.md

@@ -0,0 +1,169 @@
+# Visual Testing
+
+How does one test if the UI being rendered appears correctly to the users or how to test if each UI element appears in the right position and size? The traditional way to test the UI of the application has always been manually, which is time consuming.
+
+Visual testing with help of CodeceptJS will help in improving such use cases for the QA folks.
+
+By default CodeceptJS uses [WebDriver](/helpers/WebDriver/) helper and **Selenium** to automate browser. It is also capable of taking screenshots of the application and this could be used for visual testing.
+
+Currently there are two helpers available for Visual testing with CodeceptJS
+
+## Using Resemble helper
+[Resemble.js](https://github.com/rsmbl/Resemble.js) is a great tool for image comparison and analysis, which can be used with CodeceptJS
+### Setup
+To install the package, just run `npm install codeceptjs-resemblehelper`
+### Configuring
+This helper should be added in codecept.json/codecept.conf.js
+
+Example:
+
+```json
+{
+   "helpers": {
+     "ResembleHelper" : {
+       "require": "codeceptjs-resemblehelper",
+       "screenshotFolder" : "./tests/output/",
+       "baseFolder": "./tests/screenshots/base/",
+       "diffFolder": "./tests/screenshots/diff/"
+     }
+   }
+}
+```
+
+To use the Helper, users must provide the three parameters:
+
+`screenshotFolder` : This will always have the same value as `output` in Codecept configuration, this is the folder where webdriverIO
+saves a screenshot when using `I.saveScreenshot` method
+
+`baseFolder`: This is the folder for base images, which will be used with screenshot for comparison
+
+`diffFolder`: This will the folder where resemble would try to store the difference image, which can be viewed later.
+
+### Usage
+Details about the helper can be found on the [Github Repo](https://github.com/puneet0191/codeceptjs-resemblehelper)
+
+Base Image is compared with the screenshot image and test results are derived based on the `mismatch tolerance` level provided by the user for the comparison
+
+### Example
+Lets consider visual testing for [CodeceptJS Home](http://codecept.io)
+
+```js
+Feature('To test screen comparison with resemble Js Example test');
+
+Scenario('Compare CodeceptIO Home Page @visual-test', async (I, adminPage) => {
+    I.amOnPage("/");
+    I.saveScreenshot("Codecept_IO_Screenshot_Image.png");
+    I.seeVisualDiff("Codecept_IO_Screenshot_Image.png", {tolerance: 2, prepareBaseImage: false});
+});
+```
+In this example, we are setting the expected mismatch tolerance level as `2`
+
+`Base Image` (Generated by User)
+![Base Image](images/Codecept_IO_Base_Image.png)
+
+`Screenshot Image` (Generated by Test)
+![Screenshot Image](images/Codecept_IO_Screenshot_Image.png)
+
+Clearly the difference in both the images visible to human eye is the section about `Scenario Driven`
+
+![Difference Image](images/difference_Image_Codecept_Home.png)
+
+`Diff Image` generated by the helper clearly highlights the section which don't match
+
+![Highlight](images/Difference%20Image%20Focus.png)
+
+`Failed Test output`
+```
+To test screen comparison with resemble Js Example test --
+  Compare CodeceptIO Home Page @visual-test
+    I see Visual Diff "Codecept_IO_Screenshot_Image.png", {tolerance: 2, prepareBaseImage: false}
+MisMatch Percentage Calculated is 2.85
+  ✖ FAILED in 418ms
+
+
+-- FAILURES:
+
+  1) To test screen comparison with resemble Js Example test
+       Compare CodeceptIO Home Page @visual-test:
+
+      MissMatch Percentage 2.85
+      + expected - actual
+
+      -false
+      +true
+```
+
+`Codeceptjs-resemblehelper` basically comes with two major functions
+
+1) `seeVisualDiff` which can be used to compare two images and calculate the misMatch percentage.
+2) `seeVisualDiffForElement` which can be used to compare elements on the two images and calculate misMatch percentage.
+
+## Using Visual Knight
+
+Visual Knight is a SaaS product which strongly supports CodeceptJS with multiple use cases. It provides an user interface to handle mismatches, statistics and more. It was designed to support Designer, Product Owner and other roles which are not familiar with coding and all this tools. All captured images are saved in a secure cloud system to not mess up your git repository.
+
+### Setup
+Create an account at [Visual Knight](https://www.visual-knight.io) and install the npm package
+```npm install @visual-knight/codeceptjs -D```
+
+### Configuring
+
+```json
+{
+    "helpers": {
+        "VisualKnight": {
+            "require": "@visual-knight/codeceptjs",
+            "key": "YOUR_API_KEY",
+            "project": "YOUR_PROJECT_ID OR PROJECT_NAME"
+        }
+    }
+}
+```
+
+### Usage
+
+```javascript
+/**
+ * @param testName {string} Is provided to visual knight (must be unique)
+ * @param options {ScreenshotOptions} Contains additional settings
+ */
+I.compareFullpageScreenshot(testName, options)
+/**
+ * @param testName {string} Is provided to visual knight (must be unique)
+ * @param options {ScreenshotOptions} Contains additional settings
+ */
+I.compareViewportScreenshot(testName, options)
+/**
+ * @param cssSelector {string} Is provided to visual knight
+ * @param testName {string} Is provided to visual knight (must be unique)
+ * @param options {ScreenshotOptions} Contains additional settings
+ */
+I.compareElementScreenshot(cssSelector, testName, options)
+
+/*
+ScreenshotOptions {
+  hide?: string[] // Array of css selectors which gets hidden by "opacity: 0",
+  remove?: string[] // Array of css selectors which gets hidden by "display: none",
+  additional?: object // Data is saved as relation to the variation. (Future: can be used for filtering)
+}
+*/
+```
+
+> You can find the latest documentation here [CodeceptJS helper page](https://doc.visual-knight.io/adapters/codeceptjs)
+
+### Example
+Lets consider visual testing for [CodeceptJS Home](http://codecept.io)
+
+```js
+Feature('To test screen comparison with Visual Knight Example test');
+
+Scenario('Compare CodeceptIO Home Page @visual-test', async (I, adminPage) => {
+    I.amOnPage("/");
+    I.compareFullpageScreenshot("CodeceptIO Home Page")
+});
+```
+
+Depending of your configuration this test will fail if no baseline exists and log the link to the image to accept or automatically accept the first run as baseline.
+> You can accept the first image as baseline automatically via ```autoBaseline: true``` _default is false_
+
+### done()