An automation testing framework based on Cypress for REST API and WebUI testing using Page Object Model, Mocha, Lighthouse and CI intergrated with Github Actions, CircleCI, GitlabCI and Bitbucket Pipelines. Running with spec file. No BDD.

Table of Contents

• Changelogs
• Features
• Installation
• Basic Usage
• Write Test
• Set Up
• Author
• License

Changelogs

.....

⭐ 28/08/2021 : Update the comparing key value of API test (which not correct if return a array of object)

⭐ 30/08/2021 : Add schema validator with ajv

⭐ 31/08/2021 : CI test with Github Actions, CircleCI, Travis CI, Bitbucket Pipeleines, Gitlab Pipelines

Features

⚙️ Testing across REST API and Web UI applications

⚙️ Multiple browser automation

⚙️ Multiple reports (JSON, html) with failed screenshots attached

⚙️ CI integrated (CircleCI, Github Actions, GitlabCI, Bitbucket Pipelines)

⚙️ Performance testing with Lighthouse and Pa11y

⚙️ Visual regression testing supported

⚙️ Page Object Model

⚙️ Schema Validator with ajv

⚙️ Parallel runs with CI provider

⚙️ Parallel runs with Cypress Dashboard

⚙️ And other functions inherited from Cypress

Installation

Use npm to install the dependencies

    npm install

Basic Usage

Open cypress

    npx cypress open

Run all features

    npx cypress run 

Run all features on specific browser

    npx cypress run --browser firefox

Run all features on headless mode

    npx cypress run --headless

Run a specific feature file

    npx cypress run --spec cypress/integration/{spec.js file}

Run multiple feature files

    npx cypress run --spec "cypress/integration/atwt-api/*.spec.js"  --browser chrome

More details: https://docs.cypress.io/guides/guides/command-line

Write test

Write UI Test Case

Add page file with locators and methods

• Head to cypress/support/pages and create new .page.js file
• List all locators of elements in the page
• Write methods that supports your test validations/actions
• Export the page object

Add new test spec file and write test

• Head to cypress/integration/atwt-ui and create new .spec.js file
• Import the created page object that including the functions/validators you want to use
• "describe" tag for test suite
• "context" tag for group of test cases by condition
• "it" tag for test case
• Initialize page object and call methods/validators for testing purpose

Write API Test Case

Add new test spec file and write test

• Head to cypress/integration/atwt-api and create new .spec.js file
• Import the api-action
• "describe" tag for test suite
• "context" tag for group of test cases by condition
• "it" tag for test case
• Some bilt-in functions: makeRequest(), validateResponseKeyValue(), getResponseBodyValue(), setHeader(), ...

Validate resposne schema

• Create new schema file inside support/schema as following

• Import the schema configuration in your test spec.js file
• Call cy.validateSchema(, apiAction.getResponseBody()) to validate schema of response

Store run-time parameters

• Store the run-time parameter by adding key and value: cy.addRuntimeVariable(, )
• Get the stored parameter by cy.getRuntimeVariable()
• Remmeber to .then() since Cypress command is asynchonous

Set Up

Multiple reports option

1. Install cypress-multi-reporters

    npm install --save-dev cypress-multi-reporters

2. Install cypress-mochawesome-reporter and mocha-junit-reporter

    npm install --save-dev cypress-mochawesome-reporter,mocha-junit-reporter

3. Config reporter in cypress.json
   • Define the reports we want to use by "reporterEnabled" tag
   • For Mocha Junit Report:
     • "mochaFile" to define the output report structure
     • [hash] to generate hash string which helps distinquish report files
     • "jenkinsMode" to generate more beautiful report displayed on Jenkins
   • For Mochawesome Report:
     • "chart" to include chart counts
     • "reportPageTitle" to specify report title
     • "reportFilename" and "reportDir" to specify name and locations of output reports
     • "embeddedScreenshots" to attach screenshots for failed cases
     • "timestamp" to append timestamp to report name, prevent replacement

4. After executions, reports are located in cypress/reports
5. Head to cypress-multi-reporters for more configurations

Visual regression testing

1. Install cypress-pugin-snapshots

    npm install --save-dev cypress-plugin-snapshots

2. Add this config to cypress.json file

    "ignoreTestFiles": [
      "**/__snapshots__/*",
      "**/__image_snapshots__/*"
    ]

3. Modify /plugin/index.js with

    const { initPlugin } = require('cypress-plugin-snapshots/plugin');

    module.exports = (on, config) => {
        initPlugin(on, config);
    }

4. Import this into /support/index.js

    import 'cypress-plugin-snapshots/commands';

5. Config the settings

6. Take snapshot

    cy.toMatchSnapshot();

    cy.get(<element>).toMatchSnapshot();

7. Run the test the first time to get the base snapshots

8. To update snapshots when there are changes where expected

    npx cypress run --env updateSnapshots=true --spec cypress/integration/atwt-ui/ui-regression.spec.js --browser chrome

9. Head to cypress-plugin-snapshots for more configurations

Performance Testing with Google Lighthouse and Pa11y

1. Install cypress-audit

    npm install --save-dev cypress-audit

2. Set up prepareAudit when brower launch in /plugin/index.js

    on('before:browser:launch', (browser = {}, launchOptions) => {
        prepareAudit(launchOptions); });

3. Set up task to write test result into report files
4. Use cy.lighthouse() or cy.pa11y() to run performance testing

5. After executions, reports are located in cypress/reports

Schema Validator with ajv

1. Install ajv

    npm install --save-dev ajv

2. Add new command vlidateSchema in /support/validate-schema-command.js

    export const validateSchema = (schema, response) => {
      const ajv = new Ajv();
      const validate = ajv.compile(schema);
      const valid = validate(response);

      if (!valid) {
        getSchemaError(validate.errors).then((schemaError) => {
          throw new Error(schemaError);
        });
      } else {
        cy.log("Schema validated!");
      }
    };

3. Add the defined command in support/commands.js of Cypress

    import { validateSchema } from "./validate-schema-command";
	
    Cypress.Commands.add("validateSchema", validateSchema);

4. Create schema file in support/schema
5. Import schema file and call cy.validateSchema() to validate response schema
6. Head to AJV for more configurations

Custom response body value validation method

Parallel run with CI provider and Cypress Dashboard

Cypress Dashboard

CI	Build status	Config File	Set up Cypress Dashboard
CircleCI		config.yml	Already set up with CYPRESS_RECORD_KEY as enviroment variable in CircleCI
CircleCI - v2 Config without Orbs	Not activated	.circleci/config-without-orbs.yml	Already set up with CYPRESS_RECORD_KEY as enviroment variable in CircleCI
TravisCI		.travis.yml	Comment out code lines with record on Cypress Dashboard
GitlabCI		.gitlab-ci.yml	Comment out code lines with record on Cypress Dashboard
Github Actions		main.yml	Comment out code lines with record on Cypress Dashboard
Github Actions - Sync Bitbucket		sync-bitbucket-https.yml	To sync code to Bitbucket
Github Actions - Sync Gitlab		sync-gitlab-https.yml	To sync code to Gitlab

Author

Tuyen Nguyen - Senior QA Automation Engineer

trongtuyen96@gmail.com

License

Copyright 2021 Tuyen Nguyen

   Licensed under the Apache License, Version 2.0 (the "License");
   you may not use this file except in compliance with the License.
   You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.