Skip to content

Automated Testing

Jump to bottom
Charles Crossan edited this page Sep 7, 2019 · 16 revisions

Why Automated Testing?

Nobody likes when software is broken, but it's a really unpleasant feeling to know you broke the software for others. As a first line of defense against broken software, ChurchCRM uses automated, browser based testing. We've developed a set of scripts that emulates a real user by clicking and typing in strategic locations within the application to ensure that all of the features work as intended.

We've automated testing of some of the critical functionality. This way, components are tested every time any code is changed.

Details

There are many different automated testing strategies (unit testing, integration testing, etc). As ChurchCRM matures, so will our testing strategies; however, we're currently only using browser automation testing. This form of testing gives us the largest amount of code coverage for the smallest amount of testing code.

Toolset

  • Behat

Behat is a PHP framework for driving the various browser automation test drivers

  • MinkExtension

MinkExtension provides additional capabilities to Behat

  • Gherkin

Gherkin is the language used to write the browser automation tests. It is considered an External Domain Specific Language because it allows a QA developer to write a test for ChurchCRM without any PHP knowledge. The tests read as "plain English" Syntax Examples: https://gist.github.com/mnapoli/5848556

  • SauceLabs

SauceLabs generously provides us with hosting of a Slenium Grid. Think: a server farm running a bunch of web browsers, clicking on things automagically.

  • Selenium

Selenium is an open source project that allows QA developers to interact with browsers in a scripted, automated manner.

https://github.com/minkphp/MinkSelenium2Driver http://docs.behat.org/en/v2.5/guides/7.config.html https://michaelheap.com/behat-selenium2-webdriver-with-minkextension/ http://mink.behat.org/en/latest/drivers/selenium2.html

Tests

Page Tests

ChurchCRM uses Behat for page load testing Behavior Driven Development.

Running Behat Page Tests

  1. While SSH'd into the vagrant development box, run the following commands:
cd /vagrant
npm test
  1. You should see the result of all individual tests.

Writing Behat Page Tests

Behat page tests use the gherkin syntax.
Behat uses tests written with this syntax to execute requests and validate responses against the defined pages.

All tests should follow this outline:

Feature: Calendar
  In order to see calendar events
  As a User
  I am able to visit the calendar

  Scenario: Open the calendar
    Given I am authenticated as "admin" using "changeme"
    And  I am on "/calendar.php"
    Then I should see "Church Calendar"

Useful reference tests

Unit Tests

Unit testing is not yet fully implemented.

View Test Results

Accessing the TravisCI job

  1. On the bottom of the PR for which you want to view test results, click "Details" next to the appropriate test:

    Access to the TravisCI tests for a specific commit is also possible by clicking the "Test Status Indicator" to the right of the commit:

Viewing Job Details

  1. Click on "Details"

  2. Click one of the builds

  3. Review the logs. The bottom of the log will contain Selenium test results:

Restarting a job

  1. From the TravisCI job page, click the "Restart Job" button for either all jobs, or a specific job:

Links

Website - Demo System - Twitter @church_crm

Clone this wiki locally