TestingCombinations.md

Testing Combinations

Contents

• The Basics
  • Passing in a Reporter
• Code samples

The Basics

You can use CombinationApprovals::verifyAllCombinations to test the content of multiple containers.

This makes a kind of approval test matrix, automatically testing all combinations of a set of inputs. It's a powerful way to quickly get very good test coverage.

In this small example, all combinations of {"hello", "world"} and {1, 2, 3} are being used:

TEST_CASE("YouCanVerifyCombinationsOf2") {
    std::vector<std::string> v{"hello", "world"};
    std::vector<int> numbers{1, 2, 3};
    CombinationApprovals::verifyAllCombinations(
            [](std::string s, int i){return std::make_pair(s, i);},
            v,
            numbers);
}

^{snippet source | anchor}

The format is carefully chosen to show both inputs and outputs, to make the test results easy to interpret. The output looks like this:

(hello, 1) => (hello, 1)
(hello, 2) => (hello, 2)
(hello, 3) => (hello, 3)
(world, 1) => (world, 1)
(world, 2) => (world, 2)
(world, 3) => (world, 3)

^{snippet source | anchor}

For advice on effective formatting, see To String. As you write out larger volumes of data in your approval files, experience has shown that the choice of layout of text in approval files can make a big difference to maintainability of tests, when failures occur.

Passing in a Reporter

Note: Over releases, the position of the optional Reporter parameter to verifyAllCombinations has changed, as the code has evolved:

Release	Position of optional Reporter argument
Before v.6.0.0	The optional Reporter argument goes after all the inputs
In v.6.0.0	The optional Reporter argument should be the second argument.
After v.6.0.0	The optional Reporter argument should be the first argument.

Code samples

CombinationApprovals::verifyAllCombinations(
        []( const std::string& input1, const int input2, const double input3)
        {
            return functionThatReturnsSomethingOutputStreamable(input1, input2, input3);
        }, // This is the converter function
        listOfInput1s,
        listOfInput2s,
        listOfInput3s);

^{snippet source | anchor}

If you are using C++14 or above, you can simplify this by using auto or auto& for the lambda parameters:

CombinationApprovals::verifyAllCombinations(
        []( auto& input1, auto& input2, auto& input3)
        {
            return functionThatReturnsSomethingOutputStreamable(input1, input2, input3);
        }, // This is the converter function
        listOfInput1s,
        listOfInput2s,
        listOfInput3s);

^{snippet source | anchor}

---

Back to User Guide