Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/benchmarks.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
<a id="top"></a>
# Authoring benchmarks

> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch 2.9.0.

_Note that benchmarking support is disabled by default and to enable it,
you need to define `CATCH_CONFIG_ENABLE_BENCHMARKING`. For more details,
see the [compile-time configuration documentation](configuration.md#top)._
Expand Down
8 changes: 8 additions & 0 deletions docs/command-line.md
Original file line number Diff line number Diff line change
Expand Up @@ -277,13 +277,17 @@ either before running any tests, after running all tests - or both, depending on
## Specify the number of benchmark samples to collect
<pre>--benchmark-samples &lt;# of samples&gt;</pre>

> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch 2.9.0.

When running benchmarks a number of "samples" is collected. This is the base data for later statistical analysis.
Per sample a clock resolution dependent number of iterations of the user code is run, which is independent of the number of samples. Defaults to 100.

<a id="benchmark-resamples"></a>
## Specify the number of resamples for bootstrapping
<pre>--benchmark-resamples &lt;# of resamples&gt;</pre>

> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch 2.9.0.

After the measurements are performed, statistical [bootstrapping] is performed
on the samples. The number of resamples for that bootstrapping is configurable
but defaults to 100000. Due to the bootstrapping it is possible to give
Expand All @@ -297,6 +301,8 @@ defaults to 95%).
## Specify the confidence-interval for bootstrapping
<pre>--benchmark-confidence-interval &lt;confidence-interval&gt;</pre>

> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch 2.9.0.

The confidence-interval is used for statistical bootstrapping on the samples to
calculate the upper and lower bounds of mean and standard deviation.
Must be between 0 and 1 and defaults to 0.95.
Expand All @@ -305,6 +311,8 @@ Must be between 0 and 1 and defaults to 0.95.
## Disable statistical analysis of collected benchmark samples
<pre>--benchmark-no-analysis</pre>

> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch 2.9.0.

When this flag is specified no bootstrapping or any other statistical analysis is performed.
Instead the user code is only measured and the plain mean from the samples is reported.

Expand Down
7 changes: 7 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,8 @@ Catch's selection, by defining either `CATCH_CONFIG_CPP11_TO_STRING` or
CATCH_CONFIG_CPP17_OPTIONAL // Override std::optional support detection (checked by CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER)
CATCH_CONFIG_CPP17_BYTE // Override std::byte support detection (Catch provides a StringMaker specialization by default)

> `CATCH_CONFIG_CPP17_STRING_VIEW` was [introduced](https://github.com/catchorg/Catch2/issues/1376) in Catch 2.4.1.

Catch contains basic compiler/standard detection and attempts to use
some C++17 features whenever appropriate. This automatic detection
can be manually overridden in both directions, that is, a feature
Expand Down Expand Up @@ -211,9 +213,14 @@ By default, Catch does not stringify some types from the standard library. This
CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER // Provide StringMaker specialization for std::optional (on C++17)
CATCH_CONFIG_ENABLE_ALL_STRINGMAKERS // Defines all of the above

> `CATCH_CONFIG_ENABLE_VARIANT_STRINGMAKER` was [introduced](https://github.com/catchorg/Catch2/issues/1380) in Catch 2.4.1.

> `CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER` was [introduced](https://github.com/catchorg/Catch2/issues/1510) in Catch 2.6.0.

## Disabling exceptions

> Introduced in Catch 2.4.0.

By default, Catch2 uses exceptions to signal errors and to abort tests
when an assertion from the `REQUIRE` family of assertions fails. We also
provide an experimental support for disabling exceptions. Catch2 should
Expand Down
7 changes: 7 additions & 0 deletions docs/generators.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
<a id="top"></a>
# Data Generators

> Introduced in Catch 2.6.0.

Data generators (also known as _data driven/parametrized test cases_)
let you reuse the same set of assertions across different input values.
In Catch2, this means that they respect the ordering and nesting
Expand Down Expand Up @@ -49,6 +51,8 @@ a test case,
* `RandomFloatGenerator<Float>` -- generates random Floats from range
* `RangeGenerator<T>` -- generates all values inside a specific range

> `ChunkGenerator<T>`, `RandomIntegerGenerator<Integral>`, `RandomFloatGenerator<Float>` and `RangeGenerator<T>` were introduced in Catch 2.7.0.

The generators also have associated helper functions that infer their
type, making their usage much nicer. These are

Expand All @@ -64,6 +68,7 @@ type, making their usage much nicer. These are
* `range(start, end)` for `RangeGenerator<T>` with a step size of `1`
* `range(start, end, step)` for `RangeGenerator<T>` with a custom step size

> `chunk()`, `random()` and both `range()` functions were introduced in Catch 2.7.0.

And can be used as shown in the example below to create a generator
that returns 100 odd random number:
Expand Down Expand Up @@ -96,6 +101,8 @@ scope and thus capturing references is dangerous. If you need to use
variables inside the generator expression, make sure you thought through
the lifetime implications and use `GENERATE_COPY` or `GENERATE_REF`.**

> `GENERATE_COPY` and `GENERATE_REF` were introduced in Catch 2.7.1.

You can also override the inferred type by using `as<type>` as the first
argument to the macro. This can be useful when dealing with string literals,
if you want them to come out as `std::string`:
Expand Down
4 changes: 4 additions & 0 deletions docs/logging.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ When the last `CHECK` fails in the "Bar" test case, then only one message will b

## Logging without local scope

> [Introduced](https://github.com/catchorg/Catch2/issues/1522) in Catch 2.7.0.

`UNSCOPED_INFO` is similar to `INFO` with two key differences:

- Lifetime of an unscoped message is not tied to its own scope.
Expand Down Expand Up @@ -104,6 +106,8 @@ This semicolon will be removed with next major version. It is highly advised to

**UNSCOPED_INFO(** _message expression_ **)**

> [Introduced](https://github.com/catchorg/Catch2/issues/1522) in Catch 2.7.0.

Similar to `INFO`, but messages are not limited to their own scope: They are removed from the buffer after each assertion, section or test case, whichever comes first.

**WARN(** _message expression_ **)**
Expand Down
4 changes: 4 additions & 0 deletions docs/other-macros.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,8 @@ TEST_CASE( "SUCCEED showcase" ) {

* `STATIC_REQUIRE`

> [Introduced](https://github.com/catchorg/Catch2/issues/1362) in Catch 2.4.2.

`STATIC_REQUIRE( expr )` is a macro that can be used the same way as a
`static_assert`, but also registers the success with Catch2, so it is
reported as a success at runtime. The whole check can also be deferred
Expand Down Expand Up @@ -132,6 +134,8 @@ ANON_TEST_CASE() {

* `DYNAMIC_SECTION`

> Introduced in Catch 2.3.0.

`DYNAMIC_SECTION` is a `SECTION` where the user can use `operator<<` to
create the final name for that section. This can be useful with e.g.
generators, or when creating a `SECTION` dynamically, within a loop.
Expand Down
13 changes: 12 additions & 1 deletion docs/test-cases-and-sections.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,10 +84,13 @@ This macro maps onto ```TEST_CASE``` and works in the same way, except that the

These macros map onto ```SECTION```s except that the section names are the _something_s prefixed by "given: ", "when: " or "then: " respectively.

* **AND_GIVEN(** _something_ **)**
* **AND_WHEN(** _something_ **)**
* **AND_THEN(** _something_ **)**

Similar to ```WHEN``` and ```THEN``` except that the prefixes start with "and ". These are used to chain ```WHEN```s and ```THEN```s together.
Similar to ```GIVEN```, ```WHEN``` and ```THEN``` except that the prefixes start with "and ". These are used to chain ```GIVEN```s, ```WHEN```s and ```THEN```s together.

> `AND_GIVEN` was [introduced](https://github.com/catchorg/Catch2/issues/1360) in Catch 2.4.0.

When any of these macros are used the console reporter recognises them and formats the test case header such that the Givens, Whens and Thens are aligned to aid readability.

Expand All @@ -101,6 +104,8 @@ by types, in the form of `TEMPLATE_TEST_CASE`,

* **TEMPLATE_TEST_CASE(** _test name_ , _tags_, _type1_, _type2_, ..., _typen_ **)**

> [Introduced](https://github.com/catchorg/Catch2/issues/1437) in Catch 2.5.0.

_test name_ and _tag_ are exactly the same as they are in `TEST_CASE`,
with the difference that the tag string must be provided (however, it
can be empty). _type1_ through _typen_ is the list of types for which
Expand Down Expand Up @@ -151,6 +156,8 @@ TEMPLATE_TEST_CASE( "vectors can be sized and resized", "[vector][template]", in

* **TEMPLATE_PRODUCT_TEST_CASE(** _test name_ , _tags_, (_template-type1_, _template-type2_, ..., _template-typen_), (_template-arg1_, _template-arg2_, ..., _template-argm_) **)**

> [Introduced](https://github.com/catchorg/Catch2/issues/1468) in Catch 2.6.0.

_template-type1_ through _template-typen_ is list of template template
types which should be combined with each of _template-arg1_ through
_template-argm_, resulting in _n * m_ test cases. Inside the test case,
Expand Down Expand Up @@ -194,6 +201,8 @@ is very high and should not be encountered in practice._

* **TEMPLATE_LIST_TEST_CASE(** _test name_, _tags_, _type list_ **)**

> [Introduced](https://github.com/catchorg/Catch2/issues/1627) in Catch 2.9.0.

_type list_ is a generic list of types on which test case should be instantiated.
List can be `std::tuple`, `boost::mpl::list`, `boost::mp11::mp_list` or anything with
`template <typename...>` signature.
Expand All @@ -212,6 +221,8 @@ TEMPLATE_LIST_TEST_CASE("Template test case with test types specified inside std

## Signature based parametrised test cases

> [Introduced](https://github.com/catchorg/Catch2/issues/1609) in Catch 2.8.0.

In addition to [type parametrised test cases](#type-parametrised-test-cases) Catch2 also supports
signature base parametrised test cases, in form of `TEMPLATE_TEST_CASE_SIG` and `TEMPLATE_PRODUCT_TEST_CASE_SIG`.
These test cases have similar syntax like [type parametrised test cases](#type-parametrised-test-cases), with one
Expand Down
14 changes: 14 additions & 0 deletions docs/test-fixtures.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
<a id="top"></a>
# Test fixtures

## Defining test fixtures

Although Catch allows you to group tests together as sections within a test case, it can still be convenient, sometimes, to group them using a more traditional test fixture. Catch fully supports this too. You define the test fixture as a simple structure:

```c++
Expand Down Expand Up @@ -84,6 +86,9 @@ _While there is an upper limit on the number of types you can specify
in single `TEMPLATE_TEST_CASE_METHOD` or `TEMPLATE_PRODUCT_TEST_CASE_METHOD`,
the limit is very high and should not be encountered in practice._

## Signature-based parametrised test fixtures

> [Introduced](https://github.com/catchorg/Catch2/issues/1609) in Catch 2.8.0.

Catch2 also provides `TEMPLATE_TEST_CASE_METHOD_SIG` and `TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG` to support
fixtures using non-type template parameters. These test cases work similar to `TEMPLATE_TEST_CASE_METHOD` and `TEMPLATE_PRODUCT_TEST_CASE_METHOD`,
Expand All @@ -100,6 +105,13 @@ TEMPLATE_TEST_CASE_METHOD_SIG(Nttp_Fixture, "A TEMPLATE_TEST_CASE_METHOD_SIG bas
REQUIRE(Nttp_Fixture<V>::value > 0);
}

template<typename T>
struct Template_Fixture_2 {
Template_Fixture_2() {}

T m_a;
};

template< typename T, size_t V>
struct Template_Foo_2 {
size_t size() { return V; }
Expand All @@ -111,6 +123,8 @@ TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG(Template_Fixture_2, "A TEMPLATE_PRODUCT_TE
}
```

## Template fixtures with types specified in template type lists

Catch2 also provides `TEMPLATE_LIST_TEST_CASE_METHOD` to support template fixtures with types specified in
template type lists like `std::tuple`, `boost::mpl::list` or `boost::mp11::mp_list`. This test case works the same as `TEMPLATE_TEST_CASE_METHOD`,
only difference is the source of types. This allows you to reuse the template type list in multiple test cases.
Expand Down
4 changes: 4 additions & 0 deletions docs/tostring.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,8 @@ CATCH_TRANSLATE_EXCEPTION( MyType& ex ) {

## Enums

> Introduced in Catch 2.8.0.

Enums that already have a `<<` overload for `std::ostream` will convert to strings as expected.
If you only need to convert enums to strings for test reporting purposes you can provide a `StringMaker` specialisations as any other type.
However, as a convenience, Catch provides the `REGISTER_ENUM` helper macro that will generate the `StringMaker` specialiation for you with minimal code.
Expand Down Expand Up @@ -108,6 +110,8 @@ TEST_CASE() {

## Floating point precision

> [Introduced](https://github.com/catchorg/Catch2/issues/1614) in Catch 2.8.0.

Catch provides a built-in `StringMaker` specialization for both `float`
and `double`. By default, it uses what we think is a reasonable precision,
but you can customize it by modifying the `precision` static variable
Expand Down
94 changes: 94 additions & 0 deletions scripts/extractFeaturesFromReleaseNotes.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
#!/usr/bin/env python

#
# extractFeaturesFromReleaseNotes.py
#
# Read the release notes - docs/release-notes.md - and generate text
# for pasting in to individual documentation pages, to indicate which
# versions recent features were released in.
#
# Using the output of the file is easier than manually constructing
# the text to paste in to documentation pages.
#
# One way to use this:
# - run this script, saving the output to some temporary file
# - diff this output with the actual release notes page
# - the differences are Markdown text that can be pasted in to the
# appropriate documentation pages in the docs/ directory.
# - each release also has a github link to show which documentation files
# were changed in it.
# This can be helpful to see which documentation pages
# to add the 'Introduced in Catch ...' snippets to the relevant pages.
#

from __future__ import print_function

import re


def create_introduced_in_text(version, bug_number = None):
"""Generate text to paste in to documentation file"""
if bug_number:
return '> [Introduced](https://github.com/catchorg/Catch2/issues/%s) in Catch %s.' % (bug_number, version)
else:
# Use this text for changes that don't have issue numbers
return '> Introduced in Catch %s.' % version


def link_to_changes_in_release(release, releases):
"""
Markdown text for a hyperlink showing all edits in a release, or empty string

:param release: A release version, as a string
:param releases: A container of releases, in descending order - newest to oldest
:return: Markdown text for a hyperlink showing the differences between the give release and the prior one,
or empty string, if the previous release is not known
"""

if release == releases[-1]:
# This is the earliest release we know about
return ''
index = releases.index(release)
previous_release = releases[index + 1]
return '\n[Changes in %s](https://github.com/catchorg/Catch2/compare/v%s...v%s)' % (release, previous_release, release)


def write_recent_release_notes_with_introduced_text():
current_version = None
release_toc_regex = r'\[(\d.\d.\d)\]\(#\d+\)<br>'
issue_number_regex = r'#[0-9]+'
releases = []
with open('../docs/release-notes.md') as release_notes:
for line in release_notes:
line = line[:-1]
print(line)

# Extract version number from table of contents
match = re.search(release_toc_regex, line)
if match:
release_name = match.group(1)
releases.append(release_name)

if line.startswith('## '):
# It's a section with version number
current_version = line.replace('## ', '')

# We decided not to add released-date info for older versions
if current_version == 'Older versions':
break

print(create_introduced_in_text(current_version))
print(link_to_changes_in_release(current_version, releases))

# Not yet found a version number, so to avoid picking up hyperlinks to
# version numbers in the index, keep going
if not current_version:
continue

for bug_link in re.findall(issue_number_regex, line):
bug_number = bug_link.replace('#', '')
print(create_introduced_in_text(current_version, bug_number))


if __name__ == '__main__':
write_recent_release_notes_with_introduced_text()