Add test_matrix macro

[overhacked]

The test_matrix macro generates test cases from the Cartesian product of possible values given for test function arguments. For each argument, it supports lists of possible values, numeric ranges, or single expressions (for constant arguments). It also supports the => syntax for asserting test case output, which is handled identically to test_case, because the parsed TestCaseExpression is cloned and included in each generated TestCase before it is rendered.

Acceptance tests and basic documentation are included in this PR. I'd like to also update the Wiki documentation, and will open a parallel PR to the wiki repo.

Usage example

#[test_matrix(
    [-2, 2],
    [-4, 4]
)]
fn multiplication_tests(x: i8, y: i8) {
    let actual = (x * y).abs();

    assert_eq!(8, actual)
}

Related Issues

• Would completely resolve feature request: test_case generators #111
• Partially resolves Generate test case for each enum variant #94, if Enum variants were listed manually or could be generated by some other macro.

[overhacked]

I've just discovered that there is no way to fork a GitHub Wiki. Seriously, GitHub!? 👹 So, if you decide to merge this PR, I can send you updated Wiki Markdown files at that time. I've cloned the wiki into my forked repo, so you can see my proposed changes... just not in the nice PR interface. 🤷

[luke-biel]

Thanks for contributing :) I think I like the approach. I'll look into this over the weekend. If I didn't answer by Monday, feel free to ping me.

[overhacked]

Proposed Wiki changes are previewable at https://github.com/overhacked/test-case/wiki. Diff at https://github.com/overhacked/test-case/wiki/_compare/11bdf1109e08b9d1c54ab30ef3eff7254a5059ac...ed4b3c49a6b1072b9ffab3822e1d8a3d53cffcde.

[overhacked]

@luke-biel, if you have a second to approve running the CI workflow before Monday, I've got time over the weekend to troubleshoot any possible CI issues.

[frondeus]

@luke-biel, if you have a second to approve running the CI workflow before Monday, I've got time over the weekend to troubleshoot any possible CI issues.

Approved :)

[frondeus]

I definitely like what I see, however it might be a good idea to have this feature behind a flag. What I noticed at first glance, is that we add Itertools dependency - which might affect build times for those that are not going to use test_matrix.

What do you think?

[overhacked]

Fixed the CI issues. I didn't understand the nightly/stable cargo insta setup in the repo at first, but then I discovered scripts/test_all.sh. 😄

Also, the last commit in the series is a crazy idea that I'm not even sure I'm 💯 on board with. The itertools code base is MIT licensed, so what if, instead of feature-gating the external dependency, you just vendor the one module for making a Cartesian product out of iterators into the test-case code? It's basically self-contained in itertools; I just brought in two helper functions and changed one call from an itertools::Itertools trait function to a call to std::iter::Iterator::filter_map() that works the same.

Happy to discuss. Totally fine if you're not at all on board with it.

[luke-biel]

Finally had time to tackle this. Code looked good, didn't need to contact you. The itertools mod copy-paste was something I was considering suggesting earlier, so I'm happy it was done :)