Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow UT assert tests to specify default verbosity #1317

Open
jphickey opened this issue Oct 21, 2022 · 0 comments
Open

Allow UT assert tests to specify default verbosity #1317

jphickey opened this issue Oct 21, 2022 · 0 comments

Comments

@jphickey
Copy link
Contributor

Is your feature request related to a problem? Please describe.
By default, UT assert runs in verbose (not debug) mode, where all calls to any assert macro are added to a log, except for DEBUG messages. This means there is a record of tests that PASS, which is generally desired for coverage testing.

However, if doing fuzz testing, the point is to run a test against a large, random data set and as such there may be thousands (or even millions) of assertions. This will result in a very large log file full of "PASS" messages.

UT Assert does allow the verbosity to be set, but currently only via command line on a run-by-run basis.

Describe the solution you'd like
Allow a test to specify what its default verbosity is supposed to be. For fuzz testing this can be a lower value than the normal coverage testing default.

Describe alternatives you've considered
Currently test cases that run in a loop or otherwise would generate a large number of PASS results use a pattern where the result is checked directly in the test code, e.g.:

if (status != expected)
{
    UtAssert_Failed("message");
}

Such that an assertion is only logged if it fails. However, a side effect of this means that the "pass" counter and total test counter does not increment to reflect that something was tested/checked - it only increments if it fails. In contrast, by using the normal assert macro but reducing the verbosity means that the counters will still increment, it just doesn't write a log entry for PASS cases. Furthermore, a user could manually increase the log level if they actually do want to see them all.

Requester Info
Joseph Hickey, Vantage Systems, Inc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant