Open
Description
All modules under parsers
should have a public get_metrics
method that returns a dict documenting what metrics that parser supports. That method should also be used when constructing that parser's actual output. The parser should also have another method that returns some help text.
This should all be rendered into a summary in the help text of benchcomp run
, like this:
$ benchcomp run -h
# ...
You can use the following parsers for the `parser` key of each suite:
kani_perf
=========
Parses the output files after running the kani performance regression suite.
== Available metrics ==
* runtime
* symex_runtime
* solver_runtime
* passed
litani_cbmc
======
Parses the output of a Litani run of CBMC proofs using the AWS CBMC Starter kit.
== Available metrics ==
* runtime
* symex_runtime
* solver_runtime
* memory_kiB
* passed```