Coverage of assert statements.

[nedbat]

Originally reported by npinto (Bitbucket: npinto, GitHub: npinto)

---

Hello,

I'm trying to use coverage.py to get the coverage (duh!) of both outcomes in assert statements [1]. Am I missing something or is this not (yet) possible?

Let me know how can I contribute.

Nicolas

[1] In our code base we use many (possibly very slow) asserts during development, this is really useful in production for performance reasons since we can use python -OO to remove the execution of asserts.

---

• Bitbucket: https://bitbucket.org/ned/coveragepy/issue/160

[nedbat]

Coverage.py can't influence what code paths you execute, it merely monitors what code paths are executed. It's up to you to design a test suite that exercises all of the test cases you need tested. Maybe I don't understand your problem?

[nedbat]

Original comment by npinto (Bitbucket: npinto, GitHub: npinto)

---

But coverage.py won't tell me if my test suite elicited both path of an assert statement (True or False). Any thoughts on how to expose this in coverage.py ?

[nedbat]

I see what you mean. There are a few things that come to mind:

1. I didn't realize this, but an assert statement is actually compiled as if it were: if condition: raise AssertionError, so I suppose there's enough information in the bytecode to do what you want. Actually, I wonder why it doesn't already do what you want.

2. It's not clear to me that everyone would want to require that all their asserts go both ways to consider their code fully covered.

[nedbat]

Original comment by Anonymous

---

Thanks for the fast answer.

Re: 1)

Is there an easy way to expose this, and maybe enable this behavior with a command line option?

Re: 2)

This is a valid point, but I like using asserts since they are very concise and easily removed with python -OO for production.

I'll be happy to take a stab at the code but since I'm not "in" the codebase right now, I would appreciate pointers / strategies on how to do this optimally (i.e. that will fit your requirements to get merged).

Thanks again.

[nedbat]

Original comment by npinto (Bitbucket: npinto, GitHub: npinto)

---

Ned,

I'll be happy to contribute a pull request if you let me know how you would like this to be exposed, of course only if you think the feature is interesting.

Thanks.

Nicolas

[nedbat]

Earlier I said "I don't know why it doesn't already do this." It's because the compiled code is all associated with the same source line. For example, coverage also can't do branch coverage for a one-line if like this:

if a: print b

So something significant would have to change in coverage.py for it to be able to detect whether asserts went both ways. Possibilities include:

Preprocess the source to change assert statements

Preprocess the bytecode to fiddle with the line numbers

Change the trace function to record exceptions raised

None of these are simple, and represent a major new feature in coverage.py.

[nedbat]

Original comment by npinto (Bitbucket: npinto, GitHub: npinto)

---

Thanks Ned for the quick reply.

I'd like to try "1." (i.e. code transformation before coverage), at least get the ball rolling. What would be the best entry point to begin this?

Thanks again.

[nedbat]

Hmm, I'm reluctant to start transforming source code, especially since it would throw off the source code reporting. I'm not sure any of these approaches is a good idea, but #3 fits the current code base best. I'm honestly not sure what information is available in the trace function for tracking exceptions, but you can give it a try. Look in collector.py, line 96 to get started.

[nedbat]

Original comment by Felipe (Bitbucket: felipeochoa, GitHub: felipeochoa)

---

Not at all familiar with the coverage.py code, but given that the 3 proposed alternatives seemed so difficult let me chime in with a 4th:

Add a # pragma: raises Exception_1 Exception_2 ... pragma that lets the user hint what exceptions are potentially raised on a given line. Then, in parallel/after the existing coverage.py analysis run a simple scan to parse out try/except clauses using ast and tokenize to pull out the comments (like in your SO answer here). You could then add a branch from the pragma line to the error handler.You could start out only supporting intra-file raises pragmas for simplicity.