Description
Originally reported by: Jurko Gospodnetić (BitBucket: jurko, GitHub: jurko)
My suggestion is to make regular test output capture include output from function scoped fixture finalizers.
The same might not be desirable for fixtures with larger scopes as their finalizers are not run for every test using them, and they are not included in this proposal.
Here's a scenario that should help you understand why I think this function level fixture behaviour is desirable:
Imagine you are a new ptytest user not deeply familiar with pytest specific APIs. And you want your tests to output additional calculated information in case of failure (e.g. some test related output collected from an external process). Outputting that information to stdout will work fine, since pytest will actually display that output only for failing tests. The only remaining question is when to output that information.
If it is only a few tests, this output can simply be added in a finally clause wrapping the whole test code, but in case this is needed in a lot of tests, one tries to think of a better solution. The first thing that pops to mind is to implement a fixture that would output this information in its teardown.
You check the pytest documentation for 'teardown' and easily find that you implement this with pytest using a function scoped fixture with teardown code contained as its finalizer routine.
Now you add the finalizer, add your code and everything should work fine... but alas... it is not. You run your failing test and no output appears.
Example:
import pytest
@pytest.fixture
def passwd(request):
print("setup before yield")
def finalizer():
print("teardown after yield")
request.addfinalizer(finalizer)
return "gaga"
def test_has_lines(passwd):
print("test called (%s)" % (passwd,))
pytest.fail()
And you're pretty much out of ideas other than the ugly solution to manually wrap all your tests in a try:/finally: as suggested before.
You take another stab at the docs, and run into an experimental yield based fixture feature so you decide to try that. You rework your code, but in the end it suffers from the same problem - the teartdown output simply does not get captured.
Example:
import pytest
@pytest.yield_fixture
def passwd():
print("setup before yield")
yield "gaga"
print("teardown after yield")
def test_has_lines(passwd):
print("test called (%s)" % (passwd,))
pytest.fail()
Now you are completely out of options unless you dive into pytest source code or ask someone much more experienced with pytest who can point you in the direction of undocumented pytest report hooks or some other such expose-pytest-bowels solution.
Even if you find out how to do this you will have wasted a lot more time on this then you'd like.
Adding and/or another pytest specific solution could work, but a new user would still expect the approach above to work, and nothing in his way would warn him that this was a dead end.
My suggestion is to just let pytest consistently capture function scope fixture finalizer output (yield based or not). That should allow intuitive pytest usage using standard test framework patterns without having to search for or learn additional pytest specific functionality.
If this is added, additional pytest specific functionality could be used (if documented and made public) to improve the solution or make its results even nicer, but pytest should still allow the user to quickly solve his problem in a way consistent with typical testing framework usage.
Hope this helps.
Best regards,
Jurko Gospodnetić