Allow finer-grained testing of subprocess stderr.

[cjerdonek]

This PR adds expect_stderr_warning and expect_stderr_error arguments to PipTestEnvironment to permit finer-grained testing of stderr. It allows checking whether errors and warnings were logged in the subprocess, rather than simply checking whether stderr is non-empty.

This is most immediately to help with PR #6339.

[cjerdonek]

Also, a benefit of this independent of PR #6339 is that in cases where we expect only a warning (e.g. deprecated Python versions), we can limit ourselves to being okay only with warnings and not also logged errors. In a subsequent PR, we can try making “expect_stderr” mean only “expect warning” and not “expect error.” This will make the tests more useful.

[xavfernandez]

Passing explicitely expect_stderr_warning=False, expect_stderr_error=True should raise then (and expect_stderr_warning should default to None, meaning either False or True depending on expect_stderr_error value)

[cjerdonek]

Thanks, @xavfernandez. I incorporated the suggestion in your comment -- including adding a test case to check that the exception works. In the process of doing so, I also added a number of other test cases to check that various combinations of the arguments are working as intended.

[xavfernandez]

Thanks 👍

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.