I am currently working on a PHPUnit extension that allows setting up and working with a database in your automated tests. You can take a look at that library at cspray/database-testing-phpunit.
Currently, if an Extension throws an exception or triggers an error the tests still run. I understand the reasoning behind this, I've read through several of your responses to posted issues and appreciate explaining your rationale on certain things.
However, my use case I believe the tests should stop executing if an error occurs during database setup. I would like the test to fail with a descriptive message of what they need to do fix the problem. If database setup didn't work the rest of the tests that require that database shouldn't continue, in my opinion.
I am recommending that we allow users to define whether an extension should allow test failures. I believe this could be handled when the user registers the extension. For example:
<extensions>
<bootstrap class="Cspray\DatabaseTesting\PhpUnit\DatabaseTestingExtension" allowTestFailures="true" />
</extensions>
This value could be defaulted to "false" to preserve existing behavior. I haven't really considered the precise mechanism for how an extension would cause a failure, and don't have strong opinions on that subject.
I am willing to submit tested code that implements this feature, but before I got started on any real development work I would like buy-in from the maintainers and a clear plan for what we'd like to accomplish.
Thank you for all the hard work you put into PHPUnit. There aren't many libraries I install in every single PHP project I work on, but yours is one of them.
I am currently working on a PHPUnit extension that allows setting up and working with a database in your automated tests. You can take a look at that library at cspray/database-testing-phpunit.
Currently, if an Extension throws an exception or triggers an error the tests still run. I understand the reasoning behind this, I've read through several of your responses to posted issues and appreciate explaining your rationale on certain things.
However, my use case I believe the tests should stop executing if an error occurs during database setup. I would like the test to fail with a descriptive message of what they need to do fix the problem. If database setup didn't work the rest of the tests that require that database shouldn't continue, in my opinion.
I am recommending that we allow users to define whether an extension should allow test failures. I believe this could be handled when the user registers the extension. For example:
This value could be defaulted to
"false"to preserve existing behavior. I haven't really considered the precise mechanism for how an extension would cause a failure, and don't have strong opinions on that subject.I am willing to submit tested code that implements this feature, but before I got started on any real development work I would like buy-in from the maintainers and a clear plan for what we'd like to accomplish.
Thank you for all the hard work you put into PHPUnit. There aren't many libraries I install in every single PHP project I work on, but yours is one of them.