Engine Driver Api

NOTE: This page is a specification that was used as a starting point for creating the feature in NUnit. It needs to be reviewed and revised in order to accurately reflect what was actually built. If you take it with a grain of salt, it may still be helpful to you as documentation. This notice will be removed when the page is brought up to date.

The TestEngine uses drivers to interface with frameworks. This isolates framework-specific code from the more general code used to load and run tests. Each test framework that is used with the engine needs a driver to support it. While a driver could theoretically support multiple frameworks, we expect the normal case to be a one-to-one mapping of drivers to frameworks.

NUnit itself treats its 2.x and 3.0 framework versions as separate frameworks, each requiring a separate driver. Currently, the 3.0 driver exists and the 2.x driver is in the planning stages.

Designing an API for drivers involves compromise. The API should be simple enough to be easily implemented by those wanting to use NUnit with a particular framework. On the other hand, if it is too simple, some advanced capabilities of a given framework will not be easily accessible.

Current Implementation

The Engine Driver API is under development. Currently, it maps directly directly to the Framework API - with the addition of an Unload method - through the IFrameworkDriver

IFrameworkDriver

    public interface IFrameworkDriver
    {
        string Load();

        void Unload();

        int CountTestCases(TestFilter filter);

        string Run(ITestEventHandler listener, TestFilter filter);

        string Explore(TestFilter filter);
    }

TestFilter is a TestEngine class that implements filtering of tests for exploration, counting or execution purposes.

ITestEventHandler is a TestEngine interface that provides notifications of progress as a test executes.

The Unload method, as currently implemented, does nothing. All the other methods call directly through to the framework driver.

Unresolved Issues

As it currently exists, the driver API simply provides a more convenient interface to the framework as well as a more uniform interface to all frameworks. It's possible that it may stay this way.

However, some functionality beyond the framework primitives is needed and could be implemented either in the driver or in the engine itself. For example, how do we ensure that an assembly is loaded before we try to execute it?

• Put logic in the driver to do so?
• Put logic in the engine - above the driver - to do so?
• Require the calling program to do it.

Adding functions like this to the driver means that any future drivers might be forced to implement it, whether needed by the framework they support or not. Putting it in the engine means that every driver must respond in the same way to the sequence of calls generated by the engine. Putting it in the user program means that the logic must be re-created by everyone.

For the specific example above, it's likely that we will add the capability to the driver. Other issues that need to be resolved include:

• Reloading tests on demand or when the assembly changes.
• How to cancel test runs.
• How to handle asynchrous execution of tests - and possibly even loading.