`ITestStartEventReceiver` always runs after `[Before(Test)]` hooks on the test class

[aradalvand]

Why is it that if you apply an attribute that implements ITestStartEventReceiver to a test class which also happens to have a [Before(Test)] method, the [Before(Test)] hook inside the test class is executed prior to the attribute's ITestStartEventReceiver.OnTestStart? I would argue this is the wrong order and violates developer expectations.

This could be undesirable in a lot of (if not most) cases; generally when you're using an attribute to perform some initialization upon test start, you want to have it run before anything else in the test pipeline runs (e.g. [Before(Test)] hooks, which may rely on the attribute's ITestStartEventReceiver.OnTestStart method having initialized things).

Was this a deliberate design decision? If so, I would like to hear the rationale, and would suggest including that in the documentation.

Also, is there any kind of workaround to swap the order? If not, how should scenarios of the kind I mentioned be achieved in TUnit?

[thomhurst]

With your approach, I think the naming becomes even more confusing.

Currently Before(Test) does what it says on the tin. Runs before the test.

Then we have Receiver.OnStart which runs as a test starts.

You can use ITestRegisteredEventReceiver to run earlier.

[aradalvand]

You can use ITestRegisteredEventReceiver to run earlier.

The problem with ITestRegisteredEventReceiver is that unlike ITestStartEventReceiver which runs in parallel for more than one test (provided that no [NotInParallel] or such exists), ITestRegisteredEventReceiver runs serially.

[thomhurst]

I'd have to create something like IBeforeTestStartEventReceiver which runs before the instance level hooks.

I just don't want to get into an explosion of different hook types which make the framework confusing to use.

[aradalvand]

@thomhurst But this won't necessarily require introducing a new hook type.
How about adding a virtual property to ITestStartEventReceiver like Stage (kind of like the existing Order property) with an enum type representing two choices: Early and Late (the latter representing the current behavior and being the default so as to avoid a breaking change) and the former (Early) yielding pre-instance-hooks semantics?

[thomhurst]

@thomhurst But this won't necessarily need a new hook type. How about adding a virtual property to ITestStartEventReceiver like Stage (kind of like the existing Order property) with an enum type representing two choices: Early and Late (the latter being the current behavior and the default) and the former (Early) yielding pre-instance-hooks semantics?

Not a bad shout! I like this idea. It'll be a breaking change for .net framework versions though, because it's an interface. So I can't make it virtual. In net core I can do a default interface implementation, but for net framework users would have to do this themselves.

[aradalvand]

If avoiding a breaking change in the .NET Framework variant is paramount then we could just omit this property in the .NET Framework versions, and limit it to .NET Core onward, via a compiler preprocessor (e.g. #if NET_...).

[aradalvand]

Though it's probably safe to assume that the venn diagram of projects that are still on .NET Framework but use TUnit is already tiny anyway. So, not a big deal either way, I'd argue.

[thomhurst]

True true! I think that's a good idea. I might just make it conditional like you suggested and then it could eventually break if more need for it arises

[thomhurst]

If you wanna give v1.2 a whirl

[aradalvand]

Works brilliantly — thank you!