Allure.NUnit: Report exceptions from OneTimeSetUp methods of class fixtures

[delatrie]

I'm submitting a ...

• bug report
• feature request
• support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

If a [OneTimeSetUp] method fails, all test results of the class fixture aren't included in the report. The error, therefore, remains unreported.

What is the expected behavior?

If possible, skipped test results should be created. That might be tricky as, at first glance, NUnit doesn't provide the list of scheduled tests (it provides the list of all tests in the class instead; the user might have selected only a subset of them).

An alternative approach would be reporting the error in a surrogate test result. We already do something similar in Allure SpecFlow and Allure Reqnroll.

What is the motivation / use case for changing the behavior?

Include the information about the error that occurred during the run. That information might be important to the user.

[DenisKorbovskyTds]

Thanks @delatrie for that Feature Request and answer in #286 , is there any ETA on such improvement?

[delatrie]

Unfortunately, can't tell for sure now. Will provide you with more details as soon as I have any.

[gamerka]

For myself I made a hack adding this to my common OneTimeTearDown:

    var allure = AllureLifecycle.Instance;
    var testContext = TestContext.CurrentContext;
    var stackTrace = testContext.Result.StackTrace;
    var pattern = @"at (?<className>[\w\.]+)\.(?<methodName>[\w`$]+)";
    var preparedStackTrace = stackTrace?.Replace("_", "");

    if (preparedStackTrace == null ||
        (!preparedStackTrace.Contains("OneTimeSetUp", StringComparison.InvariantCultureIgnoreCase) &&
         !preparedStackTrace.Contains("InitFramework", StringComparison.InvariantCultureIgnoreCase))) return;

    var methodName = Regex.Match(stackTrace!, pattern).Groups["methodName"].Value;

    allure.StartTestCase(new TestResult
    {
        name = testContext.Test.DisplayName,
        fullName = testContext.Test.FullName,
        status = Status.broken,
        start = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds(),
        statusDetails = new StatusDetails
        {
            message = $"{testContext.Result.Outcome.Status} in {methodName}.\n" +
                      $"{testContext.Result.Message}",
            trace = testContext.Result.StackTrace
        }
    });

    allure.StopTestCase();
    allure.WriteTestCase();

    allure.StopTestContainer();
    allure.WriteTestContainer();

It works for my cases and in any way it just makes Allure reports not 100% green and that's the main thing.

[vladdex]

@gamerka thanks for the suggestion, I tried that but I'm getting an InvalidOperation Exception Unable to activate the test context because the fixture context is active.
Allure decorators need to be removed from the one time setup/teardown methods for this to work.

@delatrie any news on fixes on this ?

[gamerka]

@vladdex Yep, milage may vary. For me right now it works with and without decorators.