[Feature]: Focus tests

[agross]

Problem Statement

Hello, thank you for creating TUnit! I would love to see something along the lines of "focused tests" to easily limit the test run to classes or test methods without having to resort to filters.

Proposed Solution

[Focus]
public class RunsAll {
  [Test] ...
}


public class RunsSome {
  [Test] public void Skipped() {}
  [FTest] public void Runs() {}
  [Focus, Test] public void RunsToo() {}
}

Alternatives Considered

dotnet run --treenode-filter /*/*/*/*[Category=Focus]

Maybe? I find this too fiddly. I would have to read the docs every time I use these.

With [Focus] the above becomes dotnet run.

Feature Category

Test Discovery / Attributes

How important is this feature to you?

Nice to have - would improve my experience

Additional Context

I've been used to this since I had a look at rspec (a Ruby test framework).

RSpec also provides aliases – fit, fdescribe and fcontext – as a shorthand for it, describe and context with :focus metadata, making it easy to temporarily focus an example or group by prefixing an f.

Source

Contribution

• I'm willing to submit a pull request for this feature

[thomhurst]

This kinda exists already but inverted. Add the [Explicit] attribute onto tests/classes that you don't want 'focused'

[agross]

With a test suite of hundreds of tests, this would cause hundreds of Explicit tests.

Focused is mostly a temporary measure if a test fails. Instead of making all others Explicit or fiddling with --treenode-filter, I think Focus would be very valuable. This will make TUnit stand out even further w.r.t. simplicity.

[agross]

BTW, rspec also has "x" variants like xdescribe, xcontext and xit to exclude things.

I've grown to love these for temporary one-off scenarios.

[thomhurst]

Try this:

Create:

public class FocusAssemblyAttribute : SkipAttribute
{
    public FocusAssemblyAttribute() : base("This test is not focused")
    {
    }

    public override Task<bool> ShouldSkip(TestRegisteredContext context)
    {
        var hasFocusAttribute = context.TestDetails.Attributes.OfType<FocusAttribute>().Any();
        
        return Task.FromResult(!hasFocusAttribute);
    }
}

and place it in on the assembly like [assembly: FocusAssembly]

And then create:

public class FocusAttribute : Attribute;

and add it to your tests like [Focus]

[agross]

This works for simple cases.

I have a test class (A) that DependsOn another class (B).

• with the Focus on A only A will run (and fail), but not B
• without attribute on the assembly and Category("Focus") on A and --treenode-filter '/*/*/*/*[Category=Focus]', B will run before A as expected

If nothing is marked with Focus all test should run. The solution proposed above runs no tests.

[thomhurst]

Try this:

public class FocusAssemblyAttribute : SkipAttribute
{
    private static TestContext[] _focusedTests = [];
    
    public FocusAssemblyAttribute() : base("This test is not focused")
    {
    }

    [After(TestDiscovery)]
    public static void AfterTestDiscovery(TestDiscoveryContext context)
    {
        _focusedTests = context.AllTests.Where(HasFocusAttribute).ToArray();
    }

    public override Task<bool> ShouldSkip(TestRegisteredContext context)
    {
        // Run all tests if no tests are focused
        if (_focusedTests.Length == 0)
        {
            return Task.FromResult(false);
        }
        
        // Run because it's a dependency of a focused test
        if (IsDependencyOfFocusedTest(context.TestContext))
        {
            return Task.FromResult(false);
        }
        
        return Task.FromResult(!HasFocusAttribute(context.TestContext));
    }

    private static bool HasFocusAttribute(TestContext test)
    {
        return test.TestDetails.Attributes.OfType<FocusAttribute>().Any();
    }

    private static bool IsDependencyOfFocusedTest(TestContext test)
    {
        return _focusedTests.SelectMany(x => x.Dependencies.Select(x => x.TestId)).Contains(test.TestDetails.TestId);
    }
}

[agross]

Hm. The code does not compile as-is on my machine. Specifically the last 2 methods need changes:

static bool HasFocusAttribute(TestContext test)
    => test.Metadata.TestDetails.Attributes.HasAttribute<FocusAttribute>();

  static bool IsDependencyOfFocusedTest(TestContext test)
    => _focusedTests.SelectMany(x => x.Dependencies.DependsOn.Select(y => y.TestId))
                    .Contains(test.Metadata.TestDetails.TestId);

I don't know if these changes break things; they probably do.

Consider this as my focused A class from above, I've tried different placements of DependsOn:

[DependsOn<Bot>]
[Focus]
public class Freigabe
{
  [Test]
  [DependsOn<Bot>]
  [DependsOn(nameof(Bot.leitet_auf_die_Vielen_Dank_Seite_um))]
  public async Task sendet_nach_Freigabe_eine_Mail_an_den_Kunden(CancellationToken ct) {}
}

All 3 variants yield the following inside IsDependencyOfFocusedTest when inspecting _focusedTests.

• the test's DependsOn is empty
  

• the test is a dependency of itself

  

It doesn't really add up IMO. The dependency is also not run as before.

Can you make sense of it?

[agross]

I've made some examples that I could think of. None work unfortunately.

https://github.com/agross/tunit-focus

[thomhurst]

@agross The dependencies weren't being populated that early on so they were always empty. I've just pushed a fix out.

Can you try in v1.12.80 please 😄