Should ConditionalClass and similar attributes in XUnitExtensions filter tests out automatically?

[ViktorHofer]

ConditionalClass and similar attributes add a failing trait to the tests that should conditionally not run:

arcade/src/Microsoft.DotNet.XUnitExtensions/src/Discoverers/ConditionalClassDiscoverer.cs

Line 30 in 1ccd352

yield return new KeyValuePair<string, string>(XunitConstants.Category, XunitConstants.Failing);

It might make sense to automatically filter those out without relying on the test settings to exclude the failing trait (i.e. in a .runsettings file).

[Wraith2]

This is a problem which affects running tests in SqlClient and has done for many years. If there is a chance it can be fixed that would be very useful and allow tests to be run from inside visual studio which has never been useful before.

@dotnet/sqlclientdevteam may want to weigh in and identify if they consider this something important to them.

[David-Engel]

I don't know about the "failing" terminology on this issue. But we (SqlClient) do see tests run in the VS GUI that are filtered out (Skipped) in command-line tests runs by ConditionalFacts that exclude them based on environment or other logic. It would be really nice if the VS GUI behaved the same.

[akoeplinger]

@David-Engel can you please try again once you get an arcade update with Microsoft.DotNet.XUnitExtensions version 10.0.0-beta.25175.1 or later? I think I fixed the issue with ConditionalFacts not working in VS with #15667

[akoeplinger]

And just to make this issue more accurate: ConditionalFact/ConditionalTheory do not use the "failing" category, only ConditionalClass does.