[Commands] Allow skipping tests with a flag

[finagolfin]

Multiple --skip invocations can be used to skip multiple tests. Keep the environment variable around till its users switch over.

@aciidb0mb3r added the functionality to skip tests in #2181 last summer, but it was only available from an environment variable. I was unaware of that and really wanted the option to skip tests, especially when segfaulting tests cause the whole test run to stop, so I looked into implementing it and found his code. Simply adding this flag surfaces his implementation to the user and works, but iswas incomplete.

I 'd gowent further and makemade it so that --filter and ---skip could be used at the same time (the naive implementation in this WIP pull causes the skip flag to override the filter flag), so one could --filter BuildTests --skip SwiftCompilerOutputParserTests and have much more fine-grained control of what tests are run. I simply reused the colon-separated scheme of the environment variable, but something like @keith's #2800, ie allowing multiple invocations of --skip, would be more consistent.

I figured I'd implemented it by moving the .skip case to its own function skippedTests which is a fileprivate extension to [UnitTest], similar to filteredTests, then changed all three invocations of filteredTests in this module to filteredTests().skippedTests().

Before I go down that road, I thought I'd post this minimal implementation and see what the maintainers think. @keith, let me know if this would help you too.

[keith]

I think allowing multiple --skip invocations would be nicer than the : splitting for sure. I think in general this would be nice to have, but I'll leave that up to the maintainers!

[finagolfin]

@neonichu, I see that you asked for these features in SR-10800 last year, let me know what you think about the proposedthis implementation above.

[finagolfin]

Updated with the full implementation I outlined, amended OP to reflect it. @aciidb0mb3r, I simply removed the now redundant environment variable, but can add it back if there's some user who needs that till they can shift over.

I will add some tests soon, any feedback on the new behavior would be good to have first.

[finagolfin]

Added some tests, a couple more coming this evening.

[neonichu]

We should keep the environment variable for compatibility, but I think it's fine to remove it from docs.

[finagolfin]

Alright, added back the environment variable, another commit enabling some tests on non-mac platforms, and a couple more tests for this flag.

Ready for review and CI.

[MaxDesiatov]

@swift-ci please smoke test

[finagolfin]

The CI didn't run.

[MaxDesiatov]

@swift-ci please smoke test

[finagolfin]

Oh, wow, 3 min. turnaround on the self-hosted tests, this is great. 😆

[tomerd]

thank you @buttaface

[finagolfin]

Thanks for getting this in, always wanted this feature.