Abstract over test frameworks using the SBT testing interface

[andyscott]

I'd like to abstract over test interfaces so that the scala_test rule supports all test frameworks implementing the SBT testing interface.

This is relatively easy to implement using the SBT testing interface class graph utility and the new phase architecture. The new rule will then automatically discover any Scala test in the same way SBT discovers test.

The bigger issue is our migration strategy: I would like to replace the existing scala_test rule-- which only works for ScalaTest tests-- with this new generic rule. The Specs2 rules (scala_specs2_junit_test, etc) can also be removed as the new rule will run Specs2 tests.

One possible migration strategy is to introduce the new rule at a new location. During a cutover period the existing test rules can delegate to the new implementation. Once we are ready to make a breaking change, we can get rid of the old rules (just facades at this point) and rename/move the main rule over.

[ittaiz]

Interesting...
Few questions, does this support running junit tests? I found junit-interface which sounds like it but I don't know sbt-interface at all.
Also, How does it discover which classes need to run? In scala_junit_test we have an elaborate mechanism (which I want to extract actually) but still interested in hearing.

Does SBT testing interface (and the class graph utility) have cons? Reproducibility/Many dependencies/Runner performance? I'm just throwing stuff out there to better understand the trade off (or to understand there isn't a trade off 😄)

Re migration- When you say a breaking change you mean that at the cut point we'll just drop the new experimental rule location but users who have been "conservative" and haven't done anything but just kept upgrading will continue to work (minus bugs), right? No API change or anything?
Only the users who experimented with the new rule will need to move back to the "old" location (which will contain the new implementation).

[jadenPete]

Hey, folks. We've published a proposal to port some of the features from lucidsoftware/rules_scala, an alternative Scala ruleset that we maintain. Using the sbt test interface is one of them. Here's a link to the relevant proposal section that details specifically what we'd like to do, and a link to an issue we're using to track the broader effort:

The proposal section

The tracking issue for this effort

Are folks on board with our approach to solving this issue? We were hoping to take on this task in the near future.