Feature Request: Run all tests

[jeffh]

While C-c , runs either the test for the corresponding namespace or the last test namespace, it would be great if there was a hot-key to run all tests within cider-test (perhaps using C-c <). Thoughts?

That being said, cider has been great to use. Thanks a ton for all the hard work! 😄

[expez]

cider-rerun-tests, bound to C-c C-,, does what you want, provided you've loaded your tests already.

[jeffh]

Sorry if I was unclear, I want to run all tests for all namespaces. Equivalent to (run-all-tests) in clojure.test or lein test. cider-rerun-tests only reruns the previous tested namespace.

[bbatsov]

Guess we can cook something like this, unfortunately it will also require UI changes (multiple ns support).

[stig]

👍

[expez]

I'm including this in the feature list for for milestone 0.11. This is the feature I'm missing the most atm.

When this is tackled cider-test-rerun-tests should be updated too, so it reruns all the failing tests from the previous test run, instead of acting on the last tested namespace. I'm not sure how that is implemented, so we might get that synergy for free.

[bbatsov]

Btw, I hope you're also planning to work on this, as 0.11 will like be release in February and I doubt I'll work on anything significant until then.

[expez]

Yeah, I was planning on taking this one the next time I've some time for CIDER. This is unlikely to be until the second half of February, though, so with that release schedule I suppose it might not make the cut.

[jeffvalk]

This should be pretty straightforward to implement.

One question: how would the list of namespaces to be tested be determined?

[bbatsov]

I think the simplest way would be to take the first segment of whatever the current namespace is test all namespaces that share it. Alternative, with a prefix argument the user could supply a namespace prefix manually.

[expez]

One question: how would the list of namespaces to be tested be determined?

Can't we just loop over all-ns and run whatever tests we find? IIRC clojure.test stores some metadata on the ns itself, which should make this possible.

The previous solution might loop over a lot of namespaces, but we could use tools.namespace's ns-in-dir (or whatever it's called) to get the namespaces in the test folder, if that's a problem.

[jeffvalk]

I'd warn against using (all-ns). Tests can be defined in the same namespace as their functions, e.g. by adding :test metadata to a defn. This style isn't in favor, but it's certainly not incorrect. If a library author did this, you'd be running all the library's tests too; they'd be in the jar dependency.

Using find-namespaces-in-dir in tools.namespace sounds a bit better. This raises two more questions though:

1. Which directories do we specify?
2. Supposing we have a list of the "test" namespaces, do we run only those already loaded? Or do we load the ones that aren't?

[Malabarba]

Personally, I prefer the idea of taking all namespaces with the same initial prefix as the current one. For instance, if I invoked the command in cider.nrepl.middleware.debug, then this would run all tests whose namespace name starts with cider.

Using find-namespaces-in-dir is valid too, I just prefer to avoid resorting to the file system when possible.

As for loading files first. I don't think run-all-tests should mess with that. However, I do think cider is in dire need of a load-all-files command (I've never missed it, but people ask for it all the time). And then this command could be used to load all files before running the tests.

[jeffvalk]

Playing devil's advocate for a moment, I imagine most people will expect replication of lein test. I'm pretty certain that both searches the file system and loads the discovered namespaces.