cider test reporting errors under specific conditions

[RadicalZephyr]

Currently, while cider is running tests, if an error is thrown from inside a macro-generated test, but not by an assertion (i.e. an is), CIDER errors with

error in process filter: End of buffer [2 times]

It's a fairly edge case, but it's important to me right now. I have a minimal reproduction repo available.

I spent some time trying to chase this down myself, but I just got lost in the bowels of CIDER. The ideal resolution to this would be to find all the errors correctly, but it would be a big win if CIDER didn't just totally give up, and still printed the errors that were not directly inside of generated tests.

Tested under:

Linux Mint x64 17.2
Emacs 24.5.1
CIDER 0.10.0snapshot (package: 20151020.646) 
(Java 1.8.0_60, Clojure 1.7.0, nREPL 0.2.11)

and

Mac OS X 10.9.5
Emacs 24.5.1
CIDER 0.10.0snapshot (package: 20151020.646) 
(Java 1.8.0_45, Clojure 1.7.0, nREPL 0.2.11)

[expez]

(deftest test-1
  (throw (ex-info "bad error" {}))
  (is (= 1 1)))

This is actually enough to get the error reporting to fail for me with End of buffer.

[bbatsov]

@expez I do get an error, but it's not End of buffer. Can you double check this?

[bbatsov]

It's clear the we should be displaying a different summary in such cases, but otherwise I can't reproduce your problem.

[expez]

Can you double check this?

I get the same as you now. Guess we fixed the problem in the process filter while sorting out other issues / doing general refactoring.

[jeffvalk]

Anyone still seeing an error here? Neither of the repro cases above cause any misbehavior for me.

[RadicalZephyr]

I just updated to latest cider and still see the error with the macro generated test.

; CIDER 0.11.0snapshot (package: 20151228.1558) (Java 1.8.0_45, Clojure 1.7.0, nREPL 0.2.12)

for reference here's the macro code used to generate the end of buffer error.

(defn testdef-form [site-name]
  `(deftest ~(symbol (str site-name "-test"))
     (throw (ex-info "bad error" {}))
     (is (= 1 1))))

(defmacro defsite-tests []
  `(do
     ~@(map testdef-form ["a" "b"])))

(defsite-tests)

[jeffvalk]

Ok, reproduced it. The (defsite-tests) call needed to be the last thing in the buffer to get the error. I'll look into this.

[jeffvalk]

@RadicalZephyr Congrats. You were apparently the first person to run a (non-passing) test that didn't have whitespace before the form.

[RadicalZephyr]

Fun! thanks @jeffvalk