Ultra causes CIDER to hang indefinitely when running tests with some values

[daveliepmann]

Expected behavior

I run tests using CIDER, regardless of whether I have Ultra in my lein profile, regardless of the values in maps being passed from function to function, and the tests complete.

Actual behavior

With Ultra in my lein profile, CIDER hangs indefinitely when running tests which call a function that returns certain values (a decimal in one case, an integer in another; I suspect other cases exist).

Steps to reproduce the problem

1. Have {:user {:plugins [[venantius/ultra "0.5.2"]]}} in ~/.lein/profiles.clj
2. lein new decimal-bug
3. Create new function in that project's file src/decimal_bug/core.clj: (defn bar [] {:number 1}) and save the buffer
4. Replace a-test with this new test in that project's file test/decimal_bug/core_test.clj:

   (deftest decimal-test
     (testing "documenting CIDER decimal bug"
       (is (map? (bar)))))

5. M-x cider-jack-in / C-c M-j
6. M-x cider-test-run-ns-tests / C-c C-t C-n, tests complete and pass almost instantly, all is well
7. Change bar to now return {:number 1.0}
8. (Optional) For clarity, especially for folks using cider-test-show-report-on-success, add an assertion to our test, so that it is clear that the *cider-test-report* does not successfully return:

   (deftest decimal-test
     (testing "documenting CIDER decimal bug"
       (is (map? (bar)))
       (is (number? (val (first (bar)))))))

9. M-x cider-test-run-ns-tests / C-c C-t C-n, 💥 tests hang indefinitely, CIDER must be shut down manually

I could also get the tests to hang with bar returning simply 1 and decimal-test checking (number? (bar)).

All these issues with hanging seem to disappear when I remove ultra from ~/.lein/profiles.clj.

Environment & Version information

CIDER version information

;; CIDER 0.16.0 (Riga), nREPL 0.2.13
;; Clojure 1.8.0, Java 1.8.0_151

Lein/Boot version

Leiningen 2.8.1 on Java 1.8.0_151 Java HotSpot(TM) 64-Bit Server VM

Emacs version

GNU Emacs 25.3.1 (x86_64-apple-darwin17.2.0, Carbon Version 158 AppKit 1561.1) of 2018-01-05

Operating system

MacOS High Sierra 10.13.2 (17C88)

[venantius]

As a non-Emacs user, I have a bit of a naive question, which is what do you use Ultra for?

[daveliepmann]

I use Ultra for test output in the terminal. This conflict has lead me to disable it so that it doesn't interfere with test output from CIDER.

[venantius]

What happens if you set {:repl false} in your Ultra options map?

[daveliepmann]

I've never used an Ultra options map, but I tried putting it in both my lein profile and my project.clj and neither had any effect. Same problem.

[venantius]

Hmmmmm. Okay.

I'm not sure what's happening here. Based on the behavior you're describing, I'd guess that there's something happening at a lower-level library - possibly Puget or Whidbey - that relates to the formatting of decimals or other specific data types that could be interfering with CIDER. But I can't say for certain. I'm not particularly familiar with the CIDER codebase so my initial thought was that Ultra's nREPL middleware was interfering with CIDER's. But if you've disabled the nREPL middleware by setting {:repl false} then I'm not sure what the next most likely candidate would be.

[daveliepmann]

OK. In that case I think it's fair to just warn CIDER users that a conflict exists. This issue will do that (even if closed), but maybe a message in the README would help. I understand that debugging it would be a big task and it's not really necessary. Thanks for your time.

[venantius]

I'll add a small note.

On your end, what you could do to fix the problem (presumably) would be just to move Ultra to a dependency under the :test profile. That way it'll only get invoked when Lein is running tests in the terminal rather than using your normal REPL-driven development.

[venantius]

I'll leave this issue open for now in the event that an enterprising contributor comes along and feels like solving this.

[waffletower]

I managed to get this stack trace when running tests with ultra and Cider. I used lein repl and Cider's m-x cider-connect which revealed the stack trace to the console where I had started the repl. Looks like it is on the cider-nrepl side.

Exception in thread "nREPL-worker-1" java.lang.IllegalArgumentException: Cannot write value of type class java.lang.Double at clojure.tools.nrepl.bencode$eval233$fn__234.invoke(bencode.clj:326) at clojure.lang.MultiFn.invoke(MultiFn.java:233) at clojure.tools.nrepl.bencode$eval268$fn__269.invoke(bencode.clj:398) at clojure.lang.MultiFn.invoke(MultiFn.java:233) at clojure.tools.nrepl.bencode$eval258$fn__259.invoke(bencode.clj:382) at clojure.lang.MultiFn.invoke(MultiFn.java:233) at clojure.tools.nrepl.bencode$eval268$fn__269.invoke(bencode.clj:398) at clojure.lang.MultiFn.invoke(MultiFn.java:233) at clojure.tools.nrepl.bencode$eval258$fn__259.invoke(bencode.clj:382) at clojure.lang.MultiFn.invoke(MultiFn.java:233) at clojure.tools.nrepl.bencode$eval268$fn__269.invoke(bencode.clj:398) at clojure.lang.MultiFn.invoke(MultiFn.java:233) at clojure.tools.nrepl.bencode$eval258$fn__259.invoke(bencode.clj:382) at clojure.lang.MultiFn.invoke(MultiFn.java:233) at clojure.tools.nrepl.bencode$eval268$fn__269.invoke(bencode.clj:398) at clojure.lang.MultiFn.invoke(MultiFn.java:233) at clojure.tools.nrepl.bencode$eval258$fn__259.invoke(bencode.clj:382) at clojure.lang.MultiFn.invoke(MultiFn.java:233) at clojure.tools.nrepl.bencode$eval268$fn__269.invoke(bencode.clj:398) at clojure.lang.MultiFn.invoke(MultiFn.java:233) at clojure.tools.nrepl.bencode$eval268$fn__269.invoke(bencode.clj:398) at clojure.lang.MultiFn.invoke(MultiFn.java:233) at clojure.tools.nrepl.bencode$eval268$fn__269.invoke(bencode.clj:398) at clojure.lang.MultiFn.invoke(MultiFn.java:233) at clojure.tools.nrepl.transport$bencode$fn__723.invoke(transport.clj:104) at clojure.tools.nrepl.transport.FnTransport.send(transport.clj:28) at clojure.tools.nrepl.middleware.render_values$wrap_renderer$reify__1662.send(render_values.clj:30) at cider.nrepl.middleware.test$handle_test_var_query_op$fn__27195$fn__27196$fn__27197.invoke(test.clj:286) at cider.nrepl.middleware.test$handle_test_var_query_op$fn__27195$fn__27196.invoke(test.clj:279) at clojure.lang.AFn.applyToHelper(AFn.java:152) at clojure.lang.AFn.applyTo(AFn.java:144) at clojure.core$apply.invokeStatic(core.clj:657) at clojure.core$with_bindings_STAR_.invokeStatic(core.clj:1965) at clojure.core$with_bindings_STAR_.doInvoke(core.clj:1965) at clojure.lang.RestFn.invoke(RestFn.java:425) at cider.nrepl.middleware.test$handle_test_var_query_op$fn__27195.invoke(test.clj:277) at clojure.tools.nrepl.middleware.interruptible_eval$run_next$fn__1180.invoke(interruptible_eval.clj:190) at clojure.lang.AFn.run(AFn.java:22) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

[rduplain]

Following the test case helpfully provided at the start of this thread, I can reproduce the issue with the following notes:

2. lein new app decimal-bug

5.5. Refresh the project with C-c C-x.

7. Edit, then run C-c C-k to execute all changes in the buffer.

The traceback is viewable in the *nrepl-server decimal-bug* Emacs buffer.

[rduplain]

I use Emacs and the command-line. I opted to define this issue out of existence by configuring how cider-jack-in works. With the config below, CIDER skips the user profile when it loads its own REPL, while lein repl itself is left unchanged.

This will not cause any issues when keeping to the rule that projects do not depend on the lein user profile; projects have repeatable builds by including all dependencies in the project.clj. If you have a lein user profile intended to alter CIDER's behavior, consider declaring a separate profile and name it in with-profile -user,YOURPROFILE in the Emacs cider-lein-parameters variable.

In my .emacs:

; Skip :user section of ~/.lein/profiles.clj when using cider-jack-in.
(setq cider-lein-parameters
      "with-profile -user repl :headless :host localhost")

This is generally useful as projects other than Ultra run into similar conflicts when the command-line tool and IDE both attempt to format test output. For example, the humane-test-output README says:

This test output formatting works great with Emacs and cider 0.10 or lower, and when running Leiningen in a console. However, some IDEs are also bashing in their own versions of test formatting.

I do not recommend using this with cider 0.11.0 or greater or with Cursive, or LightTable, or any other IDE.

Of course, using cider-connect (as opposed to cider-jack-in) with an existing lein repl will still run into this issue, but at least the resulting exception will display in the terminal instead of the background *nrepl-server* buffer in Emacs. Personally, I often use cider-connect but do not run into any issues as I run tests in a separate shell where tests refresh automatically on file changes (viewed in a second monitor with a separate terminal running screen -x, by the way).

I'm using CIDER 0.17.0.