Ideal Beginner Editor+REPL Setup

[jefffriesen]

The editor + REPL is a significant hurdle to beginners. There are plenty of people advocating for Clojure and REPL-driven development and showing results, but those are people who already get it. They were either really persistent or had peers to help them navigate all of the incidental complexity of setting up an app, editor and REPL to become productive. If Clojurist want broader adoption, we need to simplify the setup.

I would consider myself a Clojure beginner - I've read books, watched videos, done all of the 4Clojure problems, made simle Re-frame apps. Often with a REPL. But it's been a struggle - not from the language but from the setup. So I'm going to take my beginner mindset and lay out some of the problems I've run into and imagine the setup that I would have loved to discover when I started (and would still probably use).

Alternate Universe Beginning Clojure Experience

1. Create a re-frame or backend app scaffolding from command line. Just like we have now with lein new [TEMPLATE] NAME. We might consider this done.
2. With a single command-line command, handle with dependencies and start the app and REPL. Right now, this can happen but often is at odds with working with the code in an editor.
3. Open the project in an editor (VSCode?). Execute code inline or send to a REPL - no explicit dependencies, no profiles.clj, no magic incantations in the command line, no confusion of starting the REPL in a terminal or editor. It just works.
4. When you change files, the REPL changes it's namespace automatically. You could switch to another ns just like you can now with (in-ns '...). As an experienced user, you can also turn off the auto-tracking the files. But the default REPL always represents the file you are working in.
5. The new REPL doesn't care if the file is CLJ or CLJS code. As a user, you don't really care - you just want to execute the code you're working on.
6. The REPL never gets out of sync or stale compared with the file you're working on. As soon as the file is saved, the REPL's namespace updates to reflect that.

I'm sure this may be too inflexible for some experienced users. But they already have very customizable, complicated tools that they use. And that's not really VSCode+Calva's target audience. Calva could expose options to turn some of these features off. But the idea is that beginners would get an amazing, productive experience right away.

It's steps 2 and 3 that gloss over so many implementation details and potential problems. Some things, as a beginner, that I don't want (yet) to know or care about:

• Do I start the REPL from the command line or from the editor?
• What's Piggyback? What is fighweel sidecar?
• What's the difference between nRepl and figwheel? Do I care about socket REPLs?
• What's the REPL port that I need to connect to?
• What commands from the REPL (nREPL?) do I need to issue to start the CLJS REPL?
• What's Cider?
• What's my .lein/profiles.clj file and how do I get that set up to begin working in Clojure?

I don't know how you would implement step 3. Maybe it's a single dependency that is added to either profiles.clj or the project. If that was all that would be cool. Or maybe that dependency is effectively built into the editor. I really don't know.

Part of the problem is that there is a lot of mixed information out there that is out-of-date. For example:

• Create a re-frame template from these: https://github.com/Day8/re-frame-template then run lein dev. But that's not actually how most people would do it - they want REPL integration with their editor. Which requires you to not run lein dev but run something else that is editor dependent.
• Here's how to set up an nREPL with Figwheel: https://github.com/bhauman/lein-figwheel/wiki/Using-the-Figwheel-REPL-within-NRepl. It's very long and there is old and new ways to do it. They keep saying it's "super advanced". It's not actually that hard, but requires a lot of trial and error and filtering out many outdated sources on the web. And REPL-driven development one of the greatest strengths of Clojure, so ideally it shouldn't be advanced at all.

There are a lot of people who put a lot of time into tools to make things easier. Like you guys. But it's still hard. I have a sense of why - the REPL(S) run independently of the editor, even though that's not how people generally use it. But there is still the editor/REPL divide. How do we cross it? How do we make professional tools that are easy to get started with?

It seems like you guys are thinking of similar issues and why you are putting so much work into Calva. So I thought I would share these ideas and see what you think. Are steps 1-3 unrealistic? Are 4-6 something beginners would want?

Is this an idealized version of a Clojure/Clojurescript beginner experience? Or does it look like something else?

Thanks for all of your hard work.

[pedrorgirardi]

These are great notes, Jeff! I won't be able to properly reply this today, but I will do it as soon as I can.

[PEZ]

Many thanks for this well thought-through input, @jefffriesen! You seem to share Pedro's and mine passion for lowering the barriers of entry to the Clojure world.

I'll try to find some time later today to reflect on the specifics of your suggestions/outline, but as this issue has been burning at the top of my mind a while now, I want to let you know that I find your framing of the problem(s) very accurate and I share them.

I also want to share one of my conclusions so far: I would like Calva to guide the user into Clojuresque behaviour as quickly as possible. So I am thinking about it a bit differently. Instead of hooking up support wheels on the bike, I'd like for Calva to help the wannabe biker find his balance quickly and let him bike away, wether he wants to use it only for going from A to B or be practicing tricks, like letting go of the steering handle or ride on one wheel only, or whatever.

I do not have much of an idea of how to achieve this, but I agree with you that good starter projects with as little setup as possible are important. Some months ago I created a starter project that only relies on Java and Clojure being installed, utilising the clojure deps EDN infrastructure. It's here:

https://github.com/PEZ/deps-edn-figwheel

I didn't finish it because I got sidetracked by discovering shadow-cljs, which offers a similar ease of setup experience. My starter project needs a way to connect to the REPL without using leiningen or boot. I think I know how to do this (using repl-y), but will need some quality time to do it.

I think that if we can build a sort of a playground project from this we could maybe create an on-boarding experience that quickly makes the loading of namespaces and evaluating forms and such second nature even for the beginner.

What are your thoughts on that approach? I'll be back, happy to discuss this issue further.

[PEZ]

To maybe be a bit clearer, it is steps 4-6 I am referring to. We seem to be completely on the same page regarding 1-3.

[jefffriesen]

@pedrorgirardi @PEZ -
Thanks guys for the feedback and openness to the ideas. It's cool you have been thinking along similar but maybe different lines.

I like the training wheels metaphor. I'm not sure what exactly you have in mind, but one thing that comes to mind when you said that is changing namespaces. There is a Calva: switch namespace to current file command that then sends the namespace to the repl so you can see it: (in-ns 'theapp.core) for example. Sometimes I forget the single quote and the resulting error doesn't clue me in on that at all what the problem is. Seeing your editor send that command reinforces it and offers a backup in case you forget. Is that the kind of training wheels you're thinking about, more something else?

I'm excited to work with you guys to figure out what a great experience could look like.

[PEZ]

Interesting. That is not intentional, only a result of that we only have a very clumsy way of communicating with the Terminal in vscode.

I agree it is a kind of thing that can help to guide the user, while still doing the work he needs to get done. It reminds me about that I often learn som git by looking at the git output channel in vscode.

The ideas about this on-boarding help that visit my mind are more unrealistic. 😄 Something like an interactive mode where the user is prompted for step after step in building a simple app from scratch. I am sure you have seen such interactive on-boarding guides out there. Some are awful, some are quite awesome. I have doubts it is feasible to do something like that in vscode. But maybe some kind of notebook style skeleton for an app that the user can finish by following instructions added as comments in the code? Have you looked at Maria any? This is very inspiring: https://www.youtube.com/watch?v=CUBHrS4ZzO4

We should keep in mind that visible (in-ns command sent to the REPL, because we probably will need to implement this rapid-guidance thing using several ”tools”.

[pedrorgirardi]

Hi @jefffriesen.

Sorry for taking long to get back on this.

I come from Cursive and I can't say enough good things about it. Cursive doesn't need a REPL to get autocomplete, code navigation, refactoring, formatting. The programmer doesn't need to do any setup to get all that, it's a really good experience and I don't know of any Clojure editor/IDE with those capabilities without a setup (the programmer needs to import a project.clj though).

I kinda split things into tooling and repl. Autocomplete, navigation, formatting, refactoring is tooling and it could be done without a REPL. Without a REPL (nREPL + middlewares) we would need a Clojure Language Server to get those functionalities. There is already one that seems to be very active and that might be a good option for us.
Taking the Language Server approach we (theoretically) remove the setup required to get those functionalities.

Now, what I mean by repl is an interface to my running program, so I could invoke functions, inspect its state, experiment and more.
I love the REPL development approach that we have in Clojure, and I personally find very hard to work without a REPL, but people new to Clojure & ClojureScript don't seem to 'get it' in the beginning. Also, I noticed that ClojureScript programmers interact even less with a REPL because of tools like Figwheel and Shadow CLJS.

It would be great to get the tooling part without any setup, I think that would make things much easy for newcomers. And for the repl story, we talk to a nREPL server.

All of this might not address directly your points Jeff, but that's a little bit how I think about the problem. 😀

[jefffriesen]

Thanks both of you for your comments.

So how does Cursive get the tooling? I think I remember needing to point the IDE to your local Java installation. How hard is Clojure Language Server to set up to work with VS Code?

As I understand it, you need Clojure running to get all of the tooling - hence connecting to the REPL. Cursive points to and runs Clojure (I think) to get that.

Here's a possibly terrible idea, but have you heard of GraalVM? You could probably package Clojure and Clojurescript inside GraalVM and include it with the Calva extension.
https://medium.com/graalvm/graalvm-ten-things-12d9111f307d

I know historically that sounds like a really bad approach. But this is new, alien tech. It's surprisingly lightweight and fast. You would need to run the version of Clojure that your project runs - so maybe there is a way to grab the version from a repo dynamically or something. But with it you could effectively call the entire Clojure/JVM from the Javascript that Calva has access to. That could give all of the tooling we need without first needing to connect to the REPL.

However, I agree that people generally want to do their work with a REPL. I hope I wasn't implying that. It's the complicated incantations and ordering among the project, IDE, profiles.clj and REPL steps that would be nice to avoid or simplify. Open up the project in VS Code+Calva, send a command and everything, including the REPL, starts working. I don't know if that's realistic, but I know with the current complexity, many people can't or won't get into it that want to.

So - yes - I agree the REPL has to be part of it. And tooling. How do we do that?

[PEZ]

Regarding ClojureScript programmers picking up the REPL workflow slower I think the reasons are at least twofold.

1. I think many frontend programmers have a bit less experience with different programming languages than backend programmers are. A huge generalisation, I know, but people who have been using all sorts of languages are a bit more in agreement that each language need its mindset.
2. The tooling is a bit less helpful and the middleware a bit tricker to get right.

We can try help with both.

Regardless if we go for static analysis or GraalVM or something else or stay with nREPL for the ”basics” I think it is realistic to aim for the ease of setup you are envisioning, @jefffriesen. Cider on Emacs does this pretty well by starting the real for you and injecting the middleware it needs.

I would like to know more about the GraalVM approach. You had me at ”possible terrible idea”. 😄

[jefffriesen]

I have some things to learn about GraalVM as well. I'm looking into it.

[PEZ]

Closing this for now.