Discussion: Driving this project forward?

[jsmestad]

Background

This is the best place I could think of to post this sort of thing.

I was speaking with @axelson about this project on Elixir Slack today and we both agreed this project seems to be suffering due to no clear person driving it forward. If we look at the discussions about changing CI tooling in #458 there are some great ideas but no one is pushing it over the finish line.

Questions

• Can we update the README to detail out who the maintainers of the project are?
• Should we publish a ROADMAP.md to show areas we think need to be added or revised?
  • I have seen in several PRs there has been talk about things needing to be re-done, but no official list of things we can point to.
• Would shifting this project to being a "one stop shop" for Elixir in Emacs better drive adoption than the current separate extensions? Looking at how evil-collection works where it's distributed as one package, but you can still pick-and-choose functionality.
• In terms of licensing, do we want to enforce any sort of Emacs-compatible license in hopes of being merged into Emacs core OR do we want to keep this separate long term so it's not license-restricted in what it can do?
• How should we get more volunteers to step in and help maintain the project?
• How should we document how to get an entire Elixir mode configured?
  • Should we publish instructions like in this ElixirForum thread?
• Should we combine our efforts with what ElixirLS is doing? Should we shift to a "we expect ElixirLS"?
• Do we want to support polymode for something like inline HTML templates

Closing thoughts

Back earlier this year I spoke with @axelson and @Trevoke about how to help contribute to this project. Not without quite a bit of effort on my part, I was unable to figure out who to speak with or how to go about helping push this project forward. I'd love to close the gap between this extension and what VSCode is doing.

[axelson]

First off, thanks for getting this discussion started! There's some good questions to ponder in here. Here's some initial thoughts:

Can we update the README to detail out who the maintainers of the project are?

I think this makes sense. I believe it would be @victorolinasc, @Trevoke, and myself. Not sure if I'm missing someone because I don't see a way for me to list out all the members

Should we publish a ROADMAP.md to show areas we think need to be added or revised?

I think a simple issue might be easier to manage, and we could pin it in the issues list

Would shifting this project to being a "one stop shop" for Elixir in Emacs better drive adoption than the current separate extensions? Looking at how evil-collection works where it's distributed as one package, but you can still pick-and-choose functionality.

I'm not sure about how this would end up working. My concern is that this would stretch our (as in maintainer's + community's) resources too thin. But perhaps I'm wrong about that.

In terms of licensing, do we want to enforce any sort of Emacs-compatible license in hopes of being merged into Emacs core OR do we want to keep this separate long term so it's not license-restricted in what it can do?

Quick nitpick, it's not the license that is the issue (emacs-elixir is using GPL), it is the copyright assignment to the FSF. Looking at the contributors list there are quite a few people, and I think it would be quite difficult to both contact them and get them to assign copyright to the FSF. So due to that difficulty my vote is to keep it separate and continue publishing to MELPA instead of ELPA.

How should we get more volunteers to step in and help maintain the project?

My main idea would be to make a few announcements asking for contributors, such as making a post on the ElixirForum and on Twitter. And additionally we could leave an open issue for a few months on this repo.

How should we document how to get an entire Elixir mode configured? Should we publish instructions like in this ElixirForum thread?

As long as we're not changing scope to be a "one-stop-shop" I think this is out of scope. Although linking to more comprehensive documentation would be great IMO.

Should we combine our efforts with what ElixirLS is doing? Should we shift to a "we expect ElixirLS"?

I'm generally in favor of keeping emacs-elixir separate from ElixirLS and instead aim for the two projects to be complementary (this is basically the same thinking that I posted above with respect to being a "one-stop-shop")

Do we want to support polymode for something like inline HTML templates

I think this is a good goal, but I'm not sure that much work is required to support this setup. Previously I was using polymode without too many issues (and none of them seemed specific to emacs-elixir)

[jsmestad]

Yeah good points on each item.

Now I totally appreciate this is the Emacs way and it is what you see from say using Ruby in Emacs. However when it was just alchemist.el it was WAY easier to get going with Elixir in Emacs and have a killer experience. Since the deprecation of that library things have become much more involved to get an experience anywhere near that of other editors for Elixir.

The pros / cons with the "one stop shop" idea to me is to follow more grouped convention (like alchemist.el did or a grouped package like evil-collection). I would imagine this looks like having an exunit.el, mix.el, and elixir-mode.el. The advantage is the documentation can speak to each of them as a holistic solution to getting Elixir running in Emacs. When it's a-la-carte, people struggle to get things working together as each project only documents it's responsibilities and not the interop between them. This holistic documentation has often come out of projects like Spacemacs or Doom to show the broader community how the packages can be combined and mapped to work together.

It also suffers from a users ability to discover these auxiliary libraries.

I think evil-collection has an elegant answer to this. If I want to do whatever to evil, I add (use-package evil-collection) and then configure what parts I want and which that I do not. The benefit is everything is there if I need it, but decoupled to allow for being customized.

I think right now it's quite a bit of effort to get Emacs going with all the bells and whistles for Elixir. For example, I currently have to get the following figured out:

• elixir-emacs
• mix.el (to run mix tasks)
• lsp-mode.el (or eglot.el) (for code completion)
• alchemist.el (this still provides functionality lacking from elixir-emacs and other packages)
• exunit.el (if I want test runner)
• polymode.el (if I want web-mode to work with inline liveview templates)

---

I think there should be a way to get back to that great (use-package alchemist) experience and get people off and running again hopefully driving adoption of the language. This approach should not require us to sacrifice customization that we all came to Emacs for in the first place.

[andreyorst]

Here's my two cents. Note, I'm not writing in Elixir, only looking at it, so I can be seen as a customer that can be turned off by the language support. Sort of. Anyway, I mostly write in Lisp, and very rarely in things like shell script, Lua, Perl, C, Rust, and some other languages. So I would like to look at elixir-mode from the perspective of languages that have great Emacs packages.

In my opinion, when user installs a major mode in Emacs, this mode should have essential functionality to use the language without external tooling. For example, Racket mode. This turns Emacs into pretty much complete Racket IDE - not only it provides syntax highlighting and indentation support, but it also has a REPL, completions, and a step debugger. Without such mode, users would be locked to use Geiser, which does not have a debugger, for example, so this feature is leaved out completely. Another example is Clojure, and while its major mode provides syntax highlighting and indentation handling, it also has some refactoring tools, that are meaningful for the language, and a very basic inferior clojure mode REPL. So again, it can mostly be used as a single package to get most of the work done, but more programmers use it paired with CIDER, thus making Emacs is a very capable IDE for Clojure. Same with common lisp and SLIME.

But these are Lisp languages that are known to have great support in Emacs, so what about other languages? Rust has a very basic major mode, which only provides syntax highlighting and indentation, but it also has ability to compile and run code, run Rust's linter, and code formatting. And paired with LSP support Emacs becomes a very capable IDE for Rust. This story repeats for pretty much any other language I've listed, because major modes for most of those were developed when LSP either did not exist, or was not yet adopted in Emacs.

Elixir kinda different in this regard, as it had Alchemist, which provided tons of features, but it was abandoned in favor of LSP, which actually can do less things, as it is not specific to Emacs. This gave birth to mix.el and other tools that were kinda extracted from Alchemist because LSP does not feature way to run tests for example, which can be done with Mix.

So in my opinion, major mode for Elixir should provide the following features that should not depend on any external tooling:

• syntax highlighting,
• indentation support,
• inferior IEx mode, so code can be evaluated from the buffer (kinda requires external tool but it comes with a language).

And the following features, that do require external tools:

• code formatting (currently handled with mix.el)
• code compilation, running, task running (also in mix.el)
• test running (via exunit.el)

Also a good support needed in widely used packages, such as

• Flycheck - has support via Credo
• Company mode - maybe a custom backend that can use IEx for completion? I'm aware that there's some package that runs IEx in background for completion to work, though I can't find the package, but maybe this can be a possible target?

LSP support makes last two integrations from above less needed, as it provides both completion and linting, but I think language still should be comfortable to use without depending on LSP for a bit of intelligence. Yes LSP is superior in some way, and it provides consistency, but this doesn't mean that this should be only option. Again, Clojure, there is a Clojure LSP, which evolves over time, but CIDER also is evolving and those two can actually be used together with a bit of configuration.

From my point of view Alchemist for Elixir was what CIDER is for Clojure - a well put package for interaction with a language. Which is actually a more or less unique feature of Emacs editor, to have a dedicated smart integration package for the language. Most other editors usually have generic support for each language, without too much smart stuff, and only recently LSP started to be widely used across editors to make common enchancements across those. Emacs always was different in that regard, for example it had semantic, Lisp modes had all this cool stuff around for years, which can directly compete with modern day IDEs. I believe that Elixir can be no exception here.

[Trevoke]

I like the direction andrey is going in, here.

I'm more conservative and I think the only things a major mode really needs to provide are:

• syntax highlighting
• indentation support

I'm not averse to more, but I think we should start with a conservative goal, given how much effort and expertise we have available right now.

I agree that the many packages, in the beginning, can be hard to manage for an end user, but they're infinitely easier to manage for open source maintainers, and we can always ask for unifying features if they're necessary. I much prefer a standalone mix.el package that other packages can safely depend on, for instance.

[victorolinasc]

First of all thanks to @jsmestad for this issue and the PRs and contributions made to this package. I am currently (last months) very hard at work and hadn't had the time to look into open-source more broadly unfortunately.

I agree with the general though here that a package that aims to implement a major mode shouldn't have more than syntax highlighting and indentation support. That is its main purpose and, IMHO, should be its only purpose. Both points might have a break-through in emacs devel with tree-sitter or something similar coming in the future. It is being actively discussed I think at least.

We could help the community have a "go to" package that would concentrate mix.el, iex.el, exunit.el, eex.el and all other goodies we can think of. Maybe have this centralized in a package so that it could all come bundled in might be a better experience. It is a bit contrary to the "do one thing only and do it well" but in this case it seems there are more successful examples of bundles of functionality than not.

[jsmestad]

@victorolinasc @Trevoke yeah maybe changing elixir-mode would be the wrong place to try and do things 'all in one'. I wonder if taking on reviving alchemist would be the better path forward.

[victorolinasc]

IMHO a better path would be to try an reach people responsible for mix.el, exunit.el and others to join on a new project where all of them would be maintainers together.

We could start by pinging them here for their opinion, WDYT?

At this point I believe it would be better to have a new project with this new aim that could bring the code from these smaller projects and build up a new one-stop-shop (plus the major mode) for this.

[jsmestad]

@victorolinasc I think for the new project I would aim for something as complete as alchemist was. That would mean the known extensions plus several more as there are still quite a few things missing

[victorolinasc]

@jsmestad that is what I thought too but their work could be used as a starting point IMHO. Unless there are very huge architectural changes or something like it.

I think that with more people onboard the chances of success might be higher.

[jsmestad]

Agreed

[axelson]

I'm also agreed on a new package tying everything together, and that would be as comprehensive as alchemist was.

Looking back at the original questions I think we just need to settle these two:

Can we update the README to detail out who the maintainers of the project are?

I propose adding victorolinasc, Trevoke, and myself (but please let me know if I'm missing anyone)

Should we publish a ROADMAP.md to show areas we think need to be added or revised?

I'm okay with adding a roadmap, but I'm not sure what would go on it

[viniciussbs]

Hi, guys. Any news about this topic? I would like to help, but I just started learning Emacs Lisp - even using Emacs for a long time.

IMHO a better path would be to try an reach people responsible for mix.el, exunit.el and others to join on a new project where all of them would be maintainers together.

I agree. mix.el already already offers some testing features like exunit.el, so maybe there's already some duplicated work happening.

[victorolinasc]

I think a bit has changed since we last discussed this here. IMHO we are almost at a point where the "purpose" of this project is not needed anymore... and this is actually a good thing :)

Currently, lsp-mode and similar gives us a very interesting IDE set of features with extensibility capabilities (actions). On the syntax parsing and coloring I think that the tree-sitter project is becoming the de-facto standard. Emacs seems inclined to implement this on the next big version and meanwhile the project https://emacs-tree-sitter.github.io/ seems well maintained.

Now that a tree-sitter parser is maintained by the core team, I think it will practically fulfill all the targets of this project. I think we could focus more on a better lsp experience and customizable actions as plugins with bindings into Emacs.

This would also make the devEx more standard across different editors. Of course, we are not there yet and so we might want to have everyone using a single package with everything.