Increase possibilities for yazi<->embedding editor integration

[mikavilpas]

Please describe the problem you're trying to solve

In https://github.com/mikavilpas/yazi.nvim, I have ideas for some features that I think might require allowing a deeper level of interaction with editors that want to embed yazi:

• open the currently open neovim splits in yazi tabs
• open yazi in a floating window inside neovim, then select files in yazi, and finally do a batch rename for them using the neovim that is already open
• properly forward events about the neovim terminal being resized.
  • Neovim currently has some bugs in its implementation and a similar plugin for neovim was able to work around them by using remote procedure call (rpc) calls to notify the Ranger file manager manually
• make neovim support showing images that are selected in yazi
  • right now the terminal implementation in neovim does not support images, but neovim itself does not have this limitation. If I have live access to yazi events, this can be done
• explore options related to displaying yazi in a permanent split in neovim, not a floating window that constantly needs to be closed and reopened
  • (not sure what could be done with this approach yet)
• refactor and make the currently available yazi.nvim "open in split/tab/quickfix list" commands more robust
  • right now they are done by shadowing the keymappings that are used in the yazi floating window.
  • the problem with this is that they make an assumption that the user has not changed the default yazi "open with enter" keymapping. While most users probably have not, this approach cannot be used for many things

Would you be willing to contribute this feature?

• Yes, I'll give it a shot

Describe the solution you'd like

Suggestions

Suggestion 1: programmatic access to yazi actions

The api could be disabled by default, and enabled with a command line flag such as yazi --enable-rpc-api, maybe even enabled in the user's configuration.

Currently, yazi can send events via the data distribution service (dds), and the editor integrations can listen to these events. Events can also be sent to yazi via ya pub and ya pub-static.

Even though the communication is now bidirectional, there are some difficulties with this approach:

• it's very specific to yazi. Although the communication protocol is not too complex, it's still a custom protocol
  • it's not a huge thing, but currently due to neovim limitations, I cannot read live events from yazi. The apis seem to only allow reading stdout + stderr as a combined stream while the application (yazi) is running, so right now I can only read the events after yazi has been closed.
  • contrast this to RPC support which is built in in neovim because it's a common protocol
• no yazi actions (such as "open this file") are exposed out of the box, although with plugins it seems to be possible to expose custom actions
  • yazi's api is large with lots of functionality
  • this is the same problem that the language server protocol (lsp) solves - namely, each plugin interested in this must implement their own actions. So there are m actions * n plugins while there could be m actions + n plugins if they were implemented once in yazi

Suggestion 2: allow loading additional plugins from the command line

With deeper integration between yazi and neovim, some features only make sense when both yazi and neovim are running. I love the fact that yazi is a very composable tool and I can run it from the terminal as well as in my editor.

I want myself and other users to have a good experience in both environments. I think the best way to do this would be adding a new flag such as yazi --load-plugin='~/.local/share/nvim/lazy/yazi.nvim/bundled-yazi-plugin/' which would load this additional plugin that provides yazi.nvim specific functionality. When running yazi in the terminal environment, the plugin would not get loaded.

Additional context

Benefits and ideas

Finally, I want to list some ideas and benefits that I think would come from these changes:

• Suggestion 1 (programmatic access to yazi actions)
  • would enable writing integration tests for yazi
    • I think it would make the most sense to test at this level
  • would enable plugin writers to also write integration tests for their plugins
    • it would also be really fun to explore writing helpers for development workflows
      • e.g. when I save my file, automatically restart yazi and do these 5 plugin specific actions
  • would be a great way to "dogfood" yazi's lua api
    • if full lua typing can be added (see the discussion here), the types would also be dogfooded
• Suggestion 2 (allow loading additional plugins from the command line)
  • plugin experiences can be optional and not disturb users who do not want to use them
  • performance is also not affected when running yazi in the terminal

[sxyazi]

but currently due to neovim limitations, I cannot read live events from yazi. The apis seem to only allow reading stdout + stderr as a combined stream while the application (yazi) is running, so right now I can only read the events after yazi has been closed

If Neovim doesn't provide an API for reading stdout from a running process, you can start another Yazi process in the background as a server using yazi --remote-event, and the process with which the user interacts will automatically become the client.

which would load this additional plugin that provides yazi.nvim specific functionality. When running yazi in the terminal environment, the plugin would not get loaded.

I'm not quite sure what you mean by "loading" plugins. Normally, Yazi plugins don't need to be manually loaded; they are lazily loaded automatically after user actions (such as pressing a key). Are you referring to the initialization process of plugins? If so, that would involve loading an init.lua rather than specific plugins.

[mikavilpas]

If Neovim doesn't provide an API for reading stdout from a running process, you can start another Yazi process in the background as a server using yazi --remote-event, and the process with which the user interacts will automatically become the client.

I love this idea. I think I will try it out soon.

I'm not quite sure what you mean by "loading" plugins.

I'm not sure if "plugin" is the correct term to use here. I'm looking for a way to

• get editor related features available when I use yazi inside neovim
• (ideally) have the code be loaded by yazi from a location that I can easily control. The motivation is to not force the user to handle custom installations and keeping them in sync with their neovim, yazi.nvim and yazi versions

[mikavilpas]

If Neovim doesn't provide an API for reading stdout from a running process, you can start another Yazi process in the background as a server using yazi --remote-event, and the process with which the user interacts will automatically become the client.

Tried this approach, but I quickly found out yazi could not be started when it's "hidden", possibly because it cannot draw to a screen.

Opened #1004 which could be used to work around this limitation - the idea is to start ya sub-static in addition to the already running yazi instance, and listen to incoming events without displaying a TUI.

[mikavilpas]

In comparison to the LSP (language server protocol) which uses json RPC (https://microsoft.github.io/language-server-protocol/overviews/lsp/overview/), several types of messages are supported:

1. request (editor -> server -> editor): editor asks the server, and the server responds with data
2. notice (editor -> server): the editor notifies the server but doesn't need a response
3. notice (server -> editor): the server notifies the editor but doesn't need a response

Right now, the ya sub implementation that's ongoing in #1004 should allow for type 3 (notice from server -> editor). I'm curious how you might feel about types 1 and 2.

Is this a direction you could see yazi going?

[sxyazi]

It sounds like the three modes you mentioned are already covered by the current DDS, but it's based on an asynchronous publish-subscribe model rather than a synchronous request-response model.

For example, for the first mode, editor -> server -> editor, you can use the Lua API to call ya.pub_to to send a message to the server, which then forwards the message to the corresponding Yazi instance. The instance then executes the event, and you receive the result through ya.sub_remote.

[mikavilpas]

Message passing with the DDS itself should not be an issue. My issue is that there is no known real time communication channel that I could use from neovim.

It's becoming clearer and clearer to me that at some point yazi.nvim might need its own plugin for various features. I will think about this idea more.

Maybe there's something that could be done on the plugin level to make the communication possible.

[sxyazi]

My issue is that there is no known real time communication channel that I could use from neovim.

I think adding a new ya sub to the CLI (as in the content of this PR #1004) to workaround the current limitations of nvim is the right direction. But since it's closed, I'm wondering if there's something specific that it couldn't cover in this way?

It's becoming clearer and clearer to me that at some point yazi.nvim might need its own plugin for various features.

Could you describe your needs? What are the things that DDS currently can't provide directly and must be done through plugins?

[mikavilpas]

Actually, I'm coming around to #1004 being a good approach. Let's reopen it and see if we can finish it.

Previously I was confused about a few things, but I think this approach can work for me:

• ya sub can be used to receive events from yazi in neovim in real time
• ya pub can be used to send events to yazi from neovim
• I can write a yazi plugin to act as the orchestrator for custom functionality can be used in neovim

I will think about what the best way to include the plugin would be.. For example, when running yazi inside neovim, some keybindings should probably run plugin specific functionality, but when running outside of neovim they should use the default functionality instead.

Have you thought about this kind of an approach?

[39555]

I've also played with this issue because I'm using Nushell, but yazi.nvim depends on the posix shell when running --local-events with redirection to a temp file to read events. Without the redirection, the stdout prints all stuff to the screen and breaks the UI

Starting the second process with yazi --remote-events is not possible. All of termopen jobstart, vim.uv.spawn exit with Device is not configured because yazi tries to draw the UI. --remote-events becomes useless without a --headless flag or something similar.

[sxyazi]

I've also played with this issue because I'm using Nushell, but yazi.nvim depends on the posix shell when running --local-events with redirection to a temp file to read events. Without the redirection, the stdout prints all stuff to the screen and breaks the UI

Starting the second process with yazi --remote-events is not possible. All of termopen jobstart, vim.uv.spawn exit with Device is not configured because yazi tries to draw the UI. --remote-events becomes useless without a --headless flag or something similar.

Please try #1004 to see if it meets your needs.

[sxyazi]

some keybindings should probably run plugin specific functionality

Could you give me an example - which keys need to trigger a specific plugin inside Neovim and when?