Better UI extensibility: retain webviews, provide communication channel

[pavelfeldman]

I was looking at porting some of the Chrome DevTools features over to VSCode: timeline / profiler to start with, maybe elements / sidebar.

All my controls have rich UI that I would like to surface either as editors or as views in the console drawer. I was using TextDocumentContentProvider to render my content. Some of the limitations of the extensibility system I hit were:

• Webviews are re-created on switching editor tabs (removed from DOM?). That does not work for my rich stateful controls, would be nice to retain those. I know that reparenting is impossible with the regular iframes, not sure whether webview / oopif can make it work at this point. I can look into it on the browser side if you are interested in retaining them on your side. In Chrome DevTools we use the "hideOnDetach" trick for extensions where we remove elements from render tree, while retaining them in DOM to keep them alive.

• Bi-directional communication channel between webview content and extension. I need to be able to post messages back and forth and existing uni-directional href=command: trick does not satisfy my use cases. A richer API that allows evaluating arbitrary strings (appending scripts) into webview would also be handy. I don't think these changes affect your security model, but they would enable us to do nice things.

• Contribute UI views next to Terminal and Console. For some of my controls it makes sense to surface them beside editor as an auxiliary or shell view.

[jrieken]

The design of VS Code is to not allow for UI extensions on that level but our API defined data contracts where extensions fill in data and the editor renderers it. In fact, extensions run in a separate, headless process to guarantee isolation. See https://code.visualstudio.com/docs/extensionAPI/patterns-and-principles#_our-approach-to-extensibility for all the details.

[pavelfeldman]

Yes, I'm familiar with the existing policy and I am suggesting to extend it. I was thinking of making parts of Chrome DevTools work in VSCode environment as well (I'm TL of the Chrome DevTools team). And present UI extensibility model feels rather limiting. More than that, I can get pretty far with the existing policy:

• use local server for interaction between the TextDocumentContentProvider content and extension
• refactor some of our code to re-build based on the external state to address tab switch scenario.

But the experiences I create end up sub-optimal, at the expense of the user. If you think your longer term goal is to allow richer UI extensibility, I can look into the browser limitations that prevent you from getting there. Oopifs should allow security and performance isolation sufficient to run extensions in the UI without compromising your core experience. I know reparenting is not there, but it is more important if you are willing to get there. Alternatively, if at this point you are strong on not letting features such as Chrome's performance tooling into your framework as extensions, I'll drop the idea altogether.

[egamma]

@pavelfeldman we are definitely interested in experiments in this area. I'll look into this in our side and I will get back to you.

[egamma]

@pavelfeldman having the Chrome DevTools performance tooling available inside VS Code is very interesting to us. What do you think of the approach of pushing the integration as far as we can get with the current support and then we collaborate on improving the experience and the extension API.

I suggest to start with an integration of the perf tools in an editor using the existing htmlPreview support. This is similar to what we did for the SQL extension. The SQL extension provides rich UI to present query results. The SQL externalizes the state into an express server.

I'm adding some team members that can chime in if you are game to start on this and you have more questions:
@jrieken htmlPreview API
@weinand debugger architecture and protocol
@roblourens chrome debugger extension

[jrieken]

Ok, lets brainstorm more on this. As you have said, the document provider and preview command is a way of making custom UI but was definitely not designed for that. It is what its name implies, previewing a resource. We acknowledge that it's used for building UI and time to make this more official and supported will come. We had similar ideas, in short along those lines

• have an API function ala createWebview(name:string, location: ViewColumn | OtherLocation): WebView. That function will create a part in the UI that can be positioned in some predefined places and that can be talked to. So, the 'document preview preview command'-workaround won't be needed anymore.
• Similar to an iframe or web worker a WebView will have some sort of communication channel, think of WebView.sendMessage and WebView.onMessage. That will allow easy bidi-communication between an extension and its UI piece, no more servers listening on localhost.

An API could look like this

interface Webview {
	readonly name: string;
	location: ViewColumn;
	markup: string;
	onDidReceiveMessage: Event<any>;
	sendMessage(message: any): Thenable<any>;
	dispose(): void;
}

export function createWebview(name: string): Webview;

Reasons why we haven't pursued that idea yet are

• As you know, things are implemented with a WebView which is buggy and a source of frustration. For security and process isolation reasons (out mantra is that the cursor is always blinking, you can always save) we need to run things in a separate process. I'd love to learn more about OOIF, I know they weren't there when we started with this 1.5 years ago.
• The reparenting challenge. Also, I'd love to know more about that. Please shed some light on you hideOnDetach trick. We used to have the web view sitting on the body, would hide it when not shown, and would absolutely position it when needed. Apart from slowness it is a big accessibility no-no. Also, when shuffling editors (swap column, move out, merge) around we reparent, again to keep the tab-index working.
• Apart from the technical challenges there is look and feel. Things like font-face, size, colours etc are easy'ish. Already today, the previewed document gets those style set but thinks like widget reuse is a different story. How do we make sure extension use the native context menu, how do we align the basics like buttons, file labels. etc?
• Also, there is source for conflicts when it comes to keybindings, link clicking, drag&drop etc. I'd say that something like Cmd+Q should always be handled in the host, where something like Cmd+C should maybe not.

In short, many questions and tradeoffs. @pavelfeldman - I'd like to hear your thoughts on this.