API for getting a text document without opening

[felixfbecker]

As an extension developer, I sometimes want to get the content of a textDocument identified by a URI from VS Code, but without opening it and triggering onDidOpen events.

My use case is extending the LanguageClient to support the textDocument/xfiles extension. Language servers in isolated environments need a way to request file contents without accessing the file system directly, and there should be a way to extend the LanguageClient to support these requests as well. I cannot read from disk because the URI could also be the URI of an in-memory document.

There are probably other use cases where an extension needs to inspect file contents without wanting VS Code to open them.

Current API:

		/**
		 * All text documents currently known to the system.
		 *
		 * @readonly
		 */
		export let textDocuments: TextDocument[];

		/**
		 * Opens the denoted document from disk. Will return early if the
		 * document is already open, otherwise the document is loaded and the
		 * [open document](#workspace.onDidOpenTextDocument)-event fires.
		 * The document to open is denoted by the [uri](#Uri). Two schemes are supported:
		 *
		 * file: A file on disk, will be rejected if the file does not exist or cannot be loaded, e.g. `file:///Users/frodo/r.ini`.
		 * untitled: A new file that should be saved on disk, e.g. `untitled:c:\frodo\new.js`. The language will be derived from the file name.
		 *
		 * Uris with other schemes will make this method return a rejected promise.
		 *
		 * @param uri Identifies the resource to open.
		 * @return A promise that resolves to a [document](#TextDocument).
		 */
		export function openTextDocument(uri: Uri): Thenable<TextDocument>;

Proposal:

		/**
		 * Like openTextDocument(), but does not fire an didOpen event and does not add the document to the textDocuments array.
                 * 
		 * @param uri Identifies the resource to open.
		 * @return A promise that resolves to a [document](#TextDocument).
		 */
                export function getTextDocument(uri: Uri): Thenable<TextDocument>;

[jrieken]

but without opening it and triggering onDidOpen events.

Why? I don't see that as a benefit but as additional complexity

[felixfbecker]

Many language servers need to index all files in the project. The linked protocol extensions are an effort to allow them to do that over LSP, without accessing the file system, because servers often run in an isolated environment like a Docker container. They can, for example, operate solely on the git:// protocol and provide IntelliSense directly for GitHub repos - without even checking the commit out, they can get file contents at any commit much more efficiently from git directly. Another example is working over FTP, the IDE doesn't have to download the whole project to work on it, it only needs to download the files the LS needs.

Now, to implement these requests in VS Code the workspace/files request can use the findFiles() API to return a list of all files. So far so good. Now the LS will iterate over the result and start indexing, for every file it will request the contents of that file, parse it, add the symbols to the index and then discard the content again. This textDocument/content request should only return a TextDocumentItem, but not cause the file to get "opened" in the editor. If VS Code triggered onDidOpen events, these events would get send to the LS to - which means the content will be transferred twice, one time for the request, and then with the event. All the LS wants to do is index the symbols - if onDidOpen events are triggered, it will cause VS Code and the LS to keep all these files in memory. For a project with 26k files, you would have 26k files opened and loaded in memory, twice.

I don't see why there would be added complexity - openTextDocument would just call getTextDocument, and then additionally trigger the events and add it to the opened files. I can do a PR if that would be appreciated?

[jrieken]

which means the content will be transferred twice, one time for the request, and then with the event.

That is incorrect. The document data is transferred only once and inside the extension host references are passed around.

if onDidOpen events are triggered, it will cause VS Code and the LS to keep all these files in memory.

No, the editor can decide at any time to close a document again. This usually happens when the last editor showing a document closes but also in other cases.

[felixfbecker]

Okay, so maybe I misunderstand the openTextDocument function then?

• Does it cause VS Code to "open" a text document inside the editor (eg show it to the user)?
• Does it cause VS Code to send an onDidOpen notification to language servers (that includes the content)?

[jrieken]

So, openTextDocument just loads the document into memory (think of bytes to text document). To show that document in an editor you must invoke showTextDocument.

@aeschli or @dbaeumer know best but to my knowledge onDidOpen events on the language server side are only send for documents you have expressed interest for using the documentSelector or synchronize-config

[felixfbecker]

Thanks for clearing that up @jrieken! I think I can work with that.
I will assume onDidOpen is getting sent for every document that has been showed in the editor then, to use the right terminology. It is definitely not only through synchronize?

[dbaeumer]

The language server client filters open events based on a document selector. However if the server expressed interest in lets say *.php files and the openTextDocument for a PHP file triggers an open event then that event is forwarded to the server as well.

@jrieken I assume that openTextDocument does trigger on open event.

[jrieken]

@dbaeumer Yes it does - unless it already is open.

@felixfbecker Objects closing this after clarifying things?

[felixfbecker]

Well then my assumption was correct after all

<- workspace/xfiles
-> here are all files in the workspace
<- textDocument/xcontent for 1
-> here is the content (TextDocumentItem) of text document 1
-> textDocument/didOpen here, I just opened text document 1, have the content // this should not happen
...
proceed for the next 26k text documents

I need to prevent VS Code from sending a didOpen notification to the server - I just want the text document content (TextDocumentItem)

[jrieken]

Where do you want to access the document content? In the extension host or in a language server?

[felixfbecker]

In the language client, I want to register a custom handler that uses the vscode API to get the document content, all to allow the server to get the document content.

So to the correct answer for your question would be: both. Once I have it in the former, I can enable it in the latter.

[dbaeumer]

@felixfbecker I can understand that you don't want to read the file from disk on the server side but why can't the client simply load the content of the file.