Quarto virtual documents intermittently overwrite contents of editor

[juliasilge]

Originally posted by @brndngrhm in posit-dev/positron#6867

Sometimes a quarto file in positron will get re-formatted in an unusable way. I just opened a project and a quarto doc i was working don no longer has any code chunks or YAML, and instead there many # inserted as new lines. I'm not sure what causes this and i can't seem to undo it, so i need to basically manually re-format the file by going through it, creating code chunks and then pasting code inside the code chunks

There is an additional report by @kldhana in posit-dev/positron#6867 (reply in thread):

I noticed this as well, but could not reproduce.

This looks like the Quarto virtual documents going haywire somehow, as pointed out by @DavisVaughan. In the GH discussion, it seems like it happens on saving with format-on-save turned on.

[mvwestendorp]

I have this happen as well. For me it also changes the code itself and not just adds comments like the example above.
I have the air extension installed, and it happens when format-on-save is on for quarto. I thought of this issue posit-dev/air#77 when I read the discussion and this issue,

While I had a very simple file with 1 code block where this happened, for the moment it doesn't anymore so it is not something I can reproduce consistently.

[DavisVaughan]

@mvwestendorp were you also using Positron? Or just VS Code?

[mvwestendorp]

@mvwestendorp were you also using Positron? Or just VS Code?

Forgot to mention that. I am using Positron.

[DavisVaughan]

It is worth reiterating here that it is a known thing that Air does not work in Positron on Quarto documents right now due to posit-dev/air#77 - basically Quarto ends up invoking Air, and if Air decides nothing changes in the file it will continue on to invoke positron-r's {styler} integration as well, which is wrong, and will be fixed when we remove the styler integration.

The exact details are here:
posit-dev/air#77 (comment)

[DavisVaughan]

@mvwestendorp if you have that small file with 1 code block where you saw this before and are willing to share it, it may help us reproduce (even though we know Positron support isn't working right, this still seems like a very odd bug)

[mvwestendorp]

@mvwestendorp if you have that small file with 1 code block where you saw this before and are willing to share it, it may help us reproduce (even though we know Positron support isn't working right, this still seems like a very odd bug)

Sure it is not confidential, hopefully a gist is good way to share it. If not I am happy to hear how to share it in some other way.
https://gist.github.com/mvwestendorp/a3245e30d48c12afdc326c72a8322e30

I do see the behavior again with that file. In the screenshots you see the results of repeatedly saving the file. It is still a bit inconsistent on whether it happens (it is quite odd indeed).

[sharon-wang]

Also reported here: posit-dev/positron#7063

[DavisVaughan]

Managed to reproduce while in Positron using styler from positron-r. Note I have completely uninstalled Air here, so that rules Air out. I'm mostly convinced it is a Quarto timing bug now.

All I'm doing here is repeatedly saving and hitting backspace.

Screen.Recording.2025-04-02.at.12.56.49.PM.mov

[cscheid]

Note I have completely uninstalled Air here

I'm mostly convinced it is a Quarto timing bug now.

Yes, that seems right to me.

[DavisVaughan]

OKAY Y'ALL - I did a deep dive into this, here's what I've found!

I believe this is a combination of two issues, outlined below. See The quick fix section for the TLDR

Issue 1 - positron-r maximal edits

https://github.com/posit-dev/positron/blob/bd5930a95969d33939cc48d662f001966b9768c0/extensions/positron-r/src/formatting.ts#L95-L102

positron-r's styler integration always sends back what I call a "maximal edit", i.e. it is very lazy when asked to format a document and just sends back a single edit of [edit] which replaces the whole darn document, even if nothing changed. So when you have a qmd of

words

```{r}
code
```

words2

it becomes a .vdoc.r temp doc of

#
#
#
code
#
#
#

then positron-r / styler get that WHOLE DOCUMENT (I thought it was just the individual cell, I was wrong), formats it (nothing changes here), and sends back a single text edit as TextEdit[] that would replace the WHOLE QUARTO DOCUMENT with the virtual doc contents you see above (comments and all).

So how does this ever work?? If the TextEdit is passed through "as is", then every single time you try and save it would be trying to replace the whole qmd with the weird virtual doc. But here lies the magic! On the VS Code side, it receives that single TextEdit from positron-r and minimizes it into several smaller TextEdits. Here is the chain of calls:

• Command to call the format provider that Quarto calls https://github.com/posit-dev/positron/blob/bd5930a95969d33939cc48d662f001966b9768c0/src/vs/editor/contrib/format/browser/format.ts#L475-L488
• Which asks for minimal edits https://github.com/posit-dev/positron/blob/bd5930a95969d33939cc48d662f001966b9768c0/src/vs/editor/browser/services/editorWorkerService.ts#L144-L152
• And here's the guts of the minimization https://github.com/posit-dev/positron/blob/bd5930a95969d33939cc48d662f001966b9768c0/src/vs/editor/common/services/editorSimpleWorker.ts#L218

Basically what this ends up doing is diff-ing the original contents of .vdoc.r with that TextEdit positron-r gives back, which in this case also happens to contain the exact same contents. The result is that no TextEdits are returned after minimization, and Quarto ends up saying back to VS Code "nothing to do here".

In the case where there is something to change, the minimization process is generally good enough that it takes the single massive edit from positron-r, diffs it against the original contents of .vdoc.r, and the resulting minimal TextEdits just so happen to line up with the Quarto code cells pretty much perfectly (but I think we are getting very lucky here, there are many ways this can fail!).

Okay, so if that's what is supposed to happen, what's going on in this issue? Well...

Issue 2 - .vdoc.r overwriting

Note that when we perform a format request, we go through this special local .vdoc.r path with its own early exit

quarto/apps/vscode/src/vdoc/vdoc-tempfile.ts

Lines 43 to 52 in 8ee12b5

	// if this is local then create it alongside the docPath and return a cleanup
	// function to remove it when the action is completed.
	if (local || virtualDoc.language.localTempFile) {
	const ext = virtualDoc.language.extension;
	const vdocPath = path.join(path.dirname(docPath), `.vdoc.${ext}`);
	fs.writeFileSync(vdocPath, virtualDoc.content);
	const vdocUri = Uri.file(vdocPath);
	const doc = await workspace.openTextDocument(vdocUri);
	return newVirtualDocUri(doc);
	}

This is different from the non-local in that the local document is always called vdoc.r, but in the non-local case it gets a UUID attached:

quarto/apps/vscode/src/vdoc/vdoc-tempfile.ts

Lines 90 to 97 in 8ee12b5

	function createVirtualDocTempFile(virtualDoc: VirtualDoc) {
	const ext = virtualDoc.language.extension;
	const dir = path.join(vdocTempDir, ext);
	if (!fs.existsSync(dir)) {
	fs.mkdirSync(dir);
	}
	const tmpPath = path.join(vdocTempDir, ext, ".intellisense." + uuid.v4() + "." + ext);
	fs.writeFileSync(tmpPath, virtualDoc.content);

We had seen some timing issues in the past where that UUID was important! If some other LSP request comes through that overwrites .vdoc.r before we have finished working with it, we are screwed! And that is indeed what is going on here. Note how the minimal edit computer just totally bails on attempting to minimize if canSyncModel() returns false, instead returning the original single large TextEdit that positron-r gave us

https://github.com/posit-dev/positron/blob/bd5930a95969d33939cc48d662f001966b9768c0/src/vs/editor/browser/services/editorWorkerService.ts#L144-L154

If we get caught in the scenario where our .vdoc.r has been deleted out from under us and replaced with some other contents, then it will bail. I even caught this happening:

The quick fix

So that explains it:

• positron-r sends back a whole file TextEdit
• the .vdoc.r is replaced out from under us
• the minimization of the whole file TextEdit fails
• Quarto passes through a whole file TextEdit that replaces the qmd contents with the .vdoc.r contents

I believe we can patch this by utilizing a tweaked version of createVirtualDocTempFile() here in place of just creating .vdoc.r:

quarto/apps/vscode/src/vdoc/vdoc-tempfile.ts

Lines 46 to 48 in 8ee12b5

	const ext = virtualDoc.language.extension;
	const vdocPath = path.join(path.dirname(docPath), `.vdoc.${ext}`);
	fs.writeFileSync(vdocPath, virtualDoc.content);

The main reason to do so is that it will give us vdoc-{uuid}.r, preventing future LSP requests from overwriting the current LSP request's virtual doc file, which should allow the minimization bit to work as intended.

We are also going to remove the styler integration in Positron in favor of Air. That alone would not completely solve this general problem (which does affect all languages), but it would help because Air does try to do minimal edits.

The bigger problem

I really really don't love that the formatter is handed the entire qmd document in an R-ified form where the prose is replaced by # comments.

This relies way too heavily on things working "perfectly", where the formatter must not rearrange those comments in any way, and the text edits it returns must be perfectly minimized. There is a huge margin for error here.

A reliable solution would be for Quarto to ask the formatter to format a virtual document containing a single cell at a time. Quarto would map the returned text edits from virtual document space back into qmd space, replacing the contents of the cells individually. I get that this could be slow when you have many cells to work on, but I think this is the only reliable way to do this when virtual docs are involved because there is just too much variation in how formatters behave.

The real solution is probably for Quarto to use the Notebook API of the LSP. This would allow Quarto to give each cell its own unique URI and TextDocument. So when formatting is requested of the Qmd's URI, it may be possible to just request that each individual cell just be formatted as a normal TextDocument of a particular language, I'm not totally sure. That's a lot of work, but would probably also make all other LSP features more reliable.