Escaped double quotes: \" in model responses get shown as just " in chat UI

[AlexanderLuck]

What happened?

Hi,
when I let modesl generate code containing strings with escaped " the codeGPT UI seems to strip the \ from the text for example I get as an answer:
var query = $"from(bucket:"{BucketName}") |> range(start: -1y) |> filter(fn: (r) => r._measurement == "metadata_tracking" and r.measurement_name == "{measurementName}") |> last()";

Correct would be:

var query = $"from(bucket: \"{BucketName}\") |> range(start: -1y) |> filter(fn: (r) => r._measurement == \"metadata_tracking\" and r.measurement_name == \"{measurementName}\") |> last()";

When I ask the model to fix the first response it correctly tells me the delimiters are missing and posts exactly the same with missing delimiters.

Relevant log output or stack trace

No response

Steps to reproduce

Ask in chat:
generate a c# string log message with other variables using a " delimited string

CodeGPT version

2.5.1

Operating System

None

[reneleonhardt]

@carlrobertoh I can't really find handling of escaped quotes or corresponding tests in llm-client 😅
Maybe integration tests could catch that?
When I insert \\\" into some requests and responses in OllamaClientTest everything is still green... as expected 😄

There are multiple locations dealing with escape sequences (including quotes) in CodeGPT cpp files, but I can't find tests.

[reneleonhardt]

@AlexanderLuck What service / model service are you using?
ollama run codellama doesn't generate that code including escapes with that prompt and empty context.

[carlrobertoh]

I have noticed the same. The bug is related to how the response is rendered on the screen. Each time a new message is received, it is converted into HTML (using flexmark) before being displayed to the user. Most likely, the problem lies somewhere in that process.

As a workaround, you can see the "correct" output by clicking on this icon button:

[AlexanderLuck]

I use Claude and GPT4, both have that issue.
ooks like classic pre-conversion filtering issue as @carlrobertoh describes.
It is just a minor annoyance that happens rarely so I though I'd report it in case somebody works on the UI at some point.

[AlexanderLuck]

Just checked again, also happens with user inputs, if you just put in chat:
"
repeat after me:

var query = $"from(bucket: "{BucketName}") ";
"
You will see that your own inputs are stripped of the two " as well.
Again, not a big deal, at least for me, but I thought maybe the same code filters other things for other users as well.

[alvitawa]

This bug is super annoying
Cant do any latex with codegpt anymore because of it
But gpt4o on in chat.openai.com has the same issue it seems
gpt3.5 does not

[carlrobertoh]

This will be fixed in the next release. I'm not yet entirely sure why it was added in the first place.