[Bug]: OpenAI endpoint sends/returns too many characters/tokens

[Snify89]

Is there an existing issue for the same bug?

• I have checked the existing issues.

RAGFlow workspace code commit ID

latest

RAGFlow image version

latest

Other environment information

Actual behavior

The OpenAI endpoint works great with the chat ID. I have implemented it in OpenWebUI and noticed, that response actually sends too many tokens/characters.
E.g. when the LLM answers "This is a great idea", it answers something like "This is a great idea. dea." The last part of the answer is duplicated (based on the length of the entire answer, I guess). I suppose there is a miscalculation or some sort. I haven't tested it with curl or some sort, so it might be a OpenWebUI issue but I doubt it.

Expected behavior

No response

Steps to reproduce

Implement OpenAI endpoint to OpenWebUI (or any other maybe) and chat.

Additional information

No response

[yongtenglei]

Hi,

Is there an issue with non-stream mode? We have just fixed a problem with stream mode. We typically test our API using Postman and curl and could not reproduce this error. Could you try it in non-stream mode and let us know if it works? Or, please test with our latest changes to see if that resolves the issue. You should have some understanding of how to start from the source code.

Also, if your app relies on token usage, you might encounter issues, as we currently count token usage based on the number of characters. If this is critical for you, please let us know, and we will address it promptly.

[Snify89]

Hi,

Is there an issue with non-stream mode? We have just fixed a problem with stream mode. We typically test our API using Postman and curl and could not reproduce this error. Could you try it in non-stream mode and let us know if it works? Or, please test with our latest changes to see if that resolves the issue. You should have some understanding of how to start from the source code.

Also, if your app relies on token usage, you might encounter issues, as we currently count token usage based on the number of characters. If this is critical for you, please let us know, and we will address it promptly.

Thank you very much for your response.
I will look into this again and let you know what's going on, in more detail.