[Bug]: Empty response when disabling streaming in the RagFlow API

[Peterson047]

Is there an existing issue for the same bug?

• I have checked the existing issues.

RAGFlow workspace code commit ID

448fa1c

RAGFlow image version

v0.16.0

Other environment information

Windows 11 Pro
I7 11ºGen
12Gb ram

Actual behavior

Description:
I am developing an application that will communicate with RagFlow via Make (Integromat). To achieve this, I used the following cURL request:

curl --request POST \
     --url http://{address}/api/v1/agents/{agent_id}/completions \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer <YOUR_API_KEY>' \
     --data-binary '
     {
          "question": "Hello",
          "stream": true,
          "session_id": "cb2f385cb86211efa36e0242ac120005"
     }'

Context

• For {agent_id}, I used the dialogId from the chat link.
• For session_id, I used the conversationId from the link.

The request works correctly when the stream parameter is set to true. However, the Make platform does not support this type of request, preventing the use of APIs that return responses via streaming.

By default, the RagFlow API returns responses via streaming with the "stream": true parameter. When I change it to "stream": false, the response comes back empty:

{
    "code": 100,
    "data": null,
    "message": "KeyError('content')"
}

Notes

• In Postman, the response time suggests that processing was completed, but the data field returns null.
• Am I missing any configuration, or could this be a bug in the current version?

Reference screenshot:

Additional Information

• Environment: I am using version v0.16.0 (full).
• Other relevant configurations: The chat API is Gemini 1.5 Pro.

I appreciate your support! 🚀

Expected behavior

I expected the complete response in a single return instead of streaming.

Steps to reproduce

I used the chat completion `cURL` request and changed the `stream` parameter to `false`. I expected to receive the complete response in a single return instead of streaming.

Additional information

No response

[KevinHuSh]

Do you have backend error log?
docker logs -f ragflow-server

[Peterson047]

Do you have backend error log? docker logs -f ragflow-server

"stream": false,

2025-02-12 12:35:09 2025-02-12 23:35:09,517 ERROR    18 'content'
2025-02-12 12:35:09 Traceback (most recent call last):
2025-02-12 12:35:09   File "/ragflow/.venv/lib/python3.10/site-packages/flask/app.py", line 880, in full_dispatch_request
2025-02-12 12:35:09     rv = self.dispatch_request()
2025-02-12 12:35:09   File "/ragflow/.venv/lib/python3.10/site-packages/flask/app.py", line 865, in dispatch_request
2025-02-12 12:35:09     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
2025-02-12 12:35:09   File "/ragflow/api/utils/api_utils.py", line 303, in decorated_function
2025-02-12 12:35:09     return func(*args, **kwargs)
2025-02-12 12:35:09   File "/ragflow/api/apps/sdk/session.py", line 156, in chat_completion
2025-02-12 12:35:09     for ans in rag_completion(tenant_id, chat_id, **req):
2025-02-12 12:35:09   File "/ragflow/api/db/services/conversation_service.py", line 154, in completion
2025-02-12 12:35:09     for ans in chat(dia, msg, False, **kwargs):
2025-02-12 12:35:09   File "/ragflow/api/db/services/dialog_service.py", line 376, in chat
2025-02-12 12:35:09     msg[-1]["content"], answer))
2025-02-12 12:35:09 KeyError: 'content'
2025-02-12 12:35:09 2025-02-12 23:35:09,521 INFO     18 172.18.0.6 - - [12/Feb/2025 23:35:09] "POST /api/v1/chats/2a5261d2e88e11ef9ed10242ac120005/completions HTTP/1.1" 200 -

"stream": true,

2025-02-12 12:36:57 2025-02-12 23:36:56,479 ERROR    18 LLMBundle.encode_queries can't update token usage for fa726e76e87411efbf570242ac120002/EMBEDDING used_tokens: 28
2025-02-12 12:36:57 2025-02-12 23:36:57,883 INFO     18 POST http://es01:9200/ragflow_fa726e76e87411efbf570242ac120002/_search [status:200 duration:0.125s]
2025-02-12 12:37:02 2025-02-12 23:37:02,355 INFO     18 172.18.0.6 - - [12/Feb/2025 23:37:02] "POST /api/v1/chats/2a5261d2e88e11ef9ed10242ac120005/completions HTTP/1.1" 200 -
2025-02-12 12:37:06 2025-02-12 23:37:06,755 ERROR    18 LLMBundle.encode can't update token usage for fa726e76e87411efbf570242ac120002/EMBEDDING used_tokens: 768
2025-02-12 12:37:07 2025-02-12 23:37:07,352 INFO     19 task_consumer_0 reported heartbeat: {"name": "task_consumer_0", "now": "2025-02-12T23:37:07.350+08:00", "boot_at": "2025-02-12T23:33:06.567+08:00", "pending": 0, "lag": 0, "done": 0, "failed": 0, "current": null}
2025-02-12 12:37:30 2025-02-12 23:37:30,565 ERROR    18 LLMBundle.encode_queries can't update token usage for fa726e76e87411efbf570242ac120002/EMBEDDING used_tokens: 11
2025-02-12 12:37:30 2025-02-12 23:37:30,787 INFO     18 POST http://es01:9200/ragflow_fa726e76e87411efbf570242ac120002/_search [status:200 duration:0.071s]
2025-02-12 12:37:33 2025-02-12 23:37:33,845 INFO     18 172.18.0.6 - - [12/Feb/2025 23:37:33] "POST /api/v1/chats/2a5261d2e88e11ef9ed10242ac120005/completions HTTP/1.1" 200 -
2025-02-12 12:37:37 2025-02-12 23:37:37,378 INFO     19 task_consumer_0 reported heartbeat: {"name": "task_consumer_0", "now": "2025-02-12T23:37:37.376+08:00", "boot_at": "2025-02-12T23:33:06.567+08:00", "pending": 0, "lag": 0, "done": 0, "failed": 0, "current": null}
2025-02-12 12:37:41 2025-02-12 23:37:41,408 ERROR    18 LLMBundle.encode can't update token usage for fa726e76e87411efbf570242ac120002/EMBEDDING used_tokens: 1280

[KevinHuSh]

Have you change the code?
I don't think the code is consistent with the current repo.

[adrku24]

I've tested gemini-1.5-flash-latest and got the same KeyError.
I'am using ff237f2 without any modifications.

[Peterson047]

Did you change the code? I don't think the code is consistent with the current repository.

To answer your question, I had not made any changes previously. To make sure of this, I rebuilt the project from scratch and tested again, confirming that the error still occurred.

I was able to solve the problem by modifying the construction of msg, ensuring that all messages contain the "content" key, even if it is empty on line 374 of the file:
/ragflow/api/db/services/dialog_service.py

After analyzing the logs, I realized that the error occurred because msg[-1] did not always contain "content", especially when stream=False. So, make the following change to ensure that the key is always present:

response = chat_mdl.chat(prompt, message[1:], gen_conf)
user_content = msg[-1].get("content", "[content not available]")
logging.debug("User: {}|Assistant: {}".format(user_content, response))

After this change, the error was resolved and now the behavior is consistent in both modes (stream=True and stream=False).

I sent a Pull Request with this solution, which you can check here: [#4944](#4944). ```

This should help explain the problem situation and the steps you took to resolve it.

[raikloe]

I've tested gemini-1.5-flash-latest and got the same KeyError. I'am using ff237f2 without any modifications.

I've also the same issue with gemini. The problem occurs as soon as the parameter stream is set to false in the POST request.

POST http://<IP-Address>/api/v1/chats/<chat-id>/completions
Authorization: Bearer ragflow-<token>
Content-Type: application/json

{  
"question": "<question>",  
"stream": false,  
"session_id": "<session-id>"
}

I am using the current last commit fa76974

The error output is the following:

2025-02-26 13:50:51 2025-02-26 20:50:51,644 ERROR    23 'content'
2025-02-26 13:50:51 Traceback (most recent call last):
2025-02-26 13:50:51   File "/ragflow/.venv/lib/python3.10/site-packages/flask/app.py", line 880, in full_dispatch_request
2025-02-26 13:50:51     rv = self.dispatch_request()
2025-02-26 13:50:51   File "/ragflow/.venv/lib/python3.10/site-packages/flask/app.py", line 865, in dispatch_request
2025-02-26 13:50:51     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
2025-02-26 13:50:51   File "/ragflow/api/utils/api_utils.py", line 303, in decorated_function
2025-02-26 13:50:51     return func(*args, **kwargs)
2025-02-26 13:50:51   File "/ragflow/api/apps/sdk/session.py", line 156, in chat_completion
2025-02-26 13:50:51     for ans in rag_completion(tenant_id, chat_id, **req):
2025-02-26 13:50:51   File "/ragflow/api/db/services/conversation_service.py", line 154, in completion
2025-02-26 13:50:51     for ans in chat(dia, msg, False, **kwargs):
2025-02-26 13:50:51   File "/ragflow/api/db/services/dialog_service.py", line 376, in chat
2025-02-26 13:50:51     msg[-1]["content"], answer))
2025-02-26 13:50:51 KeyError: 'content'
 

@Peterson047 Please, reopen the issue! It is not fixed yet.

[Peterson047]

@Peterson047 Please, reopen the issue! It is not fixed yet.

Did you perform a clean installation? It’s possible that you’re still using the old deploy. The error shown is on the same line as before, line 376, but the code from the branch you mentioned is now on line 314.

[raikloe]

Did you perform a clean installation?

Not in the comment above, there it was a pull of the current status.

But I have now tried a fresh installation with the same error.

[raikloe]

But I have now tried a fresh installation with the same error.

Okay, maybe it is up to me. I tried a fresh installation with the last docker image of ragflow but I guess the docker image has not the last changes in it? I will build a docker image of myself and will let you know my results.

[raikloe]

@Peterson047 Okay it was my mistake. I used the wrong process to upgrade ragflow to the latest (nightly) version. It is working now. Thanks for everything. You can close this issue it is fixed