[Issue]: About retrieve_content and code execution in call_rag_chat

[szwang-seedtech]

Describe the issue

I was following the agentchat_groupchat_RAG.ipynb https://github.com/microsoft/autogen/blob/main/notebook/agentchat_groupchat_RAG.ipynb

When I was using call_rag_chat, sometimes the agent will refuse to execute the code and terminate the conversation, but sometimes works fine. If I remove the retrieve_content part in call_rag_chat, then all the code will be executed by agents without any problem.

1. If I only listed user in register_for_execution, I will have the execution code problem occationally
2. if I listed user and other assistant agents in register_for_execution, I will have the error "openai.BadRequestError: Error code: 400 - {'error': {'message': "None is not of type 'object' - 'messages.2.function_call'", 'type': 'invalid_request_error', 'param': None, 'code': None}}" occationally

Does anyone have any clue, or is it a bug?

Steps to reproduce

No response

Screenshots and logs

This is what I mean by "refuse to execute the code and terminate the conversation"

Additional Information

No response

[gagb]

@thinkall

[thinkall]

Hi @szwang-seedtech , could you share the whole notebook of your version so that I can check the outputs and reproduce the issue? Thanks.

[szwang-seedtech]

@thinkall hello, attached is the code I am using. They were main.py and setting.json. I converted them to txt.

SETTING.txt

[thinkall]

Thanks for providing the info, but the code can't run.

with open("./settings/SEED_GPT_SETTING.json") as f:
    seed_gpt_setting = json.load(f)

with open("./settings/SEED_CONFIG_LIST.json") as f:
    seed_config_list = json.load(f)

These two files are needed.

For the code execution issue, do you mean running the code generated by LLM, or call the rag function?
For the #2, I think it's function call error.

[szwang-seedtech]

@thinkall SEED_GPT_SETTING is the SETTING file I attached. SEED_CONFIG_LIST is just a file with config_list, so that main.py can load models from there.

And yes, the code execution issue is the code generated by LLM.

Do you have any idea how I can fix the function call error? I noticed the same error in issue 1440, does it get fix in the newly released 0.2.15?

[szwang-seedtech]

For more details on "code execution issue", when I ask agents to write a code, for example "write a bubble sorting sample code", that is unrelevant to the doc I provided to the rag, agents will be busy calling retrieve_content, and only send retrieve_content with empty messages to each other and then terminate the conversation. Looks like in a retrieve_content loop.

[thinkall]

For more details on "code execution issue", when I ask agents to write a code, for example "write a bubble sorting sample code", that is unrelevant to the doc I provided to the rag, agents will be busy calling retrieve_content, and only send retrieve_content with empty messages to each other and then terminate the conversation. Looks like in a retrieve_content loop.

In your code, you're using speaker_selection_method="auto", I think this is the root cause. The speaker selection result from the LLM doesn't work well in your case. I suggest you try round_robin.

[thinkall]

@thinkall SEED_GPT_SETTING is the SETTING file I attached. SEED_CONFIG_LIST is just a file with config_list, so that main.py can load models from there.

And yes, the code execution issue is the code generated by LLM.

Do you have any idea how I can fix the function call error? I noticed the same error in issue 1440, does it get fix in the newly released 0.2.15?

The fix for 1440 is just updating the usage in the notebook. Your code is already using the latest APIs for function calling.

[szwang-seedtech]

Thank you! Your replies give me some inspiration, and I am testing it.

[szwang-seedtech]

@thinkall I added a specifc agent as the retrieve content executor, put it the first on the list and used the round_robin selection. No empty conversation between agents any more! Many thanks!