Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[2310.02170] Dynamic LLM-Agent Network: An LLM-agent Collaboration Framework with Agent Team Optimization #903

Open
1 task
ShellLM opened this issue Aug 20, 2024 · 1 comment
Labels
AI-Agents Autonomous AI agents using LLMs AI-Chatbots Topics related to advanced chatbot platforms integrating multiple AI models Algorithms Sorting, Learning or Classifying. All algorithms go here. code-generation code generation models and tools like copilot and aider human-verified <INST>NEVER PICK THIS LABEL</INST> llm Large Language Models llm-applications Topics related to practical applications of Large Language Models in various fields llm-experiments experiments with large language models MachineLearning ML Models, Training and Inference Papers Research papers Research personal research notes for a topic

Comments

@ShellLM
Copy link
Collaborator

ShellLM commented Aug 20, 2024

[2310.02170] Dynamic LLM-Agent Network: An LLM-agent Collaboration Framework with Agent Team Optimization

Preprint, under review. 21 pages

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA)

Cite as: arXiv:2310.02170 [cs.CL]

"Large language model (LLM) agents have been shown effective on a wide range of tasks, and by ensembling multiple LLM agents, their performances could be further improved. Existing approaches employ a fixed set of agents to interact with each other in a static architecture, which limits their generalizability to various tasks and requires strong human prior in designing these agents. In this work, we propose to construct a strategic team of agents communicating in a dynamic interaction architecture based on the task query. Specifically, we build a framework named Dynamic LLM-Agent Network (DyLAN) for LLM-agent collaboration on complicated tasks like reasoning and code generation. DyLAN enables agents to interact for multiple rounds in a dynamic architecture with inference-time agent selection and an early-stopping mechanism to improve performance and efficiency. We further design an automatic agent team optimization algorithm based on an unsupervised metric termed Agent Importance Score, enabling the selection of best agents based on the contribution each agent makes. Empirically, we demonstrate that DyLAN performs well in both reasoning and code generation tasks with reasonable computational cost. DyLAN achieves 13.0% and 13.3% improvement on MATH and HumanEval, respectively, compared to a single execution on GPT-35-turbo. On specific subjects of MMLU, agent team optimization in DyLAN increases accuracy by up to 25.0%."

Suggested labels

None

@ShellLM ShellLM added AI-Agents Autonomous AI agents using LLMs AI-Chatbots Topics related to advanced chatbot platforms integrating multiple AI models Algorithms Sorting, Learning or Classifying. All algorithms go here. code-generation code generation models and tools like copilot and aider llm Large Language Models llm-applications Topics related to practical applications of Large Language Models in various fields llm-experiments experiments with large language models MachineLearning ML Models, Training and Inference Papers Research papers Research personal research notes for a topic labels Aug 20, 2024
@ShellLM
Copy link
Collaborator Author

ShellLM commented Aug 20, 2024

Related content

#681 similarity score: 0.87
#847 similarity score: 0.86
#812 similarity score: 0.86
#130 similarity score: 0.84
#895 similarity score: 0.84
#896 similarity score: 0.83

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
AI-Agents Autonomous AI agents using LLMs AI-Chatbots Topics related to advanced chatbot platforms integrating multiple AI models Algorithms Sorting, Learning or Classifying. All algorithms go here. code-generation code generation models and tools like copilot and aider human-verified <INST>NEVER PICK THIS LABEL</INST> llm Large Language Models llm-applications Topics related to practical applications of Large Language Models in various fields llm-experiments experiments with large language models MachineLearning ML Models, Training and Inference Papers Research papers Research personal research notes for a topic
Projects
None yet
Development

No branches or pull requests

2 participants