Adding a Character Chat Agent #4411

treppers · 2023-05-09T17:30:58Z

Character Chat Agent

Fork of the conversational chat agent, but added in the concept of personality and fixed some issues I was getting with the output_parser. You can provide a {character_summary} that will give your bot that personality.

I was noticing some occasional weird responses with the conversation_chat agent so rewrote the entire prompt which seemed to have fixed some of them. Also noticed some errors with output_parser (but will submit that as a separate PR).

Changes with the prompt:

Moving all the TOOLS and FORMAT INSTRUCTIONS into the first System Message.
Keeping the actual Human Message to include the original input message.
Introduce a third voice called TOOL, that only speaks via the System Message
Created a More Instructions section instead of User Input to try to trigger the Final Answer
Also made sure to remove most of the you's and i's and replace it with nouns (Assistant, Human, TOOL)

Output_parser changes:

The agent sometimes doesn't want to respond with ````json` but instead will just say ````` which breaks the split function.
The agent sometimes doesn't doesn't want to follow the instructions and doesn't respond with any markdown (this is usually when the prompt contains enough information to do the completion). To fix this I put a try-except to catch the when JSON fails.
The agent sometimes runs out of token and doesn't close the markdown snippet. Created a case to handle this case.

Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:

@hwchase17

…rsonality while doing it

mbchang · 2023-05-09T21:23:25Z

langchain/agents/character_chat/prompt.py

+--------------------
+Given the entire TOOLS RESPONSE, 
+- If the USER'S ORIGINAL INPUT isn't answered using ONLY the information obtained by TOOL, try the same tool with a different input or another tool.
+- Otherwise, how would Assistant respond using information obtained from TOOL, Assistant must NOT mention the tool or tool names - the user has no context of any TOOL RESPONSES! 


It looks like this is a run-on sentence; could this be re-written? Perhaps something like:

Otherwise, to respond using information obtained from TOOL, Assistant must NOT mention the tool or tool names - the user has no context of any TOOL RESPONSES!

Good point. I think I had copied and pasted some of the conversational chat prompt in and it didn't make sense move on .

mbchang · 2023-05-09T21:28:12Z

langchain/agents/character_chat/__init__.py

@@ -0,0 +1 @@
+"""An agent designed to hold a conversation with a character in addition to using tools."""


Would it be more accurate to rephrase this as "An agent designed to hold a conversation as a given character in addition to using tools."

Definitely correct!

mbchang · 2023-05-09T21:28:21Z

langchain/agents/character_chat/base.py

@@ -0,0 +1,133 @@
+"""An agent designed to hold a conversation with a character in addition to using tools."""


Would it be more accurate to rephrase this as "An agent designed to hold a conversation as a given character in addition to using tools."

mbchang · 2023-05-09T21:28:29Z

langchain/agents/character_chat/base.py

+
+
+class CharacterChatAgent(Agent):
+    """An agent designed to hold a conversation in addition to using tools."""


Would it be more accurate to rephrase this as "An agent designed to hold a conversation as a given character in addition to using tools."

Same comment as above. Incorporating.

mbchang · 2023-05-09T21:37:18Z

langchain/agents/character_chat/prompt.py

+
+When responding to the TOOL, please output a response in one of two formats:
+
+**Option 1:**


**Option #1:** instead of **Option 1:**

Adding this into the prompt.

mbchang · 2023-05-09T21:39:24Z

langchain/agents/character_chat/prompt.py

+
+MORE INSTRUCTIONS
+--------------------
+Given the entire TOOLS RESPONSE, 


TOOLS -> TOOL

mbchang · 2023-05-09T21:40:30Z

langchain/agents/character_chat/prompt.py

+RESPONSE FORMAT INSTRUCTIONS
+----------------------------
+
+When responding to the TOOL, please output a response in one of two formats:


When responding to the TOOL -> When responding to TOOL

mbchang · 2023-05-09T22:03:24Z

Thanks @treppers for the PR! This kind of agent looks like it could make writing dialogue agents much more streamlined.

It looks like you have made a few changes to the system prompt. Would you mind sharing some concrete examples (as a jupyter notebook or code snippet) that illustrate the tangible differences between using the original prompt scheme vs your proposed prompt scheme? Such an ablation study would be helpful to know whether your proposed changes are specific to the CharacterChatAgent or whether they should be incorporated more broadly into other ConversationAgents as well. Specifically, for your following changes, do you have examples of what the response would look like with and without the change?
- Moving all the TOOLS and FORMAT INSTRUCTIONS into the first System Message.
- Keeping the actual Human Message to include the original input message.
- Introduce a third voice called TOOL, that only speaks via the System Message
- Created a More Instructions section instead of User Input to try to trigger the Final Answer
- Also made sure to remove most of the you's and i's and replace it with nouns (Assistant, Human, TOOL)
The CharacterChatAgent could be generalized to talk to not just the human but other characters. Could we replace "human" and "user" with "conversation partner" in the system prompt?

treppers · 2023-05-11T23:45:39Z

Trying to put together a notebook that will do a good compare and contrast.

That should be inbound.

I do think the Conversational Chat agent needs updating and so when I get the notebook up you can see the issues I found.

Definitely open to expanding the prompt to apply to conversation partner. Would just need to test it to make sure it doesn't think the TOOL voice is a conversation partner.

Updating this branch with a notebook to execute this agent along with the suggested tweaks. Also found a bug in the prompt that was causing the Option #1 response to include additional conversational responses in addition to the markdown.

…h Option 1 included non-markdown. added a notebook to execute this.

tonyabracadabra · 2023-05-13T14:39:35Z

langchain/agents/character_chat/base.py

+        format_instructions = character_message.format(
+            format_instructions=_output_parser.get_format_instructions()
+        )
+        final_prompt = format_instructions.format(


Suggested change

final_prompt = format_instructions.format(

system_prompt = format_instructions.format(

how about naming it to system_prompt as it is no more final?

tonyabracadabra · 2023-05-13T14:47:33Z

langchain/agents/character_chat/base.py

+        return thoughts
+
+    @classmethod
+    def from_llm_and_tools(


I saw this piece of code in several places, ever considered inheriting ConversationalChatAgent?

mbchang · 2023-05-18T00:47:33Z

Thanks for the update @treppers! Curious to see what you find in the compare-and-contrast.

efriis · 2023-11-07T04:38:23Z

Hey @mbchang ! This PR no longer lines up with the current directory structure of the library (would need to be in /libs/langchain/langchain instead of /langchain). Would you be interested in updating to the new format, or would it be better to close this and someone can work on a new PR off the current state of the library (using this as a reference implementation)?

efriis · 2023-11-07T19:52:16Z

I'll close it for now, and let me know if you'd like to reopen!

treppers added 2 commits May 9, 2023 10:17

adding a character chat agent that can use tools but also retain a pe…

dd411ad

…rsonality while doing it

change incorrectly labeled library

9f66a3e

mbchang reviewed May 9, 2023

View reviewed changes

minor tweaks to the prompt as per the PR comments. fixed an issue wit…

d746b9a

…h Option 1 included non-markdown. added a notebook to execute this.

This was referenced May 12, 2023

Option for ConversationalChatAgent to place format instruction and tools within System Message for ChatGPT 4.0 #4385

Closed

Broken intermediate output / parsing is grossly unreliable #3448

Closed

tonyabracadabra reviewed May 13, 2023

View reviewed changes

hwchase17 changed the base branch from master to harrison/character-chat-agent May 15, 2023 05:11

removed draft notebook

27a5556

dosubot bot added Ɑ: agent Related to agents module 🤖:enhancement A large net-new component, integration, or chain. Use sparingly. The largest features labels Jul 14, 2023

baskaryan assigned hinthornw and baskaryan Jul 14, 2023

leo-gan requested review from baskaryan and hwchase17 September 13, 2023 01:47

efriis closed this Nov 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding a Character Chat Agent #4411

Adding a Character Chat Agent #4411

treppers commented May 9, 2023 •

edited

Loading

mbchang May 9, 2023

treppers May 11, 2023

mbchang May 9, 2023

treppers May 11, 2023

mbchang May 9, 2023

mbchang May 9, 2023

treppers May 11, 2023

mbchang May 9, 2023

treppers May 11, 2023

mbchang May 9, 2023 •

edited

Loading

mbchang May 9, 2023

mbchang commented May 9, 2023

treppers commented May 11, 2023

tonyabracadabra May 13, 2023

tonyabracadabra May 13, 2023

mbchang commented May 18, 2023

efriis commented Nov 7, 2023

efriis commented Nov 7, 2023

		@@ -0,0 +1 @@
		"""An agent designed to hold a conversation with a character in addition to using tools."""

		@@ -0,0 +1,133 @@
		"""An agent designed to hold a conversation with a character in addition to using tools."""



		class CharacterChatAgent(Agent):
		"""An agent designed to hold a conversation in addition to using tools."""


		When responding to the TOOL, please output a response in one of two formats:

		Option 1:

	final_prompt = format_instructions.format(
	system_prompt = format_instructions.format(

Adding a Character Chat Agent #4411

Adding a Character Chat Agent #4411

Conversation

treppers commented May 9, 2023 • edited Loading

Character Chat Agent

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mbchang May 9, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mbchang commented May 9, 2023

treppers commented May 11, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mbchang commented May 18, 2023

efriis commented Nov 7, 2023

efriis commented Nov 7, 2023

treppers commented May 9, 2023 •

edited

Loading

mbchang May 9, 2023 •

edited

Loading