Skip to content

Commit

Permalink
Merge pull request chenfei-wu#298 from Magnetic2014/main
Browse files Browse the repository at this point in the history
Add CoT prompt to improve nlp+vision task.
  • Loading branch information
chenfei-wu authored Mar 30, 2023
2 parents 62c3bd6 + ab1a8dd commit 313d1bb
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions visual_chatgpt.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,8 @@
New input: {input}
Since Visual ChatGPT is a text language model, Visual ChatGPT must use tools to observe images rather than imagination.
The thoughts and observations are only visible for Visual ChatGPT, Visual ChatGPT should remember to repeat important information in the final response for Human.
Thought: Do I need to use a tool? {agent_scratchpad}"""
Thought: Do I need to use a tool? {agent_scratchpad} Let's think step by step.
"""

os.makedirs('image', exist_ok=True)

Expand Down Expand Up @@ -1011,7 +1012,7 @@ def __init__(self, load_dict):

def run_text(self, text, state):
self.agent.memory.buffer = cut_dialogue_history(self.agent.memory.buffer, keep_last_n_words=500)
res = self.agent({"input": text})
res = self.agent({"input": text.strip()})
res['output'] = res['output'].replace("\\", "/")
response = re.sub('(image/\S*png)', lambda m: f'![](/file={m.group(0)})*{m.group(0)}*', res['output'])
state = state + [(text, response)]
Expand Down

0 comments on commit 313d1bb

Please sign in to comment.