fix json parser #697

Wladastic · 2023-04-10T13:38:28Z

Background

Using gpt-3.5 the responses contain some nonsense text around the json.
This is a messy fix, don't merge before it's clean.
But you can use these changes rn to get around the issue.

Changes

added another json clean function that finds the first and last bracket and also notifies the user via speech about the issue, if it happened, as this one is important to observe and I do not want it to be a final fix.

Documentation

Test Plan

PR Quality Checklist

[X ] My pull request is atomic and focuses on a single change.
I have thouroughly tested my changes with multiple different prompts.
I have considered potential risks and mitigations for my changes.
I have documented my changes clearly and comprehensively.
I have not snuck in any "extra" small tweaks changes

onekum

Rough, but it looks functional. Also, you should functionally mark this pull request as a draft instead of simply having "draft" as a namesake. The button is in the right panel under reviewers.

scripts/main.py

Wladastic · 2023-04-10T15:22:21Z

I will try to refactor this a little further into the already existing json cleaner method.
Some users reported that they receive the same error elsewhere or the try try try is too much.

nponeccop · 2023-04-10T15:37:15Z

If it's not ready yet you can mark it as a draft PR using GitHub

Artemonim · 2023-04-10T17:11:01Z

After I download your file and replace original main.py I started getting the following error over and over again:

Traceback (most recent call last):
  File "Auto-GPT/scripts/main.py", line 329, in <module>
    assistant_reply = chat.chat_with_ai(
TypeError: chat_with_ai() takes 5 positional arguments but 6 were given

I only wrote four goals. If I use exactly the same goals with the actual git clone https://github.com/Torantulino/Auto-GPT.git file, there is no error (except the JSON Error)

Wladastic · 2023-04-10T17:37:51Z

try git pull on the master branch, the issue comes from chat.py, not my code.

Wladastic · 2023-04-10T17:39:05Z

cfg.debug is one param too much for the chat method. not sure who changed it, as my git blame is broken in vscode

Wladastic · 2023-04-10T17:55:20Z

Removing the branch made it rocksolid now.
It is creating a whole new project now and using elevenlabs.
I told it to always confirm with the other agent and I hear them both discussing right now haha

An array from google seemed to have crashed it now.

Wladastic · 2023-04-10T19:48:54Z

@Artemonim try again. I think I resolved most issues now.

Artemonim · 2023-04-10T20:27:43Z

@Artemonim try again. I think I resolved most issues now.

Is it a correct response?
Auto-GPT just write \n, not to use it

PLAN:
-  Create a 'Math' directory.\n- Create a 'workbook.markdown' file inside 'Math' directory.

Artemonim · 2023-04-10T20:32:13Z

well (input in Russian but I assume the same problem in English)

Enter 'y' to authorise command, 'y -N' to run N continuous commands, 'n' to exit program, or enter feedback for Математический решатель с автогенератором...        
Input:пожалуйста, повтори свой запрос
SYSTEM:  Human feedback: пожалуйста, повтори свой запрос
Error: Invalid JSON in assistant thoughts
 Хорошо, для того чтобы создать рабочую папку и файл, который будем использовать для сохранения математических задач и их решений, используется следующий JSON запрос: ``` { "thoughts": { "text": "Я должен создать папку 'Math' и файл 'workbook.markdown'. Это позволит мне добавлять и хранить математические задачи и их решения. Я собираюсь сохранять формулы в формате ASCII-математика.", "reasoning": "Создание папки и файла позволит мне логически организовать мои файлы и сохранять информацию в одном месте. Это позволит мне быстро находить нужные файлы, а также легко делиться ими с другими.", "plan": "- Создать папку 'Math'. \\n- В этой папке создать файл 'workbook.markdown'.", "criticism": "Я должен был выполнить это раньше.", "speak": "Я создам папку 'Math' и файл 'workbook.markdown' внутри нее." }, "command": { "name": "create_directory", "args": { "directory": "Math" } } } ```
Attempting to fix JSON by finding outermost brackets
 
Error: 
 Traceback (most recent call last): File "Auto-GPT/scripts/main.py", line 95, in print_assistant_thoughts assistant_reply_json = fix_and_parse_json(assistant_reply) File "D:\YandexDisk\Work\Auto-ChatGPT\Auto-GPT\scripts\json_parser.py", line 38, in fix_and_parse_json json_str = correct_json(json_str) File "D:\YandexDisk\Work\Auto-ChatGPT\Auto-GPT\scripts\json_utils.py", line 125, in correct_json if balanced_str := balance_braces(json_str): File "D:\YandexDisk\Work\Auto-ChatGPT\Auto-GPT\s
cripts\json_utils.py", line 79, in balance_braces raise e File "D:\YandexDisk\Work\Auto-ChatGPT\Auto-GPT\scripts\json_utils.py", line 76, in balance_braces json.loa
ds(json_string) File "C:\ProgramData\miniconda3\envs\py3.8\lib\json\__init__.py", line 357, in loads return _default_decoder.decode(s) File "C:\ProgramData\minicond
a3\envs\py3.8\lib\json\decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\ProgramData\miniconda3\envs\py3.8\lib\json\decode
r.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char
 0) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "Auto-GPT/scripts/main.py", line 98, in print_assist
ant_thoughts assistant_reply_json = attempt_to_fix_json_by_finding_outermost_brackets(assistant_reply) File "Auto-GPT/scripts/main.py", line 68, in attempt_to_fix_j
son_by_finding_outermost_brackets json_pattern = re.compile(r"\{(?:[^{}]|(?R))*\}") File "C:\ProgramData\miniconda3\envs\py3.8\lib\re.py", line 252, in compile retu
rn _compile(pattern, flags) File "C:\ProgramData\miniconda3\envs\py3.8\lib\re.py", line 304, in _compile p = sre_compile.compile(pattern, flags) File "C:\ProgramDat
a\miniconda3\envs\py3.8\lib\sre_compile.py", line 764, in compile p = sre_parse.parse(p, flags) File "C:\ProgramData\miniconda3\envs\py3.8\lib\sre_parse.py", line 9
48, in parse p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0) File "C:\ProgramData\miniconda3\envs\py3.8\lib\sre_parse.py", line 443, in _parse_sub itemsa
ppend(_parse(source, state, verbose, nested + 1, File "C:\ProgramData\miniconda3\envs\py3.8\lib\sre_parse.py", line 834, in _parse p = _parse_sub(source, state, sub
_verbose, nested + 1) File "C:\ProgramData\miniconda3\envs\py3.8\lib\sre_parse.py", line 443, in _parse_sub itemsappend(_parse(source, state, verbose, nested + 1, F
ile "C:\ProgramData\miniconda3\envs\py3.8\lib\sre_parse.py", line 823, in _parse raise source.error("unknown extension ?" + char, re.error: unknown extension ?R at position 12

After that the program did not stop and asks for confirmation

nponeccop

Looks good with regards to whitespace and unrelated changes

Wladastic · 2023-04-10T20:54:19Z

@Artemonim The printed error is intentional.
could you check what the json looks like?
+? inside a json makes no sense.
It has nothing to do with that code.

Also don't forget to pull the latest branch of mine for testing.
if you keep testing on the main branch, then of course it won't work haha

Talking of the devil, I got the same error now, regex is off, one sec.

Artemonim · 2023-04-10T21:02:41Z

don't forget to pull the latest branch of mine for testing

I use Github Desktop + PyCharm 2022.1.3 with Powershell 7.2.10 as terminal

And I'm ready to be your tester this hour 🙂

Artemonim · 2023-04-11T07:49:25Z

I noticed that the "Criticism" is now empty if there is a JSON fix message

That happens not everytime when "Apparently json was fixed.". Hmmm...

Wladastic · 2023-04-11T07:52:00Z

Maybe this attemptive fix can be added into the fix_and_parse_json or something.
@Artemonim for that issue, I checked the init message to gpt, I think we should create a fallback method.

Something like "Summarize last 5-10 history entries" and feed it into a new gpt instance, as once it starts doing those broken json messages, it will continue to do so.

@Torantulino Any ideas? I have to work now, so will be away for half a day.
A friend of mine started writing a new project and managed to avoid this exact issue by letting it give him shell commands to echo every single line of the response. everytime a line didn't begin with echo, he just asks gpt again...

Artemonim · 2023-04-11T08:02:22Z

I've update master and switch back to #697 I noticed that the "Criticism" is now empty if there is a JSON fix message

I thought I was on present master + #697, but I was only on #697. Sorry for the mistake.

Artemonim · 2023-04-11T08:31:29Z

I've merge #697 into the present master and that happened after 14 minutes of work

SYSTEM: Human feedback: Ты запущен по адресу D:\YandexDisk\Work\Auto-ChatGPT. У тебя должна быть рабочая директория D:\YandexDisk\Work\Auto-ChatGPT\auto_gpt_workspace. У тебя должен быть к ней доступ. Попробуй найти папку "hWriter" чтобы проверить моё предположение.

translation:

SYSTEM: Human feedback: You are running at D:\YandexDisk\Work\Auto-ChatGPT. You must have a working directory D:\YandexDisk\Work\Auto-ChatGPT\auto_gpt_workspace. You should have access to it. Try to find the "hWriter" folder to check my assumption.

context: It was trying to COMMAND = write_to_file ARGUMENTS = {'file': '/path/to/LTM_file.docx' but for some reason SYSTEM: Command write_to_file returned: Error: Attempted to access outside of working directory.

log:

11:22:35,324 AutoGPT INFO Error: 
: Traceback (most recent call last):
  File "Auto-GPT/scripts/main.py", line 104, in print_assistant_thoughts
    assistant_reply_json = fix_and_parse_json(assistant_reply)
  File "D:\YandexDisk\Work\Auto-ChatGPT\Auto-GPT\scripts\json_parser.py", line 51, in fix_and_parse_json
    brace_index = json_str.index("{")
ValueError: substring not found

Wladastic · 2023-04-11T08:36:56Z

@Artemonim That is a different bug.
There is an issue for that already I think.

Artemonim · 2023-04-11T08:41:51Z

~~I think I've got the accents wrong. I wanted to point to error in log, and give everything else just for context. And this is still another bug for which I should find its own issue, right?~~ Yes, it's an another error T_T

If so, then it seems your current fixes are working right

Stunspot · 2023-04-11T11:15:16Z

I just told my bot "Note: GPT-3.5 agents frequently send extraneous text with JSON messages" and I have seen only one invalid json error in the past 30 minutes since. It had been more like 1 every 5 sec.

…-json-error

Wladastic · 2023-04-11T21:54:26Z

@Torantulino
Merged with current master and works.
Together with the other json pr that helps gpt-3.5 phrase the commands better, this will work better.

claytondukes · 2023-04-11T22:20:51Z

@Torantulino
Merged with current master and works.
Together with the other json pr that helps gpt-3.5 phrase the commands better, this will work better.

I just pulled and now getting:

Using memory of type: RedisMemory
Traceback (most recent call last):
  File "scripts/main.py", line 321, in <module>
    assistant_reply = chat.chat_with_ai(
                      ^^^^^^^^^^^^^^^^^^
  File "scripts/chat.py", line 67, in chat_with_ai
    if cfg.debug:
       ^^^^^^^^^
AttributeError: 'Config' object has no attribute 'debug'

claytondukes · 2023-04-11T22:25:14Z

@Torantulino
Merged with current master and works.
Together with the other json pr that helps gpt-3.5 phrase the commands better, this will work better.

I just pulled and now getting:
Using memory of type: RedisMemory
Traceback (most recent call last):
  File "scripts/main.py", line 321, in <module>
    assistant_reply = chat.chat_with_ai(
                      ^^^^^^^^^^^^^^^^^^
  File "scripts/chat.py", line 67, in chat_with_ai
    if cfg.debug:
       ^^^^^^^^^
AttributeError: 'Config' object has no attribute 'debug'

#858 fixes it

claytondukes · 2023-04-11T22:27:01Z

#858 fixes it

Spoke too soon.

Error:
 Traceback (most recent call last): File "scripts/json_parser.py", line 55, in fix_and_parse_json return json.loads(json_str) ^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.11/3.11.2/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.11/3.11.2/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.11/3.11.2/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/decoder.py", line 353, in raw_decode obj, end = self.scan_once(s, idx) ^^^^^^^^^^^^^^^^^^^^^^ json.decoder.JSONDecodeError: Invalid \escape: line 3 column 87 (char 106) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scripts/main.py", line 76, in print_assistant_thoughts assistant_reply_json = fix_and_parse_json(assistant_reply) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "scripts/json_parser.py", line 63, in fix_and_parse_json ai_fixed_json = fix_json(json_str, JSON_SCHEMA) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "scripts/json_parser.py", line 94, in fix_json if cfg.debug: ^^^^^^^^^ AttributeError: 'Config' object has no attribute 'debug

Wladastic · 2023-04-12T08:57:28Z

#858 fixes it

Spoke too soon.

Error:
 Traceback (most recent call last): File "scripts/json_parser.py", line 55, in fix_and_parse_json return json.loads(json_str) ^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.11/3.11.2/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.11/3.11.2/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.11/3.11.2/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/decoder.py", line 353, in raw_decode obj, end = self.scan_once(s, idx) ^^^^^^^^^^^^^^^^^^^^^^ json.decoder.JSONDecodeError: Invalid \escape: line 3 column 87 (char 106) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scripts/main.py", line 76, in print_assistant_thoughts assistant_reply_json = fix_and_parse_json(assistant_reply) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "scripts/json_parser.py", line 63, in fix_and_parse_json ai_fixed_json = fix_json(json_str, JSON_SCHEMA) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "scripts/json_parser.py", line 94, in fix_json if cfg.debug: ^^^^^^^^^ AttributeError: 'Config' object has no attribute 'debug

I added my fixes.
Also set the history to -9 instead of -5 and it works MUUUCH better now.

…-json-error

Wladastic · 2023-04-12T15:20:09Z

Is this planned to be merged soon?
I already created new features and plugins and would like to create a pullrequest without having to switch back and forth 100 times haha

nponeccop · 2023-04-12T15:44:15Z

Yes, this PR is one of the few remaining PRs in the Batch 2, we were going to merge it today, but suffered from the delegation process. Now there are 2 guys with write permissions, but they are getting used to the flow. And @Torantulino probably needs some time to finally sleep. Give them some time to adapt. We may still break it before the merge though, but there are like 3 PRs left in that old batch, and you are one of them.

Wladastic · 2023-04-12T18:02:15Z

Okay, seems great :)
I have continued working on my fork so far, I added a voice prompt and telegram bot :D

momotarogrp · 2023-05-04T02:48:26Z

This bug fix is also needed for summary_memory.py
However, adding it here would require copying the exact same code as the attract_to_fix_json_by_finding_outermost_brackets function, which is not smart.

Auto-GPT\autogpt\memory_management\summary_memory.py
content_dict = json.loads(event["content"]) # JSONDecodeError

It is also problematic that a JSONDecodeError causes the process to terminate immediately

ntindle · 2023-05-04T03:01:58Z

This bug fix is also needed for summary_memory.py However, adding it here would require copying the exact same code as the attract_to_fix_json_by_finding_outermost_brackets function, which is not smart.

Auto-GPT\autogpt\memory_management\summary_memory.py content_dict = json.loads(event["content"]) # JSONDecodeError

It is also problematic that a JSONDecodeError causes the process to terminate immediately

Open a PR plz

…/github_actions/codecov/codecov-action-3 Bump codecov/codecov-action from 1 to 3

…eedback-json-error fix json parser

fix json parser - messy

66b36e9

onekum reviewed Apr 10, 2023

View reviewed changes

scripts/main.py Outdated Show resolved Hide resolved

Wladastic mentioned this pull request Apr 10, 2023

I always get : 'Error : INVALID JSON' #701

Closed

1 task

alexonab reviewed Apr 10, 2023

View reviewed changes

scripts/main.py Outdated Show resolved Hide resolved

nponeccop previously approved these changes Apr 10, 2023

View reviewed changes

fix line 328 error

bb36634

Wladastic dismissed nponeccop’s stale review via bb36634 April 10, 2023 17:40

Merge branch 'master' into fix-user-feedback-json-error

1f9d66b

Wladastic added 2 commits April 10, 2023 21:30

attempt fix

33d1fd3

add log when json was fixed.

ca4afca

Wladastic changed the title ~~Draft: fix json parser - messy~~ fix json parser Apr 10, 2023

Wladastic requested review from nponeccop, onekum and alexonab April 10, 2023 19:52

Wladastic mentioned this pull request Apr 10, 2023

Fixes Error : INVALID JSON in many cases #718

Closed

5 tasks

nponeccop previously approved these changes Apr 10, 2023

View reviewed changes

nponeccop mentioned this pull request Apr 10, 2023

PRs batch 2 #673

Closed

1 task

use regex instead of re

4ad1fc1

Wladastic dismissed nponeccop’s stale review via 4ad1fc1 April 10, 2023 20:59

nponeccop approved these changes Apr 11, 2023

View reviewed changes

Merge remote-tracking branch 'upstream/master' into fix-user-feedback…

edf364e

…-json-error

nponeccop previously approved these changes Apr 11, 2023

View reviewed changes

add my fixes

4063483

Wladastic dismissed nponeccop’s stale review via 4063483 April 12, 2023 08:56

Merge remote-tracking branch 'upstream/master' into fix-user-feedback…

c6605b0

…-json-error

nponeccop approved these changes Apr 12, 2023

View reviewed changes

Merge remote-tracking branch 'upstream/master' into fix-user-feedback…

0105dd3

…-json-error

nponeccop approved these changes Apr 12, 2023

View reviewed changes

richbeales approved these changes Apr 12, 2023

View reviewed changes

richbeales merged commit 0e004f5 into Significant-Gravitas:master Apr 12, 2023

SquareandCompass pushed a commit to SquareandCompass/Auto-GPT that referenced this pull request Oct 21, 2023

Merge pull request Significant-Gravitas#697 from microsoft/dependabot…

d3f83d8

…/github_actions/codecov/codecov-action-3 Bump codecov/codecov-action from 1 to 3

sindlinger pushed a commit to Orgsindlinger/Auto-GPT-WebUI that referenced this pull request Sep 25, 2024

Merge pull request Significant-Gravitas#697 from Wladastic/fix-user-f…

f97abca

…eedback-json-error fix json parser

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix json parser #697

fix json parser #697

Wladastic commented Apr 10, 2023 •

edited

Loading

onekum left a comment

Wladastic commented Apr 10, 2023

nponeccop commented Apr 10, 2023

Artemonim commented Apr 10, 2023

Wladastic commented Apr 10, 2023

Wladastic commented Apr 10, 2023

Wladastic commented Apr 10, 2023 •

edited

Loading

Wladastic commented Apr 10, 2023

Artemonim commented Apr 10, 2023

Artemonim commented Apr 10, 2023 •

edited

Loading

nponeccop left a comment

Wladastic commented Apr 10, 2023 •

edited

Loading

Artemonim commented Apr 10, 2023 •

edited

Loading

Artemonim commented Apr 11, 2023

Wladastic commented Apr 11, 2023 •

edited

Loading

Artemonim commented Apr 11, 2023 •

edited

Loading

Artemonim commented Apr 11, 2023

Wladastic commented Apr 11, 2023

Artemonim commented Apr 11, 2023 •

edited

Loading

Stunspot commented Apr 11, 2023

Wladastic commented Apr 11, 2023

claytondukes commented Apr 11, 2023 •

edited

Loading

claytondukes commented Apr 11, 2023

claytondukes commented Apr 11, 2023

Wladastic commented Apr 12, 2023

Wladastic commented Apr 12, 2023

nponeccop commented Apr 12, 2023

Wladastic commented Apr 12, 2023

momotarogrp commented May 4, 2023

ntindle commented May 4, 2023

fix json parser #697

fix json parser #697

Conversation

Wladastic commented Apr 10, 2023 • edited Loading

Background

Changes

Documentation

Test Plan

PR Quality Checklist

onekum left a comment

Choose a reason for hiding this comment

Wladastic commented Apr 10, 2023

nponeccop commented Apr 10, 2023

Artemonim commented Apr 10, 2023

Wladastic commented Apr 10, 2023

Wladastic commented Apr 10, 2023

Wladastic commented Apr 10, 2023 • edited Loading

Wladastic commented Apr 10, 2023

Artemonim commented Apr 10, 2023

Artemonim commented Apr 10, 2023 • edited Loading

nponeccop left a comment

Choose a reason for hiding this comment

Wladastic commented Apr 10, 2023 • edited Loading

Artemonim commented Apr 10, 2023 • edited Loading

Artemonim commented Apr 11, 2023

Wladastic commented Apr 11, 2023 • edited Loading

Artemonim commented Apr 11, 2023 • edited Loading

Artemonim commented Apr 11, 2023

I've merge #697 into the present master and that happened after 14 minutes of work

Wladastic commented Apr 11, 2023

Artemonim commented Apr 11, 2023 • edited Loading

Stunspot commented Apr 11, 2023

Wladastic commented Apr 11, 2023

claytondukes commented Apr 11, 2023 • edited Loading

claytondukes commented Apr 11, 2023

claytondukes commented Apr 11, 2023

Wladastic commented Apr 12, 2023

Wladastic commented Apr 12, 2023

nponeccop commented Apr 12, 2023

Wladastic commented Apr 12, 2023

momotarogrp commented May 4, 2023

ntindle commented May 4, 2023

Wladastic commented Apr 10, 2023 •

edited

Loading

Wladastic commented Apr 10, 2023 •

edited

Loading

Artemonim commented Apr 10, 2023 •

edited

Loading

Wladastic commented Apr 10, 2023 •

edited

Loading

Artemonim commented Apr 10, 2023 •

edited

Loading

Wladastic commented Apr 11, 2023 •

edited

Loading

Artemonim commented Apr 11, 2023 •

edited

Loading

Artemonim commented Apr 11, 2023 •

edited

Loading

claytondukes commented Apr 11, 2023 •

edited

Loading