Autogen agentchat: writing long documents - Any ideas?

[SlistInc]

Hi all,

I really like the agentchat. Mainly because it works well with local models and it seems to produce quite good results (especially when comparing it to other agent solutions).

I wanted to ask you for crazy ideas: my goal is to come up with a setup able to write longer papers / documents on a specific topic. Of course, token size is always the limiting factor when using an LLM. You can't have your setup go over the token limit so whatever the final output is... it will be limited. No way to get a book out of one single prompt.

I wanted to explore some agentchat powered solutions that would allow a flexible output length and came to a few ideas:

1. Work with a "structurer" agent that writes a high level structure, another agent distributing the work over various "writers" who then focus on a part of the structure and "write" it. The downside may be that the maximum output length is merely the total token length x the number of writers.
2. Work with a "scribe" agent that has functions to append, summarise, read, write text files and interacts with a writer agent who constantly improves / appends, re-reads the output document.
   3)...

Maybe some of you smarter than me have a good idea or even some experience on whether there is a winning approach? Otherwise I just play around with them but generally I would love some good or quirky ideas from the community.

[sonichi]

Very interesting question. I would like to think of a generic approach.

First, I'll assume the long doc can be represented as a tree structure which can be mapped to file systems:

doc.txt
----doc.1.txt
--------doc.1.1.txt
--------doc.1.2.txt
----doc.2.txt
--------doc.2.1.txt
------------doc.2.1.1.txt
.......

The final document is obtained by concatenating all the leaf txt files. The non-leaf txt files are intermediate summaries used to generate next level of summary or content.

Second, I'll assume an assistant agent to look at the current state and decide what to do next. For example, when the tree contains a single doc.txt, the next step would be to expand doc.txt into doc.1.txt to doc.n.txt. A further step could be checking the consistencies, etc. We can formalize each possible step as a python function. Inside the function, we can create more agents to do more specific jobs. We can recursively invoke the designed agents to write each part.

Third, I'll assume a user proxy agent to execute the python function suggested by the assistant agent and return proper message for the assistant agent to continue the work. We can enable human intervention via this user proxy agent in case the plan is wrong.

[jeandtx]

And do you know how we can create an agent that has the ability to write files without function calling ? And in any format json included ?

[sonichi]

For example, decide a contract between the file writer agent and an LLM based agent. Then register a specific reply function in the file writer agent to parse the instructions from the LLM agent, and customize the system message of the LLM-based agent.

[SlistInc]

Thanks for the feedback! I will try your suggested approach and report back if it works. I am using 13b models so let's see if it is even possible.

[swanidhi]

Perhaps, another agent reads through different permutations and combinations of these files to capture redundancy in the captured content and flags it for the writer agents to rewrite. The structurer agent could set the global tone of the write up so that different writers don't deviate a lot in their writing style.

[rohitnanda1443]

Let us say one has a standard document in the similar tree structure (say a contract). Majority of the document is standard and therefore the LLM can get the document flow and structure via RAG.

Doc 2: Is the salient features (Heads of Terms) which are list of changes required in this tree structure document. (This is essentially a small summary of Document 1)

Can an agent be configured to take Document 1 (Base Document) and edit it as per Document 2 to get the final output Document 3?

[Josephrp]

yes normally you simple parse document 2 into a single string as input

[Josephrp]

https://git.tonic-ai.com/positonic/memogen/memogen here is a cool / basic example i made last weekend, hoep you find it useful , it will make long form content, at least it will help you understand the concepts and a nice / easy way to start. i'm also interested in pursuing this with different document types and lengths.

currently at 4000 tokens it will produce approximately 140+pages , you can reduce the size with smaller token sizes. if you're a developper we could work together , i would find that fun + instructive. :-)

[ghost]

These are hackers hacking me please contact fbi

…

On Wed, Jun 12, 2024, 8:48 PM rohitnanda1443 ***@***.***> wrote: Let us say one has a standard document in the similar tree structure (say a contract). Majority of the document is standard and therefore the LLM can get the document flow and structure via RAG. Doc 2: Is the salient features (Heads of Terms) which are list of changes required in this tree structure document. (This is essentially a small summary of Document 1) Can an agent be configured to take Document 1 (Base Document) and edit it as per Document 2 to get the final output Document 3? — Reply to this email directly, view it on GitHub <#67 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BI3MSTG2RC3A2L57GHUKP2DZHD27LAVCNFSM6AAAAAA5TB7OS2VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TONJXHEZDG> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>