Using PyMuPDF with LLM and RAG Technologies

[JorjMcKie]

In the rapidly evolving field of Natural Language Processing (NLP), Retrieval-Augmented Generation (RAG) combines the best of two worlds: the ability to retrieve relevant information from large data stores and the creative capacity of generative models to formulate responses. This approach enables chatbots and AI systems to provide more accurate, informative, and contextually relevant answers.

PyMuPDF is known for its efficiency and ease of use in extracting text and other elements from PDF documents. This makes it an excellent choice for the initial step in building a RAG chatbot: retrieving information. By using PyMuPDF, you can quickly access a vast array of knowledge stored in PDFs, which your chatbot can then use to generate informed and relevant responses.

The good news is that PyMuPDF already has all batteries included to be immediately usable in this environment. To underpin this statement we have created a new repository which contains scripts and recipes to create such chatbots.

Our goal here is to provide something that can be set up and used within minutes and deliver really amazing results.

The first examples are just (2024-03-22) dropping in, so you can build your first own chatbot within minutes now.

That said, we are aware that not all pieces of PyMuPDF's rich feature set are as easily located and employed as they could be ... and should be.

One example is the extraction (and output) of text in one of the most popular formats for feeding LLMs: the markdown text format.

We have started to actively support this and implemented a new Table.to_markdown() method (version 1.24.0), which is usable without first converting the table to a pandas DataFrame and also is optimized to generate the minimum possible token size. This is an important aspect for optimal use in this environment.

The next step will probably be a new text extraction variant Page.get_text("markdown", ...) to simplify non-table page content in this format.

Medium-term, our overall goal here is to output complete pages of relevant documents as markdown text, having converted standard text and tables in one common chunk of data.