need check information in all documents

[be-swarm]

Hi,
I import several documents like this:
await memory.ImportDocumentAsync("D:\Temp\testia\devis-bad.pdf", tags: new() { { "document", "devis" } });
await memory.ImportDocumentAsync("D:\Temp\testia\audit.pdf", tags: new() { { "document", "audit" } });
await memory.ImportDocumentAsync("D:\Temp\testia\mar.pdf", tags: new() { { "document", "mar" } });
await memory.ImportDocumentAsync("D:\Temp\testia\mandat.pdf", tags: new() { { "document", "mandat" } });
docid=await memory.ImportDocumentAsync("D:\Temp\testia\cni.jpg", tags: new() { { "document", "cni" } });

all documents have same address except one (devis-bad)
i want to tell to MK if all documents havea the same address.
i use this prompt:
var answer = await memory.AskAsync("tell me if the address is the same in all documents");

I obtain a wrong response . MK said yes and Sources are:

• Sources:
  • mandat.pdf [mardi 18 février 2025]
  • audit.pdf [mardi 18 février 2025]

Question: Why response said yes and why only 2 documents have been taken in analyse?
what am I doing wrong?
Thanks

[dluc]

Hi @be-swarm - RAG solutions by default answer questions using the content of the files. Answers are generated looking for content that is similar to the question.

The question here is: "tell me if the address is the same in all documents". If the files contain text that is similar, that content will be used to generate an answer. However, the question is asking to analyze all files, so I would suggest a different approach using COT:

• "tell me if the address is the same in all documents"
• the user is asking to analyze all documents and see if they have the same address
• action: load all documents
• action: extract address from every document
• action: compare address
• action: use the result to reply to user

The above is not in scope of RAG, but you can reproduce that with Agents and Tools/Planners.

[be-swarm]

@dluc thanks for your response.
If I understand correctly, I should rather use semantic kernel with plugins and functions to fetch the information in order to feed the prompt?
Ex: a function that semantic kernel would call to look for information in a document so that it can be used to control it?