Testset(samples=[])

[txygjx]

I executed the following code, but the generated dataset is empty. What could be the issue? My Ragas version is 0.2.14.

code{from ragas.testset import TestsetGenerator

generator = TestsetGenerator(llm=generator_llm, embedding_model=generator_embeddings)
dataset = generator.generate_with_langchain_docs(docs, testset_size=10)
}

[sahusiddharth]

Hi @txygjx,

The issue you're facing might be due to missing or incorrect transformations when Ragas creates the knowledge graph. Here's a simple overview of how Ragas generates test sets:

First, Ragas builds a knowledge graph from your documents, consisting of nodes and relationships. From this graph, Ragas generates test questions.

There are two main types of questions:

• Single-hop questions: Simple questions based directly on individual nodes.
• Multi-hop questions: More complex questions created by connecting multiple nodes through relationships.

If your dataset is empty, the knowledge graph might lack the necessary properties or relationships to generate questions.

We are preparing a detailed tutorial on this topic, which will soon be available here:

• https://ragas--1959.org.readthedocs.build/en/1959/howtos/applications/singlehop_testset_gen/ (currently not live, coming soon)
• Single-hop query testset generation tutorial #1959

I hope this helps!