Description
Description:
I am trying to generate test set with ragas and I am getting connection error when invoking generator.generate_with_langchain_docs with the generator object. I am using llm proxy for the llm.
Ragas version: 2.0
Code to Reproduce
#ignore un necessary imports below
pip install langchain-community
import os
!pip install ragas
from ragas.llms import LangchainLLMWrapper
from ragas.embeddings import LangchainEmbeddingsWrapper
from langchain_openai import ChatOpenAI
from langchain_openai import OpenAIEmbeddings
from ragas.testset import TestsetGenerator
from langchain_community.document_loaders import DirectoryLoader
!pip install unstructured[local-inference]
pip install -qU langchain-huggingface
from langchain_community.embeddings import HuggingFaceEmbeddings
from sentence_transformers import SentenceTransformer
from sentence_transformers.util import cos_sim
from sentence_transformers.quantization import quantise_embeddings
from langchain_community.embeddings import SentenceTransformerEmbeddings
from ragas.testset import TestsetGenerator
from langchain.text_splitter import TokenTextSplitter
path = "/content"
loader = DirectoryLoader(path, glob="**/*.txt")
docs = loader.load()
embeddings = HuggingFaceEmbeddings(model_name="mixedbread-ai/mxbai-embed-large-v1")
generator_llm_candy = ChatOpenAI(model_name = "gpt-4o",
openai_api_base = "",
openai_api_key = "", # DO NOT HARDCODE YOUR KEY
model_kwargs = {'user': "" }) # DO NOT HARDCODE A USER HERE
generator_llm = LangchainLLMWrapper(generator_llm_candy)
generator_embeddings = LangchainEmbeddingsWrapper(embeddings)
generator = TestsetGenerator(llm=generator_llm, embedding_model=generator_embeddings)
dataset = generator.generate_with_langchain_docs(docs, testset_size=10)
data_frame = dataset.to_pandas()
data_frame.to_csv('file1.csv')
Error trace
ERROR:ragas.testset.transforms.engine:unable to apply transformation: Connection error.
ERROR:ragas.testset.transforms.engine:unable to apply transformation: 'headlines' property not found in this node
ERROR:ragas.testset.transforms.engine:unable to apply transformation: Connection error.
ERROR:ragas.testset.transforms.engine:unable to apply transformation: node.property('summary') must be a string, found '<class 'NoneType'>'
ERROR:ragas.testset.transforms.engine:unable to apply transformation: Node c11e1996-b069-4197-a7dd-ba85818f1641 has no summary_embedding