Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix notebook example #3142

Merged
merged 3 commits into from
Apr 19, 2023
Merged

Fix notebook example #3142

merged 3 commits into from
Apr 19, 2023

Conversation

qpleple
Copy link
Contributor

@qpleple qpleple commented Apr 19, 2023

The following calls were throwing an exception:

https://github.com/hwchase17/langchain/blob/575b717d108984676e25afd0910ccccfdaf9693d/docs/use_cases/evaluation/agent_vectordb_sota_pg.ipynb?short_path=4b3386c#L192

https://github.com/hwchase17/langchain/blob/575b717d108984676e25afd0910ccccfdaf9693d/docs/use_cases/evaluation/agent_vectordb_sota_pg.ipynb?short_path=4b3386c#L239

Exception:

---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[14], line 1
----> 1 chain_sota = RetrievalQA.from_chain_type(llm=OpenAI(temperature=0), chain_type="stuff", retriever=vectorstore_sota, input_key="question")

File ~/github/langchain/venv/lib/python3.9/site-packages/langchain/chains/retrieval_qa/base.py:89, in BaseRetrievalQA.from_chain_type(cls, llm, chain_type, chain_type_kwargs, **kwargs)
     85 _chain_type_kwargs = chain_type_kwargs or {}
     86 combine_documents_chain = load_qa_chain(
     87     llm, chain_type=chain_type, **_chain_type_kwargs
     88 )
---> 89 return cls(combine_documents_chain=combine_documents_chain, **kwargs)

File ~/github/langchain/venv/lib/python3.9/site-packages/pydantic/main.py:341, in pydantic.main.BaseModel.__init__()

ValidationError: 1 validation error for RetrievalQA
retriever
  instance of BaseRetriever expected (type=type_error.arbitrary_type; expected_arbitrary_type=BaseRetriever)

The vectorstores had to be converted to retrievers: vectorstore_sota.as_retriever() and vectorstore_pg.as_retriever().

The PR also:

  • adds the file paul_graham_essay.txt referenced by this notebook
  • adds to gitignore *.pkl and *.bin files that are generated by this notebook

Interestingly enough, the performance of the prediction greatly increased (new version of langchain or ne version of OpenAI models since the last run of the notebook): from 19/33 correct to 28/33 correct!

@vowelparrot vowelparrot merged commit 126d7f1 into langchain-ai:master Apr 19, 2023
@qpleple qpleple deleted the fix-examples branch April 20, 2023 07:23
samching pushed a commit to samching/langchain that referenced this pull request May 1, 2023
The following calls were throwing an exception:


https://github.com/hwchase17/langchain/blob/575b717d108984676e25afd0910ccccfdaf9693d/docs/use_cases/evaluation/agent_vectordb_sota_pg.ipynb?short_path=4b3386c#L192


https://github.com/hwchase17/langchain/blob/575b717d108984676e25afd0910ccccfdaf9693d/docs/use_cases/evaluation/agent_vectordb_sota_pg.ipynb?short_path=4b3386c#L239

Exception:

```
---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[14], line 1
----> 1 chain_sota = RetrievalQA.from_chain_type(llm=OpenAI(temperature=0), chain_type="stuff", retriever=vectorstore_sota, input_key="question")

File ~/github/langchain/venv/lib/python3.9/site-packages/langchain/chains/retrieval_qa/base.py:89, in BaseRetrievalQA.from_chain_type(cls, llm, chain_type, chain_type_kwargs, **kwargs)
     85 _chain_type_kwargs = chain_type_kwargs or {}
     86 combine_documents_chain = load_qa_chain(
     87     llm, chain_type=chain_type, **_chain_type_kwargs
     88 )
---> 89 return cls(combine_documents_chain=combine_documents_chain, **kwargs)

File ~/github/langchain/venv/lib/python3.9/site-packages/pydantic/main.py:341, in pydantic.main.BaseModel.__init__()

ValidationError: 1 validation error for RetrievalQA
retriever
  instance of BaseRetriever expected (type=type_error.arbitrary_type; expected_arbitrary_type=BaseRetriever)
```

The vectorstores had to be converted to retrievers:
`vectorstore_sota.as_retriever()` and `vectorstore_pg.as_retriever()`.

The PR also:
- adds the file `paul_graham_essay.txt` referenced by this notebook
- adds to gitignore *.pkl and *.bin files that are generated by this
notebook

Interestingly enough, the performance of the prediction greatly
increased (new version of langchain or ne version of OpenAI models since
the last run of the notebook): from 19/33 correct to 28/33 correct!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants