FAISS returns same score after I changed distance_strategy from Eucledian to Cosine

[stepkurniawan]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

This is how I change the distance metrics:

faiss_distance_strategy = get_faiss_distance_strategy(self.index_distance) # can be default eucledian, or DistanceStrategy.MAX_INNER_PRODUCT, or DistanceStrategy.COSINE

# langchain from_documents
self.db = self.vectorstore_obj.from_documents(split_docs, self.embedding, distance_strategy=faiss_distance_strategy) 

self.db.save_local(self.save_path)

and this is how I check the distance score:

similarity_search_with_score(query, k=top_k)

Description

Im using LangChain x FAISS. And everytime I compare the distance score between the default euclidean and cosine, it will always give me the same score, so I dont know maybe I did something wrong (or not). I was expecting different scores. Could you check please?

System Info

langchain==0.1.0
langchain-community==0.0.11
langchain-core==0.1.8

platform: windows
Python 3.11.5

[dosubot[bot]]

🤖

Based on the code you've provided, it seems like you're correctly setting the distance_strategy when creating the db object with from_documents. However, the issue might be related to how FAISS handles distance metrics.

In FAISS, the distance metric is determined when the index is created and cannot be changed afterwards. The distance_strategy parameter in LangChain's FAISS class is only used to issue a warning if it's not EUCLIDEAN_DISTANCE and _normalize_L2 is True. The actual similarity search is performed by the self.index.search method, which is a method of the faiss library and uses the distance metric that was specified when the index was created.

Therefore, if you want to use a different distance metric, you would need to create a new FAISS index with the desired distance metric. Unfortunately, there is no evidence of a method to create a new FAISS index with a specified distance metric in the provided context. Therefore, you would need to modify the LangChain code or create the index outside of LangChain and then load it.

Please note that the cosine distance is not directly supported by FAISS. However, you can achieve the effect of cosine distance by normalizing your vectors (which it appears you are doing, as _normalize_L2 is True) and then using the L2 distance.

You can refer to the source code of LangChain's FAISS class here for more details.

Regarding the from_documents method in LangChain, it uses the distance_strategy parameter to determine the strategy for calculating the distance between vectors when performing similarity searches. This parameter is passed to the from_texts method, which is called within from_documents. The distance_strategy parameter is an instance of the DistanceStrategy enum, which can take one of three values: EUCLIDEAN, COSINE, or MAX_INNER_PRODUCT. These values determine the method used to calculate the distance between vectors. The chosen strategy is used in the distance_strategy property of the PGVector class, which returns the appropriate distance function based on the value of _distance_strategy. You can refer to the source code of LangChain's from_documents method here for more details.

I hope this helps! If you have any more questions, feel free to ask.

Sources

• libs/langchain/langchain/vectorstores/faiss.py
• libs/langchain/langchain/vectorstores/pgvector.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[eyurtsev]

Related discussion: #16224

[eyurtsev]

@stepkurniawan could you check whether the vectors you're getting back from the embedder are normalized in any particular way? i.e., do they have unit norm?

[stepkurniawan]

hi @eyurtsev ,
The maximum number that I get from Inner product is 0.78... so, I believe it is normalized.
but could you guide me how to do that ?
Currently I only see that every distance is between 0 - 1, which led me think that it's normalized.

[yadav-shivani]

Hi, I am facing the similar issue, even though I setted distance_strategy to cosine still whenever I prints distance metric used it shows distance strategy is 'euclidean'.