Skip to content

Commit dcfd58b

Browse files
docs: fix semantic similarity description (cross-encoder -> bi-encoder) (#1910)
This PR updates the documentation to correctly describe the Semantic similarity. ### Issue The documentation previously stated that a **cross-encoder** was used for computing the semantic similarity score. However, after reviewing the implementation, it is clear that the current approach follows a **bi-encoder** strategy: - The ground truth and response are encoded independently - Their embeddings are then compared using cosine similarity A cross-encoder would typically process both texts together in a single forward pass (e.g., concatenating them before encoding), which is not the case in the current implementation. ### Current Implementation For example, in the current implementation: ```python embedding_1 = np.array(await self.embeddings.embed_text(ground_truth)) embedding_2 = np.array(await self.embeddings.embed_text(answer)) # Normalization factors of the above embeddings norms_1 = np.linalg.norm(embedding_1, keepdims=True) norms_2 = np.linalg.norm(embedding_2, keepdims=True) embedding_1_normalized = embedding_1 / norms_1 embedding_2_normalized = embedding_2 / norms_2 similarity = embedding_1_normalized @ embedding_2_normalized.T score = similarity.flatten() ``` This code shows that the ground truth and response are encoded separately, and their similarity is computed using cosine similarity, which is characteristic of a **bi-encoder** approach. ### Fix The term "cross-encoder" has been corrected to "bi-encoder" in the documentation to ensure consistency with the actual implementation.
1 parent 6ef4f9a commit dcfd58b

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

docs/concepts/metrics/available_metrics/semantic_similarity.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
The concept of Answer Semantic Similarity pertains to the assessment of the semantic resemblance between the generated answer and the ground truth. This evaluation is based on the `ground truth` and the `answer`, with values falling within the range of 0 to 1. A higher score signifies a better alignment between the generated answer and the ground truth.
44

5-
Measuring the semantic similarity between answers can offer valuable insights into the quality of the generated response. This evaluation utilizes a cross-encoder model to calculate the semantic similarity score.
5+
Measuring the semantic similarity between answers can offer valuable insights into the quality of the generated response. This evaluation utilizes a bi-encoder model to calculate the semantic similarity score.
66

77

88
### Example

0 commit comments

Comments
 (0)