feat: optional reranking for mix and naive query mode.

[drahnreb]

Description

Adds an optional reranking in naive and mix query mode.

Related Issues

#1414

Changes Made

Checklist

• Changes tested locally
• Code reviewed
• Documentation updated (if necessary)
• Unit tests added (if applicable)

[danielaskdd]

Reranking the results of cos similarity search can indeed improve query quality. I have a few questions that I would like you to clarify:

1. Does the current approach consider allowing users to choose different reranker options? For example, selecting the bge-reranker-v2-m3 model deployed with Ollama, or choosing the jina-reranker-v2-base-multilingual model from Jina AI.

2. At present, using QueryParam to pass the reranker function does not seem very appropriate, because Web APIs generally should not specify reranker functions via QueryParam. Is there a better way for users to choose whether to enable reranking and which reranker to use?

3. Currently, I notice that you only modified naive_query to allow it to utilize the reranker_func. Why didn’t you similarly modify mix_kg_vector_query?

4. In addition, PR Feature: Implemented an option to use DeepSeek Reasoner for intelligent semantic node re-ranking and retrieval #1400 proposes an idea to perform reranking on nodes. I would like to hear your opinion on this approach. If it is necessary to rerank both chunks and nodes, how should the interface be designed?

[drahnreb]

1. Does the current approach consider allowing users to choose different reranker options? For example, selecting the bge-reranker-v2-m3 model deployed with Ollama, or choosing the jina-reranker-v2-base-multilingual model from Jina AI.

As long as it follows the defined Signature

LightRAG/lightrag/base.py

Line 63 in 1e4ea22

reranker_func: Callable[[str, list[str]], list[str]] | None = None

it accepts a query, and list of chunks to rerank. One would need to wrap it around your specific reranker (be it API or a simple filter as illustrated) just like for the other model funcs.

2. At present, using QueryParam to pass the reranker function does not seem very appropriate, because Web APIs generally should not specify reranker functions via QueryParam. Is there a better way for users to choose whether to enable reranking and which reranker to use?

Which Web APIs do you mean - this?

LightRAG/lightrag/api/routers/query_routes.py

Line 19 in 9089539

class QueryRequest(BaseModel):

Passing a callable function directly as a parameter within a web API request introduces several fundamental problems...
Client-Side code (e.g. allowing clients to send scripts to be executed on the server) is probably not desired due to the security implications?
Otherwise we could use a configuration object and provide the set of rerankers by langchain and langchain community: https://python.langchain.com/docs/integrations/retrievers/#all-retrievers
e.g. https://api.python.langchain.com/en/latest/vertex_rank/langchain_google_community.vertex_rank.VertexAIRank.html
We would need to wrap context into langchain Documents which is not a bad thing in principle but needs some work.

3. Currently, I notice that you only modified naive_query to allow it to utilize the reranker_func. Why didn’t you similarly modify mix_kg_vector_query?

I simply missed it. You are right, it should be in both. I expected mix_kg_vector_query to call naive_query.

• Can we simplify the code and remove get_vector_context() and add a flag to naive_query instead to handle the different returns and formatting and return with query_param.only_need_context = True? The code path is almost duplicate.
• Also, if we limit to top_k = 10 a reranking is not super useful, and in any case exposing this parameter would make sense to control it when reranking or if we do not care about context limits:
  

  LightRAG/lightrag/operate.py

  Lines 1127 to 1128 in 1e4ea22

  	# Reduce top_k for vector search in hybrid mode since we have structured information from KG
  	mix_topk = min(10, query_param.top_k)

4. In addition, PR Feature: Implemented an option to use DeepSeek Reasoner for intelligent semantic node re-ranking and retrieval #1400 proposes an idea to perform reranking on nodes. I would like to hear your opinion on this approach. If it is necessary to rerank both chunks and nodes, how should the interface be designed?

I saw this PR and share the following remarks (some already mentioned):

• reranking adds query time which is not favorable all the time. Reranking over dense vectors maybe doubles the end-to-end latency (depends on top_k, reranker_func). Generally I wouldn't LLM based reasoning over nodes to rerank, as it is suggested, because it is too time consuming.

• I do disagree to use another model, if your llm_model_func allows to "reason" (with an appropriate prompt) this should be sufficient to query for more relevant nodes/edges. Similar to this reranking, there is not much conceptual difference if the context comes from a vector database or a graph.

• But ideally you would want to do the reasoning on the graph (that is what graphs are made for), which is partially (infant state) implemented via one-hop graph traversal retrieval. You could extend this with more sophisticated graph inference logics in pre-defined SPARQL clauses and LLM-based or do reranking via GNN. And offload the heavy work on the database, e.g. with more powerful providers like neo4j.

  In short: In my opinion, if we want to rerank on graph, we should leverage the fact that we have a structured database and do structured reranking before based on graph traversal and not after graph retrieval from nodes (and possibly edges).

Let me know what you @danielaskdd think, reranking is not a big feature for me but I can work on it.

[danielaskdd]

Obviously query_param is not suitable for callable object, we need to refactor the program to allow API clients to select different reranker models via query_param.