refactor: make Reranker stateless with std::variant value semantics (… (#471)

* refactor: make Reranker stateless with std::variant value semantics (#461)

Replace class hierarchy (Reranker/ScoreBasedReranker/RrfReranker/
WeightedReranker/CallbackReranker) with std::variant<RrfParams,
WeightedParams, CallbackParams> value type and a stateless free function
reranker::rerank().

Key changes:
- reranker.h: define RerankParams variant + reranker::rerank() API
- query.h: MultiQuery::reranker (shared_ptr) -> MultiQuery::rerank (value)
- schema.h: add CollectionSchema::get_field_ptr() returning FieldSchema::Ptr
- collection.cc: push field lookup to caller, pass vector<FieldSchema::Ptr>
- c_api: remove opaque zvec_reranker_t, add zvec_multi_query_set_rerank_*
- python binding: expose _RrfParams/_WeightedParams/_CallbackParams + setters
- python layer: WeightedReRanker(list[float]), remove Python rerank logic
- all tests updated to new interface

Benefits:
- Thread-safe by design: no mutable state, safe to share across threads
- Collection-decoupled: no bind_schema(), field info passed as parameter
- Simpler lifecycle: value semantics, no shared_ptr management

Closes #461

* chore: remove nightly_build.yml unrelated to reranker refactor

* chore: remove uv.lock unrelated to reranker refactor

* fix: raise ValueError when multi-query has no reranker

After the reranker stateless refactor the C++ MultiQuery rerank
strategy uses a std::variant with a default value, so the implicit
'reranker required' validation no longer triggered. Restore the
check in QueryExecutor._execute_multi_query so that a hybrid
(multi-query) request without a reranker raises ValueError.

* fix(reranker): use index_type FTS check for non-vector normalization

Replace dynamic_cast nullptr check with explicit IndexType::FTS check
and map FTS/BM25 positive scores to (0.0, 1.0) via 2*atan(score)/pi.

* refactor(reranker): move Params types into reranker namespace and qualify usages

Move RrfParams, WeightedParams, CallbackParams and RerankParams into the
zvec::reranker namespace, and add explicit reranker:: qualification at all
usage sites outside the reranker module (query.h, python/c bindings, tests).

* refactor(query): drop unused PendingQuery wrapper, use std::vector<SearchQuery> directly

* refactor(reranker): make _to_cpp_params non-abstract with default NotImplementedError

Remove @abstractmethod from RerankFunction._to_cpp_params and provide a
default implementation raising NotImplementedError. Drop the redundant
_to_cpp_params overrides from Qwen and Sentence rerankers since they use
the Python rerank path and don't need the C++ conversion.
This commit is contained in:
Cuiys
2026-06-09 12:44:17 +08:00
committed by GitHub
parent e8b888f26b
commit 0923f7c691
20 changed files with 870 additions and 924 deletions
+25 -7
View File
@@ -19,7 +19,7 @@ import numpy as np
from _zvec import _Collection, _MultiQuery
from _zvec.param import _Fts, _SearchQuery, _SubQuery
from ..extension import ReRanker
from ..extension import CallbackReRanker, ReRanker, RrfReRanker, WeightedReRanker
from ..model.convert import convert_to_py_doc
from ..model.doc import DocList
from ..model.param.query import Query
@@ -143,12 +143,18 @@ class QueryExecutor:
) -> DocList:
"""Multiple queries: send a ``_MultiQuery`` to C++.
A Python-only reranker (``_get_object()`` returns None) cannot run
inside the C++ MultiQuery, so each route is executed individually and
merged by the reranker in Python.
A Python-only reranker (e.g. a model/API-based one) cannot run inside
the C++ MultiQuery, so each route is executed individually and merged by
the reranker in Python. The built-in RRF/Weighted/Callback rerankers use
the C++ variant-based fast path.
"""
reranker = ctx.reranker
if reranker is not None and reranker._get_object() is None:
if reranker is None:
raise ValueError(
"A reranker is required to merge results from multiple queries; "
"specify the 'reranker' argument."
)
if not isinstance(reranker, (RrfReRanker, WeightedReRanker, CallbackReRanker)):
docs_list = self._execute_python_pipeline(queries, collection)
return self._merge_and_rerank(ctx, docs_list)
@@ -162,14 +168,26 @@ class QueryExecutor:
"""Assemble a C++ ``_MultiQuery`` from per-route ``_SearchQuery`` objects."""
multi_query = _MultiQuery()
multi_query.queries = [_SubQuery.from_search_query(query) for query in queries]
# num_candidates controls per-sub-query candidate count for reranking pool.
# It must NOT be limited to the final output topk; use at least the C++
# SubQuery default of 10 to ensure sufficient candidates for reranking.
_DEFAULT_NUM_CANDIDATES = 10
for sub in multi_query.queries:
sub.num_candidates = max(ctx.topk, _DEFAULT_NUM_CANDIDATES)
multi_query.topk = ctx.topk
if ctx.filter:
multi_query.filter = ctx.filter
multi_query.include_vector = ctx.include_vector
if ctx.output_fields is not None:
multi_query.output_fields = ctx.output_fields
if ctx.reranker is not None:
multi_query.reranker = ctx.reranker._get_object()
# Set rerank strategy via the C++ variant-based API.
reranker = ctx.reranker
if isinstance(reranker, RrfReRanker):
multi_query.set_rerank_rrf(reranker.rank_constant)
elif isinstance(reranker, WeightedReRanker):
multi_query.set_rerank_weighted(reranker.weights)
elif isinstance(reranker, CallbackReRanker):
multi_query.set_rerank_callback(reranker._callback)
return multi_query
def _execute_python_pipeline(
+2
View File
@@ -24,6 +24,7 @@ from .openai_function import OpenAIFunctionBase
from .qwen_embedding_function import QwenDenseEmbedding, QwenSparseEmbedding
from .qwen_function import QwenFunctionBase
from .qwen_rerank_function import QwenReRanker
from .rerank_function import RerankFunction
from .rerank_function import RerankFunction as ReRanker
from .sentence_transformer_embedding_function import (
DefaultLocalDenseEmbedding,
@@ -49,6 +50,7 @@ __all__ = [
"QwenReRanker",
"QwenSparseEmbedding",
"ReRanker",
"RerankFunction",
"RrfReRanker",
"SentenceTransformerFunctionBase",
"SparseEmbeddingFunction",
+127 -83
View File
@@ -14,134 +14,178 @@
from __future__ import annotations
from collections.abc import Callable
from typing import Optional
from typing import TYPE_CHECKING
from _zvec import _CallbackReranker, _RrfReranker, _WeightedReranker
from _zvec import _CallbackParams, _Doc, _reranker_rerank, _RrfParams, _WeightedParams
from ..model.doc import DocList
from ..model.doc import Doc, DocList
from .rerank_function import RerankFunction
if TYPE_CHECKING:
from ..model.schema import FieldSchema, VectorSchema
def _to_cpp_doc_lists(
query_results: list[list[Doc]],
) -> tuple[list[list], dict[str, Doc]]:
"""Convert Python Doc lists to C++ _Doc lists for reranker input."""
id_to_doc: dict[str, Doc] = {}
cpp_results: list[list] = []
for query_result in query_results:
cpp_list: list = []
for doc in query_result:
_doc = _Doc()
_doc.set_pk(doc.id)
_doc.set_score(doc.score if doc.score is not None else 0.0)
cpp_list.append(_doc)
if doc.id not in id_to_doc:
id_to_doc[doc.id] = doc
cpp_results.append(cpp_list)
return cpp_results, id_to_doc
def _from_cpp_docs(cpp_docs: list, id_to_doc: dict[str, Doc]) -> DocList:
"""Convert C++ rerank result _Doc list back to Python DocList."""
results: DocList = []
for _doc in cpp_docs:
doc_id = _doc.pk()
new_score = _doc.score()
original = id_to_doc.get(doc_id)
if original is not None:
results.append(original._replace(score=new_score))
else:
results.append(Doc(id=doc_id, score=new_score))
return results
class RrfReRanker(RerankFunction):
"""Re-ranker using Reciprocal Rank Fusion (RRF) for multi-vector search.
RRF combines results from multiple vector queries without requiring relevance scores.
It assigns higher weight to documents that appear early in multiple result lists.
The RRF score for a document at rank ``r`` is: ``1 / (k + r + 1)``,
where ``k`` is the rank constant.
RRF combines results from multiple vector queries without requiring
relevance scores. The RRF score for a document at rank r is:
score = 1 / (k + r + 1)
where k is the rank constant.
Args:
rank_constant (int, optional): Smoothing constant ``k`` in RRF formula.
Larger values reduce the impact of early ranks. Defaults to 60.
rank_constant: RRF smoothing constant (default: 60).
Higher values reduce the influence of rank position.
Example:
>>> reranker = RrfReRanker(rank_constant=60)
>>> merged = reranker.rerank([results_a, results_b], topn=10)
"""
def __init__(
self,
rank_constant: int = 60,
):
def __init__(self, rank_constant: int = 60):
self._rank_constant = rank_constant
# Use C++ implementation for performance
self._cpp_reranker = _RrfReranker(rank_constant)
@property
def rank_constant(self) -> int:
"""int: RRF rank constant."""
return self._rank_constant
def _get_object(self):
"""Return the underlying C++ RrfReranker instance."""
return self._cpp_reranker
def _to_cpp_params(self):
return _RrfParams(self._rank_constant)
def rerank(self, query_results: list[DocList], topn: int) -> DocList:
"""Re-rank using C++ RRF implementation.
Args:
query_results (list[DocList]): Multi-route recall results,
positionally aligned with queries.
topn (int): Number of top documents to return.
Returns:
DocList: Re-ranked documents.
"""
return self._cpp_reranker.rerank(query_results, topn)
def rerank(
self,
query_results: list[list[Doc]],
topn: int = 10,
*,
fields: list[FieldSchema | VectorSchema] | None = None, # noqa: ARG002
) -> DocList:
"""Apply RRF to combine multiple query results via C++ reranker."""
cpp_results, id_to_doc = _to_cpp_doc_lists(query_results)
cpp_docs = _reranker_rerank(self._to_cpp_params(), cpp_results, [], topn)
return _from_cpp_docs(cpp_docs, id_to_doc)
class WeightedReRanker(RerankFunction):
"""Re-ranker that combines scores from multiple vector fields using weights.
"""Re-ranker that combines scores using per-sub-query weights.
Each vector field's relevance score is normalized based on its own metric
type, then scaled by a user-provided weight. Final scores are summed across
fields. The actual re-ranking logic lives in the C++ implementation.
Each sub-query's score is normalized by metric type (automatic when used
via collection.multi_query), then multiplied by the corresponding weight.
Args:
weights (Optional[list[float]], optional): Weight per vector field,
aligned by position with the queries supplied to ``collection.query()``.
Defaults to None (treated as an empty list).
weights: Per-sub-query weights. Length must match the number of
sub-queries.
Example:
>>> reranker = WeightedReRanker([0.7, 0.3])
>>> merged = reranker.rerank([results_a, results_b], topn=10,
... fields=field_schemas)
"""
def __init__(
self,
weights: Optional[list[float]] = None,
):
self._cpp_reranker = _WeightedReranker(weights or [])
def __init__(self, weights: list[float]):
self._weights = list(weights)
@property
def weights(self) -> list[float]:
"""list[float]: Weight list for vector fields, aligned with queries."""
return self._cpp_reranker.weights
"""list[float]: Per-sub-query weights."""
return self._weights
def _get_object(self):
"""Return the underlying C++ WeightedReranker instance."""
return self._cpp_reranker
def _to_cpp_params(self):
return _WeightedParams(self._weights)
def rerank(self, query_results: list[DocList], topn: int) -> DocList:
"""Re-rank using C++ Weighted implementation.
def rerank(
self,
query_results: list[list[Doc]],
topn: int = 10,
*,
fields: list[FieldSchema | VectorSchema] | None = None,
) -> DocList:
"""Combine scores from multiple sub-queries using weighted sum via C++ reranker.
Args:
query_results (list[DocList]): Multi-route recall results,
positionally aligned with queries.
topn (int): Number of top documents to return.
query_results: Per-sub-query document lists.
topn: Maximum results to return.
fields: Per-sub-query Python FieldSchema/VectorSchema objects
(required for score normalization by metric type).
Returns:
DocList: Re-ranked documents.
Raises:
ValueError: If fields is None (required for normalization).
"""
return self._cpp_reranker.rerank(query_results, topn)
if not fields:
raise ValueError(
"WeightedReRanker.rerank() requires 'fields' for score normalization. "
"Pass field schemas via fields= parameter."
)
cpp_fields = [f._get_object() for f in fields]
cpp_results, id_to_doc = _to_cpp_doc_lists(query_results)
cpp_docs = _reranker_rerank(
self._to_cpp_params(), cpp_results, cpp_fields, topn
)
return _from_cpp_docs(cpp_docs, id_to_doc)
class CallbackReRanker(RerankFunction):
"""Re-ranker that delegates to a user-provided Python callback.
"""Re-ranker that delegates to a user-provided callback.
This bridges a Python callable into the C++ reranker interface, enabling
custom re-ranking logic to be executed within the C++ MultiQuery path.
The callback receives raw C++ ``_Doc`` objects grouped per query (as a
``list[list[_Doc]]``) and must return a ``list[_Doc]``.
The callback receives sub-query results, field schemas, and topn.
Args:
callback: A callable with signature
``(query_results: list[list[_Doc]], topn: int) -> list[_Doc]``.
(results: list[list[Doc]], fields: list, topn: int) -> list[Doc]
Example:
>>> def my_rerank(results, fields, topn):
... # custom logic
... return merged[:topn]
>>> reranker = CallbackReRanker(my_rerank)
>>> merged = reranker.rerank([results_a, results_b], topn=10)
"""
def __init__(
self,
callback: Callable,
):
def __init__(self, callback: Callable):
self._callback = callback
self._cpp_reranker = _CallbackReranker(callback)
def _get_object(self):
"""Return the underlying C++ CallbackReranker instance."""
return self._cpp_reranker
def _to_cpp_params(self):
return _CallbackParams(self._callback)
def rerank(self, query_results: list[DocList], topn: int) -> DocList:
"""Invoke the callback to re-rank documents.
Args:
query_results (list[DocList]): Multi-route recall results,
positionally aligned with queries.
topn (int): Number of top documents to return.
Returns:
DocList: Re-ranked documents.
"""
return self._callback(query_results, topn)
def rerank(
self,
query_results: list[list[Doc]],
topn: int = 10,
*,
fields: list[FieldSchema | VectorSchema] | None = None,
) -> DocList:
"""Invoke the callback to re-rank documents."""
return self._callback(query_results, fields, topn)
+29 -17
View File
@@ -13,12 +13,15 @@
# limitations under the License.
from __future__ import annotations
from typing import Optional
from typing import TYPE_CHECKING, Optional
from ..model.doc import Doc, DocList
from .qwen_function import QwenFunctionBase
from .rerank_function import RerankFunction
if TYPE_CHECKING:
from ..model.schema import FieldSchema, VectorSchema
class QwenReRanker(QwenFunctionBase, RerankFunction):
"""Re-ranker using Qwen (DashScope) cross-encoder API for semantic re-ranking.
@@ -77,51 +80,60 @@ class QwenReRanker(QwenFunctionBase, RerankFunction):
ValueError: If query is empty or API key is unavailable.
"""
QwenFunctionBase.__init__(self, model=model, api_key=api_key)
RerankFunction.__init__(self)
self._rerank_field = rerank_field
if not query:
raise ValueError("Query is required for QwenReRanker")
self._query = query
self._rerank_field = rerank_field
@property
def query(self) -> str:
"""str: Query text used for semantic re-ranking."""
return self._query
@property
def rerank_field(self) -> Optional[str]:
"""Optional[str]: Field name used as re-ranking input."""
return self._rerank_field
def rerank(self, query_results: list[DocList], topn: int) -> DocList:
@property
def query(self) -> str:
"""str: Query text used for semantic re-ranking."""
return self._query
def rerank(
self,
query_results: list[list[Doc]],
topn: int = 10,
*,
fields: list[FieldSchema | VectorSchema] | None = None, # noqa: ARG002
) -> DocList:
"""Re-rank documents using Qwen's TextReRank API.
Sends document texts to DashScope TextReRank service along with the query.
Returns documents sorted by relevance scores from the cross-encoder model.
Args:
query_results (list[DocList]): Multi-route recall results,
positionally aligned with the queries supplied to
``collection.query()``. Documents from all routes are
deduplicated and re-ranked together.
topn (int): Maximum number of documents to return after re-ranking.
query_results (list[list[Doc]]): Per-sub-query lists of retrieved
documents. Documents from all lists are deduplicated and
re-ranked together.
topn (int): Maximum number of documents to return.
fields: Unused; present for interface compatibility.
Returns:
DocList: Re-ranked documents (up to ``topn``) with updated
``score`` fields containing relevance scores from the API.
list[Doc]: Re-ranked documents (up to ``topn``) with updated ``score``
fields containing relevance scores from the API.
Raises:
ValueError: If no valid documents are found or API call fails.
Note:
- Duplicate documents (same ID) across routes are processed once
- Duplicate documents (same ID) across lists are processed once
- Documents with empty/missing ``rerank_field`` content are skipped
- Returned scores are relevance scores from the cross-encoder model
"""
if not query_results:
return []
# Accept both dict (legacy) and list formats
if isinstance(query_results, dict):
query_results = list(query_results.values())
# Collect and deduplicate documents
id_to_doc: dict[str, Doc] = {}
doc_ids: list[str] = []
+26 -28
View File
@@ -14,45 +14,43 @@
from __future__ import annotations
from abc import ABC, abstractmethod
from typing import TYPE_CHECKING
from ..model.doc import DocList
from ..model.doc import Doc, DocList
if TYPE_CHECKING:
from ..model.schema import FieldSchema, VectorSchema
class RerankFunction(ABC):
"""Abstract base class for re-ranking search results.
"""Abstract base class for reranker parameter containers.
Re-rankers refine the output of one or more vector queries by applying
a secondary scoring strategy. They are used in the ``query()`` method of
``Collection`` via the ``reranker`` parameter.
Note:
Subclasses must implement the ``rerank()`` method.
Subclasses define rerank parameters and implement _to_cpp_params()
for conversion to C++ parameter structs (used by collection fast path).
Each subclass also provides a standalone rerank() implementation.
"""
def _to_cpp_params(self):
"""Return C++ reranker params. Override in subclasses that use C++ path."""
raise NotImplementedError
@abstractmethod
def rerank(self, query_results: list[DocList], topn: int) -> DocList:
"""Re-rank documents from multi-route recall results.
def rerank(
self,
query_results: list[list[Doc]],
topn: int = 10,
*,
fields: list[FieldSchema | VectorSchema] | None = None,
) -> DocList:
"""Execute rerank on sub-query results.
Args:
query_results (list[DocList]): List of query results from
multi-route recall. Each element corresponds to a Query in the
collection.query(queries=List[Query]) call, aligned by position.
topn (int): Number of top documents to return after re-ranking.
query_results: List of per-sub-query document lists.
topn: Maximum number of results to return.
fields: Per-sub-query Python FieldSchema/VectorSchema objects
(required for WeightedReRanker score normalization).
Returns:
DocList: Re-ranked list of documents (length ≤ ``topn``),
with updated ``score`` fields.
Re-ranked document list.
"""
...
def _get_object(self):
"""Return the underlying C++ Reranker instance, if available.
This is used internally by the query executor to pass the reranker
to the C++ MultiQuery method. Subclasses that wrap a C++ reranker
should override this method.
Returns:
The C++ Reranker shared pointer, or None if not available.
"""
return None # noqa: RET501
@@ -13,13 +13,16 @@
# limitations under the License.
from __future__ import annotations
from typing import Literal, Optional
from typing import TYPE_CHECKING, Literal, Optional
from ..model.doc import Doc, DocList
from ..tool import require_module
from .rerank_function import RerankFunction
from .sentence_transformer_function import SentenceTransformerFunctionBase
if TYPE_CHECKING:
from ..model.schema import FieldSchema, VectorSchema
class DefaultLocalReRanker(SentenceTransformerFunctionBase, RerankFunction):
"""Re-ranker using Sentence Transformer cross-encoder models for semantic re-ranking.
@@ -137,13 +140,13 @@ class DefaultLocalReRanker(SentenceTransformerFunctionBase, RerankFunction):
... )
>>> # Direct rerank call (for testing)
>>> query_results = [
... [
>>> query_results = {
... "vector1": [
... Doc(id="1", score=0.9, fields={"content": "Machine learning is..."}),
... Doc(id="2", score=0.8, fields={"content": "Deep learning is..."}),
... ]
... ]
>>> reranked = reranker.rerank(query_results, topn=5)
... }
>>> reranked = reranker.rerank(query_results)
>>> for doc in reranked:
... print(f"ID: {doc.id}, Score: {doc.score:.4f}")
ID: 2, Score: 0.9234
@@ -188,14 +191,13 @@ class DefaultLocalReRanker(SentenceTransformerFunctionBase, RerankFunction):
self, model_name=model_name, model_source=model_source, device=device
)
# Initialize rerank function
RerankFunction.__init__(self)
# Initialize rerank parameters
self._rerank_field = rerank_field
# Validate query
if not query:
raise ValueError("Query is required for DefaultLocalReRanker")
self._query = query
self._rerank_field = rerank_field
self._batch_size = batch_size
# Load and validate cross-encoder model
@@ -261,22 +263,28 @@ class DefaultLocalReRanker(SentenceTransformerFunctionBase, RerankFunction):
f"from {self._model_source}: {e!s}"
) from e
@property
def query(self) -> str:
"""str: Query text used for semantic re-ranking."""
return self._query
@property
def rerank_field(self) -> Optional[str]:
"""Optional[str]: Field name used as re-ranking input."""
return self._rerank_field
@property
def query(self) -> str:
"""str: Query text used for semantic re-ranking."""
return self._query
@property
def batch_size(self) -> int:
"""int: Batch size for processing query-document pairs."""
return self._batch_size
def rerank(self, query_results: list[DocList], topn: int) -> DocList:
def rerank(
self,
query_results: list[list[Doc]],
topn: int = 10,
*,
fields: list[FieldSchema | VectorSchema] | None = None, # noqa: ARG002
) -> DocList:
"""Re-rank documents using Sentence Transformer cross-encoder model.
Evaluates each query-document pair using the cross-encoder model to compute
@@ -284,22 +292,21 @@ class DefaultLocalReRanker(SentenceTransformerFunctionBase, RerankFunction):
results are returned.
Args:
query_results (list[DocList]): Multi-route recall results,
positionally aligned with the queries supplied to
``collection.query()``. Documents from all routes are
deduplicated and re-ranked together.
topn (int): Maximum number of documents to return after re-ranking.
query_results (list[list[Doc]]): Per-sub-query lists of retrieved
documents. Documents from all lists are deduplicated and
re-ranked together.
topn (int): Maximum number of documents to return.
fields: Unused; present for interface compatibility.
Returns:
DocList: Re-ranked documents (up to ``topn``) with updated
``score`` fields containing relevance scores from the
cross-encoder model.
list[Doc]: Re-ranked documents (up to ``topn``) with updated ``score``
fields containing relevance scores from the cross-encoder model.
Raises:
ValueError: If no valid documents are found or model inference fails.
Note:
- Duplicate documents (same ID) across routes are processed once
- Duplicate documents (same ID) across fields are processed once
- Documents with empty/missing ``rerank_field`` content are skipped
- Returned scores are logits from the cross-encoder model
- Higher scores indicate higher relevance
@@ -311,19 +318,23 @@ class DefaultLocalReRanker(SentenceTransformerFunctionBase, RerankFunction):
... topn=3,
... rerank_field="content"
... )
>>> query_results = [
... [
>>> query_results = {
... "vector1": [
... Doc(id="1", score=0.9, fields={"content": "ML basics"}),
... Doc(id="2", score=0.8, fields={"content": "DL tutorial"}),
... ]
... ]
>>> reranked = reranker.rerank(query_results, topn=3)
... }
>>> reranked = reranker.rerank(query_results)
>>> len(reranked) <= 3
True
"""
if not query_results:
return []
# Accept both dict (legacy) and list formats
if isinstance(query_results, dict):
query_results = list(query_results.values())
# Collect and deduplicate documents
id_to_doc: dict[str, Doc] = {}
doc_ids: list[str] = []