refactor: change rerank interface from map-based to vector-based (#458)
* refactor: change rerank interface from map-based to vector-based (#452) - Define QueryResult = list[Doc] type alias in doc.py - Change C++ Reranker::rerank() signature from map<string, DocPtrList> to vector<DocPtrList> - Extend bind_schema() to accept field_names for index-based field lookup - Update ScoreBasedReranker/WeightedReranker/CallbackReranker implementations - Adapt collection.cc MultiQuery path to use vector<DocPtrList> - Update Python binding to expose rerank() and use vector<double> weights - Refactor Python RerankFunction interface to list[QueryResult] -> QueryResult - Remove Python-layer rerank logic from RrfReRanker/WeightedReRanker (delegate to C++) - Update query_executor to return list[list[Doc]] instead of dict - Update all related unit tests (C++ and Python) * refactor: replace list[Doc] with QueryResult type alias in executor and rerank functions * refactor: replace list[list[Doc]] with list[QueryResult] in query_executor * fix: remove unused Doc import in rerank_function.py (ruff F401) * refactor(query_executor): merge duplicate rerank return paths * refactor: RrfReRanker/WeightedReRanker.rerank() directly call C++ reranker * refactor: simplify QueryExecutor into unified class, remove Factory/subclasses/validation/concurrency * refactor: rename _VectorQuery to _SearchQuery, from_vector_query to from_search_query * refactor(query_executor): split execute into single/multi paths, rename core_vector to search_query, drop unused core_vectors * style: apply ruff formatter to test_reranker.py and query_executor.py * refactor: make rescore() private in ScoreBasedReranker hierarchy * style: apply clang-format to reranker.h * style: apply clang-format to all modified C++ files * refactor: rename private methods in QueryExecutor for clearer semantics * refactor: rename mvq to multi_query for clarity * fix: make BasicRRF test order-independent for equal scores * fix: update collection_test to use vector-based reranker interface * fix: update reranker tests to expect TypeError instead of NotImplementedError * refactor: remove PendingQuery wrapper, use SearchQuery directly in MultiQuery path * refactor: simplify MultiQuery path - remove seen_fields, merge field_names into main loop * fix: address review comments - defensive checks and remove fields param from C API - ScoreBasedReranker::rerank(): early return empty list when topn <= 0 - WeightedReranker::rescore(): null-check schema_ before use - CallbackReranker::rerank(): check callback_ is not empty before invoke - C API zvec_reranker_create_weighted(): remove unused fields parameter * fix: remove duplicate field name test (check was intentionally removed) * fix: address egolearner review comments - Rename QueryResult to DocList for clarity (见名知义) - Change docstring to #: comment for type alias - Fix output_fields check: use 'is not None' instead of truthy check (None means unset, [] means explicit empty list - different semantics) - Raise ValueError when search-by-id finds no document * refactor: remove redundant output_fields assignment in _build_search_query * refactor: address egolearner review comments (C++ refactoring) - c_api.cc: simplify weighted reranker creation with inline vector ctor - python_reranker.cc: refactor unwrap_rerank_result - take by value, early error return, move semantics - Rename C API functions for consistent naming: zvec_reranker_create_rrf -> zvec_create_rrf_reranker zvec_reranker_create_weighted -> zvec_create_weighted_reranker zvec_reranker_destroy -> zvec_destroy_reranker zvec_reranker_get_rank_constant -> zvec_get_reranker_rank_constant - reranker.h/cc: bind_schema returns Result<void>, caches vector<const FieldSchema*> to avoid repeated schema lookups in rescore - python_param.cc: rename py::arg vector_query to search_query * revert: rollback bind_schema refactoring due to thread-safety concern The field_schemas_ caching approach introduces a data race when the same WeightedReranker instance is shared across concurrent queries: bind_schema() writes field_schemas_ while rerank() reads it concurrently. Revert to storing schema_ + field_names_ and looking up fields in rescore(). Add @note thread-safety warning to WeightedReranker class documentation. * fix: unify error message format in collection.cc Change 'Vector field not found: X' to 'Invalid query: field X not found' for consistent error formatting as suggested by zhourrr. * fix: sort __all__ and remove duplicates in __init__.pyi Fix RUF022 lint error: sort __all__ alphabetically and remove duplicate entries (DenseEmbeddingFunction, ReRanker). * style: format query_executor.py with ruff formatter * fix: resolve Python test failures after FTS rebase integration - test_query_executor.py: update method names to match refactored API (_do_build -> _build_queries, _do_merge_rerank_results -> _merge_and_rerank) - test_reranker.py: fix expected exception type (TypeError from pybind11) - test_collection_fts.py: update error message match patterns - test_collection_fts_vector_hybrid.py: remove obsolete 'metrics' param, update weights from dict to positional list, adapt validation tests for multi-vector queries (now supported with reranker) - test_collection_dql.py: remove 'metrics' param, update weights format - collection.cc: distinguish FTS vs vector fields in MultiQuery path using get_fts_clause() to route field lookup correctly - reranker.cc: use get_field() instead of get_vector_field() in rescore to support FTS+vector hybrid weighted reranking * refactor: pass topn as rerank() parameter, move rerank_field to model rerankers * fix: address review comments - rename test functions and restore duplicate field check * refactor: simplify MultiQuery field lookup, let validate_and_sanitize handle type check
This commit is contained in:
@@ -824,7 +824,7 @@ class TestCollectionQuery:
|
||||
for k, v in DEFAULT_VECTOR_FIELD_NAME.items():
|
||||
multi_query_vectors.append(Query(field_name=v, vector=doc_vectors[v]))
|
||||
|
||||
rrf_reranker = RrfReRanker(topn=3)
|
||||
rrf_reranker = RrfReRanker()
|
||||
multi_query_result = full_collection.query(
|
||||
multi_query_vectors,
|
||||
reranker=rrf_reranker,
|
||||
@@ -876,8 +876,8 @@ class TestCollectionQuery:
|
||||
batchdoc_and_check(full_collection, multiple_docs, doc_num, operator="insert")
|
||||
doc_fields, doc_vectors = generate_vectordict_random(full_collection.schema)
|
||||
|
||||
metrics = {field: MetricType.IP for field in weights}
|
||||
weighted_reranker = WeightedReRanker(topn=3, weights=weights, metrics=metrics)
|
||||
weight_list = [weights[v] for v in DEFAULT_VECTOR_FIELD_NAME.values()]
|
||||
weighted_reranker = WeightedReRanker(weights=weight_list)
|
||||
|
||||
single_query_results = {}
|
||||
for k, v in DEFAULT_VECTOR_FIELD_NAME.items():
|
||||
@@ -1165,13 +1165,6 @@ class TestCollectionQuery:
|
||||
), # Non-existent ID
|
||||
"Expected exception for non-existent document ID",
|
||||
),
|
||||
(
|
||||
"Both vector and id specified (invalid combination)",
|
||||
lambda ref_dense_vector: Query(
|
||||
field_name="vector_fp32_field", vector=ref_dense_vector, id="5"
|
||||
),
|
||||
"Expected exception for specifying both vector and id",
|
||||
),
|
||||
(
|
||||
"Neither vector nor id specified",
|
||||
lambda ref_dense_vector: Query(
|
||||
|
||||
@@ -976,18 +976,6 @@ class TestCollectionQuery:
|
||||
result = collection_with_multiple_docs.query(filter="id in (1)", topk=100)
|
||||
assert len(result) == 1
|
||||
|
||||
def test_collection_query_with_vector_and_id(
|
||||
self, collection_with_single_doc: Collection, single_doc: Doc
|
||||
):
|
||||
with pytest.raises(ValueError):
|
||||
collection_with_single_doc.query(
|
||||
Query(
|
||||
field_name="dense",
|
||||
id=single_doc.id,
|
||||
vector=single_doc.vector("dense"),
|
||||
)
|
||||
)
|
||||
|
||||
def test_collection_query_with_filter_not_in(
|
||||
self, collection_with_multiple_docs: Collection, multiple_docs
|
||||
):
|
||||
@@ -1013,30 +1001,6 @@ class TestCollectionQuery:
|
||||
)
|
||||
assert len(result) == 10
|
||||
|
||||
def test_collection_query_multi_vector_with_same_field(
|
||||
self, collection_with_multiple_docs: Collection, multiple_docs
|
||||
):
|
||||
# Multi-vector query on same field without reranker should raise ValueError
|
||||
with pytest.raises(ValueError, match="Reranker is required"):
|
||||
collection_with_multiple_docs.query(
|
||||
[
|
||||
Query(field_name="dense", vector=multiple_docs[0].vector("dense")),
|
||||
Query(field_name="dense", vector=multiple_docs[1].vector("dense")),
|
||||
]
|
||||
)
|
||||
|
||||
# Same field name with reranker should also raise ValueError
|
||||
reranker = RrfReRanker(topn=10, rank_constant=60)
|
||||
with pytest.raises(ValueError, match="appears more than once"):
|
||||
collection_with_multiple_docs.query(
|
||||
[
|
||||
Query(field_name="dense", vector=multiple_docs[0].vector("dense")),
|
||||
Query(field_name="dense", vector=multiple_docs[1].vector("dense")),
|
||||
],
|
||||
topk=10,
|
||||
reranker=reranker,
|
||||
)
|
||||
|
||||
def test_collection_query_by_dense_vector(
|
||||
self, collection_with_multiple_docs: Collection, multiple_docs
|
||||
):
|
||||
@@ -1087,7 +1051,7 @@ class TestCollectionQuery:
|
||||
self, collection_with_multiple_docs: Collection, multiple_docs
|
||||
):
|
||||
"""Test multi-vector query with RRF reranker on multiple dense vectors."""
|
||||
reranker = RrfReRanker(topn=10, rank_constant=60)
|
||||
reranker = RrfReRanker(rank_constant=60)
|
||||
result = collection_with_multiple_docs.query(
|
||||
[
|
||||
Query(field_name="dense", vector=multiple_docs[0].vector("dense")),
|
||||
@@ -1106,7 +1070,7 @@ class TestCollectionQuery:
|
||||
self, collection_with_multiple_docs: Collection, multiple_docs
|
||||
):
|
||||
"""Test multi-vector query with RRF reranker on multiple sparse vectors."""
|
||||
reranker = RrfReRanker(topn=10, rank_constant=60)
|
||||
reranker = RrfReRanker(rank_constant=60)
|
||||
result = collection_with_multiple_docs.query(
|
||||
[
|
||||
Query(field_name="sparse", vector=multiple_docs[0].vector("sparse")),
|
||||
@@ -1125,7 +1089,7 @@ class TestCollectionQuery:
|
||||
self, collection_with_multiple_docs: Collection, multiple_docs
|
||||
):
|
||||
"""Test multi-vector query with RRF reranker combining dense + sparse."""
|
||||
reranker = RrfReRanker(topn=10, rank_constant=60)
|
||||
reranker = RrfReRanker(rank_constant=60)
|
||||
result = collection_with_multiple_docs.query(
|
||||
[
|
||||
Query(field_name="dense", vector=multiple_docs[0].vector("dense")),
|
||||
@@ -1141,9 +1105,7 @@ class TestCollectionQuery:
|
||||
self, collection_with_multiple_docs: Collection, multiple_docs
|
||||
):
|
||||
"""Test multi-vector query with Weighted reranker on multiple dense vectors."""
|
||||
metrics = {"dense": MetricType.IP, "dense2": MetricType.IP}
|
||||
weights = {"dense": 0.6, "dense2": 0.4}
|
||||
reranker = WeightedReRanker(topn=10, metrics=metrics, weights=weights)
|
||||
reranker = WeightedReRanker(weights=[0.6, 0.4])
|
||||
result = collection_with_multiple_docs.query(
|
||||
[
|
||||
Query(field_name="dense", vector=multiple_docs[0].vector("dense")),
|
||||
@@ -1159,9 +1121,7 @@ class TestCollectionQuery:
|
||||
self, collection_with_multiple_docs: Collection, multiple_docs
|
||||
):
|
||||
"""Test multi-vector query with Weighted reranker on multiple sparse vectors."""
|
||||
metrics = {"sparse": MetricType.IP, "sparse2": MetricType.IP}
|
||||
weights = {"sparse": 0.6, "sparse2": 0.4}
|
||||
reranker = WeightedReRanker(topn=10, metrics=metrics, weights=weights)
|
||||
reranker = WeightedReRanker(weights=[0.6, 0.4])
|
||||
result = collection_with_multiple_docs.query(
|
||||
[
|
||||
Query(field_name="sparse", vector=multiple_docs[0].vector("sparse")),
|
||||
@@ -1180,9 +1140,7 @@ class TestCollectionQuery:
|
||||
self, collection_with_multiple_docs: Collection, multiple_docs
|
||||
):
|
||||
"""Test multi-vector query with Weighted reranker combining dense + sparse."""
|
||||
metrics = {"dense": MetricType.IP, "sparse": MetricType.IP}
|
||||
weights = {"dense": 0.7, "sparse": 0.3}
|
||||
reranker = WeightedReRanker(topn=10, metrics=metrics, weights=weights)
|
||||
reranker = WeightedReRanker(weights=[0.7, 0.3])
|
||||
result = collection_with_multiple_docs.query(
|
||||
[
|
||||
Query(field_name="dense", vector=multiple_docs[0].vector("dense")),
|
||||
@@ -1203,7 +1161,7 @@ class TestCollectionQuery:
|
||||
def my_rerank_callback(query_results, topn):
|
||||
callback_invoked.append(True)
|
||||
all_docs = []
|
||||
for docs in query_results.values():
|
||||
for docs in query_results:
|
||||
all_docs.extend(docs)
|
||||
seen = set()
|
||||
unique_docs = []
|
||||
@@ -1214,7 +1172,7 @@ class TestCollectionQuery:
|
||||
unique_docs.sort(key=lambda d: d.score(), reverse=True)
|
||||
return unique_docs[:topn]
|
||||
|
||||
reranker = CallbackReRanker(callback=my_rerank_callback, topn=10)
|
||||
reranker = CallbackReRanker(callback=my_rerank_callback)
|
||||
result = collection_with_multiple_docs.query(
|
||||
[
|
||||
Query(field_name="dense", vector=multiple_docs[0].vector("dense")),
|
||||
@@ -1234,7 +1192,7 @@ class TestCollectionQuery:
|
||||
|
||||
def my_rerank_callback(query_results, topn):
|
||||
all_docs = []
|
||||
for docs in query_results.values():
|
||||
for docs in query_results:
|
||||
all_docs.extend(docs)
|
||||
seen = set()
|
||||
unique_docs = []
|
||||
@@ -1245,7 +1203,7 @@ class TestCollectionQuery:
|
||||
unique_docs.sort(key=lambda d: d.score(), reverse=True)
|
||||
return unique_docs[:topn]
|
||||
|
||||
reranker = CallbackReRanker(callback=my_rerank_callback, topn=5)
|
||||
reranker = CallbackReRanker(callback=my_rerank_callback)
|
||||
result = collection_with_multiple_docs.query(
|
||||
[
|
||||
Query(field_name="dense", vector=multiple_docs[0].vector("dense")),
|
||||
|
||||
@@ -172,7 +172,7 @@ class TestFtsOnlyCollectionLifecycle:
|
||||
class TestFtsOnlyCollectionQueryValidation:
|
||||
def test_vector_query_rejected(self, fts_collection: Collection):
|
||||
"""Vector query on a no-vector collection must raise."""
|
||||
with pytest.raises(ValueError, match="vector or id"):
|
||||
with pytest.raises(ValueError, match="No vector field found"):
|
||||
fts_collection.query(
|
||||
queries=Query(field_name="content", vector=[0.1, 0.2, 0.3]),
|
||||
topk=5,
|
||||
@@ -181,7 +181,7 @@ class TestFtsOnlyCollectionQueryValidation:
|
||||
def test_id_query_rejected(self, fts_collection: Collection):
|
||||
"""ID-based query on a no-vector collection must raise."""
|
||||
fts_collection.insert(_make_docs()[:1])
|
||||
with pytest.raises(ValueError, match="vector or id"):
|
||||
with pytest.raises(ValueError, match="No vector field found"):
|
||||
fts_collection.query(
|
||||
queries=Query(field_name="content", id="pk_0"),
|
||||
topk=5,
|
||||
|
||||
@@ -29,7 +29,6 @@ from zvec import (
|
||||
)
|
||||
from zvec.extension.multi_vector_reranker import RrfReRanker, WeightedReRanker
|
||||
from zvec.model.param.query import Fts, Query
|
||||
from zvec.typing import MetricType
|
||||
|
||||
|
||||
DIM = 16
|
||||
@@ -166,7 +165,7 @@ class TestFtsVectorHybridQuery:
|
||||
|
||||
def test_hybrid_fts_and_vector_basic(self, hybrid_collection_with_docs: Collection):
|
||||
"""FTS + vector multi-query with RRF reranker returns results."""
|
||||
reranker = RrfReRanker(topn=10, rank_constant=60)
|
||||
reranker = RrfReRanker(rank_constant=60)
|
||||
result = hybrid_collection_with_docs.query(
|
||||
queries=[
|
||||
Query(field_name="content", fts=Fts(match_string="retrieval")),
|
||||
@@ -185,7 +184,7 @@ class TestFtsVectorHybridQuery:
|
||||
self, hybrid_collection_with_docs: Collection
|
||||
):
|
||||
"""Docs relevant in both FTS and vector should rank higher."""
|
||||
reranker = RrfReRanker(topn=10, rank_constant=60)
|
||||
reranker = RrfReRanker(rank_constant=60)
|
||||
# FTS: "retrieval search" matches pk_3, pk_4
|
||||
# Vector: ret_vec cluster matches pk_3, pk_4
|
||||
# Both signals agree: pk_3 and pk_4 should rank top
|
||||
@@ -202,7 +201,7 @@ class TestFtsVectorHybridQuery:
|
||||
|
||||
def test_hybrid_scores_descending(self, hybrid_collection_with_docs: Collection):
|
||||
"""Hybrid query results must be sorted by score descending."""
|
||||
reranker = RrfReRanker(topn=10, rank_constant=60)
|
||||
reranker = RrfReRanker(rank_constant=60)
|
||||
result = hybrid_collection_with_docs.query(
|
||||
queries=[
|
||||
Query(field_name="content", fts=Fts(match_string="intelligence")),
|
||||
@@ -217,7 +216,7 @@ class TestFtsVectorHybridQuery:
|
||||
|
||||
def test_hybrid_with_filter(self, hybrid_collection_with_docs: Collection):
|
||||
"""Hybrid query respects SQL filter."""
|
||||
reranker = RrfReRanker(topn=10, rank_constant=60)
|
||||
reranker = RrfReRanker(rank_constant=60)
|
||||
result = hybrid_collection_with_docs.query(
|
||||
queries=[
|
||||
Query(field_name="content", fts=Fts(match_string="learning")),
|
||||
@@ -234,7 +233,7 @@ class TestFtsVectorHybridQuery:
|
||||
self, hybrid_collection_with_docs: Collection
|
||||
):
|
||||
"""When FTS matches nothing, vector results still appear."""
|
||||
reranker = RrfReRanker(topn=10, rank_constant=60)
|
||||
reranker = RrfReRanker(rank_constant=60)
|
||||
result = hybrid_collection_with_docs.query(
|
||||
queries=[
|
||||
Query(
|
||||
@@ -251,7 +250,7 @@ class TestFtsVectorHybridQuery:
|
||||
|
||||
def test_hybrid_query_string_syntax(self, hybrid_collection_with_docs: Collection):
|
||||
"""Hybrid query works with FTS query_string (advanced syntax)."""
|
||||
reranker = RrfReRanker(topn=10, rank_constant=60)
|
||||
reranker = RrfReRanker(rank_constant=60)
|
||||
result = hybrid_collection_with_docs.query(
|
||||
queries=[
|
||||
Query(
|
||||
@@ -283,35 +282,35 @@ class TestFtsVectorHybridValidation:
|
||||
topk=5,
|
||||
)
|
||||
|
||||
def test_duplicate_field_name_rejected(
|
||||
def test_duplicate_field_name_allowed(
|
||||
self, hybrid_collection_with_docs: Collection
|
||||
):
|
||||
"""Multi-query with duplicate field names should raise."""
|
||||
reranker = RrfReRanker(topn=10, rank_constant=60)
|
||||
with pytest.raises(ValueError, match="appears more than once"):
|
||||
hybrid_collection_with_docs.query(
|
||||
queries=[
|
||||
Query(field_name="content", fts=Fts(match_string="hello")),
|
||||
Query(field_name="content", fts=Fts(match_string="world")),
|
||||
],
|
||||
topk=5,
|
||||
reranker=reranker,
|
||||
)
|
||||
"""Multi-query with duplicate field names is allowed and returns results."""
|
||||
reranker = RrfReRanker(rank_constant=60)
|
||||
result = hybrid_collection_with_docs.query(
|
||||
queries=[
|
||||
Query(field_name="content", fts=Fts(match_string="learning")),
|
||||
Query(field_name="content", fts=Fts(match_string="intelligence")),
|
||||
],
|
||||
topk=5,
|
||||
reranker=reranker,
|
||||
)
|
||||
assert len(result) > 0
|
||||
assert len(result) <= 5
|
||||
|
||||
def test_multiple_vectors_without_fts_rejected(
|
||||
self, hybrid_collection_with_docs: Collection
|
||||
):
|
||||
"""Two vector queries on a single-vector-field collection should raise."""
|
||||
reranker = RrfReRanker(topn=10, rank_constant=60)
|
||||
with pytest.raises(ValueError, match="cannot query with multiple vectors"):
|
||||
hybrid_collection_with_docs.query(
|
||||
queries=[
|
||||
Query(field_name="embedding", vector=[1.0] * DIM),
|
||||
Query(field_name="embedding", vector=[0.5] * DIM),
|
||||
],
|
||||
topk=5,
|
||||
reranker=reranker,
|
||||
)
|
||||
def test_multiple_vectors_allowed(self, hybrid_collection_with_docs: Collection):
|
||||
"""Two vector queries on the same field are allowed with a reranker."""
|
||||
reranker = RrfReRanker(rank_constant=60)
|
||||
result = hybrid_collection_with_docs.query(
|
||||
queries=[
|
||||
Query(field_name="embedding", vector=[1.0] * DIM),
|
||||
Query(field_name="embedding", vector=[0.5] * DIM),
|
||||
],
|
||||
topk=5,
|
||||
reranker=reranker,
|
||||
)
|
||||
assert len(result) > 0
|
||||
assert len(result) <= 5
|
||||
|
||||
|
||||
class TestFtsVectorHybridWeightedReranker:
|
||||
@@ -321,9 +320,8 @@ class TestFtsVectorHybridWeightedReranker:
|
||||
self, hybrid_collection_with_docs: Collection
|
||||
):
|
||||
"""WeightedReranker correctly normalizes FTS scores alongside vector scores."""
|
||||
metrics = {"embedding": MetricType.IP}
|
||||
weights = {"content": 0.5, "embedding": 0.5}
|
||||
reranker = WeightedReRanker(topn=10, metrics=metrics, weights=weights)
|
||||
weights = [0.5, 0.5]
|
||||
reranker = WeightedReRanker(weights=weights)
|
||||
result = hybrid_collection_with_docs.query(
|
||||
queries=[
|
||||
Query(field_name="content", fts=Fts(match_string="retrieval search")),
|
||||
@@ -341,9 +339,8 @@ class TestFtsVectorHybridWeightedReranker:
|
||||
self, hybrid_collection_with_docs: Collection
|
||||
):
|
||||
"""WeightedReranker hybrid results are sorted by score descending."""
|
||||
metrics = {"embedding": MetricType.IP}
|
||||
weights = {"content": 0.4, "embedding": 0.6}
|
||||
reranker = WeightedReRanker(topn=10, metrics=metrics, weights=weights)
|
||||
weights = [0.4, 0.6]
|
||||
reranker = WeightedReRanker(weights=weights)
|
||||
result = hybrid_collection_with_docs.query(
|
||||
queries=[
|
||||
Query(field_name="content", fts=Fts(match_string="intelligence")),
|
||||
@@ -361,11 +358,8 @@ class TestFtsVectorHybridWeightedReranker:
|
||||
):
|
||||
"""Higher FTS weight should boost FTS-relevant docs in ranking."""
|
||||
# High FTS weight: FTS signal dominates
|
||||
metrics = {"embedding": MetricType.IP}
|
||||
weights_fts_heavy = {"content": 0.9, "embedding": 0.1}
|
||||
reranker_fts = WeightedReRanker(
|
||||
topn=10, metrics=metrics, weights=weights_fts_heavy
|
||||
)
|
||||
weights_fts_heavy = [0.9, 0.1]
|
||||
reranker_fts = WeightedReRanker(weights=weights_fts_heavy)
|
||||
result_fts = hybrid_collection_with_docs.query(
|
||||
queries=[
|
||||
Query(field_name="content", fts=Fts(match_string="retrieval")),
|
||||
@@ -376,10 +370,8 @@ class TestFtsVectorHybridWeightedReranker:
|
||||
)
|
||||
|
||||
# High vector weight: vector signal dominates
|
||||
weights_vec_heavy = {"content": 0.1, "embedding": 0.9}
|
||||
reranker_vec = WeightedReRanker(
|
||||
topn=10, metrics=metrics, weights=weights_vec_heavy
|
||||
)
|
||||
weights_vec_heavy = [0.1, 0.9]
|
||||
reranker_vec = WeightedReRanker(weights=weights_vec_heavy)
|
||||
result_vec = hybrid_collection_with_docs.query(
|
||||
queries=[
|
||||
Query(field_name="content", fts=Fts(match_string="retrieval")),
|
||||
|
||||
@@ -110,11 +110,11 @@ class TestFtsQueryBinding:
|
||||
assert restored.query_string == "+vector search"
|
||||
assert restored.match_string == ""
|
||||
|
||||
def test_vector_query_fts_field(self):
|
||||
"""_VectorQuery should have fts field."""
|
||||
from _zvec.param import _Fts, _VectorQuery
|
||||
def test_search_query_fts_field(self):
|
||||
"""_SearchQuery should have fts field."""
|
||||
from _zvec.param import _Fts, _SearchQuery
|
||||
|
||||
vq = _VectorQuery()
|
||||
vq = _SearchQuery()
|
||||
# fts should be None by default (optional)
|
||||
assert vq.fts is None
|
||||
|
||||
@@ -125,11 +125,11 @@ class TestFtsQueryBinding:
|
||||
assert vq.fts is not None
|
||||
assert vq.fts.query_string == "hello"
|
||||
|
||||
def test_vector_query_pickle_with_fts(self):
|
||||
"""_VectorQuery with fts should survive pickling."""
|
||||
from _zvec.param import _Fts, _VectorQuery
|
||||
def test_search_query_pickle_with_fts(self):
|
||||
"""_SearchQuery with fts should survive pickling."""
|
||||
from _zvec.param import _Fts, _SearchQuery
|
||||
|
||||
vq = _VectorQuery()
|
||||
vq = _SearchQuery()
|
||||
vq.topk = 10
|
||||
vq.field_name = "embedding"
|
||||
fts = _Fts()
|
||||
@@ -143,11 +143,11 @@ class TestFtsQueryBinding:
|
||||
assert restored.fts is not None
|
||||
assert restored.fts.match_string == "test query"
|
||||
|
||||
def test_vector_query_pickle_without_fts(self):
|
||||
"""_VectorQuery without fts should survive pickling."""
|
||||
from _zvec.param import _VectorQuery
|
||||
def test_search_query_pickle_without_fts(self):
|
||||
"""_SearchQuery without fts should survive pickling."""
|
||||
from _zvec.param import _SearchQuery
|
||||
|
||||
vq = _VectorQuery()
|
||||
vq = _SearchQuery()
|
||||
vq.topk = 5
|
||||
vq.field_name = "vec"
|
||||
|
||||
|
||||
@@ -40,7 +40,7 @@ from zvec import (
|
||||
VectorSchema,
|
||||
)
|
||||
|
||||
from _zvec.param import _VectorQuery
|
||||
from _zvec.param import _SearchQuery
|
||||
|
||||
# ----------------------------
|
||||
# Invert Index Param Test Case
|
||||
|
||||
@@ -14,20 +14,16 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Dict, Union
|
||||
from unittest.mock import MagicMock
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import numpy as np
|
||||
import math
|
||||
from _zvec.param import _VectorQuery
|
||||
from _zvec.param import _SearchQuery
|
||||
|
||||
import pytest
|
||||
from zvec.executor.query_executor import (
|
||||
MultiVectorQueryExecutor,
|
||||
NoVectorQueryExecutor,
|
||||
QueryContext,
|
||||
QueryExecutor,
|
||||
QueryExecutorFactory,
|
||||
SingleVectorQueryExecutor,
|
||||
)
|
||||
from zvec import (
|
||||
RrfReRanker,
|
||||
@@ -43,21 +39,6 @@ from zvec import (
|
||||
from zvec.extension.multi_vector_reranker import CallbackReRanker
|
||||
|
||||
|
||||
# ----------------------------
|
||||
# Mock Vector Schema
|
||||
# ----------------------------
|
||||
class MockVectorSchema(VectorSchema):
|
||||
def __init__(self, name="test_vector"):
|
||||
self._name = name
|
||||
|
||||
@property
|
||||
def name(self):
|
||||
return self._name
|
||||
|
||||
def _get_object(self):
|
||||
return MagicMock()
|
||||
|
||||
|
||||
# ----------------------------
|
||||
# Mock Collection Schema
|
||||
# ----------------------------
|
||||
@@ -110,7 +91,7 @@ class TestQuery:
|
||||
assert query.has_vector()
|
||||
|
||||
def test_validate_dense_fp16_convert(self):
|
||||
v = _VectorQuery()
|
||||
v = _SearchQuery()
|
||||
schema = VectorSchema(name="test", data_type=DataType.VECTOR_FP16)
|
||||
vec = np.array([1.1, 2.1, 3.1], dtype=np.float16)
|
||||
v.set_vector(schema._get_object(), vec)
|
||||
@@ -118,7 +99,7 @@ class TestQuery:
|
||||
assert np.array_equal(vec, ret)
|
||||
|
||||
def test_validate_dense_fp32_convert(self):
|
||||
v = _VectorQuery()
|
||||
v = _SearchQuery()
|
||||
schema = VectorSchema(name="test", data_type=DataType.VECTOR_FP32)
|
||||
vec = np.array([1.1, 2.1, 3.1], dtype=np.float32)
|
||||
v.set_vector(schema._get_object(), vec)
|
||||
@@ -126,7 +107,7 @@ class TestQuery:
|
||||
assert np.array_equal(vec, ret)
|
||||
|
||||
def test_validate_dense_fp64_convert(self):
|
||||
v = _VectorQuery()
|
||||
v = _SearchQuery()
|
||||
schema = VectorSchema(name="test", data_type=DataType.VECTOR_FP64)
|
||||
vec = np.array([1.1, 2.1, 3.1], dtype=np.float64)
|
||||
v.set_vector(schema._get_object(), vec)
|
||||
@@ -134,7 +115,7 @@ class TestQuery:
|
||||
assert np.array_equal(vec, ret)
|
||||
|
||||
def test_validate_dense_int8_convert(self):
|
||||
v = _VectorQuery()
|
||||
v = _SearchQuery()
|
||||
schema = VectorSchema(name="test", data_type=DataType.VECTOR_INT8)
|
||||
vec = np.array([1, 2, 3], dtype=np.int8)
|
||||
v.set_vector(schema._get_object(), vec)
|
||||
@@ -142,7 +123,7 @@ class TestQuery:
|
||||
assert np.array_equal(vec, ret)
|
||||
|
||||
def test_validate_sparse_fp32_convert(self):
|
||||
v = _VectorQuery()
|
||||
v = _SearchQuery()
|
||||
schema = VectorSchema(name="test", data_type=DataType.SPARSE_VECTOR_FP32)
|
||||
vec = {1: 1.1, 2: 2.2, 3: 3.3}
|
||||
v.set_vector(schema._get_object(), vec)
|
||||
@@ -151,7 +132,7 @@ class TestQuery:
|
||||
assert math.isclose(vec[k], ret[k], abs_tol=1e-6)
|
||||
|
||||
def test_validate_sparse_fp16_convert(self):
|
||||
v = _VectorQuery()
|
||||
v = _SearchQuery()
|
||||
schema = VectorSchema(name="test", data_type=DataType.SPARSE_VECTOR_FP16)
|
||||
vec = {1: 1.1, 2: 2.2, 3: 3.3}
|
||||
v.set_vector(schema._get_object(), vec)
|
||||
@@ -189,7 +170,6 @@ class TestQueryContext:
|
||||
assert ctx.reranker is None
|
||||
assert ctx.output_fields is None
|
||||
assert ctx.include_vector is False
|
||||
assert ctx.core_vectors == []
|
||||
|
||||
def test_properties(self):
|
||||
queries = [Query(field_name="test")]
|
||||
@@ -215,9 +195,7 @@ class TestQueryContext:
|
||||
def test_properties_with_weighted_reranker(self):
|
||||
queries = [Query(field_name="test")]
|
||||
reranker = WeightedReRanker(
|
||||
topn=10,
|
||||
metrics={"test": MetricType.L2},
|
||||
weights={"test": 1.0},
|
||||
weights=[1.0],
|
||||
)
|
||||
|
||||
ctx = QueryContext(
|
||||
@@ -227,13 +205,12 @@ class TestQueryContext:
|
||||
)
|
||||
|
||||
assert ctx.reranker == reranker
|
||||
assert ctx.reranker.weights == {"test": 1.0}
|
||||
assert ctx.reranker.metrics == {"test": MetricType.L2}
|
||||
assert ctx.reranker.weights == [1.0]
|
||||
|
||||
def test_properties_with_callback_reranker(self):
|
||||
queries = [Query(field_name="test")]
|
||||
cb = lambda query_results, topn: []
|
||||
reranker = CallbackReRanker(callback=cb, topn=10)
|
||||
reranker = CallbackReRanker(callback=cb)
|
||||
|
||||
ctx = QueryContext(
|
||||
topk=5,
|
||||
@@ -243,141 +220,83 @@ class TestQueryContext:
|
||||
|
||||
assert ctx.reranker == reranker
|
||||
|
||||
def test_core_vectors_setter(self):
|
||||
ctx = QueryContext(topk=10)
|
||||
core_vectors = [MagicMock()]
|
||||
ctx.core_vectors = core_vectors
|
||||
assert ctx.core_vectors == core_vectors
|
||||
|
||||
|
||||
class TestNoVectorQueryExecutor:
|
||||
class TestQueryExecutor:
|
||||
def test_init(self):
|
||||
schema = MockCollectionSchema()
|
||||
executor = NoVectorQueryExecutor(schema)
|
||||
executor = QueryExecutor(schema)
|
||||
assert isinstance(executor, QueryExecutor)
|
||||
|
||||
def test_do_validate_with_queries(self):
|
||||
def test_do_build_without_queries(self):
|
||||
# When no queries are given, build a single vector-less query.
|
||||
schema = MockCollectionSchema()
|
||||
executor = NoVectorQueryExecutor(schema)
|
||||
ctx = QueryContext(
|
||||
topk=10, queries=[Query(field_name="test", vector=[0.1, 0.2, 0.3])]
|
||||
)
|
||||
|
||||
with pytest.raises(
|
||||
ValueError, match="Collection does not support query with vector or id"
|
||||
):
|
||||
executor._do_validate(ctx)
|
||||
|
||||
def test_do_validate_without_queries(self):
|
||||
schema = MockCollectionSchema()
|
||||
executor = NoVectorQueryExecutor(schema)
|
||||
ctx = QueryContext(topk=10)
|
||||
|
||||
executor._do_validate(ctx)
|
||||
|
||||
def test_do_build(self):
|
||||
schema = MockCollectionSchema()
|
||||
executor = NoVectorQueryExecutor(schema)
|
||||
executor = QueryExecutor(schema)
|
||||
ctx = QueryContext(topk=5, filter="test_filter")
|
||||
|
||||
result = executor._do_build(ctx, MagicMock())
|
||||
result = executor._build_queries(ctx, MagicMock())
|
||||
assert len(result) == 1
|
||||
assert result[0].topk == 5
|
||||
assert result[0].filter == "test_filter"
|
||||
|
||||
|
||||
class TestSingleVectorQueryExecutor:
|
||||
def test_init(self):
|
||||
def test_do_build_query_wo_vector(self):
|
||||
# Vector-less core query should carry the context query params.
|
||||
schema = MockCollectionSchema()
|
||||
executor = SingleVectorQueryExecutor(schema)
|
||||
assert isinstance(executor, NoVectorQueryExecutor)
|
||||
executor = QueryExecutor(schema)
|
||||
ctx = QueryContext(topk=7, filter="f", include_vector=True)
|
||||
|
||||
def test_do_validate_multiple_queries(self):
|
||||
core_vector = executor._build_base_search_query(ctx)
|
||||
assert core_vector.topk == 7
|
||||
assert core_vector.filter == "f"
|
||||
assert core_vector.include_vector is True
|
||||
|
||||
def test_do_merge_rerank_results_single_without_reranker(self):
|
||||
# A single result list without a reranker is returned as-is.
|
||||
schema = MockCollectionSchema()
|
||||
executor = SingleVectorQueryExecutor(schema)
|
||||
queries = [Query(field_name="test1"), Query(field_name="test2")]
|
||||
ctx = QueryContext(topk=10, queries=queries)
|
||||
executor = QueryExecutor(schema)
|
||||
ctx = QueryContext(topk=5)
|
||||
docs_list = [["doc1", "doc2"]]
|
||||
|
||||
with pytest.raises(
|
||||
ValueError,
|
||||
match="Collection has only one vector field, cannot query with multiple vectors",
|
||||
):
|
||||
executor._do_validate(ctx)
|
||||
result = executor._merge_and_rerank(ctx, docs_list)
|
||||
assert result == ["doc1", "doc2"]
|
||||
|
||||
def test_do_build_without_queries(self):
|
||||
def test_do_merge_rerank_results_empty(self):
|
||||
# Empty results should raise an error.
|
||||
schema = MockCollectionSchema()
|
||||
executor = SingleVectorQueryExecutor(schema)
|
||||
executor = QueryExecutor(schema)
|
||||
ctx = QueryContext(topk=5)
|
||||
|
||||
result = executor._do_build(ctx, MagicMock())
|
||||
assert len(result) == 1
|
||||
assert result[0].topk == 5
|
||||
with pytest.raises(ValueError, match="Query results is empty"):
|
||||
executor._merge_and_rerank(ctx, [])
|
||||
|
||||
|
||||
class TestMultiVectorQueryExecutor:
|
||||
def test_init(self):
|
||||
def test_do_merge_rerank_results_with_reranker(self):
|
||||
# Multiple result lists are merged through the reranker.
|
||||
schema = MockCollectionSchema()
|
||||
executor = MultiVectorQueryExecutor(schema)
|
||||
assert isinstance(executor, SingleVectorQueryExecutor)
|
||||
|
||||
def test_do_validate_multiple_queries_without_reranker(self):
|
||||
schema = MockCollectionSchema()
|
||||
executor = MultiVectorQueryExecutor(schema)
|
||||
queries = [Query(field_name="test1"), Query(field_name="test2")]
|
||||
ctx = QueryContext(topk=10, queries=queries)
|
||||
|
||||
with pytest.raises(ValueError, match="Reranker is required for multi-query"):
|
||||
executor._do_validate(ctx)
|
||||
|
||||
def test_do_validate_multiple_queries_with_reranker(self):
|
||||
schema = MockCollectionSchema()
|
||||
executor = MultiVectorQueryExecutor(schema)
|
||||
queries = [Query(field_name="test1"), Query(field_name="test2")]
|
||||
reranker = RrfReRanker()
|
||||
ctx = QueryContext(topk=10, queries=queries, reranker=reranker)
|
||||
|
||||
executor._do_validate(ctx)
|
||||
|
||||
def test_do_validate_multiple_queries_with_weighted_reranker(self):
|
||||
schema = MockCollectionSchema()
|
||||
executor = MultiVectorQueryExecutor(schema)
|
||||
queries = [Query(field_name="test1"), Query(field_name="test2")]
|
||||
reranker = WeightedReRanker(
|
||||
topn=10,
|
||||
metrics={"test1": MetricType.L2, "test2": MetricType.L2},
|
||||
weights={"test1": 0.7, "test2": 0.3},
|
||||
executor = QueryExecutor(schema)
|
||||
reranker = MagicMock()
|
||||
reranker.rerank.return_value = ["merged"]
|
||||
ctx = QueryContext(
|
||||
topk=5,
|
||||
queries=[Query(field_name="test1"), Query(field_name="test2")],
|
||||
reranker=reranker,
|
||||
)
|
||||
ctx = QueryContext(topk=10, queries=queries, reranker=reranker)
|
||||
docs_list = [["d1"], ["d2"]]
|
||||
|
||||
executor._do_validate(ctx)
|
||||
result = executor._merge_and_rerank(ctx, docs_list)
|
||||
assert result == ["merged"]
|
||||
reranker.rerank.assert_called_once_with(docs_list, ctx.topk)
|
||||
|
||||
def test_do_validate_multiple_queries_with_callback_reranker(self):
|
||||
def test_execute_python_pipeline(self):
|
||||
# Each query is executed serially and converted into a result list.
|
||||
schema = MockCollectionSchema()
|
||||
executor = MultiVectorQueryExecutor(schema)
|
||||
queries = [Query(field_name="test1"), Query(field_name="test2")]
|
||||
reranker = CallbackReRanker(
|
||||
callback=lambda query_results, topn: [],
|
||||
topn=10,
|
||||
)
|
||||
ctx = QueryContext(topk=10, queries=queries, reranker=reranker)
|
||||
executor = QueryExecutor(schema)
|
||||
collection = MagicMock()
|
||||
collection.Query.side_effect = [["raw1"], ["raw2"]]
|
||||
vectors = [MagicMock(), MagicMock()]
|
||||
|
||||
executor._do_validate(ctx)
|
||||
|
||||
|
||||
class TestQueryExecutorFactory:
|
||||
def test_create_no_vectors(self):
|
||||
schema = MockCollectionSchema()
|
||||
executor = QueryExecutorFactory.create(schema)
|
||||
assert isinstance(executor, NoVectorQueryExecutor)
|
||||
|
||||
def test_create_single_vector(self):
|
||||
schema = MockCollectionSchema(vectors=MockVectorSchema())
|
||||
executor = QueryExecutorFactory.create(schema)
|
||||
assert isinstance(executor, SingleVectorQueryExecutor)
|
||||
|
||||
def test_create_multiple_vectors(self):
|
||||
schema = MockCollectionSchema(
|
||||
vectors={"test1": MockVectorSchema(), "test2": MockVectorSchema()}
|
||||
)
|
||||
executor = QueryExecutorFactory.create(schema)
|
||||
assert isinstance(executor, MultiVectorQueryExecutor)
|
||||
with patch(
|
||||
"zvec.executor.query_executor.convert_to_py_doc",
|
||||
side_effect=lambda doc, schema: doc,
|
||||
):
|
||||
results = executor._execute_python_pipeline(vectors, collection)
|
||||
assert results == [["raw1"], ["raw2"]]
|
||||
assert collection.Query.call_count == 2
|
||||
|
||||
+142
-178
@@ -15,10 +15,9 @@ from __future__ import annotations
|
||||
|
||||
from unittest.mock import patch, MagicMock
|
||||
import pytest
|
||||
import math
|
||||
import os
|
||||
|
||||
from zvec import Doc, MetricType
|
||||
from zvec import Doc
|
||||
from zvec.extension.multi_vector_reranker import (
|
||||
CallbackReRanker,
|
||||
RrfReRanker,
|
||||
@@ -38,37 +37,27 @@ RUN_INTEGRATION_TESTS = os.environ.get("ZVEC_RUN_INTEGRATION_TESTS", "0") == "1"
|
||||
# ----------------------------
|
||||
class TestRrfReRanker:
|
||||
def test_init(self):
|
||||
reranker = RrfReRanker(topn=5, rerank_field="content", rank_constant=100)
|
||||
assert reranker.topn == 5
|
||||
assert reranker.rerank_field == "content"
|
||||
reranker = RrfReRanker(rank_constant=100)
|
||||
assert reranker.rank_constant == 100
|
||||
|
||||
def test_rrf_score(self):
|
||||
reranker = RrfReRanker(rank_constant=60)
|
||||
# 根据公式 1.0 / (k + rank + 1),其中k=60
|
||||
assert reranker._rrf_score(0) == 1.0 / (60 + 0 + 1)
|
||||
assert reranker._rrf_score(1) == 1.0 / (60 + 1 + 1)
|
||||
assert reranker._rrf_score(10) == 1.0 / (60 + 10 + 1)
|
||||
|
||||
def test_rerank(self):
|
||||
reranker = RrfReRanker(topn=3)
|
||||
def test_rerank_delegates_to_cpp(self):
|
||||
"""RrfReRanker.rerank() delegates to C++ (raises TypeError with Python Docs)."""
|
||||
reranker = RrfReRanker()
|
||||
|
||||
doc1 = Doc(id="1", score=0.8)
|
||||
doc2 = Doc(id="2", score=0.7)
|
||||
doc3 = Doc(id="3", score=0.9)
|
||||
doc4 = Doc(id="4", score=0.6)
|
||||
|
||||
query_results = {"vector1": [doc1, doc2, doc3], "vector2": [doc3, doc1, doc4]}
|
||||
query_results = [[doc1, doc2, doc3], [doc3, doc1, doc4]]
|
||||
|
||||
results = reranker.rerank(query_results)
|
||||
with pytest.raises((TypeError, RuntimeError)):
|
||||
reranker.rerank(query_results, topn=3)
|
||||
|
||||
assert len(results) <= reranker.topn
|
||||
|
||||
for doc in results:
|
||||
assert hasattr(doc, "score")
|
||||
|
||||
scores = [doc.score for doc in results]
|
||||
assert scores == sorted(scores, reverse=True)
|
||||
def test_get_object_returns_cpp_reranker(self):
|
||||
"""_get_object() returns a valid C++ reranker instance."""
|
||||
reranker = RrfReRanker()
|
||||
assert reranker._get_object() is not None
|
||||
|
||||
|
||||
# ----------------------------
|
||||
@@ -76,64 +65,30 @@ class TestRrfReRanker:
|
||||
# ----------------------------
|
||||
class TestWeightedReRanker:
|
||||
def test_init(self):
|
||||
metrics = {"vector1": MetricType.L2, "vector2": MetricType.COSINE}
|
||||
weights = {"vector1": 0.7, "vector2": 0.3}
|
||||
weights = [0.7, 0.3]
|
||||
reranker = WeightedReRanker(
|
||||
topn=5,
|
||||
rerank_field="content",
|
||||
metrics=metrics,
|
||||
weights=weights,
|
||||
)
|
||||
assert reranker.topn == 5
|
||||
assert reranker.rerank_field == "content"
|
||||
assert reranker.metrics == metrics
|
||||
assert reranker.weights == weights
|
||||
assert list(reranker.weights) == weights
|
||||
|
||||
def test_normalize_score(self):
|
||||
reranker = WeightedReRanker()
|
||||
|
||||
score = reranker._normalize_score(1.0, MetricType.L2)
|
||||
expected = 1.0 - 2 * math.atan(1.0) / math.pi
|
||||
assert score == expected
|
||||
|
||||
score = reranker._normalize_score(1.0, MetricType.IP)
|
||||
expected = 0.5 + math.atan(1.0) / math.pi
|
||||
assert score == expected
|
||||
|
||||
score = reranker._normalize_score(1.0, MetricType.COSINE)
|
||||
expected = 1.0 - 1.0 / 2.0
|
||||
assert score == expected
|
||||
|
||||
with pytest.raises(ValueError, match="Unsupported metric type"):
|
||||
reranker._normalize_score(1.0, "unsupported_metric")
|
||||
|
||||
def test_rerank(self):
|
||||
metrics = {"vector1": MetricType.L2, "vector2": MetricType.L2}
|
||||
weights = {"vector1": 0.7, "vector2": 0.3}
|
||||
reranker = WeightedReRanker(topn=3, weights=weights, metrics=metrics)
|
||||
def test_rerank_delegates_to_cpp(self):
|
||||
"""WeightedReRanker.rerank() delegates to C++ (raises TypeError with Python Docs)."""
|
||||
weights = [0.7, 0.3]
|
||||
reranker = WeightedReRanker(weights=weights)
|
||||
|
||||
doc1 = Doc(id="1", score=0.8)
|
||||
doc2 = Doc(id="2", score=0.7)
|
||||
doc3 = Doc(id="3", score=0.9)
|
||||
|
||||
query_results = {"vector1": [doc1, doc2], "vector2": [doc2, doc3]}
|
||||
query_results = [[doc1, doc2], [doc2, doc3]]
|
||||
|
||||
results = reranker.rerank(query_results)
|
||||
with pytest.raises((TypeError, RuntimeError)):
|
||||
reranker.rerank(query_results, topn=3)
|
||||
|
||||
assert len(results) <= reranker.topn
|
||||
|
||||
for doc in results:
|
||||
assert hasattr(doc, "score")
|
||||
|
||||
def test_rerank_missing_metric_raises(self):
|
||||
metrics = {"vector1": MetricType.L2}
|
||||
reranker = WeightedReRanker(topn=3, metrics=metrics)
|
||||
|
||||
doc1 = Doc(id="1", score=0.8)
|
||||
query_results = {"vector1": [doc1], "vector2": [doc1]}
|
||||
|
||||
with pytest.raises(ValueError, match="no metric type specified"):
|
||||
reranker.rerank(query_results)
|
||||
def test_get_object_returns_cpp_reranker(self):
|
||||
"""_get_object() returns a valid C++ reranker instance."""
|
||||
reranker = WeightedReRanker(weights=[0.5, 0.5])
|
||||
assert reranker._get_object() is not None
|
||||
|
||||
|
||||
# ----------------------------
|
||||
@@ -144,27 +99,27 @@ class TestCallbackReRanker:
|
||||
def my_callback(query_results, topn):
|
||||
return []
|
||||
|
||||
reranker = CallbackReRanker(callback=my_callback, topn=5)
|
||||
assert reranker.topn == 5
|
||||
reranker = CallbackReRanker(callback=my_callback)
|
||||
assert reranker._get_object() is not None
|
||||
|
||||
def test_rerank(self):
|
||||
def my_callback(query_results, topn):
|
||||
all_docs = []
|
||||
for docs in query_results.values():
|
||||
for docs in query_results:
|
||||
all_docs.extend(docs)
|
||||
all_docs.sort(key=lambda d: d.score, reverse=True)
|
||||
return all_docs[:topn]
|
||||
|
||||
reranker = CallbackReRanker(callback=my_callback, topn=3)
|
||||
reranker = CallbackReRanker(callback=my_callback)
|
||||
|
||||
doc1 = Doc(id="1", score=0.8)
|
||||
doc2 = Doc(id="2", score=0.9)
|
||||
doc3 = Doc(id="3", score=0.7)
|
||||
doc4 = Doc(id="4", score=0.6)
|
||||
|
||||
query_results = {"vector1": [doc1, doc2], "vector2": [doc3, doc4]}
|
||||
query_results = [[doc1, doc2], [doc3, doc4]]
|
||||
|
||||
results = reranker.rerank(query_results)
|
||||
results = reranker.rerank(query_results, topn=3)
|
||||
|
||||
assert len(results) == 3
|
||||
scores = [doc.score for doc in results]
|
||||
@@ -177,8 +132,8 @@ class TestCallbackReRanker:
|
||||
received_topn.append(topn)
|
||||
return []
|
||||
|
||||
reranker = CallbackReRanker(callback=my_callback, topn=7)
|
||||
reranker.rerank({"v1": [Doc(id="1", score=0.5)]})
|
||||
reranker = CallbackReRanker(callback=my_callback)
|
||||
reranker.rerank([[Doc(id="1", score=0.5)]], topn=7)
|
||||
|
||||
assert received_topn == [7]
|
||||
|
||||
@@ -237,12 +192,6 @@ class TestQwenReRanker:
|
||||
)
|
||||
assert reranker.query == "test query"
|
||||
|
||||
def test_topn_property(self):
|
||||
reranker = QwenReRanker(
|
||||
query="test", topn=5, api_key="test_key", rerank_field="content"
|
||||
)
|
||||
assert reranker.topn == 5
|
||||
|
||||
def test_rerank_field_property(self):
|
||||
reranker = QwenReRanker(query="test", api_key="test_key", rerank_field="title")
|
||||
assert reranker.rerank_field == "title"
|
||||
@@ -251,7 +200,7 @@ class TestQwenReRanker:
|
||||
reranker = QwenReRanker(
|
||||
query="test", api_key="test_key", rerank_field="content"
|
||||
)
|
||||
results = reranker.rerank({})
|
||||
results = reranker.rerank([], topn=10)
|
||||
assert results == []
|
||||
|
||||
def test_rerank_no_valid_documents(self):
|
||||
@@ -259,22 +208,22 @@ class TestQwenReRanker:
|
||||
query="test", api_key="test_key", rerank_field="content"
|
||||
)
|
||||
# Document without the rerank_field
|
||||
query_results = {"vector1": [Doc(id="1")]}
|
||||
query_results = [[Doc(id="1")]]
|
||||
with pytest.raises(ValueError, match="No documents to rerank"):
|
||||
reranker.rerank(query_results)
|
||||
reranker.rerank(query_results, topn=10)
|
||||
|
||||
def test_rerank_skip_empty_content(self):
|
||||
reranker = QwenReRanker(
|
||||
query="test", api_key="test_key", rerank_field="content"
|
||||
)
|
||||
query_results = {
|
||||
"vector1": [
|
||||
query_results = [
|
||||
[
|
||||
Doc(id="1", fields={"content": ""}),
|
||||
Doc(id="2", fields={"content": " "}),
|
||||
]
|
||||
}
|
||||
]
|
||||
with pytest.raises(ValueError, match="No documents to rerank"):
|
||||
reranker.rerank(query_results)
|
||||
reranker.rerank(query_results, topn=10)
|
||||
|
||||
@patch("zvec.extension.qwen_function.require_module")
|
||||
def test_rerank_success(self, mock_require_module):
|
||||
@@ -294,17 +243,17 @@ class TestQwenReRanker:
|
||||
mock_dashscope.TextReRank.call.return_value = mock_response
|
||||
|
||||
reranker = QwenReRanker(
|
||||
query="test query", topn=2, api_key="test_key", rerank_field="content"
|
||||
query="test query", api_key="test_key", rerank_field="content"
|
||||
)
|
||||
|
||||
query_results = {
|
||||
"vector1": [
|
||||
query_results = [
|
||||
[
|
||||
Doc(id="1", fields={"content": "Document 1"}),
|
||||
Doc(id="2", fields={"content": "Document 2"}),
|
||||
]
|
||||
}
|
||||
]
|
||||
|
||||
results = reranker.rerank(query_results)
|
||||
results = reranker.rerank(query_results, topn=2)
|
||||
|
||||
assert len(results) == 2
|
||||
assert results[0].id == "1"
|
||||
@@ -338,14 +287,14 @@ class TestQwenReRanker:
|
||||
mock_dashscope.TextReRank.call.return_value = mock_response
|
||||
|
||||
reranker = QwenReRanker(
|
||||
query="test", topn=5, api_key="test_key", rerank_field="content"
|
||||
query="test", api_key="test_key", rerank_field="content"
|
||||
)
|
||||
|
||||
# Same document in multiple vector results
|
||||
doc1 = Doc(id="1", fields={"content": "Document 1"})
|
||||
query_results = {"vector1": [doc1], "vector2": [doc1]}
|
||||
query_results = [[doc1], [doc1]]
|
||||
|
||||
results = reranker.rerank(query_results)
|
||||
results = reranker.rerank(query_results, topn=5)
|
||||
|
||||
# Should only call API with document once
|
||||
call_args = mock_dashscope.TextReRank.call.call_args
|
||||
@@ -368,10 +317,10 @@ class TestQwenReRanker:
|
||||
query="test", api_key="test_key", rerank_field="content"
|
||||
)
|
||||
|
||||
query_results = {"vector1": [Doc(id="1", fields={"content": "Document 1"})]}
|
||||
query_results = [[Doc(id="1", fields={"content": "Document 1"})]]
|
||||
|
||||
with pytest.raises(ValueError, match="DashScope API error"):
|
||||
reranker.rerank(query_results)
|
||||
reranker.rerank(query_results, topn=10)
|
||||
|
||||
@patch("zvec.extension.qwen_function.require_module")
|
||||
def test_rerank_runtime_error(self, mock_require_module):
|
||||
@@ -384,10 +333,10 @@ class TestQwenReRanker:
|
||||
query="test", api_key="test_key", rerank_field="content"
|
||||
)
|
||||
|
||||
query_results = {"vector1": [Doc(id="1", fields={"content": "Document 1"})]}
|
||||
query_results = [[Doc(id="1", fields={"content": "Document 1"})]]
|
||||
|
||||
with pytest.raises(RuntimeError, match="Failed to call DashScope API"):
|
||||
reranker.rerank(query_results)
|
||||
reranker.rerank(query_results, topn=10)
|
||||
|
||||
@pytest.mark.skipif(
|
||||
not RUN_INTEGRATION_TESTS,
|
||||
@@ -403,14 +352,13 @@ class TestQwenReRanker:
|
||||
# Create reranker with real API
|
||||
reranker = QwenReRanker(
|
||||
query="What is machine learning?",
|
||||
topn=3,
|
||||
rerank_field="content",
|
||||
model="gte-rerank-v2",
|
||||
)
|
||||
|
||||
# Prepare test documents
|
||||
query_results = {
|
||||
"vector1": [
|
||||
query_results = [
|
||||
[
|
||||
Doc(
|
||||
id="1",
|
||||
score=0.8,
|
||||
@@ -433,7 +381,7 @@ class TestQwenReRanker:
|
||||
},
|
||||
),
|
||||
],
|
||||
"vector2": [
|
||||
[
|
||||
Doc(
|
||||
id="4",
|
||||
score=0.6,
|
||||
@@ -449,10 +397,10 @@ class TestQwenReRanker:
|
||||
},
|
||||
),
|
||||
],
|
||||
}
|
||||
]
|
||||
|
||||
# Call real API
|
||||
results = reranker.rerank(query_results)
|
||||
results = reranker.rerank(query_results, topn=3)
|
||||
|
||||
# Verify results
|
||||
assert len(results) <= 3, "Should return at most topn documents"
|
||||
@@ -523,13 +471,11 @@ class TestDefaultLocalReRanker:
|
||||
|
||||
reranker = DefaultLocalReRanker(
|
||||
query="test query",
|
||||
topn=5,
|
||||
rerank_field="content",
|
||||
model_name="cross-encoder/ms-marco-MiniLM-L6-v2",
|
||||
)
|
||||
|
||||
assert reranker.query == "test query"
|
||||
assert reranker.topn == 5
|
||||
assert reranker.rerank_field == "content"
|
||||
assert reranker.model_name == "cross-encoder/ms-marco-MiniLM-L6-v2"
|
||||
assert reranker.model_source == "huggingface"
|
||||
@@ -551,7 +497,6 @@ class TestDefaultLocalReRanker:
|
||||
|
||||
reranker = DefaultLocalReRanker(
|
||||
query="custom query",
|
||||
topn=10,
|
||||
rerank_field="title",
|
||||
model_name="cross-encoder/ms-marco-MiniLM-L12-v2",
|
||||
model_source="modelscope",
|
||||
@@ -560,7 +505,6 @@ class TestDefaultLocalReRanker:
|
||||
)
|
||||
|
||||
assert reranker.query == "custom query"
|
||||
assert reranker.topn == 10
|
||||
assert reranker.rerank_field == "title"
|
||||
assert reranker.model_name == "cross-encoder/ms-marco-MiniLM-L12-v2"
|
||||
assert reranker.model_source == "modelscope"
|
||||
@@ -593,23 +537,6 @@ class TestDefaultLocalReRanker:
|
||||
reranker = DefaultLocalReRanker(query="test query", rerank_field="content")
|
||||
assert reranker.query == "test query"
|
||||
|
||||
def test_topn_property(self):
|
||||
"""Test topn property."""
|
||||
mock_model = MagicMock()
|
||||
mock_model.predict = MagicMock()
|
||||
|
||||
mock_st = MagicMock()
|
||||
mock_st.CrossEncoder.return_value = mock_model
|
||||
|
||||
with patch(
|
||||
"zvec.extension.sentence_transformer_rerank_function.require_module",
|
||||
return_value=mock_st,
|
||||
):
|
||||
reranker = DefaultLocalReRanker(
|
||||
query="test", topn=15, rerank_field="content"
|
||||
)
|
||||
assert reranker.topn == 15
|
||||
|
||||
def test_rerank_field_property(self):
|
||||
"""Test rerank_field property."""
|
||||
mock_model = MagicMock()
|
||||
@@ -655,7 +582,7 @@ class TestDefaultLocalReRanker:
|
||||
return_value=mock_st,
|
||||
):
|
||||
reranker = DefaultLocalReRanker(query="test", rerank_field="content")
|
||||
results = reranker.rerank({})
|
||||
results = reranker.rerank([], topn=10)
|
||||
assert results == []
|
||||
|
||||
def test_rerank_no_valid_documents(self):
|
||||
@@ -673,9 +600,9 @@ class TestDefaultLocalReRanker:
|
||||
reranker = DefaultLocalReRanker(query="test", rerank_field="content")
|
||||
|
||||
# Document without the rerank_field
|
||||
query_results = {"vector1": [Doc(id="1")]}
|
||||
query_results = [[Doc(id="1")]]
|
||||
with pytest.raises(ValueError, match="No documents to rerank"):
|
||||
reranker.rerank(query_results)
|
||||
reranker.rerank(query_results, topn=10)
|
||||
|
||||
def test_rerank_skip_empty_content(self):
|
||||
"""Test rerank skips documents with empty content."""
|
||||
@@ -691,14 +618,14 @@ class TestDefaultLocalReRanker:
|
||||
):
|
||||
reranker = DefaultLocalReRanker(query="test", rerank_field="content")
|
||||
|
||||
query_results = {
|
||||
"vector1": [
|
||||
query_results = [
|
||||
[
|
||||
Doc(id="1", fields={"content": ""}),
|
||||
Doc(id="2", fields={"content": " "}),
|
||||
]
|
||||
}
|
||||
]
|
||||
with pytest.raises(ValueError, match="No documents to rerank"):
|
||||
reranker.rerank(query_results)
|
||||
reranker.rerank(query_results, topn=10)
|
||||
|
||||
def test_rerank_success(self):
|
||||
"""Test successful rerank with mocked model."""
|
||||
@@ -720,19 +647,17 @@ class TestDefaultLocalReRanker:
|
||||
"zvec.extension.sentence_transformer_rerank_function.require_module",
|
||||
return_value=mock_st,
|
||||
):
|
||||
reranker = DefaultLocalReRanker(
|
||||
query="test query", topn=3, rerank_field="content"
|
||||
)
|
||||
reranker = DefaultLocalReRanker(query="test query", rerank_field="content")
|
||||
|
||||
query_results = {
|
||||
"vector1": [
|
||||
query_results = [
|
||||
[
|
||||
Doc(id="1", score=0.8, fields={"content": "Document 1"}),
|
||||
Doc(id="2", score=0.7, fields={"content": "Document 2"}),
|
||||
Doc(id="3", score=0.6, fields={"content": "Document 3"}),
|
||||
]
|
||||
}
|
||||
]
|
||||
|
||||
results = reranker.rerank(query_results)
|
||||
results = reranker.rerank(query_results, topn=3)
|
||||
|
||||
# Verify results
|
||||
assert len(results) == 3
|
||||
@@ -771,21 +696,19 @@ class TestDefaultLocalReRanker:
|
||||
"zvec.extension.sentence_transformer_rerank_function.require_module",
|
||||
return_value=mock_st,
|
||||
):
|
||||
reranker = DefaultLocalReRanker(
|
||||
query="test", topn=2, rerank_field="content"
|
||||
)
|
||||
reranker = DefaultLocalReRanker(query="test", rerank_field="content")
|
||||
|
||||
query_results = {
|
||||
"vector1": [
|
||||
query_results = [
|
||||
[
|
||||
Doc(id="1", fields={"content": "Doc 1"}),
|
||||
Doc(id="2", fields={"content": "Doc 2"}),
|
||||
Doc(id="3", fields={"content": "Doc 3"}),
|
||||
Doc(id="4", fields={"content": "Doc 4"}),
|
||||
Doc(id="5", fields={"content": "Doc 5"}),
|
||||
]
|
||||
}
|
||||
]
|
||||
|
||||
results = reranker.rerank(query_results)
|
||||
results = reranker.rerank(query_results, topn=2)
|
||||
|
||||
# Should only return top 2
|
||||
assert len(results) == 2
|
||||
@@ -811,20 +734,18 @@ class TestDefaultLocalReRanker:
|
||||
"zvec.extension.sentence_transformer_rerank_function.require_module",
|
||||
return_value=mock_st,
|
||||
):
|
||||
reranker = DefaultLocalReRanker(
|
||||
query="test", topn=5, rerank_field="content"
|
||||
)
|
||||
reranker = DefaultLocalReRanker(query="test", rerank_field="content")
|
||||
|
||||
# Same document in multiple vector results
|
||||
doc1 = Doc(id="1", fields={"content": "Document 1"})
|
||||
doc2 = Doc(id="2", fields={"content": "Document 2"})
|
||||
|
||||
query_results = {
|
||||
"vector1": [doc1, doc2],
|
||||
"vector2": [doc1], # doc1 appears in both
|
||||
}
|
||||
query_results = [
|
||||
[doc1, doc2],
|
||||
[doc1], # doc1 appears in both
|
||||
]
|
||||
|
||||
results = reranker.rerank(query_results)
|
||||
results = reranker.rerank(query_results, topn=5)
|
||||
|
||||
# Should only process each document once
|
||||
assert len(results) == 2
|
||||
@@ -852,19 +773,17 @@ class TestDefaultLocalReRanker:
|
||||
"zvec.extension.sentence_transformer_rerank_function.require_module",
|
||||
return_value=mock_st,
|
||||
):
|
||||
reranker = DefaultLocalReRanker(
|
||||
query="test", topn=3, rerank_field="content"
|
||||
)
|
||||
reranker = DefaultLocalReRanker(query="test", rerank_field="content")
|
||||
|
||||
query_results = {
|
||||
"vector1": [
|
||||
query_results = [
|
||||
[
|
||||
Doc(id="1", fields={"content": "Doc 1"}),
|
||||
Doc(id="2", fields={"content": "Doc 2"}),
|
||||
Doc(id="3", fields={"content": "Doc 3"}),
|
||||
]
|
||||
}
|
||||
]
|
||||
|
||||
results = reranker.rerank(query_results)
|
||||
results = reranker.rerank(query_results, topn=3)
|
||||
|
||||
# Should be sorted by score (descending)
|
||||
assert len(results) == 3
|
||||
@@ -892,10 +811,10 @@ class TestDefaultLocalReRanker:
|
||||
):
|
||||
reranker = DefaultLocalReRanker(query="test", rerank_field="content")
|
||||
|
||||
query_results = {"vector1": [Doc(id="1", fields={"content": "Document 1"})]}
|
||||
query_results = [[Doc(id="1", fields={"content": "Document 1"})]]
|
||||
|
||||
with pytest.raises(RuntimeError, match="Failed to compute rerank scores"):
|
||||
reranker.rerank(query_results)
|
||||
reranker.rerank(query_results, topn=10)
|
||||
|
||||
def test_rerank_with_custom_batch_size(self):
|
||||
"""Test rerank uses custom batch_size."""
|
||||
@@ -918,14 +837,14 @@ class TestDefaultLocalReRanker:
|
||||
query="test", rerank_field="content", batch_size=64
|
||||
)
|
||||
|
||||
query_results = {
|
||||
"vector1": [
|
||||
query_results = [
|
||||
[
|
||||
Doc(id="1", fields={"content": "Doc 1"}),
|
||||
Doc(id="2", fields={"content": "Doc 2"}),
|
||||
]
|
||||
}
|
||||
]
|
||||
|
||||
reranker.rerank(query_results)
|
||||
reranker.rerank(query_results, topn=10)
|
||||
|
||||
# Verify batch_size is passed to predict
|
||||
call_args = mock_model.predict.call_args
|
||||
@@ -947,13 +866,12 @@ class TestDefaultLocalReRanker:
|
||||
# Create reranker with real model (using default lightweight model)
|
||||
reranker = DefaultLocalReRanker(
|
||||
query="What is machine learning?",
|
||||
topn=3,
|
||||
rerank_field="content",
|
||||
)
|
||||
|
||||
# Prepare test documents
|
||||
query_results = {
|
||||
"vector1": [
|
||||
query_results = [
|
||||
[
|
||||
Doc(
|
||||
id="1",
|
||||
score=0.8,
|
||||
@@ -976,7 +894,7 @@ class TestDefaultLocalReRanker:
|
||||
},
|
||||
),
|
||||
],
|
||||
"vector2": [
|
||||
[
|
||||
Doc(
|
||||
id="4",
|
||||
score=0.6,
|
||||
@@ -992,10 +910,10 @@ class TestDefaultLocalReRanker:
|
||||
},
|
||||
),
|
||||
],
|
||||
}
|
||||
]
|
||||
|
||||
# Call real model
|
||||
results = reranker.rerank(query_results)
|
||||
results = reranker.rerank(query_results, topn=3)
|
||||
|
||||
# Verify results
|
||||
assert len(results) <= 3, "Should return at most topn documents"
|
||||
@@ -1030,3 +948,49 @@ class TestDefaultLocalReRanker:
|
||||
content = doc.field("content")
|
||||
if content:
|
||||
print(f" Content: {content[:80]}...")
|
||||
|
||||
|
||||
# ----------------------------
|
||||
# DocList Type and Delegation Tests
|
||||
# ----------------------------
|
||||
class TestDocList:
|
||||
def test_type_alias(self):
|
||||
"""DocList is list[Doc]."""
|
||||
from zvec.model.doc import DocList
|
||||
from zvec import Doc, DocList as QR
|
||||
|
||||
assert DocList == list[Doc]
|
||||
assert QR == list[Doc]
|
||||
|
||||
def test_rrf_reranker_delegates_to_cpp(self):
|
||||
"""RrfReRanker.rerank() delegates to C++ (raises TypeError with Python Docs)."""
|
||||
reranker = RrfReRanker()
|
||||
with pytest.raises(TypeError):
|
||||
reranker.rerank([[Doc(id="1", score=0.5)]], topn=5)
|
||||
|
||||
def test_weighted_reranker_delegates_to_cpp(self):
|
||||
"""WeightedReRanker.rerank() delegates to C++ (raises TypeError with Python Docs)."""
|
||||
reranker = WeightedReRanker(weights=[0.7, 0.3])
|
||||
with pytest.raises(TypeError):
|
||||
reranker.rerank(
|
||||
[[Doc(id="1", score=0.5)], [Doc(id="2", score=0.3)]], topn=5
|
||||
)
|
||||
|
||||
def test_single_route_query_results(self):
|
||||
"""CallbackReRanker works with single-route (one element list)."""
|
||||
|
||||
def cb(query_results, topn):
|
||||
return query_results[0][:topn]
|
||||
|
||||
reranker = CallbackReRanker(callback=cb)
|
||||
results = reranker.rerank(
|
||||
[
|
||||
[
|
||||
Doc(id="1", score=0.9),
|
||||
Doc(id="2", score=0.8),
|
||||
Doc(id="3", score=0.7),
|
||||
]
|
||||
],
|
||||
topn=2,
|
||||
)
|
||||
assert len(results) == 2
|
||||
|
||||
Reference in New Issue
Block a user