Files
zvec/python/tests/test_fts_query.py
T
Cuiys c46efe1241 refactor: change rerank interface from map-based to vector-based (#458)
* refactor: change rerank interface from map-based to vector-based (#452)

- Define QueryResult = list[Doc] type alias in doc.py
- Change C++ Reranker::rerank() signature from map<string, DocPtrList> to vector<DocPtrList>
- Extend bind_schema() to accept field_names for index-based field lookup
- Update ScoreBasedReranker/WeightedReranker/CallbackReranker implementations
- Adapt collection.cc MultiQuery path to use vector<DocPtrList>
- Update Python binding to expose rerank() and use vector<double> weights
- Refactor Python RerankFunction interface to list[QueryResult] -> QueryResult
- Remove Python-layer rerank logic from RrfReRanker/WeightedReRanker (delegate to C++)
- Update query_executor to return list[list[Doc]] instead of dict
- Update all related unit tests (C++ and Python)

* refactor: replace list[Doc] with QueryResult type alias in executor and rerank functions

* refactor: replace list[list[Doc]] with list[QueryResult] in query_executor

* fix: remove unused Doc import in rerank_function.py (ruff F401)

* refactor(query_executor): merge duplicate rerank return paths

* refactor: RrfReRanker/WeightedReRanker.rerank() directly call C++ reranker

* refactor: simplify QueryExecutor into unified class, remove Factory/subclasses/validation/concurrency

* refactor: rename _VectorQuery to _SearchQuery, from_vector_query to from_search_query

* refactor(query_executor): split execute into single/multi paths, rename core_vector to search_query, drop unused core_vectors

* style: apply ruff formatter to test_reranker.py and query_executor.py

* refactor: make rescore() private in ScoreBasedReranker hierarchy

* style: apply clang-format to reranker.h

* style: apply clang-format to all modified C++ files

* refactor: rename private methods in QueryExecutor for clearer semantics

* refactor: rename mvq to multi_query for clarity

* fix: make BasicRRF test order-independent for equal scores

* fix: update collection_test to use vector-based reranker interface

* fix: update reranker tests to expect TypeError instead of NotImplementedError

* refactor: remove PendingQuery wrapper, use SearchQuery directly in MultiQuery path

* refactor: simplify MultiQuery path - remove seen_fields, merge field_names into main loop

* fix: address review comments - defensive checks and remove fields param from C API

- ScoreBasedReranker::rerank(): early return empty list when topn <= 0
- WeightedReranker::rescore(): null-check schema_ before use
- CallbackReranker::rerank(): check callback_ is not empty before invoke
- C API zvec_reranker_create_weighted(): remove unused fields parameter

* fix: remove duplicate field name test (check was intentionally removed)

* fix: address egolearner review comments

- Rename QueryResult to DocList for clarity (见名知义)
- Change docstring to #: comment for type alias
- Fix output_fields check: use 'is not None' instead of truthy check
  (None means unset, [] means explicit empty list - different semantics)
- Raise ValueError when search-by-id finds no document

* refactor: remove redundant output_fields assignment in _build_search_query

* refactor: address egolearner review comments (C++ refactoring)

- c_api.cc: simplify weighted reranker creation with inline vector ctor
- python_reranker.cc: refactor unwrap_rerank_result - take by value,
  early error return, move semantics
- Rename C API functions for consistent naming:
  zvec_reranker_create_rrf -> zvec_create_rrf_reranker
  zvec_reranker_create_weighted -> zvec_create_weighted_reranker
  zvec_reranker_destroy -> zvec_destroy_reranker
  zvec_reranker_get_rank_constant -> zvec_get_reranker_rank_constant
- reranker.h/cc: bind_schema returns Result<void>, caches
  vector<const FieldSchema*> to avoid repeated schema lookups in rescore
- python_param.cc: rename py::arg vector_query to search_query

* revert: rollback bind_schema refactoring due to thread-safety concern

The field_schemas_ caching approach introduces a data race when the same
WeightedReranker instance is shared across concurrent queries: bind_schema()
writes field_schemas_ while rerank() reads it concurrently.

Revert to storing schema_ + field_names_ and looking up fields in rescore().
Add @note thread-safety warning to WeightedReranker class documentation.

* fix: unify error message format in collection.cc

Change 'Vector field not found: X' to 'Invalid query: field X not found'
for consistent error formatting as suggested by zhourrr.

* fix: sort __all__ and remove duplicates in __init__.pyi

Fix RUF022 lint error: sort __all__ alphabetically and remove duplicate
entries (DenseEmbeddingFunction, ReRanker).

* style: format query_executor.py with ruff formatter

* fix: resolve Python test failures after FTS rebase integration

- test_query_executor.py: update method names to match refactored API
  (_do_build -> _build_queries, _do_merge_rerank_results -> _merge_and_rerank)
- test_reranker.py: fix expected exception type (TypeError from pybind11)
- test_collection_fts.py: update error message match patterns
- test_collection_fts_vector_hybrid.py: remove obsolete 'metrics' param,
  update weights from dict to positional list, adapt validation tests
  for multi-vector queries (now supported with reranker)
- test_collection_dql.py: remove 'metrics' param, update weights format
- collection.cc: distinguish FTS vs vector fields in MultiQuery path
  using get_fts_clause() to route field lookup correctly
- reranker.cc: use get_field() instead of get_vector_field() in rescore
  to support FTS+vector hybrid weighted reranking

* refactor: pass topn as rerank() parameter, move rerank_field to model rerankers

* fix: address review comments - rename test functions and restore duplicate field check

* refactor: simplify MultiQuery field lookup, let validate_and_sanitize handle type check
2026-06-04 16:03:28 +08:00

159 lines
5.2 KiB
Python

# Copyright 2025-present the zvec project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Tests for FTS (Full-Text Search) query support in the Python SDK."""
import pickle
import pytest
from zvec.model.param.query import Fts, Query
class TestFtsQueryValidation:
"""Test FTS parameter validation in Query dataclass."""
def test_fts_query_string_only(self):
"""Query with only query_string in Fts should be valid."""
q = Query(
field_name="content", fts=Fts(query_string='+hello -world "exact phrase"')
)
q._validate()
assert q.fts.query_string == '+hello -world "exact phrase"'
assert q.fts.match_string is None
assert q.has_fts() is True
def test_fts_match_string_only(self):
"""Query with only match_string in Fts should be valid."""
q = Query(field_name="content", fts=Fts(match_string="machine learning"))
q._validate()
assert q.fts.match_string == "machine learning"
assert q.fts.query_string is None
assert q.has_fts() is True
def test_fts_query_string_and_match_string_mutually_exclusive(self):
"""Cannot provide both query_string and match_string in Fts."""
q = Query(
field_name="content",
fts=Fts(query_string="+hello", match_string="hello world"),
)
with pytest.raises(ValueError, match="mutually exclusive"):
q._validate()
def test_no_fts(self):
"""Query without FTS fields should have has_fts() == False."""
q = Query(field_name="embedding", vector=[0.1, 0.2, 0.3])
assert q.has_fts() is False
def test_vector_and_fts_mutually_exclusive(self):
"""Cannot combine vector search with FTS in a single Query."""
q = Query(
field_name="embedding",
vector=[0.1, 0.2, 0.3],
fts=Fts(match_string="deep learning"),
)
with pytest.raises(ValueError, match="Cannot combine fts with vector search"):
q._validate()
def test_fts_without_vector_or_id(self):
"""Query with only FTS (no vector, no id) should be valid."""
q = Query(field_name="content", fts=Fts(query_string="hello"))
q._validate()
assert q.has_vector() is False
assert q.has_id() is False
assert q.has_fts() is True
class TestFtsQueryBinding:
"""Test FTS binding layer (_Fts)."""
def test_import_fts_query(self):
"""_Fts should be importable from _zvec.param."""
from _zvec.param import _Fts
fts = _Fts()
assert fts.query_string == ""
assert fts.match_string == ""
def test_fts_query_set_fields(self):
"""Setting fields on _Fts should work."""
from _zvec.param import _Fts
fts = _Fts()
fts.query_string = "+hello -world"
assert fts.query_string == "+hello -world"
fts2 = _Fts()
fts2.match_string = "machine learning"
assert fts2.match_string == "machine learning"
def test_fts_query_pickle(self):
"""_Fts should support pickling."""
from _zvec.param import _Fts
fts = _Fts()
fts.query_string = "+vector search"
fts.match_string = ""
data = pickle.dumps(fts)
restored = pickle.loads(data)
assert restored.query_string == "+vector search"
assert restored.match_string == ""
def test_search_query_fts_field(self):
"""_SearchQuery should have fts field."""
from _zvec.param import _Fts, _SearchQuery
vq = _SearchQuery()
# fts should be None by default (optional)
assert vq.fts is None
# set fts
fts = _Fts()
fts.query_string = "hello"
vq.fts = fts
assert vq.fts is not None
assert vq.fts.query_string == "hello"
def test_search_query_pickle_with_fts(self):
"""_SearchQuery with fts should survive pickling."""
from _zvec.param import _Fts, _SearchQuery
vq = _SearchQuery()
vq.topk = 10
vq.field_name = "embedding"
fts = _Fts()
fts.match_string = "test query"
vq.fts = fts
data = pickle.dumps(vq)
restored = pickle.loads(data)
assert restored.topk == 10
assert restored.field_name == "embedding"
assert restored.fts is not None
assert restored.fts.match_string == "test query"
def test_search_query_pickle_without_fts(self):
"""_SearchQuery without fts should survive pickling."""
from _zvec.param import _SearchQuery
vq = _SearchQuery()
vq.topk = 5
vq.field_name = "vec"
data = pickle.dumps(vq)
restored = pickle.loads(data)
assert restored.topk == 5
assert restored.field_name == "vec"
assert restored.fts is None