Alibaba lightweight in-process vector database
Go to file
lichen2015 f539580138
feat: migrate multi-vector query and reranker logic to C++ (#405)
* feat: migrate multi-vector query and reranker logic to C++

- Add Reranker base class with RrfReRanker and WeightedReRanker implementations
- Add Collection::MultiQuery interface for multi-vector queries with reranking
- Add MultiVectorQuery struct in doc.h with forward declaration for Reranker
- Add C API bindings for reranker and MultiQuery (zvec_reranker_*, zvec_multi_vector_query_*, zvec_collection_multi_query)
- Add Python binding for reranker classes with py::function bridge for callback
- Validate duplicate field names in multi-vector queries (C++ and Python consistent)
- Remove TODO comment about concurrent execution (SQLEngine is not thread-safe)
- Update collection.h MultiQuery doc comment from concurrently to sequentially
- Add C++ collection tests (6 MultiQuery test cases)
- Add C API tests (reranker functions + multi_vector_query end-to-end)
- Implement Python test cases (11 previously skipped tests now active)
- Simplify Python query_executor validation for unified duplicate field check

* style: format Python files with ruff

* fix: adapt to main branch API changes (VectorQuery->Query rename, validate_and_sanitize)

* fix: multi_vector tests now use multiple same-type vector fields (dense2, sparse2)

* fix: suppress RET501 for intentional default return None in RerankFunction._get_object

* style: ruff format test_collection.py

* refact multi vector query

* format code

* fix(multi-vector): expose SubVectorQuery in Python binding, fix tests

- Register _SubVectorQuery in pybind11 with from_vector_query() factory
- Convert _VectorQuery to _SubVectorQuery in MultiVectorQueryExecutor
- Relax RRF/Weighted score assertion tolerance from 1e-10 to 1e-6
- Fix WeightedReRanker test metric to IP (matching HnswIndexParam default)

* style: ruff format query_executor.py

* fix: define _USE_MATH_DEFINES for M_PI on Windows (MSVC)

* refactor: include reranker.h directly in query.h instead of forward declaration

* refactor(reranker): move topn from member variable to rerank() parameter

* refact code

* style(python): fix ruff UP035/UP037 in multi_vector_reranker

- import Callable from collections.abc instead of typing (UP035)
- remove redundant quotes around MetricType annotations (UP037)

* chore: trigger PR sync

* refact code

* fix(examples): restore CMakeLists.txt formatting broken by clang-format

* refactor(reranker): remove redundant metrics_ map by querying schema directly, and use insert return value to avoid duplicate set lookup

* refactor(reranker): defer schema binding to query time and remove C API callback reranker
2026-05-29 10:10:41 +08:00
.github ci: skip timing-sensitive GIL test in coverage builds (#412) 2026-05-19 10:45:42 +08:00
cmake chore: fix more warnings (#418) 2026-05-21 11:15:41 +08:00
examples feat: migrate multi-vector query and reranker logic to C++ (#405) 2026-05-29 10:10:41 +08:00
python feat: migrate multi-vector query and reranker logic to C++ (#405) 2026-05-29 10:10:41 +08:00
scripts ci: refact android ci (#330) 2026-04-15 15:37:51 +08:00
src feat: migrate multi-vector query and reranker logic to C++ (#405) 2026-05-29 10:10:41 +08:00
tests feat: migrate multi-vector query and reranker logic to C++ (#405) 2026-05-29 10:10:41 +08:00
thirdparty chore: fix more warnings (#418) 2026-05-21 11:15:41 +08:00
tools chore: fix more warnings (#418) 2026-05-21 11:15:41 +08:00
.clang-format
.clang-tidy chore: enable modernize-use-override and fix existing violations (#419) 2026-05-21 19:05:32 +08:00
.gitignore feat(ci): integrate clang-tidy for changed C/C++ files (#116) 2026-04-20 20:14:21 +08:00
.gitmodules build(thirdparty): replace CRoaring submodule with amalgamation (~292M -> ~1MB) (#381) 2026-05-12 14:55:08 +08:00
.pre-commit-config.yaml chore: enable the conventional-pre-commit run sucess and update to latest version (#111) 2026-02-25 18:03:20 +08:00
CMakeLists.txt feat: add iOS build support (#321) 2026-04-09 16:14:26 +08:00
CODE_OF_CONDUCT.md
CONTRIBUTING.md doc: add v0.3.0 release note (#312) 2026-04-03 15:47:19 +08:00
LICENSE
README.md minor: update readme for v0.4.0 (#389) 2026-05-09 11:40:30 +08:00
README_CN.md minor: update readme for v0.4.0 (#389) 2026-05-09 11:40:30 +08:00
pyproject.toml feat(test): enable parallel tests (#384) 2026-05-12 23:14:17 +08:00

README.md

English | 中文

zvec logo

Code Coverage Main License PyPI Release Python Versions npm Release

alibaba%2Fzvec | Trendshift

🚀 Quickstart | 🏠 Home | 📚 Docs | 📊 Benchmarks | 🔎 DeepWiki | 🎮 Discord | 🐦 X (Twitter)

Zvec is an open-source, in-process vector database — lightweight, lightning-fast, and designed to embed directly into applications. Battle-tested within Alibaba Group, it delivers production-grade, low-latency and scalable similarity search with minimal setup.

[!Important] 🚀 v0.4.0 (May 9, 2026)

  • Dart/Flutter SDK: Published the official zvec Flutter package with FFI bindings. Supports Android (arm64-v8a) and iOS (arm64) — no manual native compilation required.
  • iOS Build Support: Added support for building on iOS platforms, expanding cross-platform coverage.
  • Enlarged topK Limit: Relaxed the upper bound on topK to support larger-scale recall scenarios.
  • Bug Fixes: SQ8 quantizer recall drop; Windows path handling; sparse vector index ordering.

👉 Read the Release Notes | View Roadmap 📍

💫 Features

  • Blazing Fast: Searches billions of vectors in milliseconds.
  • Simple, Just Works: Install and start searching in seconds. Pure local, no servers, no config, no fuss.
  • Dense + Sparse Vectors: Work with both dense and sparse embeddings, with native support for multi-vector queries in a single call.
  • Hybrid Search: Combine semantic similarity with structured filters for precise results.
  • Durable Storage: Write-ahead logging (WAL) guarantees persistence — data is never lost, even on process crash or power failure.
  • Concurrent Access: Multiple processes can read the same collection simultaneously; writes are single-process exclusive.
  • Runs Anywhere: As an in-process library, Zvec runs wherever your code runs — notebooks, servers, CLI tools, or even edge devices.

📦 Installation

Python

Requirements: Python 3.10 - 3.14

pip install zvec

Node.js

npm install @zvec/zvec

Supported Platforms

  • Linux (x86_64, ARM64)
  • macOS (ARM64)
  • Windows (x86_64)

🛠️ Building from Source

If you prefer to build Zvec from source, please check the Building from Source guide.

One-Minute Example

import zvec

# Define collection schema
schema = zvec.CollectionSchema(
    name="example",
    vectors=zvec.VectorSchema("embedding", zvec.DataType.VECTOR_FP32, 4),
)

# Create collection
collection = zvec.create_and_open(path="./zvec_example", schema=schema)

# Insert documents
collection.insert([
    zvec.Doc(id="doc_1", vectors={"embedding": [0.1, 0.2, 0.3, 0.4]}),
    zvec.Doc(id="doc_2", vectors={"embedding": [0.2, 0.3, 0.4, 0.1]}),
])

# Search by vector similarity
results = collection.query(
    zvec.VectorQuery("embedding", vector=[0.4, 0.3, 0.3, 0.1]),
    topk=10
)

# Results: list of {'id': str, 'score': float, ...}, sorted by relevance
print(results)

📈 Performance at Scale

Zvec delivers exceptional speed and efficiency, making it ideal for demanding production workloads.

Zvec Performance Benchmarks

For detailed benchmark methodology, configurations, and complete results, please see our Benchmarks documentation.

🤝 Join Our Community

💬 DingTalk 📱 WeChat 🎮 Discord X (Twitter)
DingTalk QR Code WeChat QR Code Discord X (formerly Twitter) Follow
Scan to join Scan to join Click to join Click to follow

❤️ Contributing

We welcome and appreciate contributions from the community! Whether you're fixing a bug, adding a feature, or improving documentation, your help makes Zvec better for everyone.

Check out our Contributing Guide to get started!