Qinren Zhou
|
dbea635019
|
refactor: clarify segment-local row ID handling and tests and fixes bugs (#432)
|
2026-06-03 15:20:45 +08:00 |
|
Qinren Zhou
|
8e8bb81db0
|
refactor: tidy query APIs and execution (#431)
|
2026-06-01 21:07:35 +08:00 |
egolearner
|
02bfb31cf5
|
feat: add fts support (#408)
Add BM25-based full-text search with CJK (jieba) tokenization, supporting
query_string and match_string syntax, phrase queries, boolean operators
(AND/OR/NOT/MUST), and hybrid retrieval with existing vector search.
## Core
- BitPacked posting format with block-max WAND pruning
- Tokenizer pipeline: jieba (cut/cut_for_search/hmm/full), whitespace,
lowercase, with extensible pipeline composition
- Query parser: boolean operators, phrase queries, field scoping,
boost, MUST(+) modifier inside OR (ES query_string semantics)
- AST rewriter: dedup repeated terms with linear boost aggregation,
flatten same-type composites, canonicalize OR-with-must_not into AND
wrapper, empty-node propagation, contradiction detection
- FTS reduce/merge integrated into Optimize compaction
- Multi-segment score-descending sort
- Auto-register bundled jieba dict on SDK import
## Performance
- Block-max WAND with cached block_max_info_for (single binary search)
- AVX2/SSE bitpacked encoding with cross-arch scalar fallback
- MultiGet for batch posting retrieval and phrase position verification
- HashSkipList memtable for posting writes
- PinnableSlice zero-copy reads
- Filter pushdown into composite iterators (Disjunction/Conjunction/Phrase)
- Candidate-driven (brute-force) evaluation for selective invert filters
- Precomputed BM25 IDF weights, cached SIMD dispatch pointers
- Shortest-list anchor for phrase position matching
- Single-open per-term posting iterator
## Query
- Tokenize query terms through the same pipeline as indexing
- EmptyNode for zero-token queries (all stop-words / punctuation)
- Backslash unescape after lexing in query parser
- Schema allows collections without vector fields (FTS-only use case)
- Create/Drop Index validates supported index types
- FTS fields disallowed in SQL filter expressions
## Bindings
- C API: fts query params, brute-force ratio config
- Python SDK: FTS search, jieba dict auto-registration
## Internals
- Bypass cppjieba::Jieba to drop KeywordExtractor (~12MB fewer required files)
- Hide tokenizer pipeline from public header (Pimpl-style FtsState)
- ListColumnFamilies to avoid double-open on segment load
- Reorganized fts_column into tokenizer/, posting/, iterator/ subdirs
|
2026-06-01 15:02:54 +08:00 |
|
egolearner
|
8dcb6cbd7f
|
refactor: drop VectorQuery, unify single-target query on SearchQuery (#428)
|
2026-05-29 16:36:09 +08:00 |
lichen2015
|
e0ba23179b
|
feat: fetch() add output_fields param (#358)
|
2026-05-27 22:58:49 +08:00 |
kgeg401
|
1e25294c07
|
feat(ci): integrate clang-tidy for changed C/C++ files (#116)
* feat(ci): add clang-tidy checks for changed cpp files
* fix(ci): correct clang-tidy workflow heredoc indentation
* test: stabilize fp16 euclidean matrix comparison
* test: fix fp16 matrix CI and clang-tidy warnings
* ci: scope clang-tidy PR and filter compile-db files
* check nullptr only
* fix nullptr
* fix: format
* chore: ignore c
* fix: tests files
* fix: workflow
* fix: specify clang-tidy version
* fix: filter regex
* fix: more files
* fix: update version
* add thirdparty cache
* add pipeline dependency
* fix clang-tidy trigger
---------
Co-authored-by: kgeg401 <kgeg401@users.noreply.github.com>
Co-authored-by: jiliang.ljl <jiliang.ljl@alibaba-inc.com>
|
2026-04-20 20:14:21 +08:00 |
|
egolearner
|
e5ba11b6fe
|
feat: add hnsw-rabitq support (#69)
* feat: add hnsw-rabitq
* undefine transform
* support config rotator type
* support sample_count
* fix ut
* refactor: update interface
* refactor: update interface
* update rabitq index params
* fix interface update
* fix searcher test
* fix streamer test
* streamer support bf and add more ut
* rm env check
* add collection ut
* add schema check
* add rabitq query param binding
* add integration test
* fix local_builder
* cleanup dist calculator
* add RaBitQ-Library submodule
* cleanup rabitq converter/reformer
* add files
* disable build on mac
* disable Feature_Optimize_HNSW_RABITQ
* disable python/tests/test_collection_hnsw_rabitq.py:13
* check avx2/avx512
* fix refine
* fix mac ci
* add dimension check
* fix mac ci
* fix ci
* add missing lib
* fix centroids selection for Cosine/InnerProduct
* rename hnsw-rabitq to hnsw_rabitq
* address comments
* fix compile
* fix rabitqlib name
* fix mac compile
* check avx2/avx512 support
* runtime check again compile-time
* check AUTO_DETECT_ARCH
* rm debug log
* fix
* rabitq use avx2 by default
* add missing file
* fix search_bf/group_by dist
* address comments
* fix typo
* address comments
* rabitq support disable id_map
* address comments
|
2026-03-19 17:21:42 +08:00 |
|
Qinren Zhou
|
9314cb14a6
|
fix: use per-block filter instead of per-segment filter during query (#223)
* fix: id mismatch
* fix: remove debug string
* fix: use protected
* chore(comment): add comments
* feat: more ut
* Update vector_recall_test.cc
* fix: resolve comments
|
2026-03-16 10:55:51 +08:00 |
feihongxu0824
|
fb0b900a80
|
fix(build): replace CMAKE_SOURCE_DIR with PROJECT_ROOT_DIR for subproject builds (#195)
|
2026-03-04 20:01:52 +08:00 |
|
lichen2015
|
f1349cc91a
|
fix: remove unnecessary column_name param from the AddColumn API (#59)
|
2026-02-04 14:44:13 +08:00 |
|
feihongxu0824
|
0ebf5621a6
|
feat: support core cpp sdk (#30)
* support core cpp sdk
* fix
* fix
* fix
* fix cr
|
2026-01-26 20:01:44 +08:00 |
|
feihongxu0824
|
e957442355
|
feat: refact cpp sdk (#27)
* refact cpp sdk
---------
Co-authored-by: zhouqinren.zqr <zhouqinren.zqr@alibaba-inc.com>
|
2026-01-21 20:05:42 +08:00 |
sanyi
|
6524fabd80
|
Initial commit
|
2025-12-30 11:02:17 +08:00 |