Commit Graph
49 Commits
Author SHA1 Message Date
egolearner e5ba11b6fe feat: add hnsw-rabitq support (#69)
* feat: add hnsw-rabitq

* undefine transform

* support config rotator type

* support sample_count

* fix ut

* refactor: update interface

* refactor: update interface

* update rabitq index params

* fix interface update

* fix searcher test

* fix streamer test

* streamer support bf and add more ut

* rm env check

* add collection ut

* add schema check

* add rabitq query param binding

* add integration test

* fix local_builder

* cleanup dist calculator

* add RaBitQ-Library submodule

* cleanup rabitq converter/reformer

* add files

* disable build on mac

* disable Feature_Optimize_HNSW_RABITQ

* disable python/tests/test_collection_hnsw_rabitq.py:13

* check avx2/avx512

* fix refine

* fix mac ci

* add dimension check

* fix mac ci

* fix ci

* add missing lib

* fix centroids selection for Cosine/InnerProduct

* rename hnsw-rabitq to hnsw_rabitq

* address comments

* fix compile

* fix rabitqlib name

* fix mac compile

* check avx2/avx512 support

* runtime check again compile-time

* check AUTO_DETECT_ARCH

* rm debug log

* fix

* rabitq use avx2 by default

* add missing file

* fix search_bf/group_by dist

* address comments

* fix typo

* address comments

* rabitq support disable id_map

* address comments
2026-03-19 17:21:42 +08:00
ZeFeng Yin 432de52c5f [enhance] buffer storage (#83) 2026-03-19 15:24:46 +08:00
rayx d4ffbf06af fix: fix metric batch (#245) 2026-03-19 13:39:10 +08:00
luoxiaojian 840bcdad26 feat: Introduce turbo. (#243) 2026-03-18 20:43:18 +08:00
Qinren Zhou 3c1241f7c6 feat: enlarge indice size limit for sparse vectors (#229) 2026-03-16 17:12:24 +08:00
rayx de1e7050d7 fix/fix mips euclidean (#226)
* fix: fix mips euclidean
2026-03-16 13:41:58 +08:00
Abdur-Rahmaan Janhangeer 65b45470d6 fix: minor typo (#225) 2026-03-16 11:31:44 +08:00
Qinren Zhou 9314cb14a6 fix: use per-block filter instead of per-segment filter during query (#223)
* fix: id mismatch

* fix: remove debug string

* fix: use protected

* chore(comment): add comments

* feat: more ut

* Update vector_recall_test.cc

* fix: resolve comments
2026-03-16 10:55:51 +08:00
luoxiaojian 79387c237f feat: enable icelake and l2 batch distance for int8 quantization. (#213) 2026-03-13 17:26:22 +08:00
rayx d1e0c7b88e fix: use static array (#222)
* fix: use static array

* fix: clang-format

* fix: clang-format

* fix: clang-format
2026-03-13 13:34:28 +08:00
rayx b6a4eaddc2 fix: turn off math march if not auto detected (#220) 2026-03-12 16:12:28 +08:00
egolearnerandrayx c05974cab1 fix compile (#218)
* fix compile

* fix missing include

* Revert "fix missing include"

This reverts commit 295b52b0a5057a1c979b0316faf80e864031a0cb.

---------

Co-authored-by: rayx <rui.xing@alibaba-inc.com>
2026-03-12 14:06:04 +08:00
rayx 919f15bb36 fix: fix uint8 conversion overflow (#215) 2026-03-12 12:22:37 +08:00
a7bfd6060d refactor/march based reorganization (#193)
* add for march compatible test

* fix: remove MxN

* fix: add macro scope

* fix for android ci

* refactor: call euclidean via squared euclidean

* refactor: add match utility

* refactor: comment config out

* fix: fix dimension remainder

* fix: fix condition & config flags

* fix: remove redundant macros

* fix: fix condition error

* fix: fix macros

* fix: remove unnecessary macros

* fix: fix remaining code with vector

* fix: remove unnecessary codes

---------

Co-authored-by: Zefeng Yin <yinzefeng.yzf@alibaba-inc.com>
Co-authored-by: xufeihong.xfh <xufeihong.xfh@alibaba-inc.com>
2026-03-11 22:44:48 +08:00
Qinren Zhou 331933c75a fix: clean up crash residue (#208) 2026-03-11 15:55:13 +08:00
ZeFeng Yin ba86c1663d fix: add validator for cosine metric with int8 (#209) 2026-03-10 16:09:37 +08:00
ZeFeng Yin cfbd3ca008 fix: Replace std::vector with std::array for distance impl (#201) 2026-03-07 17:49:19 +08:00
feihongxu0824 fb0b900a80 fix(build): replace CMAKE_SOURCE_DIR with PROJECT_ROOT_DIR for subproject builds (#195) 2026-03-04 20:01:52 +08:00
feihongxu0824 43e3eeac40 feat: support android platform cross build (#90) 2026-03-04 16:42:23 +08:00
feihongxu0824 24d877da5e fix: Replace VLAs with std::vector for MSVC compatibility and stack safety (#190) 2026-03-03 10:07:21 +08:00
longwayandZeFeng Yin 4e420311df chore(flat): replace std::map with std::unordered_map for key_id_mapping_ (#153)
Reduces key lookup in get_id() from O(log N) to O(1) average, and index
load time from O(N log N) to O(N). Also lowers per-entry memory usage by
eliminating the three tree-node pointers (parent/left/right).

Co-authored-by: ZeFeng Yin <yinzefeng.yzf@alibaba-inc.com>
2026-02-28 13:10:38 +08:00
ZeFeng Yin ba6acbf48f fix: ivf optimize (#169) 2026-02-27 17:35:52 +08:00
lichen2015andyinzefeng.yzf e7ad7cc31e fix: combined indexer should use key instead of index (#87)
Co-authored-by: yinzefeng.yzf <yinzefeng.yzf@alibaba-inc.com>
2026-02-12 10:38:05 +08:00
feihongxu0824 efc6f3f7ce fix: add c++ example to macos/linux ci (#86) 2026-02-09 23:31:56 +08:00
feihongxu0824 1e09d0d34c chore: rm useless ailego module (#85) 2026-02-09 18:56:46 +08:00
lichen2015 fe90f70a3b fix: delete_by_filter should use g_doc_id (#84) 2026-02-09 17:40:51 +08:00
Qinren Zhou 0b9bd34354 minor: subsequent initialization should fail silently (#79) 2026-02-06 21:16:08 +08:00
Jalin Wang 78f08bd0cc feat: auto scalable segment meta section of mmap files (#67)
The underly core module only support a fixed-size segment due to the index file format where the section of segments meta limit the max num of data segments. This commit makes the meta section dynamically allocated and organized in a link-table manner to support much more segments.


* revert local_builder's meta cap setting

* feat: auto scalable segment meta section of mmap files

* Revert "feat: auto scalable segment meta section of mmap files"

This reverts commit 82c11e451544b912b901be33fae79e964a487b1f.

* v2: don't change to absolute offset
2026-02-06 14:58:56 +08:00
lichen2015 f1349cc91a fix: remove unnecessary column_name param from the AddColumn API (#59) 2026-02-04 14:44:13 +08:00
richyreachyandfeihongxu0824 a20d484e9e refactor(core): unify search invoke (#15)
Co-authored-by: feihongxu0824 <xufeihong.xfh@alibaba-inc.com>
2026-02-03 12:40:23 +08:00
egolearner 6df6ba8fce chore: rm IndexMeta::MetricType def (#53) 2026-02-03 11:17:02 +08:00
姚军 ea9d957893 fix: some typos in hnsw index (#50) 2026-02-03 10:41:21 +08:00
Jalin Wang ce45478dae fix: doc string & pyi (#41) 2026-02-03 10:14:44 +08:00
iaojnh 8c1819ca26 change hnsw single dist to batch dist (#40) 2026-01-29 10:18:57 +08:00
egolearner 66f137efee fix: fix MemoryReadStorage read by MemoryBlock (#38) 2026-01-27 14:11:28 +08:00
feihongxu0824 0ebf5621a6 feat: support core cpp sdk (#30)
* support core cpp sdk

* fix

* fix

* fix

* fix cr
2026-01-26 20:01:44 +08:00
Jalin Wang 0af6a62c31 fix: flat support more segments (#32)
* fix

* fix compilation
2026-01-26 16:22:37 +08:00
Jalin Wang bc90952a01 refactor: flat param string cleanup (#28)
* refactor: flat param string cleanup

* fix
2026-01-26 14:49:35 +08:00
richyreachy 614f955c5f fix: remove empty files (#33) 2026-01-26 10:08:51 +08:00
Qinren Zhou 1763564705 minor: fix some debug strings (#31) 2026-01-22 11:42:12 +08:00
feihongxu0824andzhouqinren.zqr e957442355 feat: refact cpp sdk (#27)
* refact cpp sdk

---------

Co-authored-by: zhouqinren.zqr <zhouqinren.zqr@alibaba-inc.com>
2026-01-21 20:05:42 +08:00
lichen2015 08d680bd4f fix: fix and refact segment fetch perf (#26)
* fix and refact fetch perf

* change git user name
2026-01-19 13:43:30 +08:00
iaojnh e17c4a88bc refactor: clarify HNSW 'scaling_factor' as 'm' instead of 50 (#17) 2026-01-12 20:05:29 +08:00
egolearner 89f9c378ac fix: fix default m (#20)
* fix: fix default m

* fix doc str

* fix ut
2026-01-12 10:22:58 +08:00
Jalin Wang 84b5f03f5e refactor: clarify HNSW 'm' as max neighbors of upper layer (#12)
* refactor: clarify HNSW 'm' as max neighbors of upper layer

- Distinguish l0/upper_max_neighbor_cnt
- Define l0_max_neighbor_cnt as m * l0_multiplier.

* fix ut
2026-01-08 16:31:14 +08:00
richyreachy 9c106775df fix cpu detect (#14) 2026-01-08 16:27:47 +08:00
egolearner 788ecd5ee8 fix(sqlengine): reduce memory reserve size when topk is too big (#10) 2026-01-06 20:33:08 +08:00
feihongxu0824 1d9791b57a feat(core): support cpu flag detect & dispatch (#3) 2025-12-30 18:42:47 +08:00
sanyi 6524fabd80 Initial commit 2025-12-30 11:02:17 +08:00