Commit Graph

71 Commits

Author SHA1 Message Date
Qinren Zhou 4e7fa0f37f
fix: some minor bugs (#370) 2026-04-23 16:24:59 +08:00
egolearner 2b16ed39cd
chore: rm hnsw-rabitq searcher/streamer (#367)
* chore: rm hnsw-rabitq searcher/streamer

* fix clang-tidy

* fix
2026-04-23 10:20:46 +08:00
ZeFeng Yin 4a4c02c1a5
chore: rm hnsw builder/searcher (#356) 2026-04-21 10:44:06 +08:00
feihongxu0824 bbe4ca7bae
minor: fix c example cmake and build c++ dynamic lib (#347) 2026-04-20 22:08:34 +08:00
kgeg401 1e25294c07
feat(ci): integrate clang-tidy for changed C/C++ files (#116)
* feat(ci): add clang-tidy checks for changed cpp files

* fix(ci): correct clang-tidy workflow heredoc indentation

* test: stabilize fp16 euclidean matrix comparison

* test: fix fp16 matrix CI and clang-tidy warnings

* ci: scope clang-tidy PR and filter compile-db files

* check nullptr only

* fix nullptr

* fix: format

* chore: ignore c

* fix: tests files

* fix: workflow

* fix: specify clang-tidy version

* fix: filter regex

* fix: more files

* fix: update version

* add thirdparty cache

* add pipeline dependency

* fix clang-tidy trigger

---------

Co-authored-by: kgeg401 <kgeg401@users.noreply.github.com>
Co-authored-by: jiliang.ljl <jiliang.ljl@alibaba-inc.com>
2026-04-20 20:14:21 +08:00
Qinren Zhou c17bd8876e
fix: validate query_params type (#351) 2026-04-20 10:54:06 +08:00
Qinren Zhou f602ed30ce
feat: enlarge topK limit (#348) 2026-04-17 17:51:07 +08:00
Jalin Wang 1492175906
fix: add GetLastErrorString() helper for accurate OS error messages (#341) 2026-04-16 15:30:47 +08:00
Jalin Wang 95d092e9b3
fix: relax collection path restriction (#340) 2026-04-16 15:26:15 +08:00
Zihao Wang d29bffca39
fix(quantizer): use rounded int8 values for SQ8 metadata to fix recall drop (#329)
fixes #328

Problem:
SQ8 metadata (squared_sum, sum) was computed from pre-rounded float
values, causing mismatch with actual stored int8 values. On asymmetric
datasets (e.g. OpenAI 1536D where |x_min| >> x_max), this leads to
severe recall drop.

Solution:
Move std::round before accumulating squared_sum and sum.

Co-authored-by: rayx <rui.xing@alibaba-inc.com>
2026-04-16 10:07:14 +08:00
Jalin Wang 09f93c5869
fix: skip drive root in Windows MakePath (#337)
CreateDirectoryA("C:", nullptr) returns ERROR_ACCESS_DENIED on Windows
(not ERROR_ALREADY_EXISTS), causing MakePath to fail immediately for any
path on the C: drive. Skip the CreateDirectoryA call when the intermediate
path is a bare drive root (e.g. "C:").
2026-04-14 19:01:37 +08:00
feihongxu0824 8df962171f
chore: clang format collection.cc (#319) 2026-04-07 14:10:33 +08:00
feihongxu0824 1c90b57e17
chore: fix some error message (#306) 2026-04-07 10:17:18 +08:00
lichen2015 6ac9bb0177
fix: refact c struct name (#302) 2026-04-02 14:54:03 +08:00
Jalin Wang ca4b893755
fix: misc fix for Windows(BUILD_SHARED_LIBS, CI) (#296) 2026-04-02 09:23:35 +08:00
lichen2015 7af6a31ae8
fix: c api ut crash on window platform (#298)
* fix c api ut

* remove c query_by_group
2026-04-01 15:46:15 +08:00
lichen2015 ed8c010cfd
feat: add c language support (#167)
* init branch

Add GitHub Actions release workflow for C API

add release linux-arm64

remove c api version

refact some code

add api reference

* feat(c-api): add nullable/doc-result APIs for agency migration (#234)


* Flattened index parameters structure

* refact c api code

* remove RAII guard

* refact use opaque pointer pattern

* refact version info

* refact use pure opaque pointers

* use typedef instead of enum

* fix set_last_error

* fix build yml

* fix doc.h

* remove wheel exclude

* fix c msvc link

* remove release yml

* fix c link libstdc++

* fix c link libgcc

---------

Co-authored-by: Donny/강동윤 <kdy.1997.dev@gmail.com>
2026-03-31 18:00:08 +08:00
Jalin Wang 8182cfff93
feat: windows support (#216)
Preliminary support for Windows with remaining TODOs
2026-03-30 20:47:34 +08:00
Qinren Zhou cbe9ece72a
minor: rewrite some log messages (#265) 2026-03-27 16:35:32 +08:00
Qinren Zhou ae345ad070
fix: recovery from crash during optimization (#246) 2026-03-24 10:53:16 +08:00
rayx 31aaa2febb
feat: add euclidean one2many (#188)
* add euclidean one2many implementation
2026-03-23 20:20:48 +08:00
rayx b49833bf56
Refactor/add scalar dist function and setup cmake with detailed cpu flag (#238)
* fix: add scalar

* fix: add avx512fp16

* fix: cmake config
2026-03-20 09:53:48 +08:00
egolearner e5ba11b6fe
feat: add hnsw-rabitq support (#69)
* feat: add hnsw-rabitq

* undefine transform

* support config rotator type

* support sample_count

* fix ut

* refactor: update interface

* refactor: update interface

* update rabitq index params

* fix interface update

* fix searcher test

* fix streamer test

* streamer support bf and add more ut

* rm env check

* add collection ut

* add schema check

* add rabitq query param binding

* add integration test

* fix local_builder

* cleanup dist calculator

* add RaBitQ-Library submodule

* cleanup rabitq converter/reformer

* add files

* disable build on mac

* disable Feature_Optimize_HNSW_RABITQ

* disable python/tests/test_collection_hnsw_rabitq.py:13

* check avx2/avx512

* fix refine

* fix mac ci

* add dimension check

* fix mac ci

* fix ci

* add missing lib

* fix centroids selection for Cosine/InnerProduct

* rename hnsw-rabitq to hnsw_rabitq

* address comments

* fix compile

* fix rabitqlib name

* fix mac compile

* check avx2/avx512 support

* runtime check again compile-time

* check AUTO_DETECT_ARCH

* rm debug log

* fix

* rabitq use avx2 by default

* add missing file

* fix search_bf/group_by dist

* address comments

* fix typo

* address comments

* rabitq support disable id_map

* address comments
2026-03-19 17:21:42 +08:00
ZeFeng Yin 432de52c5f
[enhance] buffer storage (#83) 2026-03-19 15:24:46 +08:00
rayx d4ffbf06af
fix: fix metric batch (#245) 2026-03-19 13:39:10 +08:00
luoxiaojian 840bcdad26
feat: Introduce turbo. (#243) 2026-03-18 20:43:18 +08:00
Qinren Zhou 3c1241f7c6
feat: enlarge indice size limit for sparse vectors (#229) 2026-03-16 17:12:24 +08:00
rayx de1e7050d7
fix/fix mips euclidean (#226)
* fix: fix mips euclidean
2026-03-16 13:41:58 +08:00
Abdur-Rahmaan Janhangeer 65b45470d6
fix: minor typo (#225) 2026-03-16 11:31:44 +08:00
Qinren Zhou 9314cb14a6
fix: use per-block filter instead of per-segment filter during query (#223)
* fix: id mismatch

* fix: remove debug string

* fix: use protected

* chore(comment): add comments

* feat: more ut

* Update vector_recall_test.cc

* fix: resolve comments
2026-03-16 10:55:51 +08:00
luoxiaojian 79387c237f
feat: enable icelake and l2 batch distance for int8 quantization. (#213) 2026-03-13 17:26:22 +08:00
rayx d1e0c7b88e
fix: use static array (#222)
* fix: use static array

* fix: clang-format

* fix: clang-format

* fix: clang-format
2026-03-13 13:34:28 +08:00
rayx b6a4eaddc2
fix: turn off math march if not auto detected (#220) 2026-03-12 16:12:28 +08:00
egolearner c05974cab1
fix compile (#218)
* fix compile

* fix missing include

* Revert "fix missing include"

This reverts commit 295b52b0a5057a1c979b0316faf80e864031a0cb.

---------

Co-authored-by: rayx <rui.xing@alibaba-inc.com>
2026-03-12 14:06:04 +08:00
rayx 919f15bb36
fix: fix uint8 conversion overflow (#215) 2026-03-12 12:22:37 +08:00
rayx a7bfd6060d
refactor/march based reorganization (#193)
* add for march compatible test

* fix: remove MxN

* fix: add macro scope

* fix for android ci

* refactor: call euclidean via squared euclidean

* refactor: add match utility

* refactor: comment config out

* fix: fix dimension remainder

* fix: fix condition & config flags

* fix: remove redundant macros

* fix: fix condition error

* fix: fix macros

* fix: remove unnecessary macros

* fix: fix remaining code with vector

* fix: remove unnecessary codes

---------

Co-authored-by: Zefeng Yin <yinzefeng.yzf@alibaba-inc.com>
Co-authored-by: xufeihong.xfh <xufeihong.xfh@alibaba-inc.com>
2026-03-11 22:44:48 +08:00
Qinren Zhou 331933c75a
fix: clean up crash residue (#208) 2026-03-11 15:55:13 +08:00
ZeFeng Yin ba86c1663d
fix: add validator for cosine metric with int8 (#209) 2026-03-10 16:09:37 +08:00
ZeFeng Yin cfbd3ca008
fix: Replace std::vector with std::array for distance impl (#201) 2026-03-07 17:49:19 +08:00
feihongxu0824 fb0b900a80
fix(build): replace CMAKE_SOURCE_DIR with PROJECT_ROOT_DIR for subproject builds (#195) 2026-03-04 20:01:52 +08:00
feihongxu0824 43e3eeac40
feat: support android platform cross build (#90) 2026-03-04 16:42:23 +08:00
feihongxu0824 24d877da5e
fix: Replace VLAs with std::vector for MSVC compatibility and stack safety (#190) 2026-03-03 10:07:21 +08:00
longway 4e420311df
chore(flat): replace std::map with std::unordered_map for key_id_mapping_ (#153)
Reduces key lookup in get_id() from O(log N) to O(1) average, and index
load time from O(N log N) to O(N). Also lowers per-entry memory usage by
eliminating the three tree-node pointers (parent/left/right).

Co-authored-by: ZeFeng Yin <yinzefeng.yzf@alibaba-inc.com>
2026-02-28 13:10:38 +08:00
ZeFeng Yin ba6acbf48f
fix: ivf optimize (#169) 2026-02-27 17:35:52 +08:00
lichen2015 e7ad7cc31e
fix: combined indexer should use key instead of index (#87)
Co-authored-by: yinzefeng.yzf <yinzefeng.yzf@alibaba-inc.com>
2026-02-12 10:38:05 +08:00
feihongxu0824 efc6f3f7ce
fix: add c++ example to macos/linux ci (#86) 2026-02-09 23:31:56 +08:00
feihongxu0824 1e09d0d34c
chore: rm useless ailego module (#85) 2026-02-09 18:56:46 +08:00
lichen2015 fe90f70a3b
fix: delete_by_filter should use g_doc_id (#84) 2026-02-09 17:40:51 +08:00
Qinren Zhou 0b9bd34354
minor: subsequent initialization should fail silently (#79) 2026-02-06 21:16:08 +08:00
Jalin Wang 78f08bd0cc
feat: auto scalable segment meta section of mmap files (#67)
The underly core module only support a fixed-size segment due to the index file format where the section of segments meta limit the max num of data segments. This commit makes the meta section dynamically allocated and organized in a link-table manner to support much more segments.


* revert local_builder's meta cap setting

* feat: auto scalable segment meta section of mmap files

* Revert "feat: auto scalable segment meta section of mmap files"

This reverts commit 82c11e451544b912b901be33fae79e964a487b1f.

* v2: don't change to absolute offset
2026-02-06 14:58:56 +08:00