Commit Graph
43 Commits
Author SHA1 Message Date
feihongxu0824 841b1ed3f9 fix: refactor parquet buffer cache ownership (#492) 2026-06-16 10:47:07 +08:00
luoxiaojian 7a5f84e648 feat: pass prefetch config (PO and PL) as search params (#482) 2026-06-12 16:33:41 +08:00
rayx 719e1cdb78 feat: support to pass in list size (#487) 2026-06-10 16:51:48 +08:00
feihongxu0824 4679a8f39d feat: support zvec core-only build (#481) 2026-06-09 23:45:42 +08:00
luoxiaojian dd4d8117df feat(entity/search): add LinearPool/BlockHeap, refac entity layout and access. (#450)
* feat(entity/search): add LinearPool/BlockHeap, split contiguous entity layout and add direct-pointer fast search path

Introduce single-heap candidate structures (LinearPool/BlockHeap) and
refactor greedy search to dispatch between pool and dual-heap paths.
Split node layout into separate vector/graph arrays for better cache
locality, add zero-copy get_vector_ptr() on the hot path, and extract
huge-page allocation into MemoryHelper. Also adds pyglass attribution.
2026-06-08 13:51:46 +08:00
feihongxu0824 faadf62bd5 fix: use project root for diskann cmake paths (#464) 2026-06-08 09:54:16 +08:00
luoxiaojian 823b5e6824 fix: resolve compiler warnings and enable -Werror across all CI platforms (#460) 2026-06-05 14:51:37 +08:00
rayx e720c1fd20 feat: add diskann index (#369) 2026-06-04 20:52:43 +08:00
luoxiaojian 45a11212d2 fix(vamana): medoid entry, concurrent build crash and prune quality. (#433)
* fix(vamana): medoid entry point, concurrent build crash and prune quality

- Add DiskANN-standard medoid (centroid-closest point) as the persisted
  entry point, computed at dump time, replacing the fixed node-0 start.
- Fix data races on node_chunks_ / dist_chunks_ / node_chunk_bases_ during
  concurrent build: add mutexes with double-checked locking and pre-reserve
  capacity so the lock-free read fast path never hits reallocation.
- Fix RobustPrune for metrics with signed internal distance (quantized int8
  cosine): introduce IndexMetric::build_distance_offset() to shift distances
  into a non-negative range, restoring a geometrically meaningful
  occlude_factor and improving graph quality / low-ef recall.
2026-06-02 16:15:57 +08:00
egolearner 3d6906ca20 refactor: apply modernize-use-override across codebase (#442)
- Remove redundant `virtual` keyword from methods already marked `override`
- Replace `virtual ~Derived()` with `~Derived() override` for derived class destructors
- Add missing `override` on methods that override base class virtuals
2026-06-02 13:49:39 +08:00
egolearner 23a1ef815e fix: validate reformer/entity ex_bits consistency in HnswRabitqStreamer (#436)
When a RabitqReformer trained with one total_bits is reused for an entity
configured with a different total_bits, the quantized data layout (ex_code
size) silently mismatches. This causes get_full_est() to interpret binary
quantization codes as floating-point factors, producing garbage distances
that sporadically drop search results.

Add an ex_bits() accessor to RabitqReformer and validate it against the
entity's ex_bits in HnswRabitqStreamer::open(), returning IndexError_Mismatch
on inconsistency. Fix the HNSWRabitqGeneral test to re-create a converter
and reformer with matching total_bits=2 for the third invocation. Add
TestExBitsMismatch to verify the mismatch is correctly rejected.
2026-06-02 11:12:15 +08:00
ZeFeng Yin 74beb2a828 feat: buffer storage write (#414) 2026-06-01 10:40:49 +08:00
ZeFeng Yin 0cb2d8830e feat: small block(4K) read for 2M chunk size (#406) 2026-05-26 11:27:06 +08:00
ZeFeng Yin d9b0920ac7 fix: ivf provider sorted by local id (#422) 2026-05-26 10:33:44 +08:00
egolearner 9aae7494ad chore: enable modernize-use-override and fix existing violations (#419)
Add modernize-use-override to .clang-tidy and apply fixes across
src/ and tests/: replace redundant virtual with override, annotate
missing override on derived methods, and drop redundant virtual on
already-overridden methods.
2026-05-21 19:05:32 +08:00
ZeFeng Yin bae0e1b763 chore: fix nightly coverage run-error (#417) 2026-05-21 11:47:49 +08:00
egolearner 9bfc4b556a chore: fix more warnings (#418) 2026-05-21 11:15:41 +08:00
rayx 3e02031eb8 refactor: remove hamming metric (#365)
* refactor: remove binary metric

* fix: fix windows
2026-05-18 19:31:20 +08:00
ZeFeng Yin ee40e5e0b7 fix hnsw context fetch (#386) 2026-05-11 11:17:40 +08:00
ZeFeng Yin d02ae2b74e fix: rollback hnsw ChunkSize to 2M (#383) 2026-05-07 11:00:00 +08:00
luoxiaojian efab064676 feat: refac entity and impl Vamana. (#371) 2026-04-30 10:22:25 +08:00
ZeFeng Yin 005680522b feat: merge vector arrow buffer (#320) 2026-04-29 19:24:50 +08:00
ZeFeng Yin 07e986d3d5 fix: hnsw chunk size init (#372) 2026-04-29 10:50:16 +08:00
egolearner 2b16ed39cd chore: rm hnsw-rabitq searcher/streamer (#367)
* chore: rm hnsw-rabitq searcher/streamer

* fix clang-tidy

* fix
2026-04-23 10:20:46 +08:00
ZeFeng Yin 4a4c02c1a5 chore: rm hnsw builder/searcher (#356) 2026-04-21 10:44:06 +08:00
Jalin Wang 8182cfff93 feat: windows support (#216)
Preliminary support for Windows with remaining TODOs
2026-03-30 20:47:34 +08:00
egolearner e5ba11b6fe feat: add hnsw-rabitq support (#69)
* feat: add hnsw-rabitq

* undefine transform

* support config rotator type

* support sample_count

* fix ut

* refactor: update interface

* refactor: update interface

* update rabitq index params

* fix interface update

* fix searcher test

* fix streamer test

* streamer support bf and add more ut

* rm env check

* add collection ut

* add schema check

* add rabitq query param binding

* add integration test

* fix local_builder

* cleanup dist calculator

* add RaBitQ-Library submodule

* cleanup rabitq converter/reformer

* add files

* disable build on mac

* disable Feature_Optimize_HNSW_RABITQ

* disable python/tests/test_collection_hnsw_rabitq.py:13

* check avx2/avx512

* fix refine

* fix mac ci

* add dimension check

* fix mac ci

* fix ci

* add missing lib

* fix centroids selection for Cosine/InnerProduct

* rename hnsw-rabitq to hnsw_rabitq

* address comments

* fix compile

* fix rabitqlib name

* fix mac compile

* check avx2/avx512 support

* runtime check again compile-time

* check AUTO_DETECT_ARCH

* rm debug log

* fix

* rabitq use avx2 by default

* add missing file

* fix search_bf/group_by dist

* address comments

* fix typo

* address comments

* rabitq support disable id_map

* address comments
2026-03-19 17:21:42 +08:00
ZeFeng Yin 432de52c5f [enhance] buffer storage (#83) 2026-03-19 15:24:46 +08:00
Qinren Zhou 3c1241f7c6 feat: enlarge indice size limit for sparse vectors (#229) 2026-03-16 17:12:24 +08:00
feihongxu0824 24d877da5e fix: Replace VLAs with std::vector for MSVC compatibility and stack safety (#190) 2026-03-03 10:07:21 +08:00
longwayandZeFeng Yin 4e420311df chore(flat): replace std::map with std::unordered_map for key_id_mapping_ (#153)
Reduces key lookup in get_id() from O(log N) to O(1) average, and index
load time from O(N log N) to O(N). Also lowers per-entry memory usage by
eliminating the three tree-node pointers (parent/left/right).

Co-authored-by: ZeFeng Yin <yinzefeng.yzf@alibaba-inc.com>
2026-02-28 13:10:38 +08:00
lichen2015andyinzefeng.yzf e7ad7cc31e fix: combined indexer should use key instead of index (#87)
Co-authored-by: yinzefeng.yzf <yinzefeng.yzf@alibaba-inc.com>
2026-02-12 10:38:05 +08:00
richyreachyandfeihongxu0824 a20d484e9e refactor(core): unify search invoke (#15)
Co-authored-by: feihongxu0824 <xufeihong.xfh@alibaba-inc.com>
2026-02-03 12:40:23 +08:00
姚军 ea9d957893 fix: some typos in hnsw index (#50) 2026-02-03 10:41:21 +08:00
iaojnh 8c1819ca26 change hnsw single dist to batch dist (#40) 2026-01-29 10:18:57 +08:00
feihongxu0824 0ebf5621a6 feat: support core cpp sdk (#30)
* support core cpp sdk

* fix

* fix

* fix

* fix cr
2026-01-26 20:01:44 +08:00
Jalin Wang 0af6a62c31 fix: flat support more segments (#32)
* fix

* fix compilation
2026-01-26 16:22:37 +08:00
Jalin Wang bc90952a01 refactor: flat param string cleanup (#28)
* refactor: flat param string cleanup

* fix
2026-01-26 14:49:35 +08:00
richyreachy 614f955c5f fix: remove empty files (#33) 2026-01-26 10:08:51 +08:00
feihongxu0824andzhouqinren.zqr e957442355 feat: refact cpp sdk (#27)
* refact cpp sdk

---------

Co-authored-by: zhouqinren.zqr <zhouqinren.zqr@alibaba-inc.com>
2026-01-21 20:05:42 +08:00
iaojnh e17c4a88bc refactor: clarify HNSW 'scaling_factor' as 'm' instead of 50 (#17) 2026-01-12 20:05:29 +08:00
Jalin Wang 84b5f03f5e refactor: clarify HNSW 'm' as max neighbors of upper layer (#12)
* refactor: clarify HNSW 'm' as max neighbors of upper layer

- Distinguish l0/upper_max_neighbor_cnt
- Define l0_max_neighbor_cnt as m * l0_multiplier.

* fix ut
2026-01-08 16:31:14 +08:00
sanyi 6524fabd80 Initial commit 2025-12-30 11:02:17 +08:00