When add_column is called on a multi-segment collection with a nullable
field and no expression, segment.cc previously sliced an Arrow ChunkedArray
with an offset that exceeded the array length, triggering SIGABRT in
Arrow's chunked_array.cc:170 assertion.
Fix the slicing logic in segment.cc to materialize null values per segment.
Add comprehensive tests in collection_test.cc and segment_test.cc covering
multi-segment add_column scenarios (nullable/non-nullable, with/without
expression, with/without unflushed data, drop+re-add).
Add modernize-use-override to .clang-tidy and apply fixes across
src/ and tests/: replace redundant virtual with override, annotate
missing override on derived methods, and drop redundant virtual on
already-overridden methods.
Add SYSTEM to target_include_directories in vendored CMake wrappers and
set INTERFACE_SYSTEM_INCLUDE_DIRECTORIES on imported / sub-project
targets so warnings from third-party headers no longer surface in our
build output. Introduces a mark_target_includes_system() helper in
cmake/utils.cmake (resolves aliases, skips missing targets) used by the
upstream-add_subdirectory wrappers (glog, gflags, googletest, yaml-cpp,
protobuf, antlr).
The COVERAGE build type uses -O0 and gcov instrumentation, which slows
C++ query execution ~5-10x. This causes test_gil_released_during_query
to exceed its 0.5s timing threshold and fail with an inconclusive result.
When a nullable scalar field has no inverted index, the forward filter path
fails to handle null values from Arrow's filter evaluation:
1. get_forward_bit(): BooleanArray::operator[] returns nullopt for null entries,
which is_filtered() treats as "no filter" (not filtered), letting null docs
through. Fix: use value_or(false) to treat null as "not matched".
2. is_matched_by_forward_filter(): reads BooleanScalar.value without checking
is_valid, which is UB for null scalars. Fix: check is_valid first.
Closes#409
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
- Set unique WORKING_DIRECTORY per test binary via cc_test()/cuda_test() to prevent filesystem path conflicts when running tests in parallel. Each test runs in ${CMAKE_BINARY_DIR}/test_tmp/${test_name}/.
- Enable parallel ctest execution in the unittest target with ProcessorCount-based --parallel flag (defaults to NPROC - 1).
- Set TEST_BINARY_DIR environment variable for crash recovery tests so they can locate helper binaries from isolated working directories.
- Update LocateDataGenerator() and LocateOptimizeGenerator() to search TEST_BINARY_DIR and TEST_BINARY_DIR/bin for helper executables.
* ci: add ccache/sccache compilation caching to speed up CI builds
- Use hendrikmuhs/ccache-action@v1.2 for Linux/macOS/iOS/Android/clang-tidy
(auto-installs ccache, manages cache, sets env vars, shows stats)
- Use mozilla-actions/sccache-action@v0.0.9 for Windows (MSVC compatible)
- Add CMAKE_C/CXX_COMPILER_LAUNCHER to all CMake build steps
- Exclude wheel build and nightly coverage workflows per decision
* ci: switch MacOS & Linux build from Unix Makefiles to Ninja generator
- Replace CMAKE_GENERATOR='Unix Makefiles' with 'Ninja' in pip build
- Replace 'make unittest -j' with 'cmake --build --target unittest --parallel'
- Add '-G Ninja' to C++ and C example cmake configure steps
- Replace 'make -j' with 'cmake --build --parallel' for examples
- Aligns with Windows and Android workflows which already use Ninja
* ci: enable parallel ctest execution with -j and --timeout
- Use CMake ProcessorCount module to detect available CPU cores
- Add -j ${NPROC} to ctest command for parallel test execution
- Add --timeout 300 to prevent individual tests from hanging CI
- Fallback to NPROC=1 when ProcessorCount returns 0
- iOS target unchanged (build-only, no test execution)
* Revert "ci: enable parallel ctest execution with -j and --timeout"
This reverts commit d196dac5f17b1f7cae443360c88bbd8c935dc145.
* fix(ci): remove sccache from Windows, fix cmake.define quote issues
Windows (05-windows-build.yml):
- Remove mozilla-actions/sccache-action: sccache incompatible with MSVC /FS flag
- Remove SCCACHE_GHA_ENABLED env var
- Remove CMAKE_C/CXX_COMPILER_LAUNCHER=sccache from build steps
- Remove 'Show sccache statistics' step
- MSVC /FS (global PDB concurrency flag) causes fatal C1041 when used with sccache
MacOS & Linux (03-macos-linux-build.yml):
- Fix cmake.define values: remove extra quotes around 'ccache' and 'ON'
- Bare values required: cmake.define.FOO=bar not cmake.define.FOO="bar"
* feat: cache key with platform and os
* ci: add compiler to ccache key to avoid cache pollution
* ci: optimize cache usage to reduce bloat
- Add max-size limits to all ccache configs (150M general, 300M Android,
100M clang-tidy) to prevent unbounded cache growth
- Remove redundant iOS full build directory cache (~1.2 GB) since ccache
already handles incremental compilation
- Fix iOS protoc cache key to use thirdparty/protobuf/** instead of
src/**, avoiding unnecessary cache misses on business code changes
* ci: trigger CI run
Rewrite file/path handling to use std::filesystem and UTF-8-safe helpers.
Switch Windows file open/create paths to wide-char APIs, replace manual
separator concatenation with PathJoin, and enable RocksDB UTF-8 filenames.
Also add UTF-8 path coverage for file IO, version manager recovery, and
collection open/flush/reopen flows.
fixes#328
Problem:
SQ8 metadata (squared_sum, sum) was computed from pre-rounded float
values, causing mismatch with actual stored int8 values. On asymmetric
datasets (e.g. OpenAI 1536D where |x_min| >> x_max), this leads to
severe recall drop.
Solution:
Move std::round before accumulating squared_sum and sum.
Co-authored-by: rayx <rui.xing@alibaba-inc.com>
lichen2015
Qinren Zhou
egolearner
ZeFeng Yin
Jason
feihongxu0824
rayx
Cuiys
luoxiaojian
Jalin Wang
kgeg401
Zihao Wang