Refactor CPU feature detection to use an explicit x86 whitelist (#258)

* fix(cpu_features): refactor architecture detection to explicit x86 whitelist

Currently, `cpu_features.cc` assumes any non-ARM architecture is x86/x64, which leads to a fatal missing `<cpuid.h>` error on architectures like RISC-V.
This commit refactors the preprocessor macros to explicitly whitelist x86 architectures (`__x86_64__`, `__i386__`, `_M_X64`, `_M_IX86`). All other architectures (RISC-V, ARM, etc.) will now safely fall back to the default zero-initialization, allowing cross-compilation to succeed.

Signed-off-by: ihb2032 <hebome@foxmail.com>

* ci: Add RISE RISC-V runner

Introduce the RISC-V CI runner provided by the RISE project.
This enables automated testing and building for the RISC-V architecture.

Signed-off-by: ihb2032 <hebome@foxmail.com>

* ci: use python 3.12 for RISC-V64

Signed-off-by: ihb2032 <hebome@foxmail.com>

* ci: add RISC-V numpy dependencies

Signed-off-by: ihb2032 <hebome@foxmail.com>

* ci: add RISC-V wheel cache

Signed-off-by: ihb2032 <hebome@foxmail.com>

* ci: use pre-built RISE numpy wheel to speed up riscv builds

Signed-off-by: ihb2032 <hebome@foxmail.com>

* ci: use pre-built RISE cmake wheel to speed up riscv builds

Signed-off-by: ihb2032 <hebome@foxmail.com>

* ci: split RISC-V build and test into separate jobs

Signed-off-by: ihb2032 <hebome@foxmail.com>

* ci: fix

Signed-off-by: ihb2032 <hebome@foxmail.com>

* ci: fix

Signed-off-by: ihb2032 <hebome@foxmail.com>

* Update hnsw_streamer_test.cc

* ci: add cache

Signed-off-by: ihb2032 <hebome@foxmail.com>

* Update hnsw_streamer_test.cc

* Update test_gil_release.py

* ci: schedule workflow to run overnight

---------

Signed-off-by: ihb2032 <hebome@foxmail.com>
Co-authored-by: ZeFeng Yin <yinzefeng.yzf@alibaba-inc.com>
This commit is contained in:
ihb2032
2026-06-02 11:51:25 +08:00
committed by GitHub
co-authored by ZeFeng Yin
parent 8d79f74214
commit df447a54e2
6 changed files with 420 additions and 29 deletions
+1
View File
@@ -18,6 +18,7 @@ on:
- '.github/workflows/_build_wheel_job.yml'
- '.github/workflows/continuous_bench.yml'
- '.github/workflows/nightly_coverage.yml'
- '.github/workflows/07-linux-riscv-build.yml'
- '.github/workflows/docker/**'
- '.github/workflows/scripts/**'
merge_group:
+3 -2
View File
@@ -88,7 +88,8 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip \
python -m pip install --upgrade pip
python -m pip install \
pybind11==3.0 \
cmake==3.30.0 \
ninja==1.11.1 \
@@ -151,4 +152,4 @@ jobs:
./c_api_field_schema_example
./c_api_index_example
./c_api_optimized_example
shell: bash
shell: bash
+341
View File
@@ -0,0 +1,341 @@
name: Nightly Linux RISC-V Build
on:
schedule:
- cron: '0 16 * * *'
workflow_dispatch:
permissions:
contents: read
env:
RISE_PYPI: https://gitlab.com/api/v4/projects/56254198/packages/pypi/simple
PIP_BREAK_SYSTEM_PACKAGES: 1
jobs:
build:
name: Build (linux-riscv64)
runs-on: ubuntu-24.04-riscv
steps:
- name: Checkout code
uses: actions/checkout@v6
with:
ref: main
submodules: recursive
- name: Install build dependencies
run: |
sudo mkdir -p /var/lib/dpkg/updates
sudo mkdir -p /var/lib/apt/lists/
sudo mkdir -p /var/cache/apt/archives/
sudo touch /var/lib/dpkg/status
sudo apt-get purge -y byobu || true
sudo apt-get update -o Dpkg::Lock::Timeout=300
sudo DEBIAN_FRONTEND=noninteractive apt-get install -yq \
-o Dpkg::Options::="--force-confdef" \
-o Dpkg::Options::="--force-confold" \
-o Dpkg::Lock::Timeout=300 \
ccache python3-pybind11 pybind11-dev
shell: bash
- name: Setup ccache
uses: hendrikmuhs/ccache-action@v1.2
with:
key: linux-riscv64-${{ runner.os }}-gcc
max-size: 150M
- name: Build from source
run: |
cd "$GITHUB_WORKSPACE"
NPROC=$(nproc 2>/dev/null || echo 2)
echo "Using $NPROC parallel jobs for builds"
cmake -S . -B build \
-G "Unix Makefiles" \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_TOOLS=ON \
-DBUILD_PYTHON_BINDINGS=ON \
-DCMAKE_C_COMPILER_LAUNCHER=ccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
make -C build -j"$NPROC"
shell: bash
- name: Archive entire workspace
run: |
cd "$GITHUB_WORKSPACE"
tar -cf linux-riscv64-workspace.tar .
shell: bash
- name: Upload workspace artifacts
uses: actions/upload-artifact@v7
with:
name: linux-riscv64-workspace
path: ${{ github.workspace }}/linux-riscv64-workspace.tar
if-no-files-found: error
cpp-tests:
name: C++ Tests
runs-on: ubuntu-24.04-riscv
needs: build
steps:
- name: Download workspace artifacts
uses: actions/download-artifact@v8
with:
name: linux-riscv64-workspace
path: ${{ github.workspace }}
- name: Extract workspace
run: |
cd "$GITHUB_WORKSPACE"
tar -xf linux-riscv64-workspace.tar
shell: bash
- name: Install test dependencies
run: |
sudo mkdir -p /var/lib/dpkg/updates
sudo mkdir -p /var/lib/apt/lists/
sudo mkdir -p /var/cache/apt/archives/
sudo touch /var/lib/dpkg/status
sudo apt-get purge -y byobu || true
sudo apt-get update -o Dpkg::Lock::Timeout=300
sudo DEBIAN_FRONTEND=noninteractive apt-get install -yq \
-o Dpkg::Options::="--force-confdef" \
-o Dpkg::Options::="--force-confold" \
-o Dpkg::Lock::Timeout=300 \
ccache python3-pybind11 pybind11-dev libgtest-dev liburing-dev
shell: bash
- name: Setup ccache
uses: hendrikmuhs/ccache-action@v1.2
with:
key: linux-riscv64-${{ runner.os }}-gcc
max-size: 150M
- name: Reconfigure build directory
run: |
cd "$GITHUB_WORKSPACE"
cmake -S . -B build \
-G "Unix Makefiles" \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_TOOLS=ON \
-DBUILD_PYTHON_BINDINGS=ON \
-DCMAKE_C_COMPILER_LAUNCHER=ccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
shell: bash
- name: Run C++ Tests
run: |
cd "$GITHUB_WORKSPACE/build"
NPROC=$(nproc 2>/dev/null || echo 2)
make unittest -j"$NPROC"
shell: bash
python-tests:
name: Python Tests
runs-on: ubuntu-24.04-riscv
needs: build
steps:
- name: Select Python
run: |
if command -v python3 >/dev/null 2>&1; then
PYTHON_BIN=python3
elif command -v python >/dev/null 2>&1; then
PYTHON_BIN=python
else
echo "No local Python interpreter found on PATH"
exit 1
fi
"$PYTHON_BIN" --version
echo "PYTHON=$PYTHON_BIN" >> "$GITHUB_ENV"
shell: bash
- name: Download workspace artifacts
uses: actions/download-artifact@v8
with:
name: linux-riscv64-workspace
path: ${{ github.workspace }}
- name: Extract workspace
run: |
cd "$GITHUB_WORKSPACE"
tar -xf linux-riscv64-workspace.tar
echo "$($PYTHON -c 'import site; print(site.USER_BASE)')/bin" >> "$GITHUB_PATH"
shell: bash
- name: Cache pip
uses: actions/cache@v4
with:
path: ~/.cache/pip
key: linux-riscv64-${{ runner.os }}-pip-${{ hashFiles('pyproject.toml') }}
restore-keys: |
linux-riscv64-${{ runner.os }}-pip-
- name: Install dependencies
run: |
sudo mkdir -p /var/lib/dpkg/updates
sudo mkdir -p /var/lib/apt/lists/
sudo mkdir -p /var/cache/apt/archives/
sudo touch /var/lib/dpkg/status
sudo apt-get purge -y byobu || true
sudo apt-get update -o Dpkg::Lock::Timeout=300
sudo DEBIAN_FRONTEND=noninteractive apt-get install -yq \
-o Dpkg::Options::="--force-confdef" \
-o Dpkg::Options::="--force-confold" \
-o Dpkg::Lock::Timeout=300 \
ccache libgtest-dev liburing-dev
$PYTHON -m pip install --upgrade pip
$PYTHON -m pip install numpy==2.2.2 cmake==3.30.0 ninja==1.11.1.1 --index-url "$RISE_PYPI"
$PYTHON -m pip install pybind11==3.0 pytest scikit-build-core setuptools_scm pytest-xdist
shell: bash
- name: Setup ccache
uses: hendrikmuhs/ccache-action@v1.2
with:
key: linux-riscv64-${{ runner.os }}-gcc
max-size: 150M
- name: Reconfigure build directory
run: |
cd "$GITHUB_WORKSPACE"
cmake -S . -B build \
-G "Unix Makefiles" \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_TOOLS=ON \
-DBUILD_PYTHON_BINDINGS=ON \
-DCMAKE_C_COMPILER_LAUNCHER=ccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
-Dpybind11_DIR="$($PYTHON -c 'import pybind11; print(pybind11.get_cmake_dir())')"
shell: bash
- name: Install from existing build directory
run: |
cd "$GITHUB_WORKSPACE"
NPROC=$(nproc 2>/dev/null || echo 2)
export SKBUILD_BUILD_DIR="$GITHUB_WORKSPACE/build"
CMAKE_GENERATOR="Unix Makefiles" \
CMAKE_BUILD_PARALLEL_LEVEL="$NPROC" \
$PYTHON -m pip install -v . \
--no-build-isolation \
--config-settings='cmake.define.BUILD_TOOLS="ON"' \
--config-settings='cmake.define.CMAKE_C_COMPILER_LAUNCHER=ccache' \
--config-settings='cmake.define.CMAKE_CXX_COMPILER_LAUNCHER=ccache'
shell: bash
- name: Run Python Tests
run: |
cd "$GITHUB_WORKSPACE"
$PYTHON -m pytest python/tests/
shell: bash
cpp-examples:
name: C++ Examples
runs-on: ubuntu-24.04-riscv
needs: build
steps:
- name: Download workspace artifacts
uses: actions/download-artifact@v8
with:
name: linux-riscv64-workspace
path: ${{ github.workspace }}
- name: Extract workspace
run: |
cd "$GITHUB_WORKSPACE"
tar -xf linux-riscv64-workspace.tar
shell: bash
- name: Install ccache
run: |
sudo mkdir -p /var/lib/dpkg/updates
sudo mkdir -p /var/lib/apt/lists/
sudo mkdir -p /var/cache/apt/archives/
sudo touch /var/lib/dpkg/status
sudo apt-get purge -y byobu || true
sudo apt-get update -o Dpkg::Lock::Timeout=300
sudo DEBIAN_FRONTEND=noninteractive apt-get install -yq \
-o Dpkg::Options::="--force-confdef" \
-o Dpkg::Options::="--force-confold" \
-o Dpkg::Lock::Timeout=300 \
ccache
shell: bash
- name: Setup ccache
uses: hendrikmuhs/ccache-action@v1.2
with:
key: linux-riscv64-${{ runner.os }}-gcc
max-size: 150M
- name: Run C++ Examples
run: |
cd "$GITHUB_WORKSPACE/examples/c++"
NPROC=$(nproc 2>/dev/null || echo 2)
mkdir -p build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_C_COMPILER_LAUNCHER=ccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
make -j "$NPROC"
./db-example
./core-example
./ailego-example
shell: bash
c-examples:
name: C Examples
runs-on: ubuntu-24.04-riscv
needs: build
steps:
- name: Download workspace artifacts
uses: actions/download-artifact@v8
with:
name: linux-riscv64-workspace
path: ${{ github.workspace }}
- name: Extract workspace
run: |
cd "$GITHUB_WORKSPACE"
tar -xf linux-riscv64-workspace.tar
shell: bash
- name: Install ccache
run: |
sudo mkdir -p /var/lib/dpkg/updates
sudo mkdir -p /var/lib/apt/lists/
sudo mkdir -p /var/cache/apt/archives/
sudo touch /var/lib/dpkg/status
sudo apt-get purge -y byobu || true
sudo apt-get update -o Dpkg::Lock::Timeout=300
sudo DEBIAN_FRONTEND=noninteractive apt-get install -yq \
-o Dpkg::Options::="--force-confdef" \
-o Dpkg::Options::="--force-confold" \
-o Dpkg::Lock::Timeout=300 \
ccache
shell: bash
- name: Setup ccache
uses: hendrikmuhs/ccache-action@v1.2
with:
key: linux-riscv64-${{ runner.os }}-gcc
max-size: 150M
- name: Run C Examples
run: |
cd "$GITHUB_WORKSPACE/examples/c"
NPROC=$(nproc 2>/dev/null || echo 2)
mkdir -p build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_C_COMPILER_LAUNCHER=ccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
make -j "$NPROC"
./c_api_basic_example
./c_api_collection_schema_example
./c_api_doc_example
./c_api_field_schema_example
./c_api_index_example
./c_api_optimized_example
shell: bash