feat(onboarding): multilingual embedder by default for new installs
Onboarding now asks the user once, on first run, whether to use the multilingual embedding model. The default answer is yes — defaulting to English-only made the recall promise effectively unreachable for any non-English content (cross-lingual cos ~0.35 vs ~0.88 for the multilingual model). The choice is written to config.json so subsequent runs pick the right EF without re-prompting; existing installs that never set the env var or ran onboarding stay on minilm for back-compat. MEMPALACE_EMBEDDING_MODEL still overrides both. Multilingual deps (huggingface_hub, tokenizers, numpy) move from the [multilingual] extra into core. The extra is kept as a no-op alias so existing install scripts keep working. The 300 MB ONNX model is still lazy-downloaded on first use, not at install time. `quick_setup` (the programmatic non-interactive path) grows an optional `embedding_model` arg so tests and benchmark scripts can pick a model without writing config.json by accident. EmbeddinggemmaONNX's "missing deps" error now points at the right recovery path (reinstall mempalace, since the deps are core) rather than the obsolete pip install mempalace[multilingual] hint. Tests: 9 new (3 _ask_embedding_model variants + 2 run_onboarding persistence + 2 quick_setup + 2 set_embedding_model round-trips). The existing 2 run_onboarding tests now patch _ask_embedding_model so they don't print to stdout.
This commit is contained in:
Igor Lins e Silva
parent
cef1c62fe7
commit
8a6537d2fb
|
|
@ -10,7 +10,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|||
|
||||
### Features
|
||||
|
||||
- **Opt-in multilingual embedding model: `embeddinggemma-300m` ONNX (q8, MRL→384-dim).** MemPalace's default embedder (`all-MiniLM-L6-v2`) is trained English-only — cross-lingual cosine similarity on parallel-translated text averages 0.35 across DE/FR/HI/IT/KO/RU (RU at 0.17, near-orthogonal). A Russian-speaking user effectively cannot find their own memories, which breaks the "100% recall" design promise from CLAUDE.md. New `EmbeddinggemmaONNX` class in [`mempalace/embedding.py`](mempalace/embedding.py) brings this to 0.88 average (validated lossless vs the Ollama gguf via direct ONNX-runtime test). Lazy-downloads `onnx-community/embeddinggemma-300m-ONNX` (~300 MB) on first use via `huggingface_hub`. Output is truncated to 384 dims via Matryoshka Representation Learning so the model is a drop-in for ChromaDB's 384-dim collections — no schema change. Sim prefix (`"task: sentence similarity | query: "`) is applied automatically. Opt-in via `MEMPALACE_EMBEDDING_MODEL=embeddinggemma` env var; default stays `minilm` for back-compat. Switching models on an existing palace requires re-embedding (different vector space) — run `mempalace repair rebuild-index` after changing the env var. Install with `pip install mempalace[multilingual]`. (#1483)
|
||||
- **Multilingual embedding by default for new installs: `embeddinggemma-300m` ONNX (q8, MRL→384-dim).** MemPalace's previous embedder (`all-MiniLM-L6-v2`) is trained English-only — cross-lingual cosine similarity on parallel-translated text averages 0.35 across DE/FR/HI/IT/KO/RU (RU at 0.17, near-orthogonal). A Russian-speaking user effectively cannot find their own memories, which breaks the "100% recall" design promise from CLAUDE.md. New `EmbeddinggemmaONNX` class in [`mempalace/embedding.py`](mempalace/embedding.py) brings this to 0.88 average (validated lossless vs the Ollama gguf via direct ONNX-runtime test). Lazy-downloads `onnx-community/embeddinggemma-300m-ONNX` (~300 MB) on first use via `huggingface_hub`. Output is truncated to 384 dims via Matryoshka Representation Learning so the model is a drop-in for ChromaDB's 384-dim collections — no schema change. Sim prefix (`"task: sentence similarity | query: "`) is applied automatically.
|
||||
|
||||
Onboarding (`python -m mempalace.onboarding`) now offers the multilingual model as the default — choosing it writes `embedding_model: embeddinggemma` to `config.json` so subsequent runs pick it up without re-prompting. Existing installs that never set the env var or ran onboarding stay on `minilm` (back-compat). `MEMPALACE_EMBEDDING_MODEL=minilm|embeddinggemma` overrides both. Switching models on an existing palace requires re-embedding — run `mempalace repair rebuild-index` after the change. (#1483)
|
||||
|
||||
- **Multilingual deps moved to core.** `huggingface_hub`, `tokenizers`, and `numpy` are now required deps so the multilingual path works out of the box after `pip install mempalace`. The `[multilingual]` extra is kept as a no-op alias for back-compat with install scripts. The 300 MB ONNX model itself is still lazy-downloaded on first use, not at install time.
|
||||
|
||||
- **Friendlier ChromaDB EF-name-mismatch error.** Switching `MEMPALACE_EMBEDDING_MODEL` on an existing palace without running `rebuild-index` previously surfaced ChromaDB's bare `Embedding function conflict: new: X vs persisted: Y` `ValueError` — accurate but didn't tell users how to recover. `ChromaBackend.get_collection()` now wraps that error and points at both options: revert the env var, or run `mempalace repair rebuild-index --palace <path>`. (#1483)
|
||||
|
||||
---
|
||||
|
|
|
|||
|
|
@ -168,7 +168,7 @@ verbatim drawer per user/assistant message, idempotent and resume-safe.
|
|||
|
||||
- Python 3.9+
|
||||
- A vector-store backend (ChromaDB by default)
|
||||
- ~300 MB disk for the default English-only embedding model. For multilingual recall (100+ languages, including non-Latin scripts), install with `pip install mempalace[multilingual]` and set `MEMPALACE_EMBEDDING_MODEL=embeddinggemma` — see the docstring at [`mempalace/embedding.py`](mempalace/embedding.py) for details and migration notes.
|
||||
- ~300 MB disk for the embedding model. Onboarding (`python -m mempalace.onboarding`) offers `embeddinggemma-300m` (multilingual, 100+ languages, recommended) or `all-MiniLM-L6-v2` (English-only, ~30 MB). See the docstring at [`mempalace/embedding.py`](mempalace/embedding.py) for details and migration notes.
|
||||
|
||||
No API key is required for the core benchmark path.
|
||||
|
||||
|
|
|
|||
|
|
@ -361,11 +361,12 @@ class MempalaceConfig:
|
|||
def embedding_model(self):
|
||||
"""Embedding model identifier.
|
||||
|
||||
Values: ``"minilm"`` (default, ChromaDB's all-MiniLM-L6-v2 — English-only),
|
||||
``"embeddinggemma"`` (multilingual, 100+ languages, requires
|
||||
``pip install mempalace[multilingual]``). Read from env
|
||||
Values: ``"minilm"`` (ChromaDB's all-MiniLM-L6-v2 — English-only),
|
||||
``"embeddinggemma"`` (multilingual, 100+ languages, default for
|
||||
new installs since onboarding writes the choice). Read from env
|
||||
``MEMPALACE_EMBEDDING_MODEL`` first, then ``embedding_model`` in
|
||||
``config.json``, then ``"minilm"`` for back-compat.
|
||||
``config.json``, then ``"minilm"`` as a back-compat fallback for
|
||||
palaces created before onboarding asked the question.
|
||||
|
||||
Switching models on an existing palace requires re-embedding
|
||||
(different vector space) — ChromaDB rejects reads when the persisted
|
||||
|
|
@ -377,6 +378,26 @@ class MempalaceConfig:
|
|||
return env_val.strip().lower()
|
||||
return str(self._file_config.get("embedding_model", "minilm")).strip().lower()
|
||||
|
||||
def set_embedding_model(self, model: str) -> None:
|
||||
"""Persist the embedding-model choice to ``config.json``.
|
||||
|
||||
Onboarding calls this once on first run. Accepts ``"minilm"`` or
|
||||
``"embeddinggemma"``; other values are normalized to lowercase and
|
||||
passed through (``embedding.get_embedding_function`` falls back to
|
||||
minilm for unrecognized values).
|
||||
"""
|
||||
self._file_config["embedding_model"] = str(model).strip().lower()
|
||||
self._config_dir.mkdir(parents=True, exist_ok=True)
|
||||
try:
|
||||
with open(self._config_file, "w", encoding="utf-8") as f:
|
||||
json.dump(self._file_config, f, indent=2, ensure_ascii=False)
|
||||
except OSError:
|
||||
pass
|
||||
try:
|
||||
self._config_file.chmod(0o600)
|
||||
except (OSError, NotImplementedError):
|
||||
pass
|
||||
|
||||
@property
|
||||
def topic_tunnel_min_count(self):
|
||||
"""Minimum number of overlapping confirmed topics required to create
|
||||
|
|
|
|||
|
|
@ -10,9 +10,11 @@ or ``embedding_model`` in ``~/.mempalace/config.json``:
|
|||
ChromaDB's default; what every existing palace was built with.
|
||||
* ``embeddinggemma`` — ``onnx-community/embeddinggemma-300m-ONNX`` (q8), 384-dim
|
||||
via Matryoshka truncation, multilingual (100+ languages). Cross-lingual cos
|
||||
~0.88 on parallel translations vs MiniLM's ~0.35. Requires
|
||||
``pip install mempalace[multilingual]``. Switching models on an existing
|
||||
palace requires ``mempalace repair rebuild-index`` (different vector space).
|
||||
~0.88 on parallel translations vs MiniLM's ~0.35. Recommended for any
|
||||
non-English use; onboarding offers it as the default. The ~300 MB ONNX
|
||||
model is lazy-downloaded from HuggingFace on first use. Switching models
|
||||
on an existing palace requires ``mempalace repair rebuild-index``
|
||||
(different vector space).
|
||||
|
||||
Supported devices (env ``MEMPALACE_EMBEDDING_DEVICE`` or ``embedding_device``
|
||||
in ``~/.mempalace/config.json``):
|
||||
|
|
@ -173,8 +175,10 @@ class EmbeddinggemmaONNX:
|
|||
from tokenizers import Tokenizer
|
||||
except ImportError as e:
|
||||
raise ImportError(
|
||||
"EmbeddinggemmaONNX requires huggingface_hub and tokenizers. "
|
||||
"Install with: pip install mempalace[multilingual]"
|
||||
"EmbeddinggemmaONNX requires huggingface_hub, tokenizers, and "
|
||||
"numpy — these ship with mempalace core, so this error usually "
|
||||
"means one was uninstalled or pinned to an incompatible version. "
|
||||
"Reinstall with: pip install --upgrade --force-reinstall mempalace"
|
||||
) from e
|
||||
|
||||
logger.info("Downloading %s/%s (cached after first run)…", _EMBEDDINGGEMMA_REPO, _EMBEDDINGGEMMA_ONNX)
|
||||
|
|
|
|||
|
|
@ -85,6 +85,31 @@ def _yn(prompt, default="y"):
|
|||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _ask_embedding_model() -> str:
|
||||
"""Return ``"embeddinggemma"`` (multilingual, default) or ``"minilm"``.
|
||||
|
||||
Defaulting to multilingual: the recall promise breaks for non-EN content
|
||||
under the English-only MiniLM default — cross-lingual cosine similarity
|
||||
collapses to roughly 0.35 (essentially random). Multilingual is the safe
|
||||
default; English-only is offered as an opt-down for users on slow links
|
||||
or who really only ever store English content.
|
||||
"""
|
||||
_header("Language support")
|
||||
print("""
|
||||
MemPalace embeds your memories into a vector space so the AI can find
|
||||
related content later. The default model supports 100+ languages,
|
||||
including non-Latin scripts (CJK, Cyrillic, Arabic, Devanagari).
|
||||
|
||||
Multilingual: ~300 MB one-time download, works across all languages.
|
||||
English-only: ~30 MB, smaller and faster, but cross-lingual recall
|
||||
is poor (a Russian memory and its English translation
|
||||
won't match each other).
|
||||
""")
|
||||
if _yn(" Use the multilingual embedding model?", default="y"):
|
||||
return "embeddinggemma"
|
||||
return "minilm"
|
||||
|
||||
|
||||
def _ask_mode() -> str:
|
||||
_header("Welcome to MemPalace")
|
||||
print("""
|
||||
|
|
@ -374,6 +399,13 @@ def run_onboarding(
|
|||
# Step 1: Mode
|
||||
mode = _ask_mode()
|
||||
|
||||
# Step 1b: Embedding model (asked once on first run; choice persists in
|
||||
# config.json so future loads don't re-prompt).
|
||||
embedding_model = _ask_embedding_model()
|
||||
from .config import MempalaceConfig
|
||||
|
||||
MempalaceConfig(config_dir=config_dir).set_embedding_model(embedding_model)
|
||||
|
||||
# Step 2: People
|
||||
people, aliases = _ask_people(mode)
|
||||
|
||||
|
|
@ -461,12 +493,17 @@ def quick_setup(
|
|||
projects: list = None,
|
||||
aliases: dict = None,
|
||||
config_dir: Path = None,
|
||||
embedding_model: str = None,
|
||||
) -> EntityRegistry:
|
||||
"""
|
||||
Programmatic setup without interactive prompts.
|
||||
Used in tests and benchmark scripts.
|
||||
|
||||
people: list of dicts {"name": str, "relationship": str, "context": str}
|
||||
embedding_model: optional ``"minilm"`` or ``"embeddinggemma"``. When set,
|
||||
writes the choice to ``config.json`` so subsequent runs pick the
|
||||
right EF. When omitted, the config stays untouched and the hard
|
||||
default (``"minilm"``) governs.
|
||||
"""
|
||||
registry = EntityRegistry.load(config_dir)
|
||||
registry.seed(
|
||||
|
|
@ -475,6 +512,10 @@ def quick_setup(
|
|||
projects=projects or [],
|
||||
aliases=aliases or {},
|
||||
)
|
||||
if embedding_model is not None:
|
||||
from .config import MempalaceConfig
|
||||
|
||||
MempalaceConfig(config_dir=config_dir).set_embedding_model(embedding_model)
|
||||
return registry
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -30,6 +30,16 @@ dependencies = [
|
|||
"chromadb>=1.5.4,<2",
|
||||
"pyyaml>=6.0,<7",
|
||||
"tomli>=2.0.0; python_version < '3.11'",
|
||||
# Multilingual embedding model deps. Onboarding now offers the
|
||||
# embeddinggemma-300m ONNX model as the default for new installs, so
|
||||
# these are required core deps (not optional extras). The 300 MB ONNX
|
||||
# model itself is lazy-downloaded on first use, not at install time.
|
||||
# Users who pick minilm during onboarding still get these installed
|
||||
# but unused — small wheel-size cost in exchange for one fewer pip
|
||||
# command in the multilingual setup path.
|
||||
"huggingface_hub>=0.20",
|
||||
"tokenizers>=0.15",
|
||||
"numpy>=1.24",
|
||||
]
|
||||
|
||||
[project.urls]
|
||||
|
|
@ -61,14 +71,9 @@ spellcheck = ["autocorrect>=2.0"]
|
|||
gpu = ["onnxruntime-gpu>=1.16"]
|
||||
dml = ["onnxruntime-directml>=1.16"]
|
||||
coreml = ["onnxruntime>=1.16"]
|
||||
# Multilingual embedding (embeddinggemma-300m, 100+ languages, MRL→384d).
|
||||
# Required when MEMPALACE_EMBEDDING_MODEL=embeddinggemma. Lazy-downloads
|
||||
# ~300 MB from HuggingFace on first use; cached under ~/.cache/huggingface/.
|
||||
multilingual = [
|
||||
"huggingface-hub>=0.20",
|
||||
"tokenizers>=0.15",
|
||||
"numpy>=1.24",
|
||||
]
|
||||
# Multilingual extra kept as a no-op alias for back-compat with
|
||||
# `pip install mempalace[multilingual]` — these deps now ship in core.
|
||||
multilingual = []
|
||||
|
||||
[dependency-groups]
|
||||
dev = ["pytest>=7.0", "pytest-cov>=4.0", "ruff>=0.4.0", "psutil>=5.9"]
|
||||
|
|
|
|||
|
|
@ -197,16 +197,16 @@ def test_cache_key_separates_models(monkeypatch):
|
|||
|
||||
|
||||
def test_missing_deps_raise_helpful_error(monkeypatch):
|
||||
"""If the user hasn't installed `mempalace[multilingual]`, the error must
|
||||
name the extra rather than just spilling a bare ImportError."""
|
||||
"""Multilingual deps now ship in core, but if a user ends up with a broken
|
||||
install (uninstalled tokenizers, incompatible pin, etc.) the error should
|
||||
tell them how to recover rather than spilling a bare ImportError."""
|
||||
|
||||
# Drop tokenizers from sys.modules and block re-import, simulating a user
|
||||
# who didn't install the multilingual extra. huggingface_hub and onnxruntime
|
||||
# are present (they ship with core), so the failure should land on tokenizers.
|
||||
# Simulate a user with a broken install: drop tokenizers from sys.modules
|
||||
# and block re-import. huggingface_hub and onnxruntime stay importable.
|
||||
monkeypatch.setitem(sys.modules, "tokenizers", None)
|
||||
|
||||
ef = embedding.EmbeddinggemmaONNX()
|
||||
with pytest.raises(ImportError, match=r"mempalace\[multilingual\]"):
|
||||
with pytest.raises(ImportError, match=r"pip install.*mempalace"):
|
||||
ef(["anything"])
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -6,6 +6,7 @@ from unittest.mock import patch
|
|||
from mempalace.onboarding import (
|
||||
DEFAULT_WINGS,
|
||||
_ask,
|
||||
_ask_embedding_model,
|
||||
_ask_mode,
|
||||
_ask_people,
|
||||
_ask_projects,
|
||||
|
|
@ -422,6 +423,7 @@ def test_run_onboarding_basic_flow(tmp_path):
|
|||
"""Test the full onboarding flow with minimal mocking."""
|
||||
with (
|
||||
patch("mempalace.onboarding._ask_mode", return_value="work"),
|
||||
patch("mempalace.onboarding._ask_embedding_model", return_value="embeddinggemma"),
|
||||
patch(
|
||||
"mempalace.onboarding._ask_people",
|
||||
return_value=([{"name": "Bob", "relationship": "boss", "context": "work"}], {}),
|
||||
|
|
@ -440,6 +442,7 @@ def test_run_onboarding_with_ambiguous_names(tmp_path):
|
|||
"""Onboarding prints a warning for ambiguous names."""
|
||||
with (
|
||||
patch("mempalace.onboarding._ask_mode", return_value="personal"),
|
||||
patch("mempalace.onboarding._ask_embedding_model", return_value="embeddinggemma"),
|
||||
patch(
|
||||
"mempalace.onboarding._ask_people",
|
||||
return_value=([{"name": "Grace", "relationship": "friend", "context": "personal"}], {}),
|
||||
|
|
@ -450,3 +453,118 @@ def test_run_onboarding_with_ambiguous_names(tmp_path):
|
|||
):
|
||||
registry = run_onboarding(directory=".", config_dir=tmp_path, auto_detect=False)
|
||||
assert "Grace" in registry.people
|
||||
|
||||
|
||||
# ── _ask_embedding_model ──────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_ask_embedding_model_defaults_to_multilingual():
|
||||
"""Default is multilingual: empty input (just Enter) accepts the default."""
|
||||
with patch("builtins.input", return_value=""):
|
||||
assert _ask_embedding_model() == "embeddinggemma"
|
||||
|
||||
|
||||
def test_ask_embedding_model_explicit_yes():
|
||||
with patch("builtins.input", return_value="y"):
|
||||
assert _ask_embedding_model() == "embeddinggemma"
|
||||
|
||||
|
||||
def test_ask_embedding_model_explicit_no():
|
||||
"""User opts down to English-only — choice is recorded as 'minilm'."""
|
||||
with patch("builtins.input", return_value="n"):
|
||||
assert _ask_embedding_model() == "minilm"
|
||||
|
||||
|
||||
# ── run_onboarding persists embedding_model ────────────────────────────
|
||||
|
||||
|
||||
def test_run_onboarding_persists_multilingual_choice(tmp_path):
|
||||
"""When user picks multilingual, embedding_model is written to config.json
|
||||
so subsequent loads pick up embeddinggemma without re-prompting."""
|
||||
import json
|
||||
|
||||
from mempalace.config import MempalaceConfig
|
||||
|
||||
with (
|
||||
patch("mempalace.onboarding._ask_mode", return_value="work"),
|
||||
patch("mempalace.onboarding._ask_embedding_model", return_value="embeddinggemma"),
|
||||
patch(
|
||||
"mempalace.onboarding._ask_people",
|
||||
return_value=([{"name": "Bob", "relationship": "boss", "context": "work"}], {}),
|
||||
),
|
||||
patch("mempalace.onboarding._ask_projects", return_value=[]),
|
||||
patch("mempalace.onboarding._ask_wings", return_value=["projects"]),
|
||||
patch("mempalace.onboarding._yn", return_value=False),
|
||||
patch("mempalace.onboarding._warn_ambiguous", return_value=[]),
|
||||
):
|
||||
run_onboarding(directory=".", config_dir=tmp_path, auto_detect=False)
|
||||
|
||||
config_file = tmp_path / "config.json"
|
||||
assert config_file.exists(), "onboarding must write config.json"
|
||||
data = json.loads(config_file.read_text())
|
||||
assert data["embedding_model"] == "embeddinggemma"
|
||||
assert MempalaceConfig(config_dir=tmp_path).embedding_model == "embeddinggemma"
|
||||
|
||||
|
||||
def test_run_onboarding_persists_minilm_choice(tmp_path):
|
||||
"""When user opts down to English-only, the choice is still persisted —
|
||||
we want the config to be explicit, not silently fall back to the hard
|
||||
default. This way changing the hard default later won't silently shift
|
||||
these users."""
|
||||
import json
|
||||
|
||||
with (
|
||||
patch("mempalace.onboarding._ask_mode", return_value="work"),
|
||||
patch("mempalace.onboarding._ask_embedding_model", return_value="minilm"),
|
||||
patch("mempalace.onboarding._ask_people", return_value=([], {})),
|
||||
patch("mempalace.onboarding._ask_projects", return_value=[]),
|
||||
patch("mempalace.onboarding._ask_wings", return_value=["projects"]),
|
||||
patch("mempalace.onboarding._yn", return_value=False),
|
||||
patch("mempalace.onboarding._warn_ambiguous", return_value=[]),
|
||||
):
|
||||
run_onboarding(directory=".", config_dir=tmp_path, auto_detect=False)
|
||||
|
||||
data = json.loads((tmp_path / "config.json").read_text())
|
||||
assert data["embedding_model"] == "minilm"
|
||||
|
||||
|
||||
# ── quick_setup writes embedding_model when provided ───────────────────
|
||||
|
||||
|
||||
def test_quick_setup_writes_embedding_model_when_provided(tmp_path):
|
||||
"""Programmatic setup honors the explicit embedding_model arg."""
|
||||
import json
|
||||
|
||||
quick_setup(
|
||||
mode="work",
|
||||
people=[{"name": "Bob", "relationship": "boss", "context": "work"}],
|
||||
config_dir=tmp_path,
|
||||
embedding_model="embeddinggemma",
|
||||
)
|
||||
data = json.loads((tmp_path / "config.json").read_text())
|
||||
assert data["embedding_model"] == "embeddinggemma"
|
||||
|
||||
|
||||
def test_quick_setup_leaves_config_alone_when_no_model_provided(tmp_path):
|
||||
"""Back-compat: callers that don't pass embedding_model don't get a
|
||||
surprise config.json write."""
|
||||
quick_setup(
|
||||
mode="work",
|
||||
people=[{"name": "Bob", "relationship": "boss", "context": "work"}],
|
||||
config_dir=tmp_path,
|
||||
)
|
||||
assert not (tmp_path / "config.json").exists()
|
||||
|
||||
|
||||
# ── MempalaceConfig.set_embedding_model ────────────────────────────────
|
||||
|
||||
|
||||
def test_set_embedding_model_persists_and_reloads(tmp_path):
|
||||
"""Setter writes to config.json and a fresh MempalaceConfig reads it back."""
|
||||
from mempalace.config import MempalaceConfig
|
||||
|
||||
MempalaceConfig(config_dir=tmp_path).set_embedding_model("embeddinggemma")
|
||||
assert MempalaceConfig(config_dir=tmp_path).embedding_model == "embeddinggemma"
|
||||
|
||||
MempalaceConfig(config_dir=tmp_path).set_embedding_model("MiniLM")
|
||||
assert MempalaceConfig(config_dir=tmp_path).embedding_model == "minilm"
|
||||
|
|
|
|||
20
uv.lock
20
uv.lock
|
|
@ -1173,7 +1173,13 @@ version = "3.3.5"
|
|||
source = { editable = "." }
|
||||
dependencies = [
|
||||
{ name = "chromadb" },
|
||||
{ name = "huggingface-hub", version = "1.8.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" },
|
||||
{ name = "huggingface-hub", version = "1.9.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" },
|
||||
{ name = "numpy", version = "2.0.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" },
|
||||
{ name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" },
|
||||
{ name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" },
|
||||
{ name = "pyyaml" },
|
||||
{ name = "tokenizers" },
|
||||
{ name = "tomli", marker = "python_full_version < '3.11'" },
|
||||
]
|
||||
|
||||
|
|
@ -1200,14 +1206,6 @@ gpu = [
|
|||
{ name = "onnxruntime-gpu", version = "1.24.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" },
|
||||
{ name = "onnxruntime-gpu", version = "1.25.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" },
|
||||
]
|
||||
multilingual = [
|
||||
{ name = "huggingface-hub", version = "1.8.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" },
|
||||
{ name = "huggingface-hub", version = "1.9.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" },
|
||||
{ name = "numpy", version = "2.0.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" },
|
||||
{ name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" },
|
||||
{ name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" },
|
||||
{ name = "tokenizers" },
|
||||
]
|
||||
spellcheck = [
|
||||
{ name = "autocorrect" },
|
||||
]
|
||||
|
|
@ -1225,8 +1223,8 @@ dev = [
|
|||
requires-dist = [
|
||||
{ name = "autocorrect", marker = "extra == 'spellcheck'", specifier = ">=2.0" },
|
||||
{ name = "chromadb", specifier = ">=1.5.4,<2" },
|
||||
{ name = "huggingface-hub", marker = "extra == 'multilingual'", specifier = ">=0.20" },
|
||||
{ name = "numpy", marker = "extra == 'multilingual'", specifier = ">=1.24" },
|
||||
{ name = "huggingface-hub", specifier = ">=0.20" },
|
||||
{ name = "numpy", specifier = ">=1.24" },
|
||||
{ name = "onnxruntime", marker = "extra == 'coreml'", specifier = ">=1.16" },
|
||||
{ name = "onnxruntime-directml", marker = "extra == 'dml'", specifier = ">=1.16" },
|
||||
{ name = "onnxruntime-gpu", marker = "extra == 'gpu'", specifier = ">=1.16" },
|
||||
|
|
@ -1235,7 +1233,7 @@ requires-dist = [
|
|||
{ name = "pytest-cov", marker = "extra == 'dev'", specifier = ">=4.0" },
|
||||
{ name = "pyyaml", specifier = ">=6.0,<7" },
|
||||
{ name = "ruff", marker = "extra == 'dev'", specifier = ">=0.4.0" },
|
||||
{ name = "tokenizers", marker = "extra == 'multilingual'", specifier = ">=0.15" },
|
||||
{ name = "tokenizers", specifier = ">=0.15" },
|
||||
{ name = "tomli", marker = "python_full_version < '3.11'", specifier = ">=2.0.0" },
|
||||
]
|
||||
provides-extras = ["dev", "spellcheck", "gpu", "dml", "coreml", "multilingual"]
|
||||
|
|
|
|||
Loading…
Reference in New Issue