memos/docs/en/open_source/modules/model_backend.md

title	desc
LLMs and Embeddings	A practical guide to configuring and using Large Language Models (LLM) and Embedders in **MemOS**.

Overview

MemOS decouples model logic from runtime config via two Pydantic factories:

Factory	Produces	Typical backends
LLMFactory	Chat model	ollama, openai, azure, qwen, deepseek, huggingface, huggingface_singleton, vllm, openai_new
EmbedderFactory	Text embedder	ollama, sentence_transformer, ark, universal_api

Both factories accept a *_ConfigFactory.model_validate(...) blob, so you can switch provider with a single backend= swap.

LLM Module

Supported LLM Backends

Backend	Notes	Example model_name_or_path
ollama	Local Ollama server	qwen3:0.6b
openai	OpenAI-compatible Chat Completions	gpt-4.1-nano
azure	Azure OpenAI Chat Completions	<your-deployment-name>
qwen	DashScope OpenAI-compatible API	qwen-plus
deepseek	DeepSeek OpenAI-compatible API	deepseek-chat / deepseek-reasoner
huggingface	Local transformers pipeline	Qwen/Qwen3-1.7B
huggingface_singleton	Same as huggingface + singleton reuse	Qwen/Qwen3-1.7B
vllm	OpenAI-compatible vLLM server	Qwen/Qwen2.5-7B-Instruct
openai_new	OpenAI Responses API wrapper	gpt-4.1

LLM Config Schema

Common fields:

Field	Type	Default	Description
model_name_or_path	str	–	Model id or local tag
temperature	float	0.7
max_tokens	int	8192
top_p / top_k	float / int	0.95 / 50
API‑specific	e.g. api_key, api_base	–	OpenAI‑compatible creds
remove_think_prefix	bool	False	Remove content within think tags from the generated text

Factory Usage

from memos.configs.llm import LLMConfigFactory
from memos.llms.factory import LLMFactory

cfg = LLMConfigFactory.model_validate({
    "backend": "ollama",
    "config": {"model_name_or_path": "qwen3:0.6b"}
})
llm = LLMFactory.from_config(cfg)

LLM Core APIs

Method	Purpose
generate(messages: list)	Return full string response
generate_stream(messages)	Yield streaming chunks

Streaming & CoT

messages = [{"role": "user", "content": "Let’s think step by step: …"}]
for chunk in llm.generate_stream(messages):
    print(chunk, end="")

::note Full code Find all scenarios in examples/basic_modules/llm.py. ::

Performance Tips

• Use qwen3:0.6b for <2 GB footprint when prototyping locally.
• Combine with KV Cache (see KVCacheMemory doc) to cut TTFT .

Embedding Module

Supported Embedder Backends

Backend	Notes	Example model_name_or_path
ollama	Local Ollama server	nomic-embed-text:latest
sentence_transformer	Local sentence-transformers	nomic-ai/nomic-embed-text-v1.5
ark	Volcano Engine Ark embeddings	<ark-model-id>
universal_api	Universal provider wrapper (e.g. OpenAI)	text-embedding-3-large

Embedder Config Schema

Shared keys: model_name_or_path, optional API creds (api_key, base_url), etc.

Factory Usage

from memos.configs.embedder import EmbedderConfigFactory
from memos.embedders.factory import EmbedderFactory

cfg = EmbedderConfigFactory.model_validate({
    "backend": "ollama",
    "config": {"model_name_or_path": "nomic-embed-text:latest"}
})
embedder = EmbedderFactory.from_config(cfg)