History

Abeautifulsnow c300336281 feat(ai): context window management and token usage tracking * @ feat(ai): add conversation context management and token usage tracking (Phase 4) ## Background Phase 4 of AI agent loop upgrade: token usage extraction from LLM responses, cost estimation display, and automatic context window management (summarization when approaching limits). ## Changes ### Backend (Rust) - New token_usage::TokenUsage — cross-turn token accumulation - New agent_loop::maybe_compact — proactive context compaction: token estimation → cut-point selection → LLM summarization → fallback summary → tool-pair integrity protection → drain/insert - New agent_loop::context_window_for_model — auto-detect context window by model - New agent_loop::is_context_length_error — detect upstream context-length errors - New agent_loop::compact_tool_result_for_context — JSON/text sampling for large results - Modified agent_loop::run_agent_loop — pre-turn compaction check, retry on context error - Modified agent_events::AgentEnd — split total_tokens into input_tokens/output_tokens - New agent_events::ContextCompacted — compaction event with metrics - New AiConfig.context_window — user-configurable context window size - New AiChatMessage.kind — context summary message marker - Modified stream_with_tools & 3 provider stream functions — extract TokenUsage - Claude: message_start(input) + message_delta(output) - OpenAI: stream_options { include_usage } → final usage chunk - Gemini: usageMetadata per chunk, keep last ### Frontend (TypeScript/Vue) - AiAssistant.vue: token + cost display row, estimateCost() with 7 model families, contextCompacted UI step, pendingCompaction deferred apply, messagesForAgentHistory post-compaction history reconstruction - EditorSettingsDialog.vue: contextWindow input (min=1000, auto-detect) - settingsStore.ts: AiConfig.contextWindow field - tauri.ts: AgentEvent type updates, AiChatMessage.kind - i18n: 6 new keys in en.ts and zh-CN.ts ## Fixed Issues - messages[0] dropped from LLM context after compaction - estimate_text_tokens hard threshold causing 2.8x estimate jump - context_compacted event field semantics mismatch with frontend insert position - validate_summary English-only keywords causing fallback on non-English LLMs - Stale agentTokens after conversation switch @ * remove cost computation logic	2026-06-15 18:55:22 +08:00
..
desktop	feat(ai): context window management and token usage tracking	2026-06-15 18:55:22 +08:00
README.md	docs(repo): add directory readmes	2026-05-18 02:50:50 +08:00

feat(ai): context window management and token usage tracking

* @
feat(ai): add conversation context management and token usage tracking (Phase 4)

## Background
Phase 4 of AI agent loop upgrade: token usage extraction from LLM responses,
cost estimation display, and automatic context window management (summarization
when approaching limits).

## Changes

### Backend (Rust)
- New token_usage::TokenUsage — cross-turn token accumulation
- New agent_loop::maybe_compact — proactive context compaction: token estimation
  → cut-point selection → LLM summarization → fallback summary
  → tool-pair integrity protection → drain/insert
- New agent_loop::context_window_for_model — auto-detect context window by model
- New agent_loop::is_context_length_error — detect upstream context-length errors
- New agent_loop::compact_tool_result_for_context — JSON/text sampling for large results
- Modified agent_loop::run_agent_loop — pre-turn compaction check, retry on context error
- Modified agent_events::AgentEnd — split total_tokens into input_tokens/output_tokens
- New agent_events::ContextCompacted — compaction event with metrics
- New AiConfig.context_window — user-configurable context window size
- New AiChatMessage.kind — context summary message marker
- Modified stream_with_tools & 3 provider stream functions — extract TokenUsage
  - Claude: message_start(input) + message_delta(output)
  - OpenAI: stream_options { include_usage } → final usage chunk
  - Gemini: usageMetadata per chunk, keep last

### Frontend (TypeScript/Vue)
- AiAssistant.vue: token + cost display row, estimateCost() with 7 model families,
  contextCompacted UI step, pendingCompaction deferred apply,
  messagesForAgentHistory post-compaction history reconstruction
- EditorSettingsDialog.vue: contextWindow input (min=1000, auto-detect)
- settingsStore.ts: AiConfig.contextWindow field
- tauri.ts: AgentEvent type updates, AiChatMessage.kind
- i18n: 6 new keys in en.ts and zh-CN.ts

## Fixed Issues
- messages[0] dropped from LLM context after compaction
- estimate_text_tokens hard threshold causing 2.8x estimate jump
- context_compacted event field semantics mismatch with frontend insert position
- validate_summary English-only keywords causing fallback on non-English LLMs
- Stale agentTokens after conversation switch
@

* remove cost computation logic

2026-06-15 18:55:22 +08:00

desktop

feat(ai): context window management and token usage tracking

2026-06-15 18:55:22 +08:00

README.md

docs(repo): add directory readmes

2026-05-18 02:50:50 +08:00

README.md

Apps

Runnable DBX applications live here.

Directories

desktop/ - the Vue frontend used by the Tauri desktop app and the Docker/web build.

The Tauri native shell remains in src-tauri/ because that is the conventional Tauri project location used by the existing build and release tooling.