xiaoxue_admin
/
orca
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update Orca CLI skills for agent worktree flow (#4529) 

Co-authored-by: Orca <help@stably.ai>
This commit is contained in:
Jinwoo Hong 2026-06-03 19:45:34 -04:00 committed by GitHub
parent 6e77c02fc6
commit 163dec7010
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
3 changed files with 272 additions and 818 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

161
skills/computer-use/SKILL.md
View File

@ -1,86 +1,59 @@
---
name: computer-use
description: Use Orca's computer-use CLI to inspect and control local desktop apps through accessibility trees, screenshots, and safe UI actions. Use when an agent needs to list desktop apps, get an app state, read visible UI, click, type, press keys, scroll, drag, set values, or perform app accessibility actions. Triggers include "computer use", "orca computer", "list apps", "get app state", "read Spotify", "read Slack", "click app", "type text", "press key", "set value", "scroll app", "drag app", and desktop app interaction tasks.
description: >-
Use Orca's computer-use CLI to inspect and operate local desktop app windows
through accessibility trees, screenshots, and safe UI actions. Use for
desktop app interaction: list apps/windows, get app state, read visible UI,
click controls, type, press keys, scroll, drag, set values, or perform
accessibility actions. Also use for browser windows, webviews, Orca app UI,
or any desktop UI outside Orca's built-in browser. Triggers include "computer
use", "orca computer", "read Spotify", "read Slack", "control/click/read in
a desktop app", and "get app state".
---
# Computer Use
Use this skill when the task should operate through Orca's desktop computer-use surface rather than native Codex computer tools, raw AppleScript, ad hoc screenshots, or direct app internals.
Use this skill for desktop UI through Orca's computer-use surface.
Routing:
- Use Orca built-in browser commands (`orca snapshot`, `orca click`, `orca fill`, etc.) only for the browser page embedded inside the Orca app.
- Use `orca computer` for native desktop apps, external browser windows, app/webview chrome, Orca settings/app UI, and any desktop UI outside Orca's embedded browser.
## Preconditions
- Prefer the public `orca computer ...` command.
- In this Orca worktree, use `./config/scripts/orca-dev computer ...` when testing the local dev runtime.
- Prefer `--json` for agent-driven calls. Screenshot image bytes are omitted from JSON and written to `screenshot.path` when present.
- Do not push, submit forms, send messages, buy items, delete data, or change account settings unless the user explicitly asked for that specific action.
- If an app contains sensitive content, read only what the user requested and avoid unnecessary screenshots or logs.
Check runtime availability first:
- Prefer `orca computer ...`; on Linux, use `orca-ide computer ...` if `orca` is unavailable. In this Orca worktree, use `./config/scripts/orca-dev computer ...` only when testing the local dev runtime.
- Prefer `--json`. Screenshot bytes are omitted from JSON and written to `screenshot.path`.
- Do not push, submit forms, send messages, buy items, delete data, change account settings, or expose secrets unless the user explicitly asked for that action.
- If an app contains sensitive content, read only what the user requested.
```bash
orca status --json
orca computer capabilities --json
```
For local development against this worktree:
```bash
./config/scripts/orca-dev status --json
```
## Core Workflow
Use a snapshot-act-snapshot loop:
1. Discover apps:
## Core Loop
```bash
orca computer list-apps --json
```
2. Get a fresh state for the target app:
```bash
orca computer get-app-state --app com.spotify.client --json
```
3. Choose an element from that state.
4. Perform one action:
```bash
orca computer click --app com.spotify.client --element-index 42 --json
```
5. Inspect the action result before deciding whether to act again. Actions return a fresh state:
```bash
orca computer click --app com.spotify.client --element-index 42 --json
```
Element indexes are scoped to the current app state. They can go stale after navigation, focus changes, scrolling, window changes, or app re-rendering. Never carry indexes across unrelated steps without refreshing state.
Use the fresh state returned by each action for the next element index. Element indexes go stale after navigation, focus changes, scrolling, window changes, or app re-rendering.
## App Selectors
Prefer bundle IDs returned by `list-apps`:
Prefer bundle IDs from `list-apps`; names are acceptable when unambiguous. Use `pid:<number>` only when bundle ID or name matching is ambiguous.
```bash
orca computer get-app-state --app com.microsoft.edgemac --json
orca computer get-app-state --app com.spotify.client --json
```
Names are acceptable when unambiguous:
```bash
orca computer get-app-state --app Spotify --json
```
Use `pid:<number>` only when bundle ID or name matching is ambiguous:
```bash
orca computer get-app-state --app pid:12345 --json
```
For apps with multiple windows or ambiguous titles, run `list-windows` first. Once you choose a window, pass the same `--window-id <id>` or `--window-index <n>` to `get-app-state` and later actions until the target window changes.
## Commands
```bash
@ -89,7 +62,9 @@ orca computer capabilities --json
orca computer list-apps --json
orca computer list-windows --app <app> --json
orca computer get-app-state --app <app> --json
orca computer get-app-state --app <app> --restore-window --json
orca computer click --app <app> --element-index <index> --json
orca computer click --app <app> --x 100 --y 100 --json
orca computer perform-secondary-action --app <app> --element-index <index> --action <name> --json
orca computer set-value --app <app> --element-index <index> --value "text" --json
orca computer type-text --app <app> --text "text" --json
@ -97,96 +72,50 @@ orca computer press-key --app <app> --key Return --json
orca computer hotkey --app <app> --key CmdOrCtrl+A --json
orca computer paste-text --app <app> --text "text" --json
orca computer scroll --app <app> (--element-index <index> | --x <x> --y <y>) --direction down --json
orca computer drag --app <app> --from-element-index <index> --to-element-index <index> --json
orca computer drag --app <app> --from-x 100 --from-y 100 --to-x 300 --to-y 300 --json
```
Use `--no-screenshot` only when pixels are not needed. Screenshots are often the only useful signal for Electron, WebView, or canvas-heavy apps with shallow accessibility trees.
Coordinates are window-local. Use coordinates from the latest screenshot/state for the same target window.
Use `--text-stdin` or `--value-stdin` for sensitive text so payloads do not land in shell history.
On Linux and Windows, action payloads still pass through a short-lived local operation file.
Use `--no-screenshot` only when pixels are not needed. Use `--text-stdin` or `--value-stdin` for sensitive text so payloads do not land in shell history. On Linux and Windows, action payloads still pass through a short-lived local operation file, so avoid sending secrets unless the user explicitly asked for them:
```bash
printf '%s' "$TEXT" | orca computer set-value --app <app> --element-index <index> --value-stdin --json
```
## Choosing Actions
## Action Rules
Prefer semantic actions over raw keyboard input:
- Use `set-value` for known editable fields.
- Use `click` for buttons, tabs, menu items, checkboxes, and other direct controls.
- Use `perform-secondary-action` only when the state lists a concrete action name and the user intent matches it.
- Use `type-text` after focusing a field and confirming the app has a focused text receiver.
- Use `press-key` for navigation keys, Return, Escape, shortcuts, or submitting a field after the state confirms the right target is active.
Why: keyboard input is process-targeted on macOS, but it still depends on the target app having a valid focused receiver. `set-value` targets the accessibility element directly and is more reliable when supported.
## Foreground And Background
Some actions work while the app is in the background. Treat this as app-dependent:
- `set-value` can work in the background when the app exposes a writable accessibility value.
- `click` and accessibility actions may work in the background for some native controls.
- `type-text` and `press-key` are targeted to the app process on macOS, but the app may ignore them unless it owns focus or already has an active text receiver.
If an action returns success but the UI did not change, do not repeat the same action blindly. Run `get-app-state` again, inspect the screenshot/tree, then switch to a more semantic action or bring/focus the target if needed.
- Prefer semantic actions: `set-value` for editable fields, `click` for controls, `perform-secondary-action` only for listed action names.
- Use `type-text` only after focusing a field and confirming the app has a focused text receiver.
- Use `press-key` for single/navigation keys such as Return, Escape, Tab, and arrows. Use `hotkey` for shortcuts; prefer `CmdOrCtrl+...` for cross-platform combos.
- Some actions work in background apps, but this is app-dependent. If success does not change the UI, refresh state and choose a more semantic action or restore/focus the window.
- Coordinates are window-local; use coordinates from the latest screenshot/state for the same target window.
## Screenshots
`get-app-state` returns an accessibility tree and, by default, a screenshot. Use both:
`get-app-state` returns tree+screenshot. Use the tree for indexes/actions and the screenshot for visual confirmation; failed capture usually means hidden, minimized, off-screen, or permission-blocked.
- Trust the tree for element indexes, names, roles, values, and actions.
- Trust the screenshot for visual confirmation, especially in Electron and WebView apps.
- If the tree is shallow, use screenshot evidence before deciding whether any action is safe.
- If screenshot capture fails or returns no image, the app may be hidden, minimized, off-screen, or have no visible window.
## App Notes
Use restore only when appropriate for the task:
```bash
orca computer get-app-state --app <app> --restore-window --json
```
## App-Specific Notes
### Browsers
For Edge, Chrome, and similar browsers, prefer setting the address/search field directly:
Browsers: for Edge, Chrome, and similar browser windows, set the address/search field directly, then press Return. Do not assume raw typing went to the address bar.
```bash
orca computer get-app-state --app com.microsoft.edgemac --json
orca computer set-value --app com.microsoft.edgemac --element-index <addressBarIndex> --value "test123" --json
orca computer press-key --app com.microsoft.edgemac --key Return --json
orca computer get-app-state --app com.microsoft.edgemac --json
```
Do not assume raw typing went to the address bar. Confirm the field or page changed after pressing Return.
Spotify: refresh after playback clicks; the UI often changes asynchronously.
### Spotify
Slack: the accessibility tree may be shallow while the screenshot contains useful information. Reading visible Slack UI is fine when requested; sending messages or triggering workflows still needs explicit permission.
Spotify state can update asynchronously after playback or network-backed search. After a playback click, run `get-app-state` before clicking again.
For search, prefer `set-value` on the search combobox, usually named like `What do you want to play?`. `type-text` may only work when Spotify owns focus and that field is already focused.
### Slack
Slack may expose a shallow accessibility tree while the screenshot contains the useful information. Reading visible Slack UI is acceptable when requested, but do not send messages or trigger workflows unless explicitly asked.
## Error Handling
## Errors
- `app_not_found`: run `list-apps` and retry with the bundle ID.
- `element_not_found`: the index is stale; run `get-app-state` again.
- `element_not_found`: index is stale; run `get-app-state` again.
- `action_failed`: inspect the element role/actions and try a more semantic action.
- Empty tree or no screenshot: the app may have no visible window, be minimized, or be blocked by permissions.
- Permission errors: the user needs to grant Accessibility or Screen Recording to `Orca Computer Use`. Run `orca computer permissions --json`, use the setup UI, then retry `orca computer get-app-state --app <bundle> --json`.
- Empty tree or no screenshot: app may have no visible window, be minimized, or need permissions.
- Permission errors: run `orca computer permissions --json`, use the setup UI, then retry.
## Safety Checks
## Next Action
Before acting, classify the action:
- Safe: read state, list apps, inspect screenshot, focus a search box, scroll, open a harmless tab.
- Needs care: typing into a focused field, pressing Return, clicking a primary button.
- Requires explicit user permission: sending messages, posting, purchasing, deleting, submitting forms, changing settings, signing in, or exposing secrets.
When uncertain, stop after `get-app-state` and report what is visible instead of acting.
Confirm Orca status unless already checked, run `orca computer capabilities --json`, then get the target app state with `orca computer get-app-state --app <app> --json`.

702
skills/orca-cli/SKILL.md
View File

@ -1,127 +1,46 @@
---
name: orca-cli
description: >-
Use the `orca` CLI to drive a running Orca editor — manage Orca worktrees;
create and manage scheduled automations; create, read, and run shell commands
in Orca-managed terminals; and automate Orca's built-in browser
(snapshot/click/fill/screenshot/tabs). Use this
instead of raw `git worktree`, ad hoc shell PTYs, or Playwright whenever the
task touches Orca state. Coding agents inside an Orca worktree should also use
it to keep the worktree comment fresh at meaningful checkpoints. Boundary with
`orchestration`: if the recipient of a terminal write is another AI agent
(Claude Code, Gemini, Codex, a worker), use `orchestration` — it is the only
correct way to send messages, nudges, replies, or task hand-offs to agents.
orca-cli writes are for non-agent terminals (shells, build/test commands);
reading or `wait`ing on any terminal — including agent terminals — stays in
orca-cli.
Use the public `orca` CLI to operate Orca-managed worktrees/workspaces,
terminals, repos, automations, worktree comments, and the browser embedded
inside the Orca app. Use when the user says "$orca-cli", "use orca cli",
"Orca worktree/workspace", "child workspace", "spawn codex/claude in a
workspace", "read/wait/send Orca terminal", "terminal send", "Orca browser", or "control the
browser inside Orca". Prefer this over raw `git worktree`, ad hoc PTYs,
Playwright, or Computer Use when the task touches Orca-managed state. Use
Computer Use for browser windows, webviews, or desktop UI outside Orca's
embedded browser.
---
# Orca CLI
Use this skill when the task should go through Orca's control plane rather than directly through `git`, shell PTYs, or ad hoc filesystem access.
Use `orca` when Orca's running editor/runtime is the source of truth. On Linux, use `orca-ide` wherever this file says `orca`.
## Platform Note
Use plain shell tools when Orca state does not matter.
On Linux, the CLI command is `orca-ide` (not `orca`) to avoid conflicting with GNOME Orca, the accessibility screen reader. Everywhere this document says `orca <subcommand>`, Linux users should substitute `orca-ide <subcommand>`. macOS and Windows are unaffected.
## When To Use
Use `orca` (or `orca-ide` on Linux) for:
- worktree orchestration inside a running Orca app
- updating the current worktree comment with meaningful progress checkpoints
- reading Orca-managed terminals and sending input to non-agent terminals
- stopping or waiting on Orca-managed terminals
- creating and managing scheduled Orca automations
- accessing repos known to Orca
Do not use `orca` / `orca-ide` when plain shell tools are simpler and Orca state does not matter.
Examples:
- creating one Orca worktree per GitHub issue
- updating the current worktree comment after a significant checkpoint, such as reproducing a bug, validating a fix, or handing off for review
- finding the Claude Code terminal for a worktree and reading its status
- checking which Orca worktrees have live terminal activity
- creating a scheduled automation that runs a prompt against a known repo or worktree
## Preconditions
- Prefer the public `orca` command first (`orca-ide` on Linux)
- Orca editor/runtime should already be running, or the agent should start it with `orca open`
- Do not begin by inspecting Orca source files just to decide how to invoke the CLI. The first step is to check whether the installed `orca` / `orca-ide` command exists.
- Do not assume a generic shell environment variable proves the agent is "inside Orca". For normal agent flows, the public CLI is the supported surface, but avoid wasting a round trip on probe-only checks when a direct Orca action would answer the question.
First verify the public CLI is installed:
## Start Here
```bash
# macOS / Windows
command -v orca
# Linux
command -v orca-ide
```
Then use the public command:
```bash
orca status --json # or orca-ide on Linux
```
If the task is about Orca worktrees or Orca terminals, do this before any codebase exploration:
```bash
command -v orca # or orca-ide on Linux
command -v orca || command -v orca-ide
orca status --json
orca worktree ps --json
orca terminal list --json
```
If the agent truly needs to confirm that the current directory is inside an Orca-managed worktree, use:
```bash
orca worktree current --json
```
If `orca` / `orca-ide` is not on PATH, say so explicitly and stop or ask the user to install/register the CLI before continuing.
## Core Workflow
1. Confirm Orca runtime availability:
```bash
orca status --json
```
If Orca is not running yet:
If Orca is not running, start it:
```bash
orca open --json
orca status --json
```
2. Discover current Orca state:
Prefer `--json` for agent-driven calls. If the CLI is missing, say so explicitly instead of inspecting source files first.
```bash
orca worktree ps --json
orca terminal list --json
```
## Worktrees
3. Resolve a target worktree or terminal handle.
An Orca worktree/workspace is Orca's tracked view of a repo checkout, its metadata, terminals, browser tabs, and UI state.
4. Act through Orca:
- `worktree create/set/rm`
- `automations list/show/create/edit/remove/run/runs`
- `terminal read/send/wait/stop`
5. When the agent reaches a significant checkpoint in the current worktree, update the Orca worktree comment so the UI reflects the latest work-in-progress:
```bash
orca worktree set --worktree active --comment "reproduced auth failure with aws sts; testing credential-chain fix" --json
```
Why: the worktree comment is Orca's lightweight, agent-writable status field. Keeping it current gives the user an at-a-glance summary of what the agent most recently proved, changed, or is waiting on.
## Command Surface
### Repo
Common commands:
```bash
orca repo list --json
@ -129,52 +48,97 @@ orca repo show --repo id:<repoId> --json
orca repo add --path /abs/repo --json
orca repo set-base-ref --repo id:<repoId> --ref origin/main --json
orca repo search-refs --repo id:<repoId> --query main --limit 10 --json
```
### Worktree
```bash
orca worktree list --repo id:<repoId> --json
orca worktree ps --json
orca worktree current --json
orca worktree show --worktree id:<worktreeId> --json
orca worktree create --repo id:<repoId> --name my-task --issue 123 --comment "seed" --json
orca worktree create --repo id:<repoId> --name related-task --parent-worktree active --json
orca worktree create --repo id:<repoId> --name independent-task --no-parent --json
orca worktree show --worktree <selector> --json
orca worktree create --repo id:<repoId> --name related-task --json
orca worktree create --name child-task --agent codex --prompt "hi" --json
orca worktree create --name independent-task --no-parent --json
orca worktree set --worktree id:<worktreeId> --display-name "My Task" --json
orca worktree set --worktree active --comment "reproduced bug; collecting logs from staging" --json
orca worktree set --worktree active --comment "waiting on review" --json
orca worktree set --worktree active --comment "reproduced bug; testing fix" --json
orca worktree rm --worktree id:<worktreeId> --force --json
```
Worktree selectors supported in focused v1:
Selectors:
- `id:<worktree-id>`
- `path:<absolute-path>`
- `branch:<branch-name>`
- `issue:<number>`
- `active` / `current` to resolve the enclosing Orca-managed worktree from the shell `cwd`
- `id:<worktreeId>`, `path:<absolutePath>`, `branch:<branchName>`, `issue:<number>`
- `active` / `current` for the enclosing Orca-managed worktree from the shell cwd
### Worktree Lineage
Lineage rules:
Worktree lineage records intent; it is not a required flag sequence. When creating a worktree from inside an Orca-managed worktree, decide whether the new work is related to the current work or independent of it.
- When creating from inside an Orca-managed worktree, Orca infers the current workspace as the parent when it can.
- Use `--parent-worktree active` when the child relationship should be explicit.
- Use `--no-parent` only when the new work is independent.
- If `--repo` is omitted, Orca infers the repo from the current Orca worktree when possible.
For related work, rely on Orca's inferred parent. Use `--parent-worktree active` when the current worktree relationship should be explicit or when the shell context might not make the intended parent obvious.
Agent/setup flags:
```bash
orca worktree create --repo id:<repoId> --name related-task --json
orca worktree create --repo id:<repoId> --name related-task --parent-worktree active --json
orca worktree create --name task --agent codex --prompt "hi" --json
orca worktree create --name task --agent claude --setup run --json
orca worktree create --name task --setup skip --json
orca worktree create --name task --run-hooks --json
```
For independent work, pass `--no-parent`.
- `--agent <id>` launches that agent in the first terminal; `--prompt <text>` sends initial work to it.
- `--setup run|skip|inherit` controls repo setup hooks. Default is `inherit`, which follows the repo's setup policy.
- `--run-hooks` is a legacy alias for `--setup run`; it also reveals/activates the new worktree.
- `--agent`, `--activate`, and `--run-hooks` reveal the new worktree. Plain create stays in the background.
- Let Orca choose setup terminal placement from repo settings, including tab vs split behavior. Do not manually create extra setup terminals.
- If an older installed CLI rejects `--agent`, `--prompt`, or `--setup`, create the worktree normally, then run `orca terminal create --worktree <selector> --command "codex"` and `orca terminal send` if a prompt is needed.
## Worktree Comments
A worktree comment is the short status text shown in Orca's workspace list/card for quick progress visibility.
Coding agents should update the active worktree comment at meaningful checkpoints:
```bash
orca worktree create --repo id:<repoId> --name independent-task --no-parent --json
orca worktree set --worktree active --comment "fix implemented; running integration tests" --json
```
A different branch, issue, or name is not enough by itself to make the work independent. Treat lineage as a record of why the workspace exists, not as a property of the branch name.
Update after meaningful state changes such as repro, fix, validation, handoff, or blocker. Keep comments short/current; failures are best-effort unless Orca state was requested.
### Automations
## Terminals
Common commands:
```bash
orca terminal list --worktree id:<worktreeId> --json
orca terminal show --terminal <handle> --json
orca terminal read --terminal <handle> --json
orca terminal read --terminal <handle> --cursor <cursor> --limit 1000 --json
orca terminal read --json
orca terminal send --terminal <handle> --text "continue" --enter --json
orca terminal send --text "echo hello" --enter --json
orca terminal wait --terminal <handle> --for exit --timeout-ms 5000 --json
orca terminal wait --terminal <handle> --for tui-idle --timeout-ms 300000 --json
orca terminal stop --worktree id:<worktreeId> --json
orca terminal create --json
orca terminal create --title "Worker" --json
orca terminal create --worktree active --command "codex" --json
orca terminal split --terminal <handle> --direction vertical --json
orca terminal split --terminal <handle> --direction horizontal --command "npm test" --json
orca terminal rename --terminal <handle> --title "New Name" --json
orca terminal switch --terminal <handle> --json
orca terminal close --terminal <handle> --json
```
Terminal rules:
- `--terminal` is optional for most commands; omitted means the active terminal in the current worktree.
- Use `terminal read` before `terminal send` unless the next input is obvious.
- Use `terminal send` only for direct terminal input or one-off prompts where no task state, inbox, or reply tracking is needed.
- For structured coordination, invoke the `orchestration` skill; it uses `orca orchestration ...` commands for messages, handoffs, task DAGs, dispatches, inbox/reply flows, and coordinator loops.
- Use `terminal wait --for tui-idle` for agent CLIs such as Claude Code, Gemini, and Codex; always pass `--timeout-ms`.
- Terminal handles are runtime-scoped. If Orca restarts or returns `terminal_handle_stale`, reacquire with `terminal list`.
- For long output, use cursor reads. After a limited tail preview, page from `oldestCursor`; after a cursor read, continue with `nextCursor` while `limited` is true and `nextCursor !== latestCursor`.
- `--direction horizontal` splits left/right. `--direction vertical` splits top/bottom.
## Automations
An automation is a scheduled Orca prompt run by a chosen provider against either a repo-created worktree or an existing workspace.
```bash
orca automations list --json
@ -182,464 +146,84 @@ orca automations show <automationId> --json
orca automations create --name "Daily review" --trigger daily --time 09:00 --prompt "Review open changes" --provider codex --repo id:<repoId> --json
orca automations create --name "Weekday triage" --trigger "0 9 * * 1-5" --prompt "Triage issues" --provider claude --repo path:/abs/repo --disabled --json
orca automations create --name "Inbox digest" --trigger hourly --prompt "Summarize unread mail" --provider codex --workspace active --reuse-session --json
orca automations edit <automationId> --name "Weekday review" --trigger weekdays --time 09:30 --fresh-session --json
orca automations edit <automationId> --trigger weekdays --time 09:30 --fresh-session --json
orca automations run <automationId> --json
orca automations runs --id <automationId> --json
orca automations remove <automationId> --json
```
Automation schedules accept `hourly`, `daily`, `weekdays`, `weekly`, a 5-field cron expression, or an RRULE string. Use `--time <HH:MM>` with `daily`, `weekdays`, or `weekly`; use `--day <0-6>` only with `weekly`, where Sunday is `0`.
Schedules accept `hourly`, `daily`, `weekdays`, `weekly`, 5-field cron, or RRULE. Use `--time <HH:MM>` with `daily`/`weekdays`/`weekly`, and `--day <0-6>` only with `weekly` where Sunday is `0`.
Use `--repo <selector>` for a new worktree per run, or `--workspace <selector>` / `--workspace-mode existing` when the automation should run in an existing Orca worktree. `--repo` and `--workspace` are mutually exclusive.
Use `--repo <selector>` for a new worktree per run, or `--workspace <selector>` / `--workspace-mode existing` for an existing Orca worktree. `--repo` and `--workspace` are mutually exclusive. Use `--reuse-session` only for existing-workspace automations; if the previous terminal is gone, Orca falls back to a fresh session. Prefer `--disabled` while testing setup.
Use `--reuse-session` only for existing-workspace automations when later runs should submit into the previous live automation terminal. Use `--fresh-session` to turn reuse back off. If the previous live terminal is gone, Orca falls back to a fresh session.
## Built-In Browser
Why: automations are persisted through the running Orca runtime, so use the CLI instead of editing automation storage files directly. Prefer `--disabled` when creating an automation during tests or setup so it cannot run before the user reviews it.
The built-in browser is Orca's embedded browser tab surface, scoped to Orca worktrees; it is not Chrome/Safari or desktop app UI.
### Terminal
These commands control only Orca's embedded browser tabs. For external Chrome/Safari/webviews or Orca app chrome/settings, use the Computer Use skill/tool. If the user explicitly asks for Orca CLI desktop control, use `orca computer ...`; do not use browser commands for desktop UI.
Use selectors to discover terminals, then use the returned handle for repeated live interaction.
```bash
orca terminal list --worktree id:<worktreeId> --json
orca terminal show --terminal <handle> --json
orca terminal read --terminal <handle> --json
orca terminal read --terminal <handle> --cursor <oldestCursor> --limit 1000 --json
orca terminal send --terminal <handle> --text "continue" --enter --json
orca terminal wait --terminal <handle> --for exit --timeout-ms 5000 --json
orca terminal wait --terminal <handle> --for tui-idle --timeout-ms 30000 --json
orca terminal stop --worktree id:<worktreeId> --json
orca terminal create --json
orca terminal create --title "My Terminal" --json
orca terminal create --worktree path:/projects/myapp --command "npm test" --json
orca terminal split --terminal <handle> --direction vertical --json
orca terminal split --terminal <handle> --direction horizontal --command "npm run dev" --json
orca terminal rename --terminal <handle> --title "New Name" --json
orca terminal switch --terminal <handle> --json
orca terminal close --terminal <handle> --json
orca terminal send --text "echo hello" --enter --json
orca terminal read --json
```
Why: `--terminal` is optional for most commands. When omitted, Orca auto-resolves to the active terminal in the current worktree (same as browser commands target the active tab). Use explicit `--terminal <handle>` when operating on a specific pane.
Why: `terminal create` creates a background session unless `--focus` is explicit. Interactive local agent commands such as bare `codex` or bare `claude` use Orca's renderer-backed terminal path so they can start at the app's measured terminal geometry without stealing focus from the user.
Why: long terminal transcripts should be read with cursors. After a limited tail preview without an input cursor, page retained transcript from `oldestCursor`; in that case `nextCursor` already equals `latestCursor` and would skip omitted output. After a cursor read, if `limited` remains true and `nextCursor !== latestCursor`, continue with the returned `nextCursor`. Cursor reads default to the retained transcript size; `--limit` can request a smaller page. If `truncated` is true, older output has already fallen out of the retained buffer; use `oldestCursor` as the earliest available cursor.
Why: terminal handles are runtime-scoped and may go stale after reloads. If Orca returns `terminal_handle_stale`, reacquire a fresh handle with `terminal list`.
Why: `--direction horizontal` splits the pane **left and right** (new pane appears to the right). `--direction vertical` splits the pane **top and bottom** (new pane appears below). This matches VS Code's split convention. Default is horizontal.
## Agent Guidance
- If the user says to create/manage an Orca worktree, use `orca worktree ...`, not raw `git worktree ...`.
- If the user says to create/manage a scheduled Orca automation, use `orca automations ...`, not direct persistence edits.
- Treat Orca as the source of truth for Orca worktree and terminal tasks. Do not mix Orca-managed state with ad hoc git worktree commands unless Orca explicitly cannot perform the requested action.
- Prefer `--json` for all machine-driven use.
- Use `worktree ps` as the first summary view when many worktrees may exist.
- Use `worktree current` or `--worktree active` when the agent is already running inside the target worktree.
- When creating a worktree from an existing workspace, choose lineage based on intent: related work should keep parent context, independent work should use `--no-parent`.
- Let Orca infer the parent when the current/caller workspace is the right parent; use `--parent-worktree active` when making that relationship explicit is useful.
- Treat `orca worktree set --worktree active --comment ... --json` as a default coding-agent behavior whenever the agent reaches a meaningful checkpoint in the current Orca-managed worktree; the user does not need to explicitly ask for each update.
- Update the worktree comment at significant checkpoints, not every trivial command. Good checkpoints include reproducing a bug, confirming a hypothesis, starting a risky migration, finishing a meaningful implementation slice, switching from investigation to fix, or blocking on external input.
- Write comments as short status snapshots of the current state, for example `debugging AWS CLI profile resolution`, `confirmed flaky test is caused by temp-dir race`, or `fix implemented; running integration tests`.
- Prefer optimistic execution over probe-first flows for checkpoint updates: if `orca` (or `orca-ide` on Linux) is on `PATH`, call `orca worktree set --worktree active --comment ... --json` directly at the checkpoint instead of spending an extra cycle on `orca worktree current`.
- If that direct update fails because Orca is unavailable or the shell is not inside an Orca-managed worktree, continue the main task and treat the comment update as best-effort unless the user explicitly made Orca state part of the task.
- Use `orca worktree current --json` only when the agent actually needs the worktree identity for later logic, not as a preflight before every comment update.
- Orca only injects `ORCA_WORKTREE_PATH`-style variables for some setup-hook flows, so they are not a general detection contract for agents.
- Use `terminal list` to reacquire handles after Orca reloads.
- Use `terminal read` before `terminal send` unless the next input is obvious.
- For long agent responses, use `terminal read --json` with `oldestCursor`, `nextCursor`, `--cursor`, and `--limit` instead of relying on the default human preview. After a limited tail preview, start at `oldestCursor`; after a cursor read, continue with `nextCursor` only while `limited` is true and `nextCursor !== latestCursor`. Treat `truncated` as a signal that the requested cursor was older than the retained output.
- Use `terminal wait --terminal <handle> --for exit` only when the task actually depends on process completion.
- Use `terminal wait --terminal <handle> --for tui-idle` to wait for an agent CLI (Claude Code, Gemini, Codex, etc.) to finish its current task. This detects the working→idle OSC title transition. Always pass `--timeout-ms` as a safety net — unsupported CLIs will hang until timeout.
- Use `terminal create` to spin up new terminal tabs programmatically, optionally with a `--command` for startup (e.g. `--command "claude"` to launch Claude Code) and `--title` for labeling. In local Orca sessions, `--command "codex"` is routed through Orca's visible terminal path automatically so Codex does not start as a headless/background PTY. After creating a `--command` terminal, use `terminal wait --for tui-idle` to wait for the agent to boot before dispatching.
- Use `terminal split` to create split panes within an existing terminal tab. Pass `--command` to run a command in the new pane.
- Prefer Orca worktree selectors over hardcoded paths when Orca identity already exists.
- If the user asks for CLI UX feedback, test the public `orca` / `orca-ide` command first. Only inspect `src/cli` or use `node out/cli/index.js` if the public command is missing or the task is explicitly about implementation internals.
- If a command fails, prefer retrying with the public `orca` / `orca-ide` command before concluding the CLI is broken, unless the failure already came from the CLI itself.
## Browser Automation
The `orca` CLI (or `orca-ide` on Linux) also drives the built-in Orca browser. The core workflow is a **snapshot-interact-re-snapshot** loop:
1. **Snapshot** the page to see interactive elements and their refs.
2. **Interact** using refs (`@e1`, `@e3`, etc.) to click, fill, or select.
3. **Re-snapshot** after interactions to see the updated page state.
Use a snapshot-interact-re-snapshot loop:
```bash
orca goto --url https://example.com --json
orca snapshot --json
# Read the refs from the snapshot output
orca click --element @e3 --json
orca snapshot --json
```
### Element Refs
Refs like `@e1`, `@e5` are short identifiers assigned to interactive page elements during a snapshot. They are:
- **Assigned by snapshot**: Run `orca snapshot` to get current refs.
- **Scoped to one tab**: Refs from one tab are not valid in another.
- **Invalidated by navigation**: If the page navigates after a snapshot, refs become stale. Re-snapshot to get fresh refs.
- **Invalidated by tab switch**: Switching tabs with `orca tab switch` invalidates refs. Re-snapshot after switching.
If a ref is stale, the command returns `browser_stale_ref` — re-snapshot and retry.
### Worktree Scoping
Browser commands default to the **current worktree** — only tabs belonging to the agent's worktree are visible and targetable. Tab indices are relative to the filtered tab list.
Common commands:
```bash
# Default: operates on tabs in the current worktree
orca goto --url <url> --json
orca back --json
orca reload --json
orca snapshot --json
# Explicitly target all worktrees (cross-worktree access)
orca snapshot --worktree all --json
# Tab indices are relative to the worktree-filtered list
orca tab list --json # Shows tabs [0], [1], [2] for this worktree
orca tab switch --index 1 --json # Switches to tab [1] within this worktree
```
If no tabs are open in the current worktree, commands return `browser_no_tab`.
### Stable Page Targeting
For single-agent flows, bare browser commands are fine: Orca will target the active browser tab in the current worktree.
For concurrent or multi-process browser automation, prefer a stable page id instead of ambient active-tab state:
1. Run `orca tab list --json`.
2. Read `tabs[].browserPageId` from the result.
3. Pass `--page <browserPageId>` to follow-up commands like `snapshot`, `click`, `goto`, `screenshot`, `tab switch`, or `tab close`.
Why: active-tab state and tab indices can change while another Orca CLI process is working. `browserPageId` pins the command to one concrete tab.
```bash
orca tab list --json
orca snapshot --page page-123 --json
orca click --page page-123 --element @e3 --json
orca screenshot --page page-123 --json
orca tab switch --page page-123 --json
orca tab close --page page-123 --json
```
If you also pass `--worktree`, Orca treats it as extra scoping/validation for that page id. Without `--page`, commands still fall back to the current worktree's active tab.
### Navigation
```bash
orca goto --url <url> [--json] # Navigate to URL, waits for page load
orca back [--json] # Go back in browser history
orca forward [--json] # Go forward in browser history
orca reload [--json] # Reload the current page
```
### Observation
```bash
orca snapshot [--page <browserPageId>] [--json] # Accessibility tree snapshot with element refs
orca screenshot [--page <browserPageId>] [--format <png|jpeg>] [--json] # Viewport screenshot (base64)
orca full-screenshot [--page <browserPageId>] [--format <png|jpeg>] [--json] # Full-page screenshot (base64)
orca pdf [--page <browserPageId>] [--json] # Export page as PDF (base64)
```
### Interaction
```bash
orca click --element <ref> [--page <browserPageId>] [--json] # Click an element by ref
orca dblclick --element <ref> [--page <browserPageId>] [--json] # Double-click an element
orca fill --element <ref> --value <text> [--page <browserPageId>] [--json] # Clear and fill an input
orca type --input <text> [--page <browserPageId>] [--json] # Type at current focus (no element targeting)
orca select --element <ref> --value <value> [--page <browserPageId>] [--json] # Select dropdown option
orca check --element <ref> [--page <browserPageId>] [--json] # Check a checkbox
orca uncheck --element <ref> [--page <browserPageId>] [--json] # Uncheck a checkbox
orca scroll --direction <up|down> [--amount <pixels>] [--page <browserPageId>] [--json] # Scroll viewport
orca scrollintoview --element <ref> [--page <browserPageId>] [--json] # Scroll element into view
orca hover --element <ref> [--page <browserPageId>] [--json] # Hover over an element
orca focus --element <ref> [--page <browserPageId>] [--json] # Focus an element
orca drag --from <ref> --to <ref> [--page <browserPageId>] [--json] # Drag from one element to another
orca clear --element <ref> [--page <browserPageId>] [--json] # Clear an input field
orca select-all --element <ref> [--page <browserPageId>] [--json] # Select all text in an element
orca keypress --key <key> [--page <browserPageId>] [--json] # Press a key (Enter, Tab, Escape, etc.)
orca upload --element <ref> --files <paths> [--page <browserPageId>] [--json] # Upload files to a file input
```
### Tab Management
```bash
orca tab list [--json] # List open browser tabs
orca tab switch (--index <n> | --page <browserPageId>) [--json] # Switch active tab (invalidates refs)
orca tab create [--url <url>] [--json] # Open a new browser tab
orca tab close [--index <n> | --page <browserPageId>] [--json] # Close a browser tab
```
### Wait / Synchronization
```bash
orca wait [--timeout <ms>] [--json] # Wait for timeout (default 1000ms)
orca wait --selector <css> [--state <visible|hidden>] [--timeout <ms>] [--json] # Wait for element
orca wait --text <string> [--timeout <ms>] [--json] # Wait for text to appear on page
orca wait --url <substring> [--timeout <ms>] [--json] # Wait for URL to contain substring
orca wait --load <networkidle|load|domcontentloaded> [--timeout <ms>] [--json] # Wait for load state
orca wait --fn <js-expression> [--timeout <ms>] [--json] # Wait for JS condition to be truthy
```
After any page-changing action, pick one:
- Wait for specific content: `orca wait --text "Dashboard" --json`
- Wait for URL change: `orca wait --url "/dashboard" --json`
- Wait for network idle (catch-all for SPA navigation): `orca wait --load networkidle --json`
- Wait for an element: `orca wait --selector ".results" --json`
Avoid bare `orca wait --timeout 2000` except when debugging — it makes scripts slow and flaky.
### Data Extraction
```bash
orca exec --command "get text @e1" [--json] # Get visible text of an element
orca exec --command "get html @e1" [--json] # Get innerHTML
orca exec --command "get value @e1" [--json] # Get input value
orca exec --command "get attr @e1 href" [--json] # Get element attribute
orca exec --command "get title" [--json] # Get page title
orca exec --command "get url" [--json] # Get current URL
orca exec --command "get count .item" [--json] # Count matching elements
```
### State Checks
```bash
orca exec --command "is visible @e1" [--json] # Check if element is visible
orca exec --command "is enabled @e1" [--json] # Check if element is enabled
orca exec --command "is checked @e1" [--json] # Check if checkbox is checked
```
### Page Inspection
```bash
orca eval --expression <js> [--json] # Evaluate JS in page context
```
### Cookie Management
```bash
orca cookie get [--url <url>] [--json] # List cookies
orca cookie set --name <n> --value <v> [--domain <d>] [--json] # Set a cookie
orca cookie delete --name <n> [--domain <d>] [--json] # Delete a cookie
```
### Emulation
```bash
orca viewport --width <w> --height <h> [--scale <n>] [--mobile] [--json]
orca geolocation --latitude <lat> --longitude <lng> [--accuracy <m>] [--json]
```
### Request Interception
```bash
orca intercept enable [--patterns <list>] [--json] # Start intercepting requests
orca intercept disable [--json] # Stop intercepting
orca intercept list [--json] # List paused requests
```
> **Note:** Per-request `intercept continue` and `intercept block` are not yet supported.
> They will be added once agent-browser supports per-request interception decisions.
### Console / Network Capture
```bash
orca capture start [--json] # Start capturing console + network
orca capture stop [--json] # Stop capturing
orca console [--limit <n>] [--json] # Read captured console entries
orca network [--limit <n>] [--json] # Read captured network entries
```
### Mouse Control
```bash
orca exec --command "mouse move 100 200" [--json] # Move mouse to coordinates
orca exec --command "mouse down left" [--json] # Press mouse button
orca exec --command "mouse up left" [--json] # Release mouse button
orca exec --command "mouse wheel 100" [--json] # Scroll wheel
```
### Keyboard
```bash
orca exec --command "keyboard inserttext \"text\"" [--json] # Insert text bypassing key events
orca exec --command "keyboard type \"text\"" [--json] # Raw keystrokes
orca exec --command "keydown Shift" [--json] # Hold key down
orca exec --command "keyup Shift" [--json] # Release key
```
### Frames (Iframes)
Iframes are auto-inlined in snapshots — refs inside iframes work transparently. For scoped interaction:
```bash
orca exec --command "frame @e3" [--json] # Switch to iframe by ref
orca exec --command "frame \"#iframe\"" [--json] # Switch to iframe by CSS selector
orca exec --command "frame main" [--json] # Return to main frame
```
### Semantic Locators (alternative to refs)
When refs aren't available or you want to skip a snapshot:
```bash
orca exec --command "find role button click --name \"Submit\"" [--json]
orca exec --command "find text \"Sign In\" click" [--json]
orca exec --command "find label \"Email\" fill \"user@test.com\"" [--json]
orca exec --command "find placeholder \"Search\" type \"query\"" [--json]
orca exec --command "find testid \"submit-btn\" click" [--json]
```
### Dialogs
`alert` and `beforeunload` are auto-accepted. For `confirm` and `prompt`:
```bash
orca exec --command "dialog status" [--json] # Check for pending dialog
orca exec --command "dialog accept" [--json] # Accept
orca exec --command "dialog accept \"text\"" [--json] # Accept with prompt input
orca exec --command "dialog dismiss" [--json] # Dismiss/cancel
```
### Extended Commands (Passthrough)
```bash
orca exec --command "<agent-browser command>" [--json]
```
The `exec` command provides access to agent-browser's full command surface. Useful for commands without typed Orca handlers:
```bash
orca exec --command "set device \"iPhone 14\"" --json # Emulate device
orca exec --command "set offline on" --json # Toggle offline mode
orca exec --command "set media dark" --json # Emulate color scheme
orca exec --command "network requests" --json # View tracked network requests
orca exec --command "help" --json # See all available commands
```
**Important:** Do not use `orca exec --command "tab ..."` for tab management. Use `orca tab list/create/close/switch` instead — those operate at the Orca level and keep the UI synchronized.
### `fill` vs `type`
- **`fill`** targets a specific element by ref, clears its value first, then enters text. Use for form fields.
- **`type`** types at whatever currently has focus. Use for search boxes or after clicking into an input.
If neither works on a custom input component, try:
```bash
orca focus --element @e1 --json
orca exec --command "keyboard inserttext \"text\"" --json # bypasses key events
```
### Browser Error Codes
| Error Code | Meaning | Recovery |
| ----------------------- | -------------------------------------------- | -------------------------------------------------------------------------------------------- |
| `browser_no_tab` | No browser tab is open in this worktree | Open a tab, or use `--worktree all` to check other worktrees |
| `browser_stale_ref` | Ref is invalid (page changed since snapshot) | Run `orca snapshot` to get fresh refs |
| `browser_tab_not_found` | Tab index does not exist | Run `orca tab list` to see available tabs |
| `browser_error` | Error from the browser automation engine | Read the message for details; common causes: element not found, navigation timeout, JS error |
### Browser Worked Example
Agent fills a login form and verifies the dashboard loads:
```bash
# Navigate to the login page
orca goto --url https://app.example.com/login --json
# See what's on the page
orca snapshot --json
# Output includes:
# [@e1] text input "Email"
# [@e2] text input "Password"
# [@e3] button "Sign In"
# Fill the form
orca fill --element @e1 --value "user@example.com" --json
orca fill --element @e2 --value "s3cret" --json
# Submit
orca click --element @e3 --json
# Verify the dashboard loaded
orca snapshot --json
# Output should show dashboard content, not the login form
```
### Browser Troubleshooting
**"Ref not found" / `browser_stale_ref`**
Page changed since the snapshot. Run `orca snapshot --json` again, then use the new refs.
**Element exists but not in snapshot**
It may be off-screen or not yet rendered. Try:
```bash
orca screenshot --json
orca full-screenshot --json
orca pdf --json
orca click --element <ref> --json
orca fill --element <ref> --value <text> --json
orca type --input <text> --json
orca select --element <ref> --value <value> --json
orca check --element <ref> --json
orca scroll --direction down --amount 1000 --json
orca snapshot --json
# or wait for it:
orca wait --text "..." --json
orca snapshot --json
orca hover --element <ref> --json
orca focus --element <ref> --json
orca keypress --key Enter --json
orca upload --element <ref> --files <paths> --json
orca wait --text <text> --json
orca wait --url <substring> --json
orca wait --selector <css> --json
orca wait --load networkidle --json
orca eval --expression <js> --json
orca tab list --json
orca tab create --url <url> --json
orca tab switch --index <n> --json
orca tab close --index <n> --json
orca cookie get --json
orca capture start --json
orca console --limit 50 --json
orca network --limit 50 --json
orca exec --command "help" --json
```
**Click does nothing / overlay swallows the click**
Modals or cookie banners may be blocking. Snapshot, find the dismiss button, click it, then re-snapshot.
Browser rules:
**Fill/type doesn't work on a custom input**
Some components intercept key events. Use `keyboard inserttext`:
- Re-snapshot after navigation, tab switches, clicks that change the page, and any `browser_stale_ref`.
- Refs like `@e1` are assigned by `snapshot`, scoped to one tab, and invalidated by navigation or tab switch.
- Browser commands default to the current worktree and its active tab. Use `--worktree all` only intentionally.
- For concurrent browser work, run `orca tab list --json`, read `tabs[].browserPageId`, and pass `--page <browserPageId>` on later commands.
- Use typed tab commands (`orca tab list/create/close/switch`), not `orca exec --command "tab ..."`, so Orca keeps UI state synchronized.
- Prefer `wait --text`, `--url`, `--selector`, or `--load` after async page changes instead of bare timeouts.
- Less common workflows can use typed commands above or `orca exec --command "<agent-browser command>"` passthrough.
- If `fill` or `type` fails on a custom input, try `orca focus --element @e1 --json` then `orca inserttext --text "text" --json`.
```bash
orca focus --element @e1 --json
orca exec --command "keyboard inserttext \"text\"" --json
```
Common recoveries:
**`browser_no_tab` error**
No browser tab is open in the current worktree. Open one with `orca tab create --url <url> --json`.
- `browser_no_tab`: open a tab with `orca tab create --url <url> --json`.
- `browser_stale_ref`: run `orca snapshot --json` and retry with fresh refs.
- `browser_tab_not_found`: run `orca tab list --json` before switching or closing.
### Auto-Switch Worktree
## Next Action
Browser commands automatically activate the target worktree in the Orca UI when needed. If the agent issues a browser command targeting a worktree that isn't currently active, Orca will switch to that worktree before executing the command.
### Tab Create Auto-Activation
When `orca tab create` opens a new tab, it is automatically set as the active tab for the worktree. Subsequent commands (`snapshot`, `click`, etc.) will target the newly created tab without needing an explicit `tab switch`.
### Browser Agent Guidance
- Always snapshot before interacting with elements.
- After navigation (`goto`, `back`, `reload`, clicking a link), re-snapshot to get fresh refs.
- After switching tabs, re-snapshot.
- If you get `browser_stale_ref`, re-snapshot and retry with the new refs.
- Use `orca tab list` before `orca tab switch` to know which tabs exist.
- For concurrent browser workflows, prefer `orca tab list --json` and reuse `tabs[].browserPageId` with `--page` on later commands.
- Use `orca wait` to synchronize after actions that trigger async updates (form submits, SPA navigation, modals) instead of arbitrary sleeps.
- Use `orca eval` as an escape hatch for interactions not covered by other commands.
- Use `orca exec --command "help"` to discover extended commands.
- Worktree scoping is automatic — you'll only see tabs from your worktree by default.
- Bare browser commands without `--page` still target the current worktree's active tab, which is convenient but less robust for multi-process automation.
- Tab creation auto-activates the new tab — no need for `tab switch` after `tab create`.
- Browser commands auto-switch the active worktree if needed — no manual worktree activation required.
## Important Constraints
- Orca CLI only talks to a running Orca editor.
- Terminal handles are ephemeral and tied to the current Orca runtime. If Orca restarts, handles change.
- `terminal wait` supports `--for exit` (wait for process exit) and `--for tui-idle` (wait for a recognized agent CLI like Claude Code, Gemini, or Codex to finish its current task, detected via OSC title transitions). `tui-idle` defaults to a 5-minute timeout if `--timeout-ms` is not specified. Real coding tasks routinely take 15-60 minutes — always pass `--timeout-ms` explicitly.
- Orca is the source of truth for worktree/terminal state; do not duplicate that state with manual assumptions.
- The public `orca` command (`orca-ide` on Linux) is the interface users experience. Agents should validate and use that surface, not repo-local implementation entrypoints.
- The default bounded `terminal read` preview is for status monitoring. For retained transcript extraction, use `terminal read --json` with `oldestCursor`/`nextCursor`, `--cursor`, and `--limit`.
## References
See these docs in this repo when behavior is unclear:
- `docs/orca-cli-focused-v1-status.md`
- `docs/orca-cli-v1-spec.md`
- `docs/orca-runtime-layer-design.md`
Confirm `orca status --json` unless already checked this turn, then choose the narrowest command for the job: `worktree ps/current/create`, `terminal list/read/wait/send`, `automations list`, or built-in browser `snapshot`.

227
skills/orchestration/SKILL.md
View File

@ -1,220 +1,161 @@
---
name: orchestration
description: >-
Use for Orca agent-to-agent coordination: send/ask/reply between agent
terminals, dispatch tasks to worker agents, wait for worker_done or
escalation messages, manage task DAGs with dependencies, run decision
gates, operate coordinator loops, or decompose a spec into parallel subtasks.
Use `orca-cli` instead for terminal control, shell commands, browser
automation, worktree management, and reading or waiting on terminals.
Use Orca orchestration for structured multi-agent coordination: threaded
messages, blocking ask/reply flows, task dispatch, worker_done/escalation
waits, task DAGs, decision gates, coordinator loops, or decomposing work
across agents. Use `orca-cli` instead for ordinary terminal control,
lightweight terminal prompts, shell commands, Orca worktree management,
reading or waiting on terminals, and automation of the browser embedded inside
Orca. Use Computer Use for browser windows, webviews, Orca app UI, or desktop
UI outside Orca's embedded browser.
---
# Orca Inter-Agent Orchestration
Use this skill when the task involves coordinating multiple coding agents through Orca's orchestration system. For basic terminal and worktree management, use the `orca-cli` skill instead.
Orchestration is Orca's structured coordination layer for agent messages, task ownership, dispatch state, and worker completion tracking.
Use this skill when coordination state matters. For lightweight terminal prompts or basic worktree/terminal/built-in-browser control, use `orca-cli`.
## When To Use
- You need to send messages between agent terminals
- You need to decompose a spec into parallel subtasks with dependencies
- You need to dispatch tasks to worker agents with structured feedback
- You need to act as a coordinator managing a multi-agent workflow
- You need to create decision gates for human-in-the-loop checkpoints
## When Not To Use
Use `orca-cli` instead for ordinary terminal control, shell commands, browser automation, worktree management, or reading/waiting on terminals.
- Send/reply/ask between agent terminals with persistent messages.
- Dispatch structured tasks to workers and wait for `worker_done` or `escalation`.
- Track task DAGs with dependencies.
- Run coordinator loops or decision gates.
## Preconditions
- Orca must be running (`orca status --json` should return `runtime: true`).
- The `orca` CLI must be on PATH (`orca-ide` on Linux; installed via Settings > Browser > Enable Orca CLI).
- `orca status --json` should show a running runtime.
- `orca` must be on PATH (`orca-ide` on Linux).
- The orchestration experimental feature must be enabled in Settings > Experimental.
- All `orca orchestration` commands are RPC calls to the running Orca runtime — they require an active Orca session.
- `orca orchestration` commands are RPC calls to the running Orca runtime.
## Ownership And Handoff Boundaries
## Ownership
Orchestration messages and tasks are runtime-global. The authority for a worker completion is the active dispatch context (`taskId` + `dispatchId` + assignee handle), not the filesystem worktree by itself. Cross-worktree coordination is valid when a live coordinator intentionally owns the task graph.
Orchestration messages and tasks are runtime-global. Completion authority comes from the active dispatch context: `taskId` + `dispatchId` + assignee handle.
Do not treat a copied or injected preamble as automatic parentage for new work. First classify the situation:
Classify inherited context before sending lifecycle messages:
- **Coordinated subtask**: a live coordinator owns the DAG and is waiting on this dispatch. Use the exact `worker_done`, heartbeat, `ask`, and escalation flow from the preamble, even if the coordinator terminal is in another worktree.
- **Full handoff**: the original actor intentionally delegated ownership and does not want to monitor the work. Finish the current assignment in the current session. Create a new coordinator only when the user asks for orchestration or you deliberately decompose fresh subtasks in the current worktree; if you spawn workers, pass your current-worktree coordinator handle and use a current-worktree selector such as `--worktree active`.
- Coordinated subtask: a live coordinator owns the DAG and waits on this dispatch. Follow the preamble exactly, including `worker_done`, heartbeat/status, `ask`, and `escalation`.
- Full handoff: the original actor delegated ownership and is not monitoring. Finish in the current session. Create a new coordinator only when the user asks or you deliberately decompose fresh subtasks; if spawning workers, use your current-worktree coordinator handle and a selector such as `--worktree active`.
When the handoff type is unclear, inspect `orca orchestration task-list --json`, `orca orchestration dispatch-show`, and `orca terminal list --json` for the task, dispatch, and handle ownership before sending lifecycle messages. If you still cannot tell whether the remote handle owns an active dispatch, ask the current owner instead of silently completing a task into an unrelated workstream.
If unclear, inspect orchestration state before sending lifecycle messages:
Why: a stale or copied cross-worktree `worker_done` can make an unrelated feature coordinator responsible for work it intentionally delegated away. Conversely, refusing every cross-worktree completion would break legitimate orchestrated DAGs, so the decision must follow dispatch ownership, not location alone.
```bash
orca orchestration task-list --json
orca terminal list --json
# If inherited context includes a task id:
orca orchestration dispatch-show --task <task_id> --json
```
## Command Surface
### Messaging
Inter-agent messaging via persistent SQLite-backed mail store. Messages are delivered automatically when the recipient agent goes idle (push-on-idle).
## Messaging
```bash
orca orchestration send --to <handle|@group> --subject <text> [--from <handle>] [--body <text>] [--type <type>] [--priority <level>] [--thread-id <id>] [--payload <json>] [--json]
orca orchestration check [--terminal <handle>] [--unread] [--types <type,...>] [--inject] [--wait] [--timeout-ms <n>] [--json]
orca orchestration reply --id <msg_id> --body <text> [--from <handle>] [--json]
orca orchestration ask --to <handle> --question <text> [--options <csv>] [--timeout-ms <n>] [--from <handle>] [--json]
orca orchestration inbox [--limit <n>] [--json]
```
Why: `--from` auto-resolves via the `ORCA_TERMINAL_HANDLE` environment variable injected into every Orca-managed terminal. Omit it unless impersonating another terminal.
Rules:
Why: `--inject` formats messages as readable banners with priority indicators (`[HIGH]`, `[URGENT]`) for agent prompt injection. Use `--json` for machine-readable output.
- Omit `--from` unless impersonating another terminal; Orca auto-resolves it from the current terminal.
- While supervising workers manually, use `check --wait --types worker_done,escalation,decision_gate --timeout-ms <n>` instead of sleep/poll loops. Reply to `decision_gate` messages with `orca orchestration reply --id <msg_id> --body <answer> --json`, then keep waiting.
- Use `ask` when a worker needs a blocking answer from the coordinator; it waits for the reply and returns the answer directly.
- `check --wait` returns one message at a time. If N workers may finish together, loop N times and dispatch newly ready tasks after each completion.
- Group addresses include `@all`, `@idle`, `@claude`, `@codex`, `@opencode`, `@gemini`, and `@worktree:<id>`.
- Message types include `status`, `dispatch`, `worker_done`, `merge_ready`, `escalation`, `handoff`, `decision_gate`, and `heartbeat`.
Why: `--wait` blocks until a matching message arrives or the timeout expires (default 2 minutes). This replaces sleep+poll loops. If unread messages already exist, returns immediately. Combine with `--types` to wait for specific message types (e.g. `--wait --types worker_done --timeout-ms 120000`).
## Tasks And Dispatch
**Message types**: `status` (general), `dispatch` (assign work), `worker_done` (signal completion), `merge_ready` (branch ready for merge), `escalation` (issue requiring attention), `handoff` (pass work to another agent), `decision_gate` (human-in-the-loop).
**Priority levels**: `normal`, `high`, `urgent`.
**Group addresses** resolve to terminal handles:
| Group | Resolves To |
| ---------------- | ----------------------------------------- |
| `@all` | All terminal handles except sender |
| `@idle` | Handles where the agent is currently idle |
| `@claude` | Handles running Claude Code |
| `@codex` | Handles running Codex |
| `@opencode` | Handles running OpenCode |
| `@gemini` | Handles running Gemini |
| `@worktree:<id>` | All handles in a specific worktree |
Group messages fan out: one message per recipient, shared `thread_id`, independent read tracking.
### Tasks
Task tracking with DAG dependencies. A task becomes `ready` when all tasks in its `deps` array are `completed`.
A task is the work item, a dispatch assigns it to a terminal, and a gate blocks progress until a coordinator or user decision is recorded.
```bash
orca orchestration task-create --spec <text> [--deps <json_array>] [--parent <task_id>] [--json]
orca orchestration task-list [--status <status>] [--ready] [--json]
orca orchestration task-update --id <task_id> --status <status> [--result <json>] [--json]
```
**Task statuses**: `pending` (waiting on deps), `ready` (deps met, dispatchable), `dispatched` (assigned to a terminal), `completed`, `failed`, `blocked` (waiting on a decision gate).
Why: when a task is marked `completed`, the runtime automatically promotes any pending tasks whose deps are now all satisfied to `ready`. This is the DAG resolution step.
### Dispatch
Dispatch assigns a ready task to a terminal. Optionally injects the task spec + preamble into the terminal so the agent knows how to communicate back.
```bash
orca orchestration dispatch --task <task_id> --to <handle> [--from <handle>] [--inject] [--json]
orca orchestration dispatch-show --task <task_id> [--json]
```
Why: `--inject` sends a preamble that teaches the agent how to use `orca orchestration send --type worker_done` to report completion. All agents have `orca` (or `orca-ide` on Linux) on PATH and can execute shell commands. The preamble maximizes structured feedback but the system works without it (coordinator falls back to idle detection + output reading).
Task statuses: `pending`, `ready`, `dispatched`, `completed`, `failed`, `blocked`.
Why: `--inject` requires a recognized agent CLI (e.g. Claude Code) running in the target terminal. If the terminal is a bare shell, omit `--inject` and send the prompt manually with `terminal send`.
Dispatch rules:
Why: dispatch contexts are separate from tasks (sling pattern). A task can be dispatched, fail, and be re-dispatched to a different terminal — the task stays clean while dispatch contexts track retry state.
- `--inject` sends the task spec plus preamble into a recognized agent CLI so it can report `worker_done`.
- If the target is a bare shell, omit `--inject`, dispatch for tracking if needed, then send the prompt manually with `orca terminal send --terminal <handle> --text <prompt> --enter --json`.
- After 3 consecutive failures on one task, the dispatch context circuit-breaks and the task is marked failed.
**Circuit breaker**: After 3 consecutive failures on a task, the dispatch context is marked `circuit_broken`. The task is marked `failed` to prevent infinite retry loops.
### Decision Gates
Human-in-the-loop decision points that block a task until resolved.
## Gates And Coordinator
```bash
orca orchestration gate-create --task <task_id> --question <text> [--options <json_array>] [--json]
orca orchestration gate-resolve --id <gate_id> --resolution <text> [--json]
orca orchestration gate-list [--task <task_id>] [--status <status>] [--json]
```
Why: creating a gate blocks the task and completes its active dispatch. Resolving a gate sets the task back to `ready` with the resolution context included in the next dispatch preamble.
**Gate statuses**: `pending`, `resolved`, `timeout`.
### Coordinator
Start an automated coordinator loop that dispatches ready tasks, processes `worker_done`/`escalation` messages, and advances the task DAG.
```bash
orca orchestration run --spec <text> [--from <handle>] [--poll-interval-ms <n>] [--max-concurrent <n>] [--worktree <selector>] [--json]
orca orchestration run-stop [--json]
```
Why: `run` returns immediately with a run ID. The coordinator loop runs in the background inside the Orca runtime. Query progress via `orca orchestration task-list`. Only one coordinator can run at a time.
`run` returns immediately with a run ID. Query progress with `task-list`. Use `ask` for worker-to-coordinator questions; it creates a `decision_gate` message that the coordinator answers with `reply`. Use `gate-create` only for coordinator-managed task DAG decisions, not for answering a worker's `ask`.
**Coordinator phases**: `decomposing` → `dispatching` → `monitoring` → `merging` → `done`.
Recovery only: `orca orchestration reset --tasks|--messages|--all --json` clears runtime-global orchestration state. Do not run it during active coordination unless explicitly abandoning that state.
### Lifecycle
## Worker Terminals
For tracked orchestration work, create the worker without a prompt, wait for idle, then dispatch with `--inject`:
```bash
orca orchestration reset [--all] [--tasks] [--messages] [--json]
orca worktree create --name <task-name> --agent codex --json
orca terminal list --worktree id:<newWorktreeId> --json
orca terminal wait --terminal <handle> --for tui-idle --timeout-ms 60000 --json
orca orchestration dispatch --task <task_id> --to <handle> --inject --json
```
Why: `--all` is the default if no flags provided. `--tasks` clears tasks, dispatch contexts, decision gates, and coordinator runs but preserves messages.
Read the new worktree id from `worktree create`, then use `terminal list` to get the agent terminal handle. Omit `--repo` only when running inside an Orca-managed worktree; otherwise pass `--repo <selector>`. `--agent` reveals the new worktree and launches the selected agent in its first terminal, so do not create a separate startup terminal. Use `--setup run|skip|inherit` when setup behavior matters, and `--no-parent` for independent work.
### Terminal Commands for Coordinators
Use `orca worktree create --prompt ...` or `orca terminal send ...` only for untracked/lightweight prompts. Those paths do not attach `taskId`/`dispatchId`; the worker should not send lifecycle messages unless the prompt supplies a live orchestration preamble.
Coordinators need these terminal commands to spawn agents, monitor progress, and read output. Full terminal documentation lives in the `orca-cli` skill — this is the subset required for orchestration workflows.
Other terminal commands coordinators often need:
```bash
orca terminal list [--worktree <selector>] [--json]
orca terminal create [--worktree <selector>] [--title <text>] [--command <cmd>] [--json]
orca terminal split --terminal <handle> [--direction horizontal|vertical] [--command <cmd>] [--json]
orca terminal read [--terminal <handle>] [--json]
orca terminal send [--terminal <handle>] --text <text> [--enter] [--json]
orca terminal wait [--terminal <handle>] --for <exit|tui-idle> [--timeout-ms <n>] [--json]
orca terminal show --terminal <handle> [--json]
orca terminal stop [--terminal <handle>] [--json]
orca terminal close [--terminal <handle>] [--json]
orca terminal wait --terminal <handle> --for tui-idle --timeout-ms <n> --json
orca terminal read --terminal <handle> --json
orca terminal send --terminal <handle> --text <text> --enter --json
```
Why: `--terminal` is optional for most commands. When omitted, Orca auto-resolves to the active terminal in the current worktree.
If an older CLI rejects `worktree create --agent`, create the worktree normally, then run `orca terminal create --worktree <selector> --command "codex" --json` or `--command "claude"`.
Why: `--command "claude"` launches Claude Code in the new terminal. In local Orca sessions, `--command "codex"` launches Codex through Orca's visible terminal path automatically so Codex does not start as a headless/background PTY. After creating a `--command` terminal, use `terminal wait --for tui-idle` to wait for the agent to boot before dispatching.
Why: `--for tui-idle` detects the working→idle OSC title transition for recognized agent CLIs (Claude Code, Gemini, Codex, etc.). Always pass `--timeout-ms` — real coding tasks routinely take 15-60 minutes.
Why: `--direction horizontal` splits left/right (new pane to the right). `--direction vertical` splits top/bottom (new pane below). Default is horizontal.
Why: terminal handles are runtime-scoped. If Orca restarts, handles go stale. Re-acquire with `terminal list`.
Why: the 120-line terminal output buffer (`terminal read`) is for status monitoring, not result extraction. Prefer structured `worker_done` payloads over parsing terminal output.
Wait for `tui-idle` before dispatching. Always pass `--timeout-ms`; real coding tasks can take 15-60 minutes. If `check --wait` times out with no `worker_done` or `escalation`, fall back to `terminal wait --for tui-idle`, then `terminal read`.
## Agent Guidance
- When dispatched with a valid live preamble, **send `worker_done` exactly once to the owning coordinator**. The `dispatchId` in the payload is the completion authority.
- Treat a preamble inherited through terminal history or a full handoff as stale unless the current prompt explicitly keeps that coordinator in the loop.
- If blocked or unable to complete a task, send an `escalation` message to the owning coordinator only when ownership is valid; otherwise report the blocker in the current session.
- Use `orca orchestration check` to read incoming messages from the coordinator or other agents. Messages are delivered automatically when you go idle, but you can also poll explicitly.
- Treat `orca orchestration` commands the same way you treat `git` or `npm` — they are CLI tools available in your shell.
- The coordinator uses `orca orchestration task-list --ready` as its external memory. Prefer querying orchestration state over tracking it in your context window.
- For multi-agent coordination, prefer the **inter-worktree** pattern (each agent in its own worktree) for parallel implementation tasks. Use **intra-worktree** (split panes, shared files) for complementary tasks where agents don't edit the same files.
- When acting as coordinator: discover existing agents with `terminal list`, create tasks with `task-create`, dispatch with `dispatch --inject`, and wait for `worker_done` messages via `check --wait --types worker_done,escalation --timeout-ms 300000`.
- When acting as coordinator: prefer `check --wait` over sleep+poll loops. `--wait` blocks until a message arrives, eliminating wasted time. Always pass `--timeout-ms` as a safety net. If the wait times out with no messages, fall back to `terminal wait --for tui-idle` and then reading terminal output.
- `check --wait` returns one message at a time. If N workers finish near-simultaneously, call `check --wait` N times in a loop to collect all results. After each return, mark the task complete (which auto-promotes dependents) and dispatch the next wave before looping back to wait.
- After receiving `worker_done` from a terminal, that terminal is guaranteed idle — skip the `terminal wait --for tui-idle` round-trip and dispatch the next task immediately.
- Terminal handles are ephemeral and runtime-scoped. If Orca restarts mid-workflow, all handles go stale. Re-acquire them with `terminal list` before continuing.
- Keep dependency chains to 3-4 steps maximum. Prefer parallel waves of independent tasks over deep sequential chains.
- Insert decision gates (`gate-create`) between phases for human oversight on risky operations.
- Workers with a valid live preamble must send `worker_done` exactly once, even on failure:
`orca orchestration send --to <coordinator_handle> --type worker_done --subject "<short status>" --body "<3-sentence summary: what you did, what you found, what's left>" --payload '{"taskId":"<task_id>","dispatchId":"<dispatch_id>","filesModified":["path/a"],"reportPath":"<optional>"}' --json`
- For long tasks, send heartbeat/status only when the preamble asks for it, including both IDs:
`orca orchestration send --to <coordinator_handle> --type heartbeat --subject "alive" --payload '{"taskId":"<task_id>","dispatchId":"<dispatch_id>","phase":"implementing"}' --json`
- If blocked before completion, use `ask`; use `escalation` only when ownership is valid and the coordinator must intervene.
- Treat preambles inherited through terminal history or full handoffs as stale unless the current prompt explicitly keeps that coordinator in the loop.
- Coordinators should use `task-list --ready` as external memory, dispatch parallel waves, and avoid dependency chains deeper than 3-4 steps.
- Prefer inter-worktree workers for parallel implementation; use split panes in one worktree only for complementary tasks that will not edit the same files.
## Coordinator Worked Example
Dispatch a task to a fresh Claude Code terminal and wait for completion:
## Example
```bash
# 1. Create a terminal running Claude Code
orca terminal create --worktree active --title "worker-1" --command "claude" --json
# → handle: term_abc123
# 2. Wait for Claude Code to boot (tui-idle fires when the prompt appears)
orca terminal wait --terminal term_abc123 --for tui-idle --timeout-ms 60000 --json
# 3. Create and dispatch a task with preamble injection
orca worktree create --name login-css-worker --agent claude --json
orca terminal list --worktree id:<newWorktreeId> --json
orca terminal wait --terminal <handle> --for tui-idle --timeout-ms 60000 --json
orca orchestration task-create --spec "Fix the login button CSS" --json
# → id: task_def456
orca orchestration dispatch --task task_def456 --to term_abc123 --inject --json
# 4. Block until the worker reports back (no sleep loops needed)
orca orchestration check --wait --types worker_done,escalation --timeout-ms 300000 --json
# → returns immediately when worker sends worker_done
# 5. If --wait timed out with no messages, fall back to idle detection
orca terminal wait --terminal term_abc123 --for tui-idle --timeout-ms 60000 --json
orca terminal read --terminal term_abc123 --json
orca orchestration dispatch --task <task_id> --to <handle> --inject --json
orca orchestration check --wait --types worker_done,escalation,decision_gate --timeout-ms 300000 --json
```
## Next Action
Coordinator: confirm `orca status --json`, inspect `task-list`/`dispatch-show` if inheriting state, then choose either a manual loop (`task-create` -> worker -> `dispatch --inject` -> `check --wait`) or `orchestration run`.
Worker: if the current prompt contains a live dispatch preamble, do the task, use `ask` for blocking questions, and send `worker_done` once with the required payload. If the preamble is stale or absent, do not send lifecycle messages; inspect state or treat the prompt as an ordinary handoff.