* fix(startup): stop a duplicate headless serve from crash-looping and leaking AppImage mounts
A second Orca launch that loses the single-instance lock called app.quit()
before `ready`. That quit is deferred, so the doomed process kept booting into
Chromium's Linux display initialization, failed with "Missing X server or
$DISPLAY", and died with SIGSEGV. systemd read that as a crash and restarted it
forever; each restart re-mounted the AppImage and left the squashfuse mount
behind, until the host hit the 1000-mount FUSE ceiling and every later launch
failed.
The lock-losing launch now calls app.exit(3), which terminates synchronously
before any display init. Exit code 3 is a stable "another process already owns
this userData profile" contract, and the documented systemd unit uses
RestartPreventExitStatus=3 plus a real StartLimitIntervalSec/StartLimitBurst
window so a permanently failing launch can no longer retry unbounded.
Second-instance argv is now forwarded to the owner, and a duplicate `orca serve`
no longer asks the live headless server to open a desktop window. Desktop
activation for ordinary launches and macOS dock re-activation is unchanged.
Closes#11935
* docs(headless): clear the start limit before the scripted service starts
StartLimitIntervalSec=300/StartLimitBurst=5 rate-limits operator starts too, so
after a crash-loop trips the burst systemd refuses a plain `systemctl start` for
the rest of the window. The Upgrade and Roll back scripts run under
`set -euo pipefail`, so that refusal aborted the rollback mid-flight and left the
server down on the exact recovery path the doc prescribes.
Both scripts (and their EXIT-trap recoveries) now run `systemctl reset-failed`
first, the unit reference explains the interaction, and the crash-loop bullet
points at it for manual starts.
Co-authored-by: Orca <help@stably.ai>
* test(startup): reproduce the #11935 duplicate-serve crash loop under real Electron
The committed coverage for #11935 was source-text greps, so nothing gated the
mechanism the fix rests on: pre-`ready` `app.quit()` is deferred, which is why
the lock-losing headless `orca serve` kept booting into Linux display init.
This runs two real Electron processes against one disposable profile. The
duplicate executes the lock-loss gate's own `app.*` statement, lifted out of
`src/main/index.ts`, so reverting to `app.quit()` fails the test. It also feeds
the owner's real forwarded argv through `shouldActivateDesktopForSecondInstance`.
Also record why the activation predicate matches `--serve` and not the `serve`
subcommand: an AppImage launched as `orca serve` exits at the CLI redirect
before requesting the lock.
* test(startup): wait for the owner process to exit before removing its profile
Windows holds the profile's handles for a beat after SIGKILL, so an immediate
rmSync can fail with EBUSY/EPERM.
Co-authored-by: Orca <help@stably.ai>
* test(startup): pass the fixture marker path by env, not argv
Chromium reorders argv and the duplicate's argv is itself under test, so a
trailing positional was the wrong channel for it.
Co-authored-by: Orca <help@stably.ai>
* test(startup): only the activation case waits on the owner notification
The exit-contract cases assert on the duplicate's own already-terminated
process, so they should not block on cross-process delivery.
Co-authored-by: Orca <help@stably.ai>
* test(startup): drop the staged lock race, keep the real-Electron gate contract
CI proved the two-process form cannot work on a display-less Linux runner:
Chromium's ProcessSingleton needs the browser IO thread, which needs `ready`,
which needs a display. The pre-`ready` owner looked stale and the duplicate took
the lock (`expected [ 'DUPLICATE_WON_LOCK' ] to include 'DUPLICATE_LOST_LOCK'`).
Lock acquisition and argv forwarding are already covered in
single-instance-lock.test.ts. What only a real process can settle is what the
loser does next, so that is all this file now runs -- display-independent.
Co-authored-by: Orca <help@stably.ai>
---------
Co-authored-by: Orca <help@stably.ai>
Adds `orca skills install` and `orca skills update` so skills can be set up without the GUI — SSH hosts, containers, CI. Previously `orca skills` had only `list` and `get`, so there was no headless path.
**Agent targeting is scoped explicitly rather than delegated to detection.** The `skills` CLI decides which agents to install into, and with `-y` and zero detected agents it takes `targetAgents = validAgents` — all ~75. That is not a corner case for a headless CLI: a fresh SSH box or container with no agent installed is the normal starting state. Measured on a bare host, the unscoped command created **52 top-level agent directories and 54 junctions** (one real payload in `~/.agents/skills`, the rest links) on Windows, and 52/53 on macOS.
The CLI now passes `--agent` derived from Orca's own detection, mapped to the `skills` key namespace, plus `universal`. Supplying `--agent` makes `runAdd` use it directly and never call `detectInstalledAgents()`, so the fan-out branch is unreachable. On a bare host it now refuses with `No coding agent detected on this host` and exit 1, creating nothing. Same command with scoping: **1 directory, 0 junctions.**
`universal` alone would under-install — Claude Code is not in that set, and 19 of 28 mapped keys write agent-private homes `universal` never touches. `--agent '*'` is the bug itself. The mapping is hedged three ways: `null` for any agent whose key could not be confirmed, `satisfies Record<TuiAgent, …>` so a new Orca agent is a compile error, and a test pinning every mapped key against the CLI's own valid list.
Fixed during review — two holes that each restored the full fan-out through a different door:
- `--agent ','` trimmed to nothing, which skipped the refusal *and* emitted no `--agent`.
- `--agent -y` passed an emptiness check, and the vendor CLI silently drops `-`-leading values, re-emptying its list.
The real invariant is argument *shape*, not emptiness, and it is now enforced at the choke point in `buildAgentFeatureSkillInstallArgs`, so no caller can emit `-y` without a usable target. `*` remains allowed — asking for every agent explicitly is a choice, not an accident. Verified with 51 hostile inputs through the built binary, each recorded argv replayed through the vendor's own parser.
Also fixed: the `ORCA_CLI_CWD` refusal now runs before target resolution (it was quoting the wrong host's agent list), and `--dry-run` is refused in a forwarded shell rather than printing a command naming the wrong machine.
Validated on a real Windows host across PowerShell 7, PowerShell 5.1, cmd.exe and Git Bash: `.cmd` shims route through `cmd.exe` and `.exe` shims spawn directly (proved with instrumented shims, not inferred), the ENOENT path produces an actionable error rather than a silent failure, and `skills update` genuinely restores a corrupted skill byte-for-byte.
Known, not addressed here — both upstream behaviours this only forwards: a partial install failure exits 0, and "no installed skills found" exits 0. Both are invisible to the headless callers this feature exists for.
Co-authored-by: scastanoh21 <scastanoh21@gmail.com>
* fix(linux): restore Ubuntu 20.04 launch by pinning node-pty glibc symbols (#9902)
The bundled node-pty pty.node is compiled from source in release CI on
ubuntu-latest (glibc 2.39). glibc's 2.32-2.34 libpthread/libutil merge
relocated openpty/forkpty (GLIBC_2.34) and pthread_sigmask (GLIBC_2.32)
into libc under new symbol versions, so the from-source build bound to
versions absent on Ubuntu 20.04 (glibc 2.31). The main process imports
node-pty at startup, so the app crashed on launch. pty.node is the sole
blocker (Electron needs GLIBC_2.25; other native modules <= 2.17).
- Patch node-pty: a .symver shim pins the 3 symbols to their pre-merge
version (GLIBC_2.2.5 x64 / GLIBC_2.17 arm64), and Linux-only ldflags
force libutil.so.1/libpthread.so.0 back into DT_NEEDED. Guarded to
Linux; macOS/Windows untouched.
- Add a packaging gate (verify-linux-glibc-floor.cjs, afterPack): reads
each bundled native binary's objdump -p version needs and fails the
Linux build if any strong GLIBC_/GLIBCXX_/CXXABI_ node exceeds stock
Ubuntu 20.04 (glibc 2.31 / GLIBCXX_3.4.28 / CXXABI_1.3.12). Catches
GLIBC_ABI_DT_RELR, rejects GLIBC_PRIVATE, skips weak needs, fail-closed.
- Docs + tests; the lazy sherpa-onnx speech prebuilt (GLIBCXX_3.4.29,
never loaded at launch) is a documented libstdc++-floor exemption.
* fix(linux): assert DT_NEEDED provider deps in the glibc-floor gate
Harden the packaging gate (flagged in adversarial re-eval): the version-floor
check alone can false-pass if the patch's forced `-l:libutil.so.1` ever silently
drops — the pinned openpty@GLIBC_2.2.5 still resolves from libc's compat alias at
build time, but fails to load on Ubuntu 20.04 where openpty/forkpty live only in
libutil. The gate now also asserts that any binary importing openpty/forkpty
keeps libutil.so.1 in DT_NEEDED. Validated on a real symver-pinned .so with
libutil dropped (now fails) vs. present (passes). Documents the recommended
real-host smoke-test follow-up.
* docs(headless-server): add upgrade SOP for orca serve on Linux
The headless Linux guide covered install/run/systemd but had no upgrade
section, leaving operators to guess how to move to a new AppImage without
losing state.
Add an "Upgrade" section documenting the manual SOP (serve mode never
auto-updates) and one troubleshooting bullet:
- State lives under the service user's ~/.config (orca + Orca dirs),
independent of /opt/orca, and orca-data.json is forward-migrated on load,
so a forward upgrade is safe.
- Replace the binary with an atomic same-filesystem rename (download to
.new, verify, mv) — never curl -o over the FUSE-mounted live binary.
- Back up the whole .config before upgrading, because rollback is NOT
binary-only safe: an older build strips newer orca-data.json fields it
doesn't recognize, and the .bak.* ring is corruption-recovery, not a
pre-upgrade copy.
- Note there is no headless version command; track the release tag instead.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(headless-server): harden the orca serve upgrade/rollback runbook
Address CodeRabbit review on #9575:
- Fail closed: run the upgrade block under `set -euo pipefail`, remove any stale
`.new` file before download, and gate the atomic `mv` on an explicit ELF check
so a failed/partial/non-ELF download can never be promoted.
- Keep /opt/orca/VERSION tied to the installed binary: a single `TAG` variable
drives both the download URL and the recorded VERSION, saved as VERSION.prev on
upgrade and restored on rollback so the audit file never drifts.
- Crash-loop troubleshooting now points to Roll back first (restores the
pre-upgrade orca-data.json) instead of re-running Upgrade.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(headless): harden server upgrade SOP
---------
Co-authored-by: fanyunqian.1 <fanyunqian.1@bytedance.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
* docs: add headless Linux server guide
* docs: add ldd/appimage-extract tip for diagnosing missing libraries
Salvaged from #6817 before closing it as a duplicate.
Co-authored-by: Orca <help@stably.ai>
---------
Co-authored-by: Jinwoo-H <jinwoo@stably.ai>
Co-authored-by: Orca <help@stably.ai>
Neil
Sebastián Castaño
OrcaWin
Yunqian Fan
Dhilip Subramanian