xiaoxue_admin
/
orca
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix(remote): preserve terminal recovery across control refresh (#11513) 

* fix(remote): recover stalled terminal streams

* fix(i18n): localize manual disconnect error

* fix(remote): park paired terminals with host snapshots

* test(remote): mock authoritative resync snapshots

* fix(terminal): defer startup mounts until hydration

* fix(remote): raise paired terminal stream capacity

* fix(remote): harden terminal recovery lifecycle

* fix(remote): preserve calls across control refresh

* test(remote): harden paired recovery oracle

* test(workspace): seed Jira source context

* test(remote): assert raw host terminal identities

* test(terminal): keep restore sentinels atomic

* test(terminal): keep restore sentinel on one row

---------

Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
This commit is contained in:
OrcaWin 2026-07-30 03:05:10 -07:00 committed by GitHub
parent 9eede0084d
commit ab665a3ce7
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
9 changed files with 152 additions and 43 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

28
config/reliability-gates.jsonc
View File

@ -3087,15 +3087,16 @@
"providers": ["paired-runtime", "ssh"],
"coveredPlatforms": ["macos"],
"coveredProviders": ["paired-runtime"],
"coverageNotes": "Deterministic headed macOS runs launch an isolated Orca desktop server and a separate paired web client. A byte-identical headless run uses an isolated `orca serve` host. Both create six real paired host PTYs with bounded high-output scrollback, ordinary-park five client xterms without enabling the lossy retention budget, assert bounded cells/heap/timer lag, then restore 5,000 rows on the original PTY including output produced while parked and continued input/output. Separate headed/headless ACK-starvation tests recover one stalled stream without replacing its PTY. A capability-disabled run proves legacy hosts retain the prior lossy limit/TTL fallback. Unit tests cover exact-owner capability routing, raw-stream release, singleton side-effect facts with timed handoff cleanup, provider-authoritative snapshots, 128 active streams plus retry after a 129th-stream capacity rejection, capacity-pressure backoff, full split-leaf remint reconciliation, truncation, and manual-disconnect queue fencing. Linux, Windows, mixed-version paired hosts, live SSH, and production-scale paired hosts remain gaps.",
"coverageNotes": "Deterministic headed macOS runs launch an isolated Orca desktop server and a separate paired web client. A byte-identical headless run uses an isolated `orca serve` host. Both create six real paired host PTYs with bounded high-output scrollback, ordinary-park five client xterms without enabling the lossy retention budget, assert bounded cells/heap/timer lag, then restore 5,000 rows on the original PTY including output produced while parked and continued input/output. The headed oracle additionally keeps every remote target workspace and terminal unmounted in the host renderer before and after client recovery. Separate headed/headless ACK-starvation tests recover one stalled stream without replacing its PTY. A capability-disabled run proves legacy hosts retain the prior lossy limit/TTL fallback. Unit tests cover exact-owner capability routing, raw-stream release, singleton side-effect facts with timed handoff cleanup, provider-authoritative snapshots, 128 active streams plus retry after a 129th-stream capacity rejection, capacity-pressure backoff, full split-leaf remint reconciliation, truncation, and manual-disconnect queue fencing. Linux, Windows, mixed-version paired hosts, live SSH, and production-scale paired hosts remain gaps.",
"motivatingLinks": [
"https://github.com/stablyai/orca/issues/8652",
"https://github.com/stablyai/orca/pull/10625"
],
"invariant": "A host advertising terminal.paired-parking.v1 keeps the PTY and bounded authoritative history alive while an ordinary hidden-view park destroys the client xterm and releases its raw per-PTY stream. Reveal must restore up to the requested 5,000 rows, parked-time side effects/output, the same PTY identity, and continued input/output. Hosts without the capability must retain the existing limit/TTL force-parking fallback. A paired terminal stream whose delivery credits stop progressing must replace only that stream; command silence must first probe authoritative state and replace only if that probe times out. Manual disconnect must retain pairing while preventing queued or passive calls and subscriptions from recreating transport until explicit Connect.",
"oracle": "Run one byte-identical six-terminal oracle against an isolated headed desktop host and an isolated headless `orca serve` host. Stage at least 1,000,000 xterm cells, enable ordinary parking with the lossy retention budget disabled, require exactly one mounted manager and five parked tabs, at most 45% retained cells, no more than 16 MiB heap growth, and under 500 ms timer drift. While a tab is parked, require authoritative terminal.read to observe new PTY output; reveal it and require the original PTY, a marker within the requested 5,000-row history, the parked marker, and post-reveal input/output. Admit 128 active streams, reject the 129th as retryable, release one stream, then require the retry to attach and publish a snapshot without multiplying retained subscribers. Disable terminal.paired-parking.v1 and require the same oracle to fail before parking, while the legacy limit-one fallback separately passes. Also preserve truncated first paint, ACK-starved same-PTY recovery, responsive silent-command snapshot probes, dead-stream probe timeout recovery, and queued manual-disconnect fencing.",
"invariant": "A host advertising terminal.paired-parking.v1 keeps the PTY and bounded authoritative history alive while an ordinary hidden-view park destroys the client xterm and releases its raw per-PTY stream. Reveal must restore up to the requested 5,000 rows, parked-time side effects/output, the same PTY identity, and continued input/output. Hosts without the capability must retain the existing limit/TTL force-parking fallback. A paired terminal stream whose delivery credits stop progressing must replace only that stream; command silence must first probe authoritative state and replace only if that probe times out. A successful status probe may replace a pre-ready shared-control socket without rejecting or duplicating calls already waiting for that transport. Manual disconnect must retain pairing while preventing queued or passive calls and subscriptions from recreating transport until explicit Connect.",
"oracle": "Run one byte-identical six-terminal oracle against an isolated headed desktop host and an isolated headless `orca serve` host. Stage at least 1,000,000 xterm cells, enable ordinary parking with the lossy retention budget disabled, require exactly one mounted manager and five parked tabs, at most 45% retained cells, no more than 16 MiB heap growth, and under 500 ms timer drift. In headed mode, require the host renderer to remain on its original workspace with zero target terminal managers mounted throughout client park, reveal, and live I/O. While a tab is parked, require authoritative terminal.read to observe new PTY output; reveal it and require the original PTY, a marker within the requested 5,000-row history, the parked marker, and post-reveal input/output. Withhold the first encrypted shared-control ready frame, start one RPC, trigger a status-probe refresh, and require two connections, one host delivery, successful response, and zero retained request bytes. Admit 128 active streams, reject the 129th as retryable, release one stream, then require the retry to attach and publish a snapshot without multiplying retained subscribers. Disable terminal.paired-parking.v1 and require the same oracle to fail before parking, while the legacy limit-one fallback separately passes. Also preserve truncated first paint, ACK-starved same-PTY recovery, responsive silent-command snapshot probes, dead-stream probe timeout recovery, and queued manual-disconnect fencing.",
"commands": [
"pnpm exec vitest run --config config/vitest.config.ts src/main/runtime/orca-runtime.test.ts src/main/runtime/rpc/terminal-multiplex.test.ts src/main/runtime/rpc/terminal-subscribe-buffer.test.ts src/renderer/src/components/terminal-pane/parked-terminal-byte-watcher.test.ts src/renderer/src/components/terminal-pane/terminal-side-effect-facts-handler.test.ts src/renderer/src/components/terminal-pane/pty-connection.test.ts src/renderer/src/components/terminal-pane/remote-runtime-pty-transport.test.ts src/renderer/src/components/terminal-pane/terminal-hidden-view-parking.test.ts src/renderer/src/components/terminal-pane/terminal-hidden-worktree-retention.test.ts src/renderer/src/components/terminal-pane/terminal-parked-tab-watchers.test.ts src/renderer/src/components/terminal-pane/terminal-parked-watcher-reconciliation.test.ts src/renderer/src/components/terminal-pane/terminal-parked-watcher-partial-reconciliation.test.ts src/renderer/src/components/terminal-pane/terminal-parking-e2e-overrides.test.ts src/renderer/src/runtime/remote-runtime-terminal-stall-recovery.test.ts src/renderer/src/runtime/runtime-client-events.test.ts src/renderer/src/web/web-preload-api.test.ts src/main/ipc/runtime-environments.test.ts",
"pnpm exec vitest run --config config/vitest.config.ts src/shared/remote-runtime-shared-control-connection.test.ts",
"pnpm exec vitest run --config config/vitest.config.ts src/renderer/src/runtime/remote-runtime-terminal-parse-backpressure.test.ts",
"ORCA_E2E_WEB_CLIENT=1 SKIP_BUILD=1 pnpm exec playwright test tests/e2e/paired-remote-terminal-truncated-tail-first-paint.spec.ts --config tests/playwright.config.ts --project electron-headful --workers=1",
"ORCA_E2E_WEB_CLIENT=1 ORCA_E2E_DISABLE_PAIRED_TERMINAL_PARKING=1 SKIP_BUILD=1 pnpm exec playwright test tests/e2e/paired-remote-terminal-truncated-tail-first-paint.spec.ts --config tests/playwright.config.ts --project electron-headful --workers=1",
@ -3124,6 +3125,7 @@
"src/renderer/src/runtime/remote-runtime-terminal-parse-backpressure.test.ts",
"src/renderer/src/runtime/runtime-client-events.test.ts",
"src/renderer/src/components/terminal-pane/remote-runtime-pty-transport.test.ts",
"src/shared/remote-runtime-shared-control-connection.test.ts",
"tests/e2e/paired-remote-terminal-truncated-tail-first-paint.spec.ts",
"tests/e2e/paired-remote-terminal-stall-recovery.spec.ts",
"tests/e2e/headless-paired-remote-terminal-stall-recovery.spec.ts",
@ -3152,7 +3154,8 @@
{
"file": "tests/e2e/paired-remote-terminal-retention-memory.spec.ts",
"assertions": [
"ordinary-parks paired terminals and restores authoritative host scrollback"
"ordinary-parks paired terminals and restores authoritative host scrollback",
"client recovery completes while every target terminal remains unmounted in the host renderer"
]
},
{
@ -3276,7 +3279,7 @@
"command": "ORCA_E2E_WEB_CLIENT=1 SKIP_BUILD=1 pnpm exec playwright test tests/e2e/paired-remote-terminal-retention-memory.spec.ts --config tests/playwright.config.ts --project electron-headful --workers=1",
"result": "passed",
"durationSeconds": 14.5,
"summary": "Six real paired worktrees and host PTYs ordinary-parked five client xterms with the lossy budget disabled, released at least 55% of staged cells within heap and timer-lag budgets, then restored requested row 4,000, parked-time output, and live I/O on the original PTY."
"summary": "Six real paired worktrees and host PTYs ordinary-parked five client xterms with the lossy budget disabled, released at least 55% of staged cells within heap and timer-lag budgets, then restored requested row 4,000, parked-time output, and live I/O on the original PTY while every target terminal remained unmounted in the host renderer."
},
{
"date": "2026-07-29",
@ -3304,6 +3307,15 @@
"result": "passed",
"durationSeconds": 18.6,
"summary": "A host without terminal.paired-parking.v1 preserved the existing lossy limit-one fallback, restored retained output, and continued PTY input/output."
},
{
"date": "2026-07-29",
"runner": "local",
"platform": "macos",
"command": "pnpm exec vitest run --config config/vitest.config.ts src/shared/remote-runtime-shared-control-connection.test.ts",
"result": "passed",
"durationSeconds": 3.16,
"summary": "All 27 encrypted shared-control tests passed. The new oracle was red on rc.1-equivalent behavior with the exact Refreshing remote runtime control transport error, then passed with one replacement connection, one delivered host RPC, and zero retained pending bytes."
}
],
"runtimeBudget": {
@ -4301,7 +4313,7 @@
"providers": ["ssh", "remote-runtime", "wsl"],
"coveredPlatforms": ["macos"],
"coveredProviders": ["ssh", "remote-runtime"],
"coverageNotes": "Deterministic renderer coverage proves startup publishes the state returned by ssh.connect, retained native and runtime SSH payloads are admitted through production routes only with valid complete authority, stale cleanup cannot unregister a replacement runtime terminal, direct SSH Git and folder panes clear and retry by exact authority, one authority chain stops after two automatic attempts even when each timeout exceeds the rolling window, rejected acknowledgements mutate no store maps, and one shared exact attempt admits every concurrent split-pane spawn and reattach while preserving the first PTY as the tab fallback. A later sibling failure rotates the tab once, stale callbacks from the prior attempt mutate no state, split remount activity suppression is counted per leaf, primary PTY exit promotes a bound survivor or preserves an empty continuation gap for a late sibling, and primary, non-primary, or null-PTY detach preserves exact authority on both resulting tabs. Intentional pane disposal cancels its settlement timer without breaking StrictMode remount timeout ownership. Target snapshot hydration/reconnect preserves sibling SSH/local/WSL/runtime state, and a mounted remote-runtime terminal survives repeated transport partitions without changing PTY identity. Direct SSH coordinator tests cover immediate terminal finalization, hydration correction, damping, bounded retry, and telemetry non-interference. Client/server heartbeat tests cover timer suspension, socket generations fence stale callbacks, cold restored-terminal attachment retries, cached pixels remain unhealthy until authoritative replay, automatic retries stop after one minute, manual reconnect preserves the PTY, and pane closure releases recovery UI state. Current macOS Electron journeys against an ephemeral Linux Docker SSH target cover exact-authority repo/worktree hydration, live terminal recovery after disconnect/reconnect, and eager six-terminal remount after renderer reload. A Windows remote-runtime smoke covers reachability and PTY round-trip. Multi-target live fanout, paired-close, WSL, and patched live partition journeys remain gaps.",
"coverageNotes": "Deterministic renderer coverage proves startup publishes the state returned by ssh.connect, retained native and runtime SSH payloads are admitted through production routes only with valid complete authority, stale cleanup cannot unregister a replacement runtime terminal, direct SSH Git and folder panes clear and retry by exact authority, one authority chain stops after two automatic attempts even when each timeout exceeds the rolling window, rejected acknowledgements mutate no store maps, and one shared exact attempt admits every concurrent split-pane spawn and reattach while preserving the first PTY as the tab fallback. A later sibling failure rotates the tab once, stale callbacks from the prior attempt mutate no state, split remount activity suppression is counted per leaf, primary PTY exit promotes a bound survivor or preserves an empty continuation gap for a late sibling, and primary, non-primary, or null-PTY detach preserves exact authority on both resulting tabs. Intentional pane disposal cancels its settlement timer without breaking StrictMode remount timeout ownership. Target snapshot hydration/reconnect preserves sibling SSH/local/WSL/runtime state, and a mounted remote-runtime terminal survives repeated transport partitions without changing PTY identity. A real encrypted-WebSocket oracle proves a successful reachability probe can replace a pre-ready shared-control socket without rejecting or duplicating the waiting RPC. Direct SSH coordinator tests cover immediate terminal finalization, hydration correction, damping, bounded retry, and telemetry non-interference. Client/server heartbeat tests cover timer suspension, socket generations fence stale callbacks, cold restored-terminal attachment retries, cached pixels remain unhealthy until authoritative replay, automatic retries stop after one minute, manual reconnect preserves the PTY, and pane closure releases recovery UI state. Current macOS Electron journeys against an ephemeral Linux Docker SSH target cover exact-authority repo/worktree hydration, live terminal recovery after disconnect/reconnect, and eager six-terminal remount after renderer reload. A Windows remote-runtime smoke covers reachability and PTY round-trip. Multi-target live fanout, paired-close, WSL, and patched live partition journeys remain gaps.",
"motivatingLinks": [
"https://github.com/stablyai/orca/pull/6951",
"https://github.com/stablyai/orca/pull/6955",
@ -4309,8 +4321,8 @@
"https://github.com/stablyai/orca/pull/7009",
"https://github.com/stablyai/orca/pull/8597"
],
"invariant": "SSH, WSL, and remote-runtime restore paths must treat provider listing failures and unknown liveness as unknown, not dead, while still avoiding duplicate spawn and clearing expired relay leases exactly once. Direct SSH reconnect must atomically clear only exact-target live PTY bindings, preserve relay identity, retry Git and folder panes without paired close or provider shutdown, and allow at most two automatic attempts in one authority chain even when each settlement exceeds the rolling window. A rejected acknowledgement mutates no store map. A successful exact split-pane spawn or reattach must retain that attempt as shared live authority until sibling leaves settle; the first success cannot consume sibling authority, a sibling failure can start at most one second tab-wide attempt, and prior-attempt callbacks become inert after rotation. Once the retry budget is exhausted, a failure cannot start attempt three or revoke attempt-two authority from siblings that may still settle. Primary PTY exit must promote a bound survivor or preserve exact authority through an empty activation gap, and split detach must project that authority to both resulting tabs. Hydrated PTY hints cannot supersede a current exact-attempt owner, and target snapshot hydration/reconnect cannot reset sibling SSH, local, WSL, or runtime-owned state. Every restored remote terminal must preserve its provider PTY identity, including the authoritative incarnation returned by a successful session-ID reattach. After a recoverable partition the same authenticated runtime must reattach the same PTY, reject detached input, apply the latest viewport, and report healthy only after authoritative replay. Automatic PTY recovery stops after one bounded minute without a fatal terminal error; a manual reconnect starts a newly fenced epoch against the same PTY, and closed panes retain no recovery UI state. One capability-gated terminal-create mutation must produce at most one host PTY across an unknown response outcome, remain manually retryable after cutoff, and never let a stale completion replace a newer pane lifecycle.",
"oracle": "Deterministic tests cover bounded stale-handle replacement, suspended heartbeat clocks, cold and established subscription failure, ten partition/recovery cycles, automatic-recovery cutoff, manual reconnect, and exact direct SSH binding recovery. They assert one atomic store publication clears only exact-target PTY indexes, null-PTY activation remains unchanged, relay identity survives, Git and folder panes retry symmetrically, another target/local/WSL/runtime panes remain byte-identical through target snapshot hydration and reconnect, only an accepted exact failure or timeout starts the second attempt, two 31-second timeouts cannot start a third settlement-triggered attempt, rejected stale/mismatched acknowledgements preserve every store map, and concurrent split-pane spawn and reattach callbacks both commit under the same attempt ID after the first success replaces pending state with live shared authority. A sibling failure revokes that shared authority and starts exactly one second attempt; duplicate failures and late first-attempt PTY callbacks preserve the second attempt and every state map. Attempt-two failure retains continuation authority for later siblings, primary exit promotes a bound survivor or preserves the lease until a late sibling binds, and primary plus non-primary detach retain exact authority and history on both resulting tabs. Both remount callbacks consume split-count activity suppression, intentional dispose emits no failure/timeout, and a same-attempt StrictMode remount still owns one timeout. Hydration clears an untrusted PTY hint without clearing its current pending owner, healthy current-authority bindings suppress correction, hydration finalizes once, and reconnect emits no paired close lifecycle. A provider-level session-ID reattach returns an incarnation, then a legacy exit without an incarnation must resolve to that returned identity rather than minting a fallback identity. Tests also assert one unsubscribe per remote-runtime epoch, observable recovery phases, stable PTY identity, resumed snapshot/output/input, no healthy state before replay, no retry or input after cutoff, a new manual epoch against the same PTY, quiet recovery UI with an explicit Reconnect action, pane-close state cleanup, one stable create mutation id, old-runtime no-retry behavior, cross-process PTY adoption, and bounded in-flight coordination.",
"invariant": "SSH, WSL, and remote-runtime restore paths must treat provider listing failures and unknown liveness as unknown, not dead, while still avoiding duplicate spawn and clearing expired relay leases exactly once. Direct SSH reconnect must atomically clear only exact-target live PTY bindings, preserve relay identity, retry Git and folder panes without paired close or provider shutdown, and allow at most two automatic attempts in one authority chain even when each settlement exceeds the rolling window. A rejected acknowledgement mutates no store map. A successful exact split-pane spawn or reattach must retain that attempt as shared live authority until sibling leaves settle; the first success cannot consume sibling authority, a sibling failure can start at most one second tab-wide attempt, and prior-attempt callbacks become inert after rotation. Once the retry budget is exhausted, a failure cannot start attempt three or revoke attempt-two authority from siblings that may still settle. Primary PTY exit must promote a bound survivor or preserve exact authority through an empty activation gap, and split detach must project that authority to both resulting tabs. Hydrated PTY hints cannot supersede a current exact-attempt owner, and target snapshot hydration/reconnect cannot reset sibling SSH, local, WSL, or runtime-owned state. Every restored remote terminal must preserve its provider PTY identity, including the authoritative incarnation returned by a successful session-ID reattach. After a recoverable partition the same authenticated runtime must reattach the same PTY, reject detached input, apply the latest viewport, and report healthy only after authoritative replay. A successful one-shot reachability probe may replace a pre-ready shared-control socket, but waiting RPCs must continue onto the replacement under their original deadline without duplicate host delivery or retained request bytes. Automatic PTY recovery stops after one bounded minute without a fatal terminal error; a manual reconnect starts a newly fenced epoch against the same PTY, and closed panes retain no recovery UI state. One capability-gated terminal-create mutation must produce at most one host PTY across an unknown response outcome, remain manually retryable after cutoff, and never let a stale completion replace a newer pane lifecycle.",
"oracle": "Deterministic tests cover bounded stale-handle replacement, suspended heartbeat clocks, cold and established subscription failure, ten partition/recovery cycles, automatic-recovery cutoff, manual reconnect, and exact direct SSH binding recovery. They assert one atomic store publication clears only exact-target PTY indexes, null-PTY activation remains unchanged, relay identity survives, Git and folder panes retry symmetrically, another target/local/WSL/runtime panes remain byte-identical through target snapshot hydration and reconnect, only an accepted exact failure or timeout starts the second attempt, two 31-second timeouts cannot start a third settlement-triggered attempt, rejected stale/mismatched acknowledgements preserve every store map, and concurrent split-pane spawn and reattach callbacks both commit under the same attempt ID after the first success replaces pending state with live shared authority. A sibling failure revokes that shared authority and starts exactly one second attempt; duplicate failures and late first-attempt PTY callbacks preserve the second attempt and every state map. Attempt-two failure retains continuation authority for later siblings, primary exit promotes a bound survivor or preserves the lease until a late sibling binds, and primary plus non-primary detach retain exact authority and history on both resulting tabs. Both remount callbacks consume split-count activity suppression, intentional dispose emits no failure/timeout, and a same-attempt StrictMode remount still owns one timeout. Hydration clears an untrusted PTY hint without clearing its current pending owner, healthy current-authority bindings suppress correction, hydration finalizes once, and reconnect emits no paired close lifecycle. A provider-level session-ID reattach returns an incarnation, then a legacy exit without an incarnation must resolve to that returned identity rather than minting a fallback identity. The shared-control oracle withholds the first encrypted ready frame, starts one RPC, triggers the successful-probe refresh, then requires exactly two client connections, one host request, a successful response, zero pending calls, and zero retained request bytes. Tests also assert one unsubscribe per remote-runtime epoch, observable recovery phases, stable PTY identity, resumed snapshot/output/input, no healthy state before replay, no retry or input after cutoff, a new manual epoch against the same PTY, quiet recovery UI with an explicit Reconnect action, pane-close state cleanup, one stable create mutation id, old-runtime no-retry behavior, cross-process PTY adoption, and bounded in-flight coordination.",
"commands": [
"pnpm exec vitest run --config config/vitest.config.ts src/main/providers/ssh-pty-provider-reattach-incarnation.test.ts --reporter=dot",
"pnpm exec vitest run --config config/vitest.config.ts src/renderer/src/startup/ssh-startup-reconnect.test.ts src/renderer/src/lib/resolved-worktree-execution-host.test.ts src/renderer/src/components/terminal/background-terminal-worktree-mount.test.ts src/renderer/src/runtime/sync-runtime-graph-scheduling.test.ts src/renderer/src/components/terminal-pane/use-terminal-pane-lifecycle.test.ts src/renderer/src/components/terminal-pane/pty-connection.test.ts src/renderer/src/components/terminal-pane/remote-runtime-pty-transport.test.ts src/renderer/src/runtime/remote-runtime-session-tabs-inflight.test.ts src/renderer/src/runtime/web-session-terminal-handle-events.test.ts src/renderer/src/store/slices/terminal-pty-identity-replacement.test.ts",

29
src/shared/remote-runtime-shared-control-connection.test.ts
View File

@ -138,6 +138,29 @@ describe('RemoteRuntimeSharedControlConnection', () => {
expect(open).toHaveBeenCalledOnce()
})
it('keeps a waiting request alive when a reachability probe replaces its pre-ready socket', async () => {
const server = await createServer({ suppressReadyFrameCount: 1 })
const connection = new RemoteRuntimeSharedControlConnection(server.pairing)
const response = connection.request('worktree.ps', undefined, 1000)
await vi.waitFor(() => expect(server.connectionCount()).toBe(1))
connection.reconnectNow()
await expect(response).resolves.toMatchObject({
ok: true,
result: { method: 'worktree.ps' }
})
expect(server.connectionCount()).toBe(2)
expect(server.requests.map(({ method }) => method)).toEqual(['worktree.ps'])
expect(connection.getDiagnostics().pendingRequestCount).toBe(0)
expect(getRemoteRuntimeRequestAdmissionEvidence()).toEqual({
pendingRequestCount: 0,
retainedBytes: 0
})
connection.close()
})
it('logs unknown response ids without breaking pending requests', async () => {
const warn = vi.spyOn(console, 'warn').mockImplementation(() => undefined)
const server = await createServer({ sendUnknownResponseBeforeResponse: true })
@ -665,6 +688,7 @@ async function createServer(
closeAfterFirstStreamingResponse?: boolean
closeBeforeResponse?: boolean
suppressReadyFrame?: boolean
suppressReadyFrameCount?: number
// Why: half-open simulation — the socket stays open but never answers
// protocol pings, like a wedged tunnel that swallows frames silently.
disableAutoPong?: boolean
@ -696,7 +720,10 @@ async function createServer(
serverKeyPair.secretKey,
publicKeyFromBase64(hello.publicKeyB64)
)
if (options.suppressReadyFrame) {
if (
options.suppressReadyFrame ||
connectionCount <= (options.suppressReadyFrameCount ?? 0)
) {
return
}
ws.send(JSON.stringify({ type: 'e2ee_ready' }))

22
src/shared/remote-runtime-shared-control-connection.ts
View File

@ -130,7 +130,10 @@ export class RemoteRuntimeSharedControlConnection {
return
}
// Why: a successful one-shot status probe proves the restarted endpoint is reachable; replace even a stuck CONNECTING/awaiting-ready socket instead of waiting behind stale backoff.
this.closeSocket(remoteRuntimeUnavailableError('Refreshing remote runtime control transport.'))
this.closeSocket(
remoteRuntimeUnavailableError('Refreshing remote runtime control transport.'),
true
)
this.open()
}
@ -208,7 +211,11 @@ export class RemoteRuntimeSharedControlConnection {
sendEncrypted: (payload) => this.sendEncrypted(payload),
markReady: () => {
this.lastConnectedAt = Date.now()
this.scheduleReconnectAttemptReset()
this.readyStableReset.schedule({
getState: () => this.state,
getSocket: () => this.ws,
reset: () => this.reconnect.resetAttempt()
})
},
replaySubscriptions: () => this.replaySubscriptions()
})
@ -293,7 +300,7 @@ export class RemoteRuntimeSharedControlConnection {
}
}
private closeSocket(error?: Error): void {
private closeSocket(error?: Error, preserveReadyWaitersAndPendingRequests = false): void {
closeSharedControlSocket({
environmentId: this.options.environmentId,
state: this.state,
@ -304,18 +311,11 @@ export class RemoteRuntimeSharedControlConnection {
socketCleanup: this.socketCleanup,
ws: this.ws,
error,
preserveReadyWaitersAndPendingRequests,
clearReadyStableTimer: () => this.readyStableReset.clear()
})
this.ws = this.sharedKey = null
this.socketCleanup = null
this.state = 'closed'
}
private scheduleReconnectAttemptReset(): void {
this.readyStableReset.schedule({
getState: () => this.state,
getSocket: () => this.ws,
reset: () => this.reconnect.resetAttempt()
})
}
}

4
src/shared/remote-runtime-shared-control-socket-close.ts
View File

@ -19,6 +19,7 @@ export function closeSharedControlSocket(args: {
readyWaiters: SharedControlReadyWaiter[]
lastClose: { code: number; reason: string } | null
error?: Error
preserveReadyWaitersAndPendingRequests?: boolean
clearReadyStableTimer: () => void
}): void {
if (args.ws || args.socketCleanup) {
@ -39,6 +40,7 @@ export function closeSharedControlSocket(args: {
subscriptions: args.subscriptions,
socketCleanup: args.socketCleanup,
ws: args.ws,
error: args.error
error: args.error,
preserveReadyWaitersAndPendingRequests: args.preserveReadyWaitersAndPendingRequests
})
}

10
src/shared/remote-runtime-shared-control-state.ts
View File

@ -186,9 +186,15 @@ export function closeSharedControlSocketState(args: {
socketCleanup: (() => void) | null
ws: { close: () => void } | null
error?: Error
preserveReadyWaitersAndPendingRequests?: boolean
}): void {
rejectSharedControlReadyWaiters(args.readyWaiters, args.error ?? remoteRuntimeUnavailableError())
rejectAllSharedControlPendingRequests(args.pendingRequests, args.error)
if (!args.preserveReadyWaitersAndPendingRequests) {
rejectSharedControlReadyWaiters(
args.readyWaiters,
args.error ?? remoteRuntimeUnavailableError()
)
rejectAllSharedControlPendingRequests(args.pendingRequests, args.error)
}
markSharedControlSubscriptionsUnsent(args.subscriptions)
try {
args.socketCleanup?.()

38
tests/e2e/helpers/paired-terminal-parking-oracle.ts
View File

@ -1,7 +1,10 @@
import type { Page } from '@stablyai/playwright-test'
import type { RuntimeTerminalRead } from '../../../src/shared/runtime-types'
import { TERMINAL_PAIRED_PARKING_RUNTIME_CAPABILITY } from '../../../src/shared/protocol-version'
import { toWebTerminalSurfaceTabId } from '../../../src/shared/terminal-surface-id'
import {
toHostSessionTabId,
toWebTerminalSurfaceTabId
} from '../../../src/shared/terminal-surface-id'
import {
readPairedRetentionSample,
startRendererLagProbe
@ -16,11 +19,6 @@ const MAX_RETAINED_CELL_FRACTION = 0.45
const MAX_EVICTION_LAG_MS = 500
const MAX_HEAP_GROWTH_BYTES = 16 * 1024 * 1024
type PairedTerminalParkingSeed = {
fallbackWorktreeId: string
repoId: string
}
type RemoteTab = {
marker: string
originalPtyId: string
@ -44,7 +42,8 @@ async function callRuntime<TResult>(page: Page, method: string, params: unknown)
export async function runPairedTerminalParkingOracle(
page: Page,
seed: PairedTerminalParkingSeed
seed: { fallbackWorktreeId: string; repoId: string },
options: { hostPage?: Page } = {}
): Promise<void> {
const fixture = createPairedTerminalParkingFixture()
const createdWorktreeIds: string[] = []
@ -132,6 +131,7 @@ export async function runPairedTerminalParkingOracle(
.toContain(`FILLED:${created.marker}`)
remoteTabs.push({ ...created, originalPtyId })
}
await expectHostTerminalsUnmounted(options.hostPage, seed.fallbackWorktreeId, remoteTabs)
await page.evaluate(() => window.__store?.getState().setActiveView('tasks'))
await expect
@ -263,6 +263,7 @@ export async function runPairedTerminalParkingOracle(
await expect
.poll(() => getTerminalContent(page), { timeout: 30_000 })
.toContain(`LIVE:${liveMarker}`)
await expectHostTerminalsUnmounted(options.hostPage, seed.fallbackWorktreeId, remoteTabs)
} finally {
for (const tab of remoteTabs) {
await callRuntime(page, 'terminal.closeTab', { terminal: tab.terminal }).catch(
@ -282,3 +283,26 @@ export async function runPairedTerminalParkingOracle(
fixture.dispose()
}
}
async function expectHostTerminalsUnmounted(
hostPage: Page | undefined,
activeWorktreeId: string,
remoteTabs: RemoteTab[]
): Promise<void> {
if (!hostPage) {
return
}
await expect
.poll(
() =>
hostPage.evaluate(
(tabIds) => ({
activeWorktreeId: window.__store?.getState().activeWorktreeId,
mountedCount: tabIds.filter((tabId) => window.__paneManagers?.has(tabId)).length
}),
remoteTabs.map(({ tabId }) => toHostSessionTabId(tabId))
),
{ timeout: 30_000 }
)
.toEqual({ activeWorktreeId, mountedCount: 0 })
}

45
tests/e2e/new-workspace-linked-item-project-switch.spec.ts
View File

@ -20,6 +20,7 @@ import type { Page } from '@stablyai/playwright-test'
import { test, expect } from './helpers/orca-app'
import { waitForActiveWorktree, waitForSessionReady } from './helpers/store'
import type { LinkedWorkItemSummary } from '../../src/renderer/src/lib/new-workspace'
import type { TaskSourceContext } from '../../src/shared/task-source-context'
const SECOND_PROJECT_NAME = 'linked-item-second-project'
@ -53,20 +54,52 @@ async function addSecondProject(page: Page, repoPath: string): Promise<void> {
async function openComposerWithLinkedWorkItem(
page: Page,
linkedWorkItem: LinkedWorkItemSummary,
prefilledName: string
prefilledName: string,
taskSourceContext: TaskSourceContext | null = null
): Promise<void> {
await page.evaluate(
({ linkedWorkItem, prefilledName }) => {
({ linkedWorkItem, prefilledName, taskSourceContext }) => {
const store = window.__store
if (!store) {
throw new Error('window.__store is not available')
}
store.getState().openModal('new-workspace-composer', { linkedWorkItem, prefilledName })
store.getState().openModal('new-workspace-composer', {
linkedWorkItem,
prefilledName,
taskSourceContext
})
},
{ linkedWorkItem, prefilledName }
{ linkedWorkItem, prefilledName, taskSourceContext }
)
}
async function getJiraSourceContext(page: Page): Promise<TaskSourceContext> {
return page.evaluate(() => {
const state = window.__store?.getState()
const activeWorktreeId = state?.activeWorktreeId
const setup = state?.projectHostSetups.find((candidate) =>
state.worktreesByRepo[candidate.repoId]?.some((worktree) => worktree.id === activeWorktreeId)
)
if (!setup) {
throw new Error('Active project host setup is unavailable')
}
return {
kind: 'task-source',
provider: 'jira',
projectId: setup.projectId,
hostId: setup.hostId,
projectHostSetupId: setup.id,
repoId: setup.repoId,
providerIdentity: {
provider: 'jira',
siteId: 'e2e-jira-site',
siteUrl: 'https://example.atlassian.net',
projectKey: 'RDG'
}
}
})
}
async function switchComposerProject(page: Page, projectName: string): Promise<void> {
const composer = page.getByRole('dialog')
const combobox = composer.getByRole('combobox', { name: 'Project' })
@ -94,6 +127,7 @@ test.describe('New workspace composer linked item across project switches', () =
})
test('keeps a Jira issue linked when the project changes', async ({ orcaPage }) => {
const jiraSourceContext = await getJiraSourceContext(orcaPage)
await openComposerWithLinkedWorkItem(
orcaPage,
{
@ -104,7 +138,8 @@ test.describe('New workspace composer linked item across project switches', () =
url: 'https://example.atlassian.net/browse/RDG-344',
jiraIdentifier: 'RDG-344'
},
'rdg-344-nuxtjs-nextjs'
'rdg-344-nuxtjs-nextjs',
jiraSourceContext
)
const composer = orcaPage.getByRole('dialog')

14
tests/e2e/paired-remote-terminal-retention-memory.spec.ts
View File

@ -17,17 +17,21 @@ test('ordinary-parks paired terminals and restores authoritative host scrollback
if (!active) {
throw new Error('Paired retention host has no active seeded worktree')
}
return { repoId: active.repoId, worktreeIds: worktrees.map((worktree) => worktree.id) }
return { activeWorktreeId: active.id, repoId: active.repoId }
})
const offer = await createRuntimeDesktopPairingOffer(orcaPage)
const client = await launchPairedWebClient(electronApp, offer, {
terminalParkingDelayMs: 100
})
try {
await runPairedTerminalParkingOracle(client.page, {
fallbackWorktreeId: seed.worktreeIds[0]!,
repoId: seed.repoId
})
await runPairedTerminalParkingOracle(
client.page,
{
fallbackWorktreeId: seed.activeWorktreeId,
repoId: seed.repoId
},
{ hostPage: orcaPage }
)
} finally {
await client.dispose()
}

5
tests/e2e/terminal-long-table-fixtures.ts
View File

@ -89,7 +89,7 @@ await writeStdout('LONG_TABLE_SCROLL_RESTORE_${runId}\\n')
export function emojiFixtureMarkdownTableScript(table: string, runId: string): string {
const marker = `EMOJI_FIXTURE_TABLE_RESTORE_${runId}`
const widthMarker = `EMOJI_FIXTURE_TABLE_WIDTH_${runId}`
const widthMarker = `${marker} WIDTH`
return `
const table = ${JSON.stringify(table)}
const minimumWidths = [2, 5, 4, 7, 7, 4, 3, 4]
@ -215,12 +215,11 @@ for (const line of rendered) {
}
await writeStdout('\\x1b[?2026l')
await writeStdout('${widthMarker}:' + generatedTableWidth + '\\r\\n')
await writeStdout('${marker}\\r\\n')
`
}
export function emojiFixtureTableWidthMarker(runId: string): string {
return `EMOJI_FIXTURE_TABLE_WIDTH_${runId}:`
return `EMOJI_FIXTURE_TABLE_RESTORE_${runId} WIDTH:`
}
export function narrowSignerMarkdownTableScript(runId: string): string {