* ci(pr): run E2E when a PR touches tests/e2e paths
Regression specs under tests/e2e never ran on PR CI — only schedule and
release called e2e.yml — so a red regression test could merge green.
Path-filter and workflow_call the E2E suite when E2E-relevant files change.
Use merge-base diffs so base-branch drift does not false-trigger E2E, fail
the detector when git diff cannot compute the PR range, and pin
least-privilege contents:read on both the detector and reusable E2E workflow.
Closes#10518
Co-authored-by: Wooseong Kim <innocarpe@gmail.com>
Co-authored-by: Orca <help@stably.ai>
* ci(pr): make the E2E path gate actually block, and match the real config path
Two fixes to the new path-filtered E2E job.
The gate did not gate. pr.yml's `verify` job is the required check, and it
enumerates its dependencies explicitly — `e2e` was in neither `needs` nor the
result list, so a failing shard left `verify` green. That reproduces the exact
hole this job exists to close: a red spec merges green, just with a red box
further down the page. Add `e2e` to both.
Because the job is path-filtered, `skipped` is the normal result on a PR that
touches no E2E files and has to keep passing. That allowance is checked after
the strict loop rather than inside it, so it can never leak to the six jobs
that are always required.
The `playwright.` pattern matched nothing. The config is
tests/playwright.config.ts — beside tests/e2e/, not inside it — so no tracked
file starts with `playwright.` and editing the runner config would silently
skip E2E. Anchor it at `tests/playwright.`.
Adds a contract test alongside the existing release-e2e one. Verified it fails
when either fix is reverted, and simulated the gate across
success/skipped/failure/cancelled plus the skip-must-not-mask-a-real-failure
case.
* test(ci): close two gaps in the E2E gate contract
CodeRabbit was right on both counts — verified by reverting each and watching
the contract stay green.
The path filter was unasserted, so `e2e` could lose its `if:` and run on every
PR — the cost the filter exists to avoid — without failing anything.
The strict-loop check hardcoded four of the six required jobs, so dropping
GIT_COMPATIBILITY or SHELL_CONTRACTS left them unenforced while the contract
passed. Derive the list from verify.needs instead, so a newly added required
job that misses the loop fails here rather than silently going unchecked.
* ci(pr): land the E2E path gate advisory instead of blocking
The E2E suite is currently failing every scheduled run on main — 22 of the last
22 — so making verify depend on it would block any PR touching tests/e2e/**,
including the PRs that fix the suite. This PR's own run reproduced that: 3 of 12
shards failed on specs unrelated to it (agent-session resume, Jira linking,
plugin containment, terminal artifacts).
So the job runs and reports on E2E-path PRs but is left out of verify.needs for
now. The detector, the tests/playwright. path fix, and the contract tests are
unaffected — those stand on their own and were the substance of the review.
Flipping to blocking is a three-line change once the suite is green; the exact
wiring, including why the skipped allowance must sit outside the strict loop, is
recorded on verify's Require-successful-checks step. The contract test pins the
advisory choice so it reads as deliberate rather than as the unwired-gate bug it
originally caught, and still fails if the path filter, the strict-loop coverage,
or the config path regress.
---------
Co-authored-by: Wooseong Kim <innocarpe@gmail.com>
Co-authored-by: Orca <help@stably.ai>
Reviewed with an independent reproduction. Added the allowlist entry that unblocked verify:localization-coverage on main, the 4th drifted step, and a parity gate that fails when pnpm lint's chain contains a script absent from pr.yml.
* fix(skills): source released history from the committed ledger, not a tag walk
verify:skill-bundle-manifest rebuilt the entire released-skill history by
walking every local refs/tags/v* on each run and demanded byte-equality with
the committed artifacts. Output was therefore a function of (skill bytes x
local tag set x release timing), so any clone holding stray, deleted, or fork
tags the committed artifacts predate rebuilt a divergent registry and failed
lint. This was the 4th instance of one failure class (#8637 -> #9119 version
bumps -> #9778 new tags -> local tag drift), each patched with a new tolerance
rather than removing the tag coupling.
Fix: the committed snapshot-registry + release-mapping ARE the released history;
trust them instead of re-deriving from tags.
- releasedHistoryFromCommitted() seeds generation from the committed ledger,
dropping the floating unreleased tail (entries beyond what the mapping names).
verify and --write are now pure functions of working-tree bytes with zero tag
access. The tag walk survives only behind --rebuild-from-tags (disaster
recovery), off the everyday path.
- --release <version> + appendReleaseRow() perform the O(1) append of one
mapping row at release cut (dedupes vs the last row, strips the v-prefix) --
the single authoritative point where working-tree bytes become an immutable
released revision.
- release-cut.yml runs generate --release "$VERSION" before the release commit
(Node built-ins only, no install needed); pr.yml drops fetch-depth: 0 from the
lint job since verify no longer needs tag history.
Recognition is unaffected: the runtime uses knownSnapshots = registry.skills
(all entries, incl. the tail committed at PR-merge time), so a missing mapping
row only loses a version label, never recognition or the update nudge.
Trade-off: lint no longer cross-checks committed historical snapshots against
tags. A hand-edit to an old released entry is still caught by the runtime
manifest<->registry consistency check when the current manifest points at it,
and can be audited anytime with --rebuild-from-tags.
Verified: verify passes committed-sourced; --write is zero-diff (byte parity);
a planted stray v-tag no longer changes output; edit-stub -> --write -> --release
appends the correct single row; double --release is idempotent;
--rebuild-from-tags reproduces the committed artifacts. Generator tests 14 pass/
1 skip; runtime skill-bundle-artifacts + freshness-inventory 14 pass; bundled
skill guides verify passes.
* fix(skills): keep one release-mapping row per version on a re-cut
A cut that pushed the version bump to main but died before pushing the
tag is re-cut at the same version. If skills changed in between, the
second --release appended a duplicate row, and the stale one named
revisions that tag never ships — which verify-skill-update-roundtrip
then pairs with the tag's real bytes.
Overwrite the trailing row instead (the tag is absent, so that version
was never published). Refuse only when an earlier row claims the
version, which the cut workflow already rejects upstream, so this
cannot wedge a recovering cut.
* Add safe skill freshness detection
* Accept observed copy-mode rail outcomes
* chore(skills): regenerate snapshot artifacts for the merged guide content
The rebase onto main picked up the reviewed guide fixes (#8624), so the
current manifest hashes and a new appended snapshot generation must
match those bytes; the registry keeps all prior snapshots so existing
installs classify as outdated rather than unrecognized.
* fix(skills): canonicalize snapshot file order and guard released history
Historical snapshots kept git ls-tree byte-order while the working-tree
walk and runtime observation use the sorted depth-first order, so any
future multi-file skill would misclassify older installs as unrecognized
and churn spurious registry revisions; all producers now share one
canonical order (no digest changes for today's single-file packages).
Also rejects executable files from shipped skills (Windows observation
cannot see execute bits, which would misclassify pristine Windows
installs) and adds an explicit append-only invariant for released
snapshots so a generation-logic change cannot rewrite them silently.
* fix(skills): throttle focus rescans and correct self-blocked placement copy
Every window focus re-read and re-hashed all installed packages, and the
nudge and panel each forced their own trailing rescan for one event; a
15s cooldown plus a shared invalidation latch keep one bounded scan per
event while install-change events stay immediate. Bundle artifacts are
now loaded once per run instead of re-parsed on every scan. A read-only
or otherwise unsupported outdated placement now explains that it blocks
itself instead of blaming a phantom sibling placement; the supported
topology set moved to shared so eligibility and copy cannot drift.
* feat(skills): move freshness surfacing to a lingering toast and update modal
The Skills page has been unreachable since its toolbox menu entry was
removed (#4535), so surfacing freshness there buried the feature behind
its own nudge. The nudge now lingers until acted on (ignoring it records
nothing; only the explicit close persists dismissal keys) and opens an
update modal hosting the pre-filled editable terminal, an honest
current/blocked summary, and the per-placement rows in a collapsed
Details section. A compact 'Check for skill updates' row in CLI settings
is the manual re-entry point. Skills page restored to main; design-doc
surfacing section records the venue decision.
* fix(skills): mount update dialog inside the link-routing provider and fold freshness into the setup rails
The dialog hosts a live terminal pane that requires the link-routing
preference context; mounted outside the provider it crashed the renderer
the moment an eligible update existed (caught by live QA — unit tests
mock the terminal). It now mounts inside the provider behind its own
recoverable boundary.
The separate 'Check for skill updates' settings button is gone: the
setup rails' own pill now carries freshness (Update available / Up to
date, falling back to Installed for blocked or unrecognized copies and
for non-local runtimes the local-only scan cannot vouch for), and
Re-check refreshes both installation detection and the freshness
inventory. Wired for the CLI, Orchestration, Computer Use, and
Per-Workspace Environments rails.
* fix(skills): use the sleek scrollbar style in the update dialog
* chore(skills): regenerate manifest for merged main (v1.4.142-rc.1)
Main advanced to 1.4.142-rc.1 with a v1.4.141 release, so the embedded
appVersion and release mapping were stale on the PR's merged tree. Only
appVersion and the new release entry change; no snapshot digests move
(released history preserved).
* fix(skills): bound and batch freshness work
* fix(skills): harden freshness integrity checks
* fix(skills): accept observed copy topology outcomes
* chore(skills): regenerate manifest for current main
* fix(skills): preserve update terminal lifecycle
* chore(skills): regenerate manifest for current main
* fix(skills): fail closed on stale freshness scans
* chore(skills): regenerate manifest for current main
* fix(skills): preserve freshness safety under focus churn
* feat(skills): group the update modal by skill with plain-language status
The Update skills modal now lists only skills that will update or that can't
(with why), grouped by skill with their install locations nested underneath —
no more one row per placement.
- Statuses collapse to "Update available" / "Can't update" at the skill level.
- A location's problem is a chip (Duplicate, Unrecognized, Inaccessible, Read
only, In a repo, External/Broken link, Plugin cache) with a hover tooltip
that explains what it means for the user and what to do.
- Up-to-date, unrecognized-only, and unreadable-only skills are hidden; a
current/unrecognized/etc. location only appears when it explains a shown skill.
- Copy is de-jargoned (drops "copy"/"placement"/"snapshot"/"official copy") and
names the mechanism as the npx skills update command, not "Orca's update".
- Rename the section to "Update details"; drop the unreachable newer-known state.
Renderer-only: derivation is a pure module (groupSkillFreshness) with unit tests;
no IPC or main-process change. Locales updated for all five languages.
* chore(skills): regenerate manifest for current main (v1.4.143-rc.0)
* feat(skills): don't let a duplicate block the update; clearer skipped copy
- Eligibility: a clean standalone duplicate no longer poisons the whole
name — the canonical copy still updates and the duplicate is flagged;
a duplicate-only skill stays unoffered.
- Update modal: "Can't update" -> "Skipped" with a reason-specific
sentence (edited/read-only/in-a-repo/plugin/link); chips describe only
the location state; footer "Check now" -> "Re-check".
- Settings sidebar nav pills go amber "Update available" when a skill is
updatable, matching the setup cards.
- Localized new strings across en/es/ja/ko/zh.
* chore(skills): regenerate manifest for merged main (v1.4.144-rc.1)
* Add version-matched bundled skill guides
* Clarify skill freshness rollout PRs
* Add canonical skills show alias
* fix(skills): address guide review feedback
* fix(skills): make guide commands cross-platform
* fix(skills): apply the ORCA convention to the emulator guides
Review follow-up: the emulator guides still instructed literal
`orca emulator ...` in sh fences with no Linux disambiguation, so on
unmanaged Linux they could launch the GNOME screen reader — the exact
failure the executable-selection preamble prevents. Both emulator
guides now carry the preamble and ORCA placeholder across fences,
tables, and prose, and the cross-platform safety test covers all four
converted guides. Also replaces computer-use's "unless a block names a
shell" carve-out, which contradicted its own POSIX example, with the
unconditional placeholder rule.
oxlint already fails any file over max-lines that is not suppressed, so the
only way to grow past the budget is to add an eslint/oxlint-disable max-lines
comment or a per-file max-lines bump in mobile/.oxlintrc.json. This adds a CI
gate that freezes the current set of suppressions (config/max-lines-baseline.txt,
355 grandfathered entries) and fails the build when a NEW one appears — with a
loud, actionable message pointing at 'split the file'. Existing oversized files
are untouched; the baseline may only shrink (pnpm check:max-lines-ratchet --prune).
Wired into the root lint script and as a dedicated pr.yml step. Unit-tested
(15 cases) and verified against all three failure paths + clean-tree pass.
Co-authored-by: Orca <help@stably.ai>
* Add rerun action for failed automations
Adds a recoverable failed automation rerun action in the run detail view, with pending-state handling and focused view-state coverage.
Design doc: docs/failed-automation-rerun-action.md
* Rebuild Electron binary before PR tests
- Ensure Node-based unit tests can import `electron` after install blocks package postinstall
- Generate Electron's required `path.txt` in the PR workflow
The e2e reusable workflow defaulted to github.ref, which on
pull_request events is refs/pull/N/merge. GitHub does not compute
that ref when the PR has conflicts or before the merge commit is
ready, causing actions/checkout to fail with "couldn't find remote
ref refs/pull/N/merge" for reasons unrelated to the code.
Pass github.event.pull_request.head.sha from pr.yml so e2e checks
out the PR head directly.
Motivating failure: https://github.com/stablyai/orca/actions/runs/24768962365
Seen on #937.
The default oxlint output buries filenames in multi-line blocks, making
CI errors like "File has too many lines" hard to diagnose. The github
format produces ::error annotations with file paths front and center.
Brennan Benson
Neil
Henry Su
Jinjing
Jinwoo Hong