chore: remove stray mock folder and design-notes docs (#9641)
* chore: remove stray agent-dashboard-mocks folder Design mockups (index.html + RESEARCH.md) accidentally committed alongside the agent dashboard popout feature in #9604. These are scratch design artifacts and don't belong in the shipped repo. * chore: remove stray notes/ design docs Internal skill design/research scratch docs committed in #8624. Two of the three are explicitly marked superseded/folded, none are referenced anywhere in the codebase. Planning artifacts, not shipped documentation.
This commit is contained in:
Brennan Benson
GitHub
parent
88c78611b7
commit
d834cc6303
|
|
@ -1,709 +0,0 @@
|
|||
# Skill Auto-Update Design
|
||||
|
||||
Status: SUPERSEDED in direction by `skill-freshness-design.md` (2026-07-13): Phases 2-4
|
||||
(background writes, WSL, SSH reconcilers) are retired; Phase-1 detection and all empirical
|
||||
findings below remain valid inputs. Original status: revised after design, OSS review, and
|
||||
macOS + Windows + Linux empirical validation (2026-07-12). Phase 1 + background updates were
|
||||
implemented on PR #8496 (branch brennanb2025/skill-auto-update-research), now archived as
|
||||
reference for the write machinery. Windows validation confirmed the skills CLI writes CRLF (drove the
|
||||
text-normalized package-identity rule); Linux confirmed verbatim-LF installs and the
|
||||
`XDG_STATE_HOME` lock-location rule.
|
||||
|
||||
## Problem
|
||||
|
||||
Orca ships agent skills in `skills/` (orca-cli, orchestration, computer-use, orca-linear,
|
||||
linear-tickets, orca-per-workspace-env, orca-emulator, orca-emulator-android). Users install
|
||||
them with the skills CLI:
|
||||
|
||||
```sh
|
||||
npx skills add https://github.com/stablyai/orca --skill <names> --global
|
||||
```
|
||||
|
||||
Nothing then keeps those installations aligned with the Orca release they describe. Settings
|
||||
surface a manual `npx skills update <name> --global` command, but users receive no update
|
||||
signal. A stale skill can therefore tell an agent to use commands that are wrong or unsafe for
|
||||
the Orca binary it is driving.
|
||||
|
||||
## Goals
|
||||
|
||||
1. Keep an Orca skill current after Orca has safely adopted or installed that physical copy.
|
||||
2. Never overwrite a modified, unknown, externally managed, project-scoped, or third-party
|
||||
skill.
|
||||
3. Never install a skill the user did not request.
|
||||
4. Work on macOS, Linux, Windows, WSL, and SSH, with reconciliation performed on the host
|
||||
where the agent reads the skill.
|
||||
5. Coexist with the skills CLI, symlink and copy installs, dotfile managers, read-only or
|
||||
generated configurations, and multiple Orca builds sharing a home directory.
|
||||
6. Keep skill content compatible with the Orca app release that is allowed to manage it.
|
||||
|
||||
Non-goals: updating third-party skills; replacing the skills CLI as the normal install path;
|
||||
managing repo-scoped `.agents/skills` or `.claude/skills`; mutating plugin caches; merging user
|
||||
edits into a new skill release.
|
||||
|
||||
## Research conclusion
|
||||
|
||||
The common package-manager pattern is not to scan arbitrary same-named directories and infer
|
||||
that they are writable. The updater manages packages inside an ownership boundary established
|
||||
at install or adoption time, records package identity outside user content, stages a complete
|
||||
replacement, and treats external links/custom installations as unmanaged.
|
||||
|
||||
The skills CLI lock is useful supporting evidence, but it is not an ownership ledger:
|
||||
|
||||
- The global lock is `$XDG_STATE_HOME/skills/.skill-lock.json` when `XDG_STATE_HOME` is set,
|
||||
otherwise `~/.agents/.skill-lock.json`.
|
||||
- A v3 entry records source, source URL/type, Git ref, path within the repo, an upstream folder
|
||||
hash, and timestamps.
|
||||
- It does not record a physical install path, symlink/copy topology, per-skill global agent
|
||||
placements, the current on-disk hash, local modifications, or app compatibility.
|
||||
- `skills update` compares the stored source hash with upstream and then re-runs installation.
|
||||
It does not prove that installed files are unchanged first.
|
||||
|
||||
Orca therefore reads supported lock versions as a provenance hint but never treats a lock
|
||||
entry by itself as permission to write. Orca never writes the foreign lock format.
|
||||
|
||||
## Decision: detect, adopt, then manage
|
||||
|
||||
Only a physical destination recorded in Orca's management ledger is eligible for background
|
||||
writes. A destination enters that ledger in one of two ways:
|
||||
|
||||
1. Orca's install UI invokes the skills CLI and then verifies and records the resulting
|
||||
physical installation.
|
||||
2. A legacy installation is adopted after its complete on-disk package matches a known,
|
||||
released Orca snapshot and its topology is eligible. Phase 1 makes this an explicit
|
||||
“Manage and update” action. Background auto-update never silently claims a newly
|
||||
discovered path.
|
||||
|
||||
This one-time adoption cost is intentional. Without it, no app can distinguish an official
|
||||
copy from a same-named user copy or establish which manager is allowed to replace it.
|
||||
|
||||
Do not rely on users discovering adoption passively in settings. When the bounded inventory
|
||||
finds exact official snapshots in eligible topologies, show one non-repeating banner/toast:
|
||||
“N installed Orca skills can be kept up to date,” with a one-click review/adoption action.
|
||||
Prioritize recording ownership immediately after every successful Orca-driven skills CLI
|
||||
install so new installations never require a later adoption step.
|
||||
|
||||
“Non-repeating” is scoped per eligible destination snapshot, not globally or per session. Store
|
||||
a dismissed-adoption tuple containing host identity, physical destination identity, skill name,
|
||||
and matched snapshot digest. Do not prompt again for that unchanged tuple, but allow a future
|
||||
prompt when a newly installed skill or genuinely different official snapshot creates a tuple
|
||||
the user has never dismissed. Removing another candidate alone does not clear prior dismissals.
|
||||
|
||||
## Build and release artifacts
|
||||
|
||||
Package `skills/` into app resources on every supported platform and generate a current bundle
|
||||
manifest. For each skill it contains:
|
||||
|
||||
- canonical name and repo-relative source path;
|
||||
- release revision and Orca app version;
|
||||
- deterministic whole-package digest (composed from the per-file identities below);
|
||||
- every regular file's relative path, size, executable bit, a per-file text/binary
|
||||
classification, an exact-byte SHA-256, and — for text files — a text-normalized SHA-256 with
|
||||
line endings folded to LF;
|
||||
- the upstream Git tree SHA when available;
|
||||
- minimum/maximum compatible app version if a skill is not backward-compatible;
|
||||
- schema version.
|
||||
|
||||
The build rejects absolute paths, traversal, case-colliding paths, special files, and symlinks
|
||||
inside the shipped package. Executable modes and the exact bytes of binary files are part of
|
||||
package identity. For text files, identity is the line-ending-normalized content, not the exact
|
||||
bytes: supported installers apply platform- and Git-config-dependent EOL translation, so an
|
||||
exact-byte hash is not stable across hosts. This is validated, not hypothetical — the skills CLI
|
||||
writes CRLF on a default Windows Git install (see Empirical validation), so a macOS-built
|
||||
exact-byte hash never matches a Windows install and would misclassify every Windows copy as
|
||||
modified. Because the whole-package digest composes the per-file normalized-or-exact hashes, one
|
||||
official snapshot has a single identity across macOS, Linux, and Windows and across
|
||||
`core.autocrlf` settings.
|
||||
|
||||
Maintain a compact, checked-in registry of every generated Orca skill snapshot plus a separate
|
||||
release mapping that identifies which revisions actually shipped. Historical file bytes are
|
||||
unnecessary; historical paths and hashes are sufficient to prove that an existing package is
|
||||
an exact official snapshot and to map a skills CLI folder hash to an Orca release. Only a
|
||||
revision present in the release mapping is eligible as legacy-install provenance; an
|
||||
unreleased candidate cannot be adopted merely because it appeared on main.
|
||||
|
||||
Release revision assignment is mechanical, not a hand-edited field. The manifest generator
|
||||
compares each package digest with the latest generated registry entry: unchanged content keeps
|
||||
its revision; changed content appends the next integer. The generator is the only writer of
|
||||
registry entries, and existing entries are immutable. Pull-request CI verifies generated
|
||||
output, but the same generation/monotonicity check must rerun against the merge-queue head and
|
||||
on main pushes so two independently green PRs cannot record different content with the same
|
||||
revision. Release creation adds the current revisions to the release mapping and fails unless
|
||||
main's generated registry is current and the packaged manifest exactly matches it.
|
||||
|
||||
Do not rely on `metadata.version` inside `SKILL.md` as the authority. Installers can transform
|
||||
frontmatter, users can edit it, and a value inside the package cannot prove the rest of the
|
||||
package is intact.
|
||||
|
||||
The exact-snapshot model depends on supported skills CLI installations preserving the shipped
|
||||
package. Add a release CI round trip on macOS, Linux, and Windows that installs representative
|
||||
single- and multi-file Orca skills through a pinned supported CLI version in both symlink and
|
||||
copy/fallback shapes, then compares paths, bytes, and applicable executable modes with the
|
||||
generated bundle manifest. Also exercise the newest CLI as an early-warning job. The bundle
|
||||
manifest remains generated from Orca's shipped source; if an installer intentionally transforms
|
||||
content, model that installation shape explicitly or mark it ineligible rather than silently
|
||||
changing the authoritative digest. A mismatch blocks background-update rollout for that shape.
|
||||
macOS verbatim behavior and Windows CRLF translation are both confirmed (see Empirical
|
||||
validation); the round trip must assert LF-normalized identity holds — not exact bytes — and
|
||||
cover `core.autocrlf` on/off plus the copy-fallback and junction shapes as a regression guard.
|
||||
|
||||
## Eligible roots and topology
|
||||
|
||||
Use an explicit registry of global, user-owned skill roots. Do not derive writable roots from
|
||||
all discovery sources: discovery also includes repo roots and the Codex plugin cache, neither
|
||||
of which this feature may mutate.
|
||||
|
||||
Classify every discovered physical destination before offering adoption:
|
||||
|
||||
| Installation topology | Behavior |
|
||||
| --- | --- |
|
||||
| Canonical `~/.agents/skills/<name>` with a known official snapshot | Eligible for adoption |
|
||||
| Provider symlink/junction resolving to that canonical copy | Dedupe; manage the canonical copy once |
|
||||
| Independent copy in an approved global provider root with a known official snapshot | Eligible for separate adoption |
|
||||
| Modified, incomplete, or unknown same-named copy | Never auto-write; show diff/replacement action |
|
||||
| Symlink/junction into dotfiles, a checkout, Nix/Home Manager output, network storage, or another external tree | Unmanaged; never write through the link |
|
||||
| Broken/dangling provider symlink (target missing) | Inaccessible; never adopt or write through; offer provenance-verified repair only with consent |
|
||||
| Read-only/generated root | Detection only |
|
||||
| Repo-scoped skill or plugin cache | Out of scope; never mutate |
|
||||
|
||||
Hardlinks, directory junctions, case-insensitive aliases, and symlinked parent directories must
|
||||
be deduplicated by physical identity where the host API exposes it, with normalized real paths
|
||||
as the fallback. Revalidate the entry type and resolved parent immediately before every
|
||||
mutation so a link swap cannot redirect a verified write.
|
||||
|
||||
Management and agent visibility are separate. Updating a canonical `~/.agents/skills` copy
|
||||
does not make it visible to an agent that reads only its provider-specific root. If no verified
|
||||
provider link/copy exists, settings must say “managed but not visible to <agent>” rather than
|
||||
reporting the provider as current. Creating or repairing a provider link is a separate,
|
||||
provenance-verified, user-approved follow-up; auto-update never invents a missing placement.
|
||||
|
||||
## Orca management ledger
|
||||
|
||||
Store state in Orca-owned application state on the execution host, never inside a user skill
|
||||
directory. Use one record per adopted physical destination:
|
||||
|
||||
- stable execution-host identity and user/home identity;
|
||||
- logical root kind and unresolved destination path;
|
||||
- last verified physical identity, entry type, and resolved path;
|
||||
- skill name, source, source path, and source ref/hash evidence;
|
||||
- installed release revision and whole-package digest;
|
||||
- per-file paths, hashes, modes, and the digest Orca last wrote;
|
||||
- last attempted bundle fingerprint, outcome, and error category;
|
||||
- adoption source and timestamp.
|
||||
|
||||
Local, WSL distro, and SSH records are isolated. SSH identity must include the persisted target
|
||||
identity plus the resolved remote user/home; an in-memory provider object or relay connection
|
||||
ID is not durable identity. Host-side state lets a remote remember ownership across desktop
|
||||
reinstalls and reconnects.
|
||||
|
||||
Corrupt, missing, migrated, or mismatched state fails closed. It can be reconstructed only by
|
||||
the same exact-snapshot adoption rules; it never grants ownership from a name alone.
|
||||
|
||||
## Detection and adoption
|
||||
|
||||
For each Orca skill found in an approved global root:
|
||||
|
||||
1. `lstat` the logical path and classify its topology without following external targets for
|
||||
mutation.
|
||||
2. Dedupe aliases that resolve to the same eligible canonical destination.
|
||||
3. Read the supported skills CLI lock as optional evidence. Bind a lock entry only to the
|
||||
canonical installation shape it describes; never apply one name-level entry to every
|
||||
same-named copy.
|
||||
4. Hash the complete physical package — exact bytes for binary/executable files, LF-normalized
|
||||
content for text files — and compare with the released-manifest registry, so installer EOL
|
||||
translation never misclassifies an official Windows install as modified.
|
||||
5. Classify it as current, update available, newer known release, modified, unknown,
|
||||
externally managed, or inaccessible.
|
||||
6. Offer “Manage and update” only for an exact known official snapshot in an eligible
|
||||
topology. Show a diff and explicit destructive replacement action for modified/unknown
|
||||
content; that action is not adoption and must preserve a backup until success.
|
||||
|
||||
Never send skill contents, diffs, paths, or user edits to telemetry or normal logs.
|
||||
|
||||
## Reconcile algorithm for adopted destinations
|
||||
|
||||
For each adopted physical destination whose bundle revision is newer and app-compatible:
|
||||
|
||||
1. Acquire an Orca transaction lock scoped to the execution host and destination. Dedupe
|
||||
in-flight work within the process as well. Other managers do not honor this lock, so also
|
||||
revalidate immediately before publish.
|
||||
2. Re-read topology and hash the installed package. Continue only when it equals the exact
|
||||
digest in the ledger. Any local edit, extra file, missing file, skills CLI update, or link
|
||||
change turns the destination into a conflict and cancels the write.
|
||||
3. Stage the complete bundled package in a unique directory under the reserved transaction
|
||||
workspace on the same filesystem. Write exact bytes and modes, then validate the staged
|
||||
package against the manifest.
|
||||
4. Recheck the live destination digest and identity after staging. If either changed, delete
|
||||
the staging directory and report a concurrent modification.
|
||||
5. Publish using the strongest package-level replacement supported by that host. Consumers
|
||||
require a fixed skill path, so a universally atomic directory swap is not possible,
|
||||
especially on Windows. Where package-level replacement is unavailable, first preserve a
|
||||
complete rollback copy, then use this required order:
|
||||
1. publish every new/changed non-`SKILL.md` file with same-directory temp + rename;
|
||||
2. publish `SKILL.md` with same-directory temp + rename as the semantic commit marker;
|
||||
3. unlink only obsolete files recorded in the old ledger, rechecking that each still has
|
||||
its old verified hash, then remove only empty recorded directories.
|
||||
This order ensures a removed file cannot make the new digest permanently unreachable and
|
||||
avoids deleting an asset while the new entry point is not yet present. It does not make a
|
||||
multi-file update atomic: a reader may briefly observe the old entry point with new assets,
|
||||
or the new entry point with harmless obsolete assets. Skills requiring cross-file atomicity
|
||||
must use package-level replacement; if the host cannot provide it, skip and retry rather
|
||||
than use the in-place fallback. Do not claim stronger consistency than the host provides.
|
||||
6. Verify the live package digest. Only then update the destination ledger and remove the
|
||||
backup. On failure, restore the old package when possible, retain the prior ledger digest,
|
||||
and leave the destination retryable.
|
||||
7. Notify discovery and show one aggregated toast for successfully updated skills. Running
|
||||
agent sessions pick changes up at their next skill load/session start.
|
||||
|
||||
Never repair equal-revision drift automatically. Equal revision plus different content is a
|
||||
conflict, not proof of a partial write. Never downgrade a known newer release.
|
||||
|
||||
Line endings are a rendering of text content, not part of it. The bundle ships LF, but supported
|
||||
installers write platform-native endings (validated: CRLF on default Windows Git). Compare text
|
||||
provenance and drift on LF-normalized content; stage and publish text files in the destination's
|
||||
existing EOL convention, defaulting to what a fresh supported install would produce on that host
|
||||
when adopting a copy that has none; write binary and executable files as exact bytes. This keeps
|
||||
an updated file byte-shaped like a fresh CLI install, so an EOL difference alone never counts as
|
||||
a conflict and Orca and the skills CLI do not reclassify each other's writes.
|
||||
|
||||
### Transaction workspace and crash recovery
|
||||
|
||||
Staging and rollback packages must be on the same filesystem/volume as the live destination,
|
||||
but must not appear as candidate skills. Use an Orca-reserved transaction root adjacent to the
|
||||
skill root when possible (for example, beside `skills/`, not as another child skill), verify
|
||||
same-filesystem identity, and fall back to a reserved child only when the skill root is itself
|
||||
a mount boundary. Both general skill discovery and updater inventory must hard-exclude the
|
||||
reserved transaction root; a leading dot alone is not an exclusion rule.
|
||||
|
||||
Each transaction directory contains an Orca marker with schema version, transaction ID,
|
||||
destination identity, creation time, and an atomically advanced transaction phase before it
|
||||
receives skill files. On host startup and before reconciliation, sweep only marked orphan
|
||||
transactions whose owning lock is absent/stale: restore a verified rollback package when the
|
||||
marker/ledger phase says publication was incomplete, otherwise remove the verified
|
||||
staging/backup directory. Never delete an unmarked directory based on its name, age, or
|
||||
resemblance to a skill. Coordinate cleanup with the same destination lock so one Orca process
|
||||
cannot sweep another process's live transaction.
|
||||
|
||||
### Removed files and retired skills
|
||||
|
||||
Package-level replacement naturally omits files removed by a newer managed package. The
|
||||
in-place fallback explicitly removes old-ledger files after publishing the new `SKILL.md`, as
|
||||
specified above. Never delete an unrecorded extra file from a live destination; its presence
|
||||
causes the pre-publish digest check to fail before any write.
|
||||
|
||||
Retired skills are detection/prompt-only in the initial implementation. A future cleanup may
|
||||
delete only individually recorded, unchanged files and then empty directories. It must never
|
||||
recursively delete a skill directory or follow a link target.
|
||||
|
||||
## Fast path and triggers
|
||||
|
||||
State is per destination, not one success bit for an entire host. A failed, inaccessible, or
|
||||
partially reconciled destination remains retryable even when other destinations succeeded.
|
||||
|
||||
At launch, after first paint:
|
||||
|
||||
1. Perform a bounded inventory of approved global roots to detect newly installed, removed,
|
||||
or topology-changed Orca skills.
|
||||
2. For adopted destinations, skip content hashing only when that destination already records
|
||||
successful reconciliation with the current bundle and its cheap identity/stat signature is
|
||||
unchanged.
|
||||
3. Hash only new, changed, failed, or bundle-mismatched candidates.
|
||||
|
||||
Also run/invalidate on:
|
||||
|
||||
- successful Orca-driven skill installation;
|
||||
- `notifyInstalledAgentSkillsChanged()` after an install/update terminal exits;
|
||||
- WSL distro first activation;
|
||||
- SSH connection after the host runtime is ready;
|
||||
- restart into a newly installed Orca app version.
|
||||
|
||||
Do not reconcile on the updater's “download complete” event: the running process still owns
|
||||
the old app resources until restart. Coalesce triggers and cap concurrency so launch, WSL, and
|
||||
SSH activation cannot fan out unbounded filesystem or network work.
|
||||
|
||||
## Host execution
|
||||
|
||||
- **macOS/Linux:** use Node filesystem APIs and platform app-state paths; do not shell out.
|
||||
- **Windows:** use `path` APIs, preserve exact bytes, support junction/copy topology, handle
|
||||
case-insensitive identity and long/UNC paths, and retry bounded `EPERM`/`EBUSY` replacement
|
||||
failures. A skipped destination remains retryable and never advances its ledger digest. A
|
||||
long-running agent may keep an obsolete file open; if its hash-verified unlink still fails
|
||||
after bounded retries, roll back the whole update and retry after the handle is released.
|
||||
- **WSL:** run reconciliation inside the selected distro through a host-side runtime/RPC
|
||||
operation. Do not mix Windows UNC mutation semantics with Linux locks, modes, and renames.
|
||||
State is scoped to distro plus Linux user/home.
|
||||
- **SSH:** run discovery, hashing, staging, locking, and publication on the remote host through
|
||||
the runtime/filesystem abstraction. Transfer only the selected bundled package and manifest.
|
||||
Do not assemble shell commands. Support Linux, macOS, and Windows SSH targets, and fail closed
|
||||
when the remote runtime lacks a required safe filesystem primitive.
|
||||
|
||||
## Multiple writers and app compatibility
|
||||
|
||||
The same global roots may be touched by stable Orca, a development build, and the skills CLI.
|
||||
Orca cannot guarantee that one shared global package simultaneously matches two incompatible
|
||||
app binaries.
|
||||
|
||||
- Production stable Orca is the only automatic writer by default.
|
||||
- Main-process runtime identity is authoritative: `app.isPackaged`, the signed release channel,
|
||||
and the resolved user-data/home roots determine whether writes are allowed. Renderer build
|
||||
flags alone are insufficient. An unpackaged build or development channel is detection-only
|
||||
unless both skill home and user-data roots are explicitly isolated from production.
|
||||
- A stable app writes only a bundle declared compatible with that app version.
|
||||
- A newer known installed release is never downgraded.
|
||||
- If `npx skills update` changes an adopted package, the next Orca check sees a ledger-digest
|
||||
mismatch and stops managing it until the new content matches a known released snapshot and
|
||||
is explicitly re-adopted.
|
||||
- Provenance and drift comparison fold text line endings to LF, so a stable app and the skills
|
||||
CLI never treat each other's platform-native EOL output as a conflict; only real content
|
||||
changes do.
|
||||
- Skills should remain backward-compatible across supported stable app versions where
|
||||
practical; compatibility metadata is still required for exceptions.
|
||||
|
||||
This avoids version ping-pong. A monotonic number inside user content alone cannot solve
|
||||
multiple incompatible writers.
|
||||
|
||||
## Consent and settings UX
|
||||
|
||||
The background setting is “Keep managed Orca agent skills up to date.” It controls only
|
||||
already adopted/Orca-installed destinations and may default on. It does not authorize claiming
|
||||
new paths.
|
||||
|
||||
Settings show each physical installation as one of:
|
||||
|
||||
- managed and current;
|
||||
- managed, update available;
|
||||
- known official copy, available to manage;
|
||||
- modified/unknown, review required;
|
||||
- externally managed/read-only;
|
||||
- inaccessible or update failed.
|
||||
|
||||
Managed and known-snapshot rows show the released skill revision, Orca app release, and a short
|
||||
digest for human/support diagnosis. Do not add a second, non-authoritative version marker to
|
||||
`SKILL.md`; it can be transformed independently of the package and mistaken for write
|
||||
authority.
|
||||
|
||||
Every successful background batch produces one toast. Conflicts and failures remain visible in
|
||||
settings without repeated error toasts. Explicit replacement shows a local diff, warns that it
|
||||
discards edits, and keeps a rollback backup until verification succeeds.
|
||||
|
||||
## Empirical validation (2026-07-12, macOS + Windows + Linux)
|
||||
|
||||
Both load-bearing assumptions were tested on a real developer machine against live installed
|
||||
skills, not deferred to Phase 1 telemetry. Results are recorded so the evidence travels with
|
||||
the design.
|
||||
|
||||
- **Verbatim install (decisive).** A clean-room `npx skills add https://github.com/stablyai/orca
|
||||
--skill orca-cli orchestration --global --yes` into a throwaway home produced files
|
||||
byte-identical to `origin/main`: equal Git blob hashes, equal byte counts, zero CRLF, exactly
|
||||
one `SKILL.md` per skill, and no injected or stripped files. The CLI does not transform content
|
||||
on macOS, so exact-content provenance is viable.
|
||||
- **Historical-snapshot match on real stale installs.** The machine's genuinely stale `orca-cli`,
|
||||
`computer-use`, and `orchestration` have on-disk bytes that exist verbatim as committed Git
|
||||
blobs in repo history. Exact-content adoption against a released-snapshot registry would
|
||||
recognize real, messy installs — not just freshly installed ones.
|
||||
- **Release-mapping guard justified by data.** Those stale blobs are reachable at commits dated
|
||||
after their recorded install time, i.e. identical bytes existed in a checkout/branch before or
|
||||
independently of shipping. Presence in history is therefore not proof of an official release;
|
||||
adoption must gate on the release mapping, and content identity — never timestamps — is the
|
||||
arbiter. This is exactly the hole the separate release mapping closes.
|
||||
- **Topology and dedup.** Provider skills under `~/.claude/skills` are symlinks (both relative
|
||||
`../../.agents/...` and absolute forms) into canonical `~/.agents/skills`; realpath resolution
|
||||
collapses provider and canonical to one physical destination, confirming physical-identity
|
||||
dedup yields a single write. A live broken/dangling provider symlink was also present (target
|
||||
missing), which is why the topology table has an explicit inaccessible row.
|
||||
- **Lockfile corroboration present (macOS).** Lock entries carry `source: stablyai/orca` and a
|
||||
folder hash, usable only as corroboration, consistent with the design.
|
||||
|
||||
**Windows (2026-07-12, validated on a real machine via the handoff below).** A clean-room
|
||||
`npx skills add ... --skill orca-cli --global` on a default, non-Developer-Mode Windows install
|
||||
produced:
|
||||
|
||||
- **CRLF translation, not byte-identical.** Installed `SKILL.md` was 21180 bytes with 318 CR
|
||||
bytes; repo-main source was 20862 bytes with 0 CR. The size delta equals the CR count exactly,
|
||||
i.e. a pure LF→CRLF translation with identical text. This is why text-file package identity is
|
||||
LF-normalized rather than exact-byte; an exact-byte model would have adopted nothing on Windows.
|
||||
- **Copy-fallback shape.** `.agents\skills\orca-cli` was a plain directory copy (no link), and no
|
||||
`.claude` provider copy was created (Developer Mode off, process unelevated). Confirms the
|
||||
Windows copy path and the need to manage independent per-root copies, not only a canonical
|
||||
symlink target.
|
||||
- **Same file set.** Only `SKILL.md` in both source and install — no injected or stripped files.
|
||||
- **Lockfile populated.** The lock recorded `orca-cli`; the known Windows empty-lockfile bug did
|
||||
not reproduce on this machine/version. Corroboration signal is therefore sometimes available on
|
||||
Windows, but the design still relies on content-match as primary since it is not guaranteed.
|
||||
|
||||
Still gated behind the cross-platform CI round trip as a regression guard, and untested:
|
||||
`core.autocrlf=false` on Windows (would install LF), and the junction/symlink shape under
|
||||
Developer Mode. LF-normalized identity covers the autocrlf variance by construction.
|
||||
|
||||
**Linux (2026-07-12, throwaway Docker container, reached over SSH).** Ran the same clean-room
|
||||
install on `Linux 6.12 aarch64` (node 22, npx 10.9, git 2.39). The container served SSH (sshd
|
||||
listening); the CLI check was executed on the box, since SSH transport does not change what the
|
||||
CLI writes to disk:
|
||||
|
||||
- **Verbatim LF, like macOS.** `orca-cli` and `orchestration` installed byte-identical to
|
||||
`origin/main`: equal SHA-256, exact byte counts (20862 / 22850), **CR=0**, only `SKILL.md` in
|
||||
each dir. Linux needs no separate manifest shape — it is covered by the LF identity.
|
||||
- **Symlink topology.** `~/.claude/skills/orca-cli` is a relative symlink
|
||||
(`../../.agents/skills/orca-cli`) into the canonical `~/.agents/skills` copy, matching macOS;
|
||||
realpath dedup collapses provider and canonical to one write.
|
||||
- **XDG lockfile path confirmed, with a sharper rule.** With `XDG_STATE_HOME` unset the lock is
|
||||
`~/.agents/.skill-lock.json` (`source: stablyai/orca`); with `XDG_STATE_HOME` set the lock is
|
||||
at `$XDG_STATE_HOME/skills/.skill-lock.json` and **not** at `~/.agents` — it moves, it is not
|
||||
duplicated. So the corroboration reader must resolve `XDG_STATE_HOME` and read the single
|
||||
correct location; checking only `~/.agents` finds no lock at all on such hosts. This is a
|
||||
property of the Linux/host environment, not of Orca's SSH transport, so it applies equally to
|
||||
native Linux, WSL, and SSH Linux targets — the reconciler must resolve the remote host's
|
||||
`XDG_STATE_HOME` when reading lock corroboration remotely.
|
||||
|
||||
Not exercised here: Orca's own remote reconciler over SSH (Phase 4, unbuilt — nothing to drive
|
||||
yet). This validated the Linux CLI install shape and the remote lock-location rule the reconciler
|
||||
will depend on.
|
||||
|
||||
## Windows validation handoff
|
||||
|
||||
Result (2026-07-12): **FAIL — CRLF translation**, resolved by LF-normalized text identity (see
|
||||
Build and release artifacts and Empirical validation). The procedure is retained for CI
|
||||
regression and for the still-untested `core.autocrlf=false` and Developer-Mode junction shapes.
|
||||
|
||||
Give this to an agent on a Windows machine. It is self-contained, non-destructive (sandboxes the
|
||||
skills CLI to a throwaway profile), and requires only PowerShell, Node/npx, and Git. It answers
|
||||
one question: does `npx skills add` on Windows write skill bytes identical to the repo source, or
|
||||
does it translate line endings / change the file set — and where does the lockfile land.
|
||||
|
||||
Goal and pass/fail:
|
||||
|
||||
- PASS (design safe as written): the installed `SKILL.md` is byte-identical to the repo source
|
||||
(equal SHA-256, equal byte count, zero CRLF), and the skill directory contains the same file
|
||||
set as the source.
|
||||
- FAIL (design must model the Windows shape explicitly or mark it ineligible): the installed file
|
||||
differs only by CRLF/line endings, or the file set differs, or the lockfile `skills` object is
|
||||
empty after a successful install (the known Windows lockfile-not-written failure), which means
|
||||
lockfile corroboration is unavailable on Windows and content-match must carry provenance alone.
|
||||
|
||||
Run this in PowerShell and paste the full transcript back:
|
||||
|
||||
```powershell
|
||||
$ErrorActionPreference = 'Stop'
|
||||
$sandbox = Join-Path $env:TEMP ("orca-skilltest-" + [Guid]::NewGuid().ToString('N'))
|
||||
New-Item -ItemType Directory -Path $sandbox | Out-Null
|
||||
# Sandbox the CLI's global install to a throwaway profile so real skills are untouched.
|
||||
$old = @{ USERPROFILE=$env:USERPROFILE; HOME=$env:HOME; XDG_STATE_HOME=$env:XDG_STATE_HOME }
|
||||
$env:USERPROFILE = $sandbox; $env:HOME = $sandbox; Remove-Item Env:XDG_STATE_HOME -ErrorAction SilentlyContinue
|
||||
try {
|
||||
npx --yes skills add https://github.com/stablyai/orca --skill orca-cli --global --yes 2>&1 | Tee-Object "$sandbox\install.log" | Out-Null
|
||||
|
||||
$installed = Join-Path $sandbox '.agents\skills\orca-cli\SKILL.md'
|
||||
$truth = Join-Path $sandbox 'truth-SKILL.md'
|
||||
# Ground truth = exact bytes Git stores on main (LF), fetched without transformation.
|
||||
Invoke-WebRequest 'https://raw.githubusercontent.com/stablyai/orca/main/skills/orca-cli/SKILL.md' -OutFile $truth
|
||||
|
||||
function Info($label,$f){
|
||||
if(!(Test-Path $f)){ Write-Host "$label`: MISSING"; return }
|
||||
$bytes=[IO.File]::ReadAllBytes($f)
|
||||
$crlf=($bytes | Where-Object {$_ -eq 13}).Count
|
||||
Write-Host ("{0}: sha256={1} bytes={2} CR={3}" -f $label,(Get-FileHash $f -Algorithm SHA256).Hash.Substring(0,16),$bytes.Length,$crlf)
|
||||
}
|
||||
Write-Host "`n=== byte fidelity ==="
|
||||
Info 'installed' $installed
|
||||
Info 'repo-main ' $truth
|
||||
$same = (Get-FileHash $installed -Algorithm SHA256).Hash -eq (Get-FileHash $truth -Algorithm SHA256).Hash
|
||||
Write-Host ("VERDICT: {0}" -f ($(if($same){'VERBATIM (pass)'}else{'DIFFERS (inspect CR counts: CRLF-only diff = autocrlf translation)'})))
|
||||
|
||||
Write-Host "`n=== link shape (junction/symlink/copy) for provider + canonical ==="
|
||||
foreach($p in @("$sandbox\.claude\skills\orca-cli","$sandbox\.agents\skills\orca-cli")){
|
||||
if(Test-Path $p){ $i=Get-Item $p; Write-Host ("{0} -> LinkType={1} Target={2}" -f $p,$i.LinkType,($i.Target -join ',')) }
|
||||
else { Write-Host "$p -> (absent)" }
|
||||
}
|
||||
|
||||
Write-Host "`n=== file set in installed skill dir (extra/stripped files?) ==="
|
||||
Get-ChildItem (Join-Path $sandbox '.agents\skills\orca-cli') -Recurse -File | ForEach-Object { $_.FullName.Substring($sandbox.Length) }
|
||||
|
||||
Write-Host "`n=== lockfile location + whether skills object populated (Windows #-not-written bug) ==="
|
||||
foreach($lp in @("$sandbox\.agents\.skill-lock.json", "$env:XDG_STATE_HOME\skills\.skill-lock.json")){
|
||||
if($lp -and (Test-Path $lp)){
|
||||
$j=Get-Content $lp -Raw | ConvertFrom-Json
|
||||
Write-Host ("{0} -> skills keys: {1}" -f $lp, (($j.skills.PSObject.Properties.Name) -join ','))
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
$env:USERPROFILE=$old.USERPROFILE; $env:HOME=$old.HOME; if($old.XDG_STATE_HOME){$env:XDG_STATE_HOME=$old.XDG_STATE_HOME}
|
||||
Remove-Item $sandbox -Recurse -Force -ErrorAction SilentlyContinue
|
||||
Write-Host "`n(sandbox removed; your real skills were never touched)"
|
||||
}
|
||||
```
|
||||
|
||||
Also report, in words: (1) is Windows Developer Mode on (decides whether junctions or copy
|
||||
fallback occurred)? (2) the VERDICT line, (3) whether CR counts differ between installed and
|
||||
repo-main (CRLF-only difference = `core.autocrlf` translation → design must treat the Windows
|
||||
install shape as its own manifest or mark it ineligible), (4) the link shapes, (5) the file set,
|
||||
(6) whether any lockfile `skills` object was populated. If Developer Mode can be toggled, run the
|
||||
block once with it on and once off to capture both junction and copy-fallback shapes.
|
||||
|
||||
## Historical rollout (superseded; do not implement)
|
||||
|
||||
The steps below preserve the retired write-based rollout for research context only. The
|
||||
active rollout is defined in `skill-freshness-design.md` and uses read-only detection plus a
|
||||
user-invoked, targeted `npx skills update <names...> --global` command; it has no ledger,
|
||||
adoption flow, background updater, transactional writer, WSL reconciler, or SSH reconciler.
|
||||
|
||||
1. **Detection and adoption:** ship current/historical manifests, lockfile parsing, topology
|
||||
classification, destination-scoped ledger, settings states, and an explicit “Manage and
|
||||
update” action. No background writes.
|
||||
2. **Local background updates:** enable adopted destinations on native macOS/Linux/Windows,
|
||||
including transaction, rollback, concurrency, and restart tests.
|
||||
3. **WSL:** move the same host-side reconciler into the distro runtime and validate Linux
|
||||
semantics independently of UNC discovery.
|
||||
4. **SSH:** expose the reconciler through remote runtime/RPC and validate Linux/macOS/Windows
|
||||
remote hosts, reconnects, and multiple desktop clients.
|
||||
|
||||
Do not advance phases based only on unit tests. Each phase needs a real-host package update,
|
||||
failure injection between every transaction boundary, and proof that modified/external content
|
||||
was not written.
|
||||
|
||||
## Implementation touchpoints
|
||||
|
||||
Grounded in the current codebase (verified 2026-07-12); an implementer starts here. Follow the
|
||||
repo naming rule — concrete domain names, no `helpers`/`utils`.
|
||||
|
||||
Main process (skill engine):
|
||||
|
||||
- `src/main/skills/skill-discovery-sources.ts` — the approved writable-root registry lives here
|
||||
or beside it. Today it enumerates `~/.codex/skills`, `~/.agents/skills`, `~/.claude/skills`
|
||||
(sourceKind `home`) plus a Codex plugin cache (`plugin`) and repo roots. The updater must
|
||||
include only `home` roots and exclude the plugin cache and repo roots.
|
||||
- `src/main/skills/discovery.ts` and `src/main/ipc/skills.ts` (`registerSkillsHandlers(store)`,
|
||||
wired in `src/main/ipc/register-core-handlers.ts`) — extend discovery to classify topology and
|
||||
provenance, and add IPC for ledger states, the adoption action, and explicit replace. The
|
||||
handler already receives the persistence `store`.
|
||||
- New modules, e.g. `src/main/skills/skill-manifest.ts`, `skill-ledger.ts`, `skill-reconcile.ts`
|
||||
— manifest load, content identity (LF-normalized text / exact-byte binary), transaction
|
||||
workspace, publish/rollback.
|
||||
- `src/shared/skill-metadata.ts` — existing top-level-only frontmatter parser; reuse for
|
||||
name/description in settings. It need not read a version (identity is manifest-based, not
|
||||
`metadata.version`).
|
||||
|
||||
State and host identity:
|
||||
|
||||
- `src/main/persistence.ts` (`store`, host-partitioned `orca-data.json`) — home for the
|
||||
management ledger; already host-aware and passed to the skills handler.
|
||||
- `src/shared/execution-host.ts` — `ExecutionHostId = 'local' | ssh:<id> | runtime:<id>`; add a
|
||||
`wsl:<distro>` variant (none today) so ledger records key per host. Do not key off
|
||||
`src/main/git/git-capability-state.ts`: it is in-memory only and scopes SSH by provider object
|
||||
identity, which is not durable across reconnects or restarts.
|
||||
|
||||
Renderer and UX:
|
||||
|
||||
- The ~13 setting/feature surfaces that today print raw `npx skills ...` strings from
|
||||
`src/shared/agent-feature-install-commands.ts` (CliSection, OrchestrationPane, BrowserUsePane,
|
||||
EphemeralVmsPane, ComputerUseSkillSetupPanel, the linear/emulator CTAs, and the feature-wall /
|
||||
feature-tip cards) become ledger-state rows with adoption/update actions.
|
||||
- `src/renderer/src/hooks/useInstalledAgentSkills.ts` — `notifyInstalledAgentSkillsChanged()` is
|
||||
the post-write refresh signal, already listened to on focus and on the install event.
|
||||
|
||||
Build and CI:
|
||||
|
||||
- `config/electron-builder.config.cjs` (+ `config/scripts/electron-builder-config.test.mjs`) —
|
||||
extraResources is already contract-tested; add `skills/` bundling and the generated manifest.
|
||||
- Manifest generator, released-snapshot registry, and the monotonic-revision check run on PR and
|
||||
re-run on merge-queue / main pushes (the localization-catalog check in `pnpm lint` is the
|
||||
precedent for a generated-output gate).
|
||||
- Cross-platform round trip asserts LF-normalized identity (not exact bytes) for text files and
|
||||
exact bytes for binary/executable files.
|
||||
|
||||
Tests:
|
||||
|
||||
- `tests/e2e/settings-skill-detection.spec.ts` — extend for update-available / adoption /
|
||||
conflict / rollback states.
|
||||
- Keep filesystem-transaction and host-isolation coverage as deterministic integration tests
|
||||
below E2E, per the Test matrix.
|
||||
|
||||
### Phase 1 definition of done
|
||||
|
||||
Ship-ready, with no background writes, when:
|
||||
|
||||
- The bundled manifest and released-snapshot registry are generated and CI-verified (monotonic
|
||||
per-skill revisions, immutable history, release mapping).
|
||||
- Discovery classifies every `home`-root Orca skill as current / update-available / newer-known /
|
||||
modified / unknown / externally-managed / inaccessible, using LF-normalized text identity.
|
||||
- The ledger records adopted destinations in the host-partitioned store, keyed by
|
||||
`ExecutionHostId` (plus `wsl:<distro>` where applicable).
|
||||
- Settings shows those states; a proactive non-repeating adoption nudge exists; "Manage and
|
||||
update" and explicit destructive replace (with backup and diff) work on the local host.
|
||||
- Orca-driven installs auto-record ownership.
|
||||
- No path is written in the background and no path is adopted silently.
|
||||
|
||||
Phase 1 exists to validate the two field assumptions before the background writer is built:
|
||||
the byte/EOL identity match rate on real installs, and the adoption take-rate on the nudge.
|
||||
|
||||
## Test matrix
|
||||
|
||||
### Content and provenance
|
||||
|
||||
- exact current and historical official snapshot;
|
||||
- edited, missing, extra, truncated, and mode-changed file;
|
||||
- foreign same-name skill;
|
||||
- equal revision/different content;
|
||||
- newer known and unknown release;
|
||||
- missing, corrupt, old-version, XDG-located, and spoofed skills CLI lock;
|
||||
- lost/corrupt Orca ledger and app reinstall;
|
||||
- source ref/path changes and repo/skill rename;
|
||||
- supported pinned and newest skills CLI round trips match bundle identity on macOS, Linux, and
|
||||
Windows, including symlink and copy/fallback installation: exact bytes/modes for binary and
|
||||
executable files, LF-normalized content for text files;
|
||||
- CRLF vs LF install (`core.autocrlf` on and off) adopts and stays managed via normalized
|
||||
identity; an EOL-only difference is never a conflict; a real content change still is;
|
||||
- Orca-published text files keep the destination's existing EOL convention and do not trigger a
|
||||
skills-CLI re-update loop.
|
||||
- two same-skill PRs that independently change content cannot pass the merge queue/main
|
||||
monotonicity gate with one revision;
|
||||
- adoption dismissal suppresses the same destination snapshot but a newly installed official
|
||||
destination remains eligible for one proactive prompt.
|
||||
|
||||
### Topology
|
||||
|
||||
- canonical copy with provider symlinks;
|
||||
- independent copy mode and Windows symlink-to-copy fallback;
|
||||
- parent-directory symlink, relative/absolute skill symlink, Windows junction, hardlink, and
|
||||
case-variant alias;
|
||||
- external dotfiles/chezmoi/stow target;
|
||||
- Nix/Home Manager/generated and read-only roots;
|
||||
- network/UNC home and long Windows paths;
|
||||
- repo-scoped and plugin-cache same-name skills remain untouched;
|
||||
- partial provider presence and custom provider home.
|
||||
|
||||
### Transactions and concurrency
|
||||
|
||||
- failure after stage, backup, publish, verify, ledger write, and cleanup;
|
||||
- crash-orphaned staging and rollback directories are excluded from discovery, recovered or
|
||||
swept from their markers, and never offered for adoption;
|
||||
- unmarked lookalike directories under or near the reserved transaction path are never swept;
|
||||
- `EPERM`, `EBUSY`, disk full, permission loss, and process crash;
|
||||
- two Orca windows and duplicate triggers;
|
||||
- stable/dev attempts and isolated dev home;
|
||||
- skills CLI or user mutation before stage, during stage, and immediately before publish;
|
||||
- failed destinations retry without reprocessing successful siblings.
|
||||
|
||||
### Hosts and E2E
|
||||
|
||||
- native macOS, Linux, and Windows;
|
||||
- multiple WSL distros/users, distro shutdown mid-update, and state isolation;
|
||||
- SSH Linux/macOS/Windows, disconnect/reconnect mid-update, old remote runtime, two desktop
|
||||
clients, and host/user identity changes;
|
||||
- settings update-available, adoption, managed-current, conflict, rollback, and retry states;
|
||||
- proactive adoption nudge is non-repeating, opens review, and never claims a path by itself;
|
||||
- canonical-only installs report provider visibility accurately and provider-link repair stays
|
||||
separately consented;
|
||||
- post-restart app update uses the new bundle, never the pre-restart bundle.
|
||||
|
||||
Extend `tests/e2e/settings-skill-detection.spec.ts`, but keep filesystem transaction and
|
||||
host-isolation coverage below E2E as deterministic integration tests.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Exact platform paths and schema migration policy for the host-local management ledger.
|
||||
- Whether the skills CLI can expose a supported machine-readable placement/ownership API in
|
||||
the future; until then its private lock remains read-only supporting evidence.
|
||||
- Which custom agent homes Orca can identify from the actual launch environment rather than
|
||||
ambient desktop environment variables.
|
||||
- Exact provider-link repair scope after a canonical install is managed but not visible to an
|
||||
agent; this remains separately consented from content updates.
|
||||
- Toast copy and whether settings should link to a per-skill changelog.
|
||||
|
|
@ -1,262 +0,0 @@
|
|||
# Skill Freshness: Thin Stubs + Read-Only Detection
|
||||
|
||||
Status: adopted direction 2026-07-13. Phase 1 (guide sources + binary-served CLI) is
|
||||
implemented, pending release; detection and stub migration are not implemented. This is the
|
||||
authoritative plan. It supersedes the write phases of `skill-auto-update-design.md` (Phases
|
||||
2–4: background updates, WSL, SSH) and the migration section of
|
||||
`skill-guide-indirection-design.md`. The detection and content-identity research in those
|
||||
notes still applies and is referenced below.
|
||||
|
||||
## Problem (unchanged)
|
||||
|
||||
Orca ships agent skills that teach coding agents to drive the `orca` CLI. Users install them
|
||||
with `npx skills add stablyai/orca --global`. Installed copies are frozen files; the Orca
|
||||
binary keeps moving. A stale skill tells an agent to use commands that are wrong or unsafe
|
||||
for the binary it is driving.
|
||||
|
||||
## Decision
|
||||
|
||||
Three moves, replacing the in-app write machinery entirely:
|
||||
|
||||
1. **Structural fix — content lives in the binary.** All version-sensitive skill content is
|
||||
served by the CLI (`orca skills get <topic>`), compiled at build time from authoritative
|
||||
`skill-guides/` sources. Generated `skills/<name>/SKILL.md` files are the installable
|
||||
discovery surface and become permanent thin stubs. Staleness becomes impossible for the
|
||||
content that matters, rather than mitigated.
|
||||
2. **Residual freshness — read-only detection, ecosystem-rail updates.** Orca detects
|
||||
outdated official copies (content-addressed, LF-normalized identity) and pre-fills a
|
||||
targeted `npx skills update <names...> --global` command in a terminal. The user reviews
|
||||
and runs the skills CLI's own update command; Orca never submits it automatically and
|
||||
never writes a byte into a skill directory.
|
||||
3. **No persistent ownership or update state.** No ownership ledger, no adoption consent,
|
||||
no background writer, no transactional publish/rollback stack, no settings toggle. The
|
||||
one content migration (fat skill → stub) rides the same user-invoked npx rail. A small
|
||||
dismissed-nudge set remains in app state; it grants no write authority.
|
||||
|
||||
### Why the write machinery was dropped (decision record)
|
||||
|
||||
- **Maintenance-per-use.** The transactional mutation stack (staging, rollback, crash
|
||||
recovery, orphan sweeping, ownership ledger, installer attribution) is ~3.2K production
|
||||
lines before its tests. It is the most correctness-critical code in the tree, built to
|
||||
write into user-owned directories forever. Under the stub model it would run meaningfully
|
||||
once. Every module is permanent Windows/WSL/SSH edge-case surface for a one-shot job.
|
||||
- **Trust posture.** Ecosystem discourse (2026) consistently favors pinning and reviewable,
|
||||
user-invoked updates over silent writes into `$HOME`; Orca has first-hand precedent of
|
||||
user backlash from writing into user-owned config directories. Read-only detection has no
|
||||
trust cost at all.
|
||||
- **The rail already exists.** `npx skills update <names...> --global` is the ecosystem's
|
||||
documented remedy, and once `skills/` contains stubs it delivers the migration without an
|
||||
Orca-owned writer. The rail is treated as an external dependency with a tested contract,
|
||||
not assumed trustworthy from its lock file alone.
|
||||
- The full write implementation exists, reviewed and green, on branch
|
||||
`brennanb2025/skill-auto-update-research` (PR #8496, closed as superseded). If in-app
|
||||
writes are ever genuinely needed, start from that branch, not from scratch.
|
||||
|
||||
## Design
|
||||
|
||||
### A. Binary-served guides
|
||||
|
||||
```sh
|
||||
orca skills list # one line per topic: name + when to use
|
||||
orca skills get <topic> # full version-matched guide, markdown to stdout
|
||||
orca skills get <topic> --full # include bundled reference docs, if any
|
||||
```
|
||||
|
||||
- Topics are the skill names (orca-cli, orchestration, computer-use, …).
|
||||
- Full version-sensitive content lives in `skill-guides/<topic>.md`. A generator embeds it
|
||||
in a concrete CLI module and emits the installable `skills/<name>/SKILL.md` projection.
|
||||
During the pre-stub release this projection may remain fat; after migration it is a stub.
|
||||
- The generated-output-current check uses the existing generated-artifact-gate pattern used
|
||||
by the manifest verifier. It asserts that generated guide data and installable projections
|
||||
match their sources and that every stub topic resolves against the compiled guide table.
|
||||
No network or runtime filesystem lookup is required. Unknown topic → nonzero exit + topic
|
||||
list.
|
||||
- Authoritative guides, generated projections, and embedded TypeScript are pinned to LF, and
|
||||
the generator normalizes input before embedding it. Detection still normalizes text identity
|
||||
because already-installed Windows copies may retain the historical CRLF shape.
|
||||
- Topic names are append-only and aliased forever: a stub installed in 2026 must still
|
||||
resolve in 2028. Renames add an alias, never remove one.
|
||||
- Verb is `skills get` (not `guide`) to match the convention agents are already taught by
|
||||
other tools in the wild.
|
||||
|
||||
### B. Stub format and command resolution
|
||||
|
||||
One stub per skill — frontmatter descriptions are the agent-routing layer and stay
|
||||
per-skill registry entries.
|
||||
|
||||
```markdown
|
||||
---
|
||||
name: orca-cli
|
||||
description: <unchanged per-skill trigger copy — the discovery surface>
|
||||
allowed-tools: <the supported Orca CLI command names>
|
||||
---
|
||||
|
||||
# Orca CLI
|
||||
|
||||
This file is a discovery stub, not the usage guide. The full, version-matched reference
|
||||
lives in the `orca` binary itself.
|
||||
|
||||
Before using Orca commands, resolve the Orca CLI for this session and load the guide once:
|
||||
|
||||
<resolved-orca-cli> skills get orca-cli
|
||||
|
||||
Don't guess subcommands or flags from memory or from cached copies of this skill — they
|
||||
change between Orca releases; the command above always matches the installed binary that
|
||||
will handle subsequent Orca commands.
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- The permanent body says when to engage Orca, how to resolve its CLI, and where to fetch
|
||||
the version-matched guide. It does not carry the changing command reference.
|
||||
- A stub must never blindly invoke bare `orca` outside an Orca-managed terminal on Linux;
|
||||
that name commonly resolves to the GNOME Orca screen reader. The contract must cover
|
||||
packaged `orca`, Linux/WSL `orca-ide`, SSH relay `orca`, and development `orca-dev`, and
|
||||
`allowed-tools` must cover every command the resolution contract can select.
|
||||
- **Linux command decision (2026-07-13):** do not install a uniform global bare `orca` alias;
|
||||
it would shadow or risk launching the GNOME Orca screen reader. Keep `orca-ide` outside
|
||||
managed Linux terminals, the existing managed-terminal/SSH `orca` shims, and `orca-dev` for
|
||||
development. The permanent stub therefore needs the short resolver exercised by the spike.
|
||||
- First-generation stubs are hybrid: a minimal safe bootstrap plus the guide pointer. Thin
|
||||
them further only after pointer compliance and old-binary behavior are measured. If
|
||||
`skills get` is unavailable, the hybrid must provide a bounded legacy workflow and tell the
|
||||
user that updating Orca restores the full version-matched guide; it must not dead-end or
|
||||
invite the agent to guess the missing command surface.
|
||||
|
||||
### C. Read-only detection (kept from Phase 1, slimmed)
|
||||
|
||||
Kept as-is:
|
||||
|
||||
- Bundled `skills/` packages + current manifest + released-snapshot registry + release
|
||||
mapping, with the generation script and merge-queue monotonicity gate (static data + CI,
|
||||
not runtime machinery).
|
||||
- LF-normalized text identity / exact-byte binary identity (the Windows CRLF finding
|
||||
stands: exact-byte matching would misclassify every Windows install as modified).
|
||||
- Bounded inventory work limits, topology classification (symlink dedup, external links,
|
||||
plugin caches and repo scopes excluded), and the launch / focus / post-install triggers.
|
||||
- The skills-CLI round-trip CI on macOS/Linux/Windows — extended from current-install tests
|
||||
to historical-fat-install → targeted global update → stub migration. The matrix covers
|
||||
LF/CRLF and provider aliases as a positive convergence contract, plus independent-copy
|
||||
observation that accepts only unchanged historical or exact-current bytes. Post-update
|
||||
bytes, not exit status, decide.
|
||||
|
||||
Slimmed:
|
||||
|
||||
- Statuses collapse to: `current`, `outdated` (exact match of an older released snapshot),
|
||||
`newer-known`, `unrecognized`, and `inaccessible`. Without a ledger, Orca cannot honestly
|
||||
distinguish a locally modified official copy from unrelated same-named content;
|
||||
`unrecognized` says it may be edited or from another source. All `managed-*` states, the
|
||||
ledger, adoption eligibility, and attribution are removed.
|
||||
- Status and action eligibility are separate. External links, read-only locations, plugin
|
||||
caches, repo scopes, and unsupported topologies remain informational even when their bytes
|
||||
match an official snapshot.
|
||||
- Dismissal state for the nudge is a simple local dismissed-set keyed by
|
||||
(physical identity, skill, bundled revision) in app state — not a consent ledger.
|
||||
|
||||
### D. Surfacing
|
||||
|
||||
- **Surfaces (venue decision 2026-07-14):** a lingering toast, an update modal, and the
|
||||
existing Settings setup rails for CLI, Orchestration, Computer Use, and Per-Workspace
|
||||
Environments. Their installed pills carry safe freshness status, while their existing
|
||||
Update and Re-check actions remain the per-skill path. The Skills page was
|
||||
de-linked by #4535 (2026-06-02) — its only entry, the sidebar toolbox menu, was removed —
|
||||
so it is no longer a venue; the freshness surface moved off it entirely. The behavior
|
||||
contracts below (name-scoped eligibility, no auto-run, dismissal keys, re-inventory
|
||||
triggers) are unchanged; only the venue moved.
|
||||
- **Per-placement rows** (read-only): name, status badge, one-line explanation. `newer-known`,
|
||||
`unrecognized`, `inaccessible`, and unsupported-topology rows are informational. They live in
|
||||
the modal's collapsed **Details** section (auto-expanded when a placement is blocked).
|
||||
- **Name-scoped update eligibility:** eligibility is computed across all discovered placements
|
||||
of a name, not per row. Offer a name only when at least one placement is `outdated` and every
|
||||
placement is an exact `current` or `outdated` official snapshot in a topology the validated
|
||||
rail actually converges. With skills CLI 1.5.17 that means the canonical global copy and
|
||||
provider aliases to it. Independent provider copies are informational and poison the offer:
|
||||
empirical copy-mode testing produced both stale and converged provider copies in otherwise
|
||||
equivalent 1.5.17 environments, so that topology is not deterministic enough to offer. One
|
||||
`newer-known`, unrecognized, external, read-only,
|
||||
inaccessible, repo/plugin, independent-copy, or otherwise unsupported placement poisons the
|
||||
update offer for that name entirely.
|
||||
- The action combines only eligible outdated Orca names into
|
||||
`npx skills update <names...> --global` and opens the update modal's editable terminal with
|
||||
that command pre-filled, leaving execution to the user. Never use an unscoped bulk update and
|
||||
never auto-submit the command. Re-inventory after terminal exit, modal close, or focus; only
|
||||
observed bytes, not the skills CLI exit status, determine success. When the eligible set
|
||||
empties and every placement is `current`, the modal shows an up-to-date state; if placements
|
||||
remain outdated-but-blocked or unrecognized, it says so honestly instead.
|
||||
- **One lingering, non-repeating nudge**: count only eligible outdated skill names and offer the
|
||||
same targeted action, which opens the update modal. The toast lingers (no auto-close) until the
|
||||
user opens the modal or explicitly dismisses it; ignoring it (app quit) records nothing, so a
|
||||
still-outdated skill may prompt once more next launch. A later inventory that resolves or blocks
|
||||
the offered tuple retracts the stale toast without recording a dismissal. An outdated name
|
||||
poisoned by another placement remains visible in the modal's Details but never produces an
|
||||
unsafe nudge action. Dismissal is recorded per (physical identity, name, bundled revision) only
|
||||
on explicit dismissal, so a newly outdated official placement or genuinely newer stub revision
|
||||
may prompt once more. No
|
||||
toggle — nothing automatic happens that would need one.
|
||||
|
||||
### E. Migration (fat → stub)
|
||||
|
||||
1. **Implemented, pending release:** from a fresh main-based PR, add authoritative guide
|
||||
sources, generated embedded data, `orca skills list/get`, aliases, generated-output checks,
|
||||
and local/SSH/WSL/dev tests. Keep distributed skills fat and ship this release first.
|
||||
2. From a separate PR, land slim read-only detection and Skills-page/nudge UI, including the
|
||||
name-scoped targeted update action and the real migration-rail CI. Keep distributed
|
||||
skills fat.
|
||||
3. Run the pointer-compliance spike against the released guide-serving binary, not a checkout
|
||||
artifact. The binary must be publicly released before a stub PR merges because the skills
|
||||
CLI installs from repository main, independently of Orca's desktop release train.
|
||||
4. In one PR, convert only `orca-cli` to a first-generation hybrid stub and keep any final
|
||||
thinning of that stub in the same change. This bumps its registry revision like any content
|
||||
change. Existing users see an `outdated` exact snapshot and may run the targeted global
|
||||
update; users of pre-guide binaries retain the hybrid bootstrap.
|
||||
5. Cut an RC before the stable release and use that validation window to measure compliance,
|
||||
task success, old-binary behavior, and token cost. Ship the thin form in stable only if those
|
||||
gates pass; otherwise retain the hybrid. Convert the remaining skills gradually in later PRs.
|
||||
Users who ignore the nudge keep working with their existing fat skills.
|
||||
|
||||
## Spike gate (before any stub ships)
|
||||
|
||||
Using the released guide-serving binary, install the proposed hybrid `orca-cli` stub in a
|
||||
test home and run real agents (Claude Code, Codex) on representative Orca tasks. Measure:
|
||||
|
||||
- how often the agent resolves the correct packaged/Linux/WSL/SSH/dev command and fetches the
|
||||
guide before its first Orca command;
|
||||
- task success versus the fat skill;
|
||||
- old-binary failure behavior; and
|
||||
- net token cost (stub preload + one fetch versus fat preload).
|
||||
|
||||
Then test a thinner stub against the same corpus. Nothing converts, and the hybrid stub does
|
||||
not thin, until the relevant variant passes.
|
||||
|
||||
## Risks and open questions
|
||||
|
||||
- **npx rail reliability.** The skills CLI update path has had false "up to date" results,
|
||||
global Windows failures, missing global lock tracking, lossy lock migration, and copy-mode
|
||||
topology changes. The historical-fat → targeted-global-update → stub CI is a release gate,
|
||||
not an early-warning job. Detection always re-checks bytes after the user updates, so a
|
||||
failed or no-op update re-surfaces `outdated` instead of lying. Minimum validated version is
|
||||
1.5.17: 1.5.16 failed the provider-alias convergence contract, while 1.5.17 copy convergence
|
||||
still varies by environment. CI pins 1.5.17 and probes latest; monitor and contribute upstream
|
||||
fixes before broadening eligibility.
|
||||
- **Trigger-copy iteration slows.** Improvements to stub descriptions reach existing
|
||||
installs only when users run the npx command. Acceptable at stub-change cadence; the
|
||||
compiled guides (the content that matters) are exempt by construction.
|
||||
- **Multi-file skills.** Current shipped packages are single-file. If a future skill needs
|
||||
scripts/assets, either the binary serves them (`--full` / `--script`) or that skill
|
||||
accepts the fat-file decay model. Decide when it happens.
|
||||
- **Remote hosts.** Detection ships local-host-only. Stubs make remote _content_ a non-issue:
|
||||
SSH/WSL launchers forward to the host's bundled CLI, so the guide matches the command
|
||||
surface that will handle subsequent requests. Remote stub installs can lag on trigger
|
||||
copy, which is the accepted residual. The WSL/SSH reconciler phases of the old design are
|
||||
retired, not deferred.
|
||||
- **Agent Skills spec evolution** (frontmatter fields, allowed-tools syntax) is the most
|
||||
likely future cause of a real stub update wave; the nudge path covers it.
|
||||
|
||||
## Relationship to prior notes
|
||||
|
||||
- `skill-auto-update-design.md`: Problem statement, empirical CLI-behavior findings
|
||||
(verbatim-LF mac/linux, CRLF Windows, XDG lock location, symlink topology, released-blob
|
||||
provenance) and the Phase-1 detection design remain valid inputs. Phases 2–4 (background
|
||||
writes, WSL, SSH reconcilers) are retired by this document.
|
||||
- `skill-guide-indirection-design.md`: The stub/CLI contract and prior-art survey are
|
||||
folded in here; its migration-via-in-app-updater section is superseded by §E.
|
||||
|
|
@ -1,148 +0,0 @@
|
|||
# Skill Guide Indirection (Thin Stubs + `orca skills get`)
|
||||
|
||||
Status: FOLDED INTO `skill-freshness-design.md` (2026-07-13) — read that instead. The stub/CLI
|
||||
contract and prior-art survey carried over; the migration-via-in-app-updater section here is
|
||||
superseded (migration now rides `npx skills update`, no in-app writes).
|
||||
|
||||
## Principle
|
||||
|
||||
Version-sensitive content must not live in distributed files; only discovery metadata should.
|
||||
Every hard problem in the current system — staleness, adoption consent, installer attribution,
|
||||
transactional replacement, remote-host reconciliation — descends from shipping full skill
|
||||
bodies as mutable files that must track the installed Orca binary. Move the bodies into the
|
||||
binary and the problems shrink to a residue the existing machinery already handles.
|
||||
|
||||
## Design
|
||||
|
||||
### 1. The binary serves the instructions
|
||||
|
||||
New CLI surface (topic names match skill names):
|
||||
|
||||
```sh
|
||||
orca skills list # enumerate available guides, one line each
|
||||
orca skills get <topic> # full version-matched guide for one skill, markdown to stdout
|
||||
orca skills get <topic> --full # include bundled reference docs, if any
|
||||
```
|
||||
|
||||
- Content is authored in `skill-guides/<topic>.md`. A generator embeds those authoritative
|
||||
sources in the CLI and emits `skills/<name>/SKILL.md` as an installable projection;
|
||||
`skills/` is generated output, not an authoring source.
|
||||
- Output contract: plain markdown on stdout, exit 0; unknown topic exits nonzero with the
|
||||
topic list. No network, no filesystem reads outside the binary's own resources.
|
||||
- Verb choice: `skills get` (not `guide`) to match the convention agents are already being
|
||||
taught by other tools (see Prior art).
|
||||
|
||||
### 2. Historical stub sketch (superseded; do not copy)
|
||||
|
||||
This sketch records the indirection idea only. The resolver and first-generation hybrid stub
|
||||
contract in `skill-freshness-design.md` are authoritative and must cover packaged `orca`,
|
||||
Linux/WSL `orca-ide`, SSH `orca`, and development `orca-dev` without blindly invoking bare
|
||||
`orca` on Linux.
|
||||
|
||||
```markdown
|
||||
---
|
||||
name: orca-cli
|
||||
description: <unchanged per-skill trigger copy — this is the discovery surface>
|
||||
allowed-tools: <all supported Orca CLI command names>
|
||||
---
|
||||
|
||||
# Orca CLI
|
||||
|
||||
This file is a discovery stub, not the usage guide. The full, version-matched reference
|
||||
lives in the `orca` binary itself.
|
||||
|
||||
Before using Orca commands, resolve the CLI for this session and load the guide once:
|
||||
|
||||
<resolved-orca-cli> skills get orca-cli
|
||||
|
||||
Don't guess subcommands or flags from memory or from cached copies of this skill — they
|
||||
change between Orca releases; the command above always matches the installed binary.
|
||||
```
|
||||
|
||||
Stub rules:
|
||||
- Body is deliberately version-independent: it says when to engage Orca and where to fetch
|
||||
the how — never the how itself. A stub should survive many releases unchanged.
|
||||
- `allowed-tools` must cover every executable that the authoritative resolver can select.
|
||||
- Stub must not ship before the binary that serves its topic: gate stub rollout on the
|
||||
release that includes `skills get` (a stub pointing at a command that does not exist
|
||||
is worse than a fat skill). Enforce with a build check: every stub topic must resolve
|
||||
against the compiled guide table.
|
||||
- Stub should degrade honestly when no supported Orca command is on PATH and must retain a
|
||||
bounded legacy bootstrap for binaries that predate `skills get`.
|
||||
|
||||
### 3. What this retires, what it keeps
|
||||
|
||||
Retired / collapsed:
|
||||
- The ownership ledger, adoption and installer-attribution flows, background updater,
|
||||
transactional publish/rollback/orphan sweep, and all automatic writes into user-owned
|
||||
skill directories.
|
||||
- Phases 3–4 of skill-auto-update-design.md (WSL/SSH remote file reconcilers). Wherever the
|
||||
skill is useful the `orca` binary is present, and the remote binary serves the guide
|
||||
matching its own host's version. No remote file-sync problem remains.
|
||||
|
||||
Kept (read-only):
|
||||
- Bounded discovery, LF-normalized content identities, the released-snapshot registry,
|
||||
release mapping, and CI gates. Statuses are `current`, `outdated`, `newer-known`,
|
||||
`unrecognized`, and `inaccessible`; no ledger is needed to compute them.
|
||||
- Name-scoped eligibility across every placement. One newer, unrecognized, external,
|
||||
read-only, repo-scoped, plugin, or inaccessible placement poisons the update offer for
|
||||
that skill name.
|
||||
- The skills-CLI round-trip CI, extended to prove historical fat installs migrate to stubs
|
||||
through targeted global updates across supported hosts and topologies.
|
||||
- Read-only settings rows and a dismissible nudge that pre-fill a targeted
|
||||
`npx skills update <eligible-names...> --global` command. Orca never submits it or writes
|
||||
into a skill directory.
|
||||
|
||||
## Prior art (verified live 2026-07-13)
|
||||
|
||||
- vercel-labs/agent-browser — canonical stub + `agent-browser skills get core`; docs frame
|
||||
it explicitly: "the installed SKILL.md rarely changes, while the CLI always serves content
|
||||
matching its own version." Stub self-describes as a discovery stub that "cannot change
|
||||
between releases."
|
||||
- Canner/WrenAI (skills/wren/SKILL.md) — independent (non-Vercel) adopter: "The actual
|
||||
workflow guides … live inside the `wren` CLI itself, so they always match the installed
|
||||
wrenai version (no skill cache, no version drift)." Uses `wren skills list` /
|
||||
`wren skills get <topic>` / `--full` — the verb convention to match.
|
||||
- vercel-labs/zerolang (skills/zero/SKILL.md) — "This file is only a discovery stub… ask the
|
||||
installed compiler for the skill content that matches that exact binary." Adds the nuance
|
||||
of warning agents not to replace a pinned binary.
|
||||
- vercel/next.js (skills/next-dev-loop/SKILL.md) — consumes the pattern: instructs agents to
|
||||
"run `agent-browser skills get core` once for the version-matched usage guide — don't
|
||||
guess subcommands from memory." Normalization signal.
|
||||
- Ecosystem discourse (Snyk threat model, HN, vercel-labs/skills issues #500/#542, Anthropic
|
||||
skill-trust guidance) demands pinning + reviewable updates and condemns silent pulls from
|
||||
mutable remotes. Stub indirection satisfies the audit-once trust model: the audited file
|
||||
never changes meaning; served content is exactly as trusted as the installed binary.
|
||||
|
||||
## Migration plan
|
||||
|
||||
0. Release `orca skills list/get` first from authoritative `skill-guides/` sources while
|
||||
distributed skills remain fat. No stub may reach repository main before a public binary
|
||||
can serve it.
|
||||
1. Add read-only freshness detection, name-scoped update eligibility, the targeted
|
||||
user-invoked `npx skills update <names...> --global` action, and migration-rail CI. Keep
|
||||
distributed skills fat.
|
||||
2. Spike pointer compliance against the released guide-serving binary with Claude Code and
|
||||
Codex, including Linux/WSL/SSH/dev command resolution, old-binary fallback, task success,
|
||||
and token cost.
|
||||
3. Convert only `orca-cli` to a first-generation hybrid stub. Existing exact official fat
|
||||
copies become eligible for the targeted ecosystem update rail; users who ignore the
|
||||
nudge retain their existing skills.
|
||||
4. Cut an RC, measure the gates, and thin the hybrid only if it passes. Convert remaining
|
||||
skills gradually in later PRs.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Compliance failure mode: if agents skim the stub and skip the fetch, options are stronger
|
||||
stub wording, frontmatter `description` nudging ("requires running orca skills get"),
|
||||
or hybrid stubs carrying a minimal command table plus the pointer. Spike decides.
|
||||
- Multi-file skills: current shipped packages are single-file; if a future skill needs
|
||||
scripts/assets, decide whether the binary serves them (`--script <name>` like WrenAI) or
|
||||
they stay in the package (then that skill keeps the fat-update path).
|
||||
- Topic/verb naming: `orca skills get` collides conceptually with the `skills` installer
|
||||
CLI; confirm no confusion in agent behavior during the spike.
|
||||
- Old binaries: a user can hold a stub while running an older orca without `skills get`
|
||||
(downgrade case). Stub wording should fail gracefully ("if the command is missing, update
|
||||
Orca"); acceptable residual.
|
||||
- Whether settings should surface "guide served by binary" as a distinct row state so
|
||||
support can tell stub-era installs from fat-era ones at a glance.
|
||||
Loading…
Reference in New Issue