Add ito-inference and ito-training skills (delegate to canonical Itô backend) (#2700)

Two ECC skills chaining off an ito-compute booking, per the Full-Stack Harness
Engineering Plan (2026-08-06):

- ito-inference: serve a model on booked GPUs via ecc ito serve (Layer 0.2).
- ito-training: run a staged, eval-gated training pipeline via ecc ito train
  (Layer 0.3).

Both match the existing ito-compute skill: origin ECC, delegate to the canonical
CLI/backend, implement no parallel serving/training stack, chain off a completed
booking, and never book, reserve, or spend. They report the missing capability
while the desk serve-on-booking / training-run backends are scaffolds.

Co-authored-by: Affaan Mustafa <affaan@itomarkets.com>
This commit is contained in:
Kierkegaarde e/con 2026-08-07 12:46:06 -04:00 committed by GitHub
parent 9aac8585ab
commit fd27a0ec9f
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 119 additions and 0 deletions

View File

@ -0,0 +1,59 @@
---
name: ito-inference
description: Serve a model on a completed Itô compute booking through the canonical Itô backend. Use after ito-compute has booked GPU nodes and the user wants an OpenAI-compatible endpoint on that metal. Chains off a booking record; ECC implements no serving stack of its own.
metadata:
origin: ECC
---
# Itô Inference
Serve a model on rented Itô metal by delegating to the canonical Itô compute
backend (Layer 0.2). ECC does not implement a parallel serving stack, launch
adapter, or inference server, and does no browser automation. This skill chains
off a **completed booking** produced by `ito-compute`; it never books, reserves,
or spends.
## Prerequisite
A completed booking from the `ito-compute` skill: booking id, node IPs, SSH
access, GPU SKU, node count, and fabric, already recorded in harness memory.
Without a booking record, stop — this skill does not provision.
## Delegation
ECC calls the canonical backend through the `ecc ito` bridge; it never
re-implements serving. Authenticate once with `ecc ito login` (device
authorization; no key in arguments, files, logs, or chat), exactly as
`ito-compute` documents.
```sh
ecc ito serve \
--booking <booking-id> \
--model <hf-model-id> \
[--quantization <scheme>] \
[--ttft-ms <target>] [--tpot-ms <target>]
```
The `--ttft-ms` / `--tpot-ms` SLO is optional; supplying it turns on
disaggregated prefill/decode, which is off by default.
## What the backend does (Layer 0.2)
The desk backend, not ECC, runs the stages, and this skill only reports them:
1. Fabric gate — never launch on unverified metal. Blocks below 80% of
fabric-expected bus bandwidth; advisory between 80% and 92%; fails loud on
silent NCCL socket fallback.
2. Weights download and shard to the serving layout (desk-side sharded cache
keyed by model, quantization, TP degree).
3. Topology plan (AIConfigurator): TP inside the NVLink domain, PP across nodes;
engine flags emitted as a reviewable file before launch.
4. Launch (vLLM, Dynamo when disaggregating) under systemd, warmup, SLO canary,
and registration of the endpoint URL and config to Graphiti memory.
## Unavailable today
The serving operation is not yet wired: the canonical CLI's `inference` verb and
the desk `serve-on-booking` backend are scaffolds. Until they land, this skill
reports the missing capability and stops. Never substitute a local runner or a
purchase endpoint.

View File

@ -0,0 +1,60 @@
---
name: ito-training
description: Run an ML training job on a completed Itô compute booking through the canonical Itô backend. Use after ito-compute has booked GPU nodes and the user wants pre-training, fine-tuning, or RL on that metal. Chains off a booking record; ECC implements no training stack of its own.
metadata:
origin: ECC
---
# Itô Training
Run training work on rented Itô metal by delegating to the canonical Itô compute
backend (Layer 0.3). ECC does not implement a parallel training stack, trainer,
or scheduler, and does no browser automation. This skill chains off a
**completed booking** from `ito-compute`; it never books, reserves, or spends.
## Prerequisite
A completed booking from the `ito-compute` skill (booking id, node IPs, SSH,
GPU SKU, node count, fabric) in harness memory. Without one, stop.
## Delegation
ECC calls the canonical backend through the `ecc ito` bridge; it never
re-implements training. Authenticate once with `ecc ito login`, as
`ito-compute` documents. Never put a key or token in arguments, files, logs, or
chat.
```sh
ecc ito train \
--booking <booking-id> \
--model-size <e.g. 8B> \
--data <data-ref> \
--target <capability> \
--budget-usd <ceiling> \
[--post-training sft|dpo|rlvr]
```
## What the backend does (Layer 0.3)
The desk backend runs a staged, eval-gated pipeline; this skill reports stage
gates and never overrides one:
1. Data prep — manifest, dedup, decontamination against the eval suite;
150M-ladder decision job as the cheap pre-check for custom data.
2. Parallelism and precision — selected from model size, node count, fabric;
wasteful combinations refused.
3. Checkpointing and fault tolerance — async DCP, torchft; detect < 10 min,
resume < 15 min. Loss-spike restart is a proposed, human-gated action.
4. Curriculum and eval gates — staged pretrain / mid-train / long-context /
post-training, each with a fixed eval battery; a failed gate stops the run.
5. Post-training — SFT → DPO → RLVR (GRPO with DAPO stability fixes),
trainer/rollout separation with bounded staleness.
Emits desk telemetry (goodput, interruption rate, checkpoint bandwidth) so the
desk prices training blocks honestly.
## Unavailable today
Not yet wired: the canonical CLI's `run` verb and the desk `training-run`
backend are scaffolds. Until they land, this skill reports the missing
capability and stops. Never substitute a local trainer or a purchase endpoint.