Add ito-inference and ito-training skills (delegate to canonical Itô backend) (#2700)
Two ECC skills chaining off an ito-compute booking, per the Full-Stack Harness Engineering Plan (2026-08-06): - ito-inference: serve a model on booked GPUs via ecc ito serve (Layer 0.2). - ito-training: run a staged, eval-gated training pipeline via ecc ito train (Layer 0.3). Both match the existing ito-compute skill: origin ECC, delegate to the canonical CLI/backend, implement no parallel serving/training stack, chain off a completed booking, and never book, reserve, or spend. They report the missing capability while the desk serve-on-booking / training-run backends are scaffolds. Co-authored-by: Affaan Mustafa <affaan@itomarkets.com>
This commit is contained in:
parent
9aac8585ab
commit
fd27a0ec9f
|
|
@ -0,0 +1,59 @@
|
|||
---
|
||||
name: ito-inference
|
||||
description: Serve a model on a completed Itô compute booking through the canonical Itô backend. Use after ito-compute has booked GPU nodes and the user wants an OpenAI-compatible endpoint on that metal. Chains off a booking record; ECC implements no serving stack of its own.
|
||||
metadata:
|
||||
origin: ECC
|
||||
---
|
||||
|
||||
# Itô Inference
|
||||
|
||||
Serve a model on rented Itô metal by delegating to the canonical Itô compute
|
||||
backend (Layer 0.2). ECC does not implement a parallel serving stack, launch
|
||||
adapter, or inference server, and does no browser automation. This skill chains
|
||||
off a **completed booking** produced by `ito-compute`; it never books, reserves,
|
||||
or spends.
|
||||
|
||||
## Prerequisite
|
||||
|
||||
A completed booking from the `ito-compute` skill: booking id, node IPs, SSH
|
||||
access, GPU SKU, node count, and fabric, already recorded in harness memory.
|
||||
Without a booking record, stop — this skill does not provision.
|
||||
|
||||
## Delegation
|
||||
|
||||
ECC calls the canonical backend through the `ecc ito` bridge; it never
|
||||
re-implements serving. Authenticate once with `ecc ito login` (device
|
||||
authorization; no key in arguments, files, logs, or chat), exactly as
|
||||
`ito-compute` documents.
|
||||
|
||||
```sh
|
||||
ecc ito serve \
|
||||
--booking <booking-id> \
|
||||
--model <hf-model-id> \
|
||||
[--quantization <scheme>] \
|
||||
[--ttft-ms <target>] [--tpot-ms <target>]
|
||||
```
|
||||
|
||||
The `--ttft-ms` / `--tpot-ms` SLO is optional; supplying it turns on
|
||||
disaggregated prefill/decode, which is off by default.
|
||||
|
||||
## What the backend does (Layer 0.2)
|
||||
|
||||
The desk backend, not ECC, runs the stages, and this skill only reports them:
|
||||
|
||||
1. Fabric gate — never launch on unverified metal. Blocks below 80% of
|
||||
fabric-expected bus bandwidth; advisory between 80% and 92%; fails loud on
|
||||
silent NCCL socket fallback.
|
||||
2. Weights download and shard to the serving layout (desk-side sharded cache
|
||||
keyed by model, quantization, TP degree).
|
||||
3. Topology plan (AIConfigurator): TP inside the NVLink domain, PP across nodes;
|
||||
engine flags emitted as a reviewable file before launch.
|
||||
4. Launch (vLLM, Dynamo when disaggregating) under systemd, warmup, SLO canary,
|
||||
and registration of the endpoint URL and config to Graphiti memory.
|
||||
|
||||
## Unavailable today
|
||||
|
||||
The serving operation is not yet wired: the canonical CLI's `inference` verb and
|
||||
the desk `serve-on-booking` backend are scaffolds. Until they land, this skill
|
||||
reports the missing capability and stops. Never substitute a local runner or a
|
||||
purchase endpoint.
|
||||
|
|
@ -0,0 +1,60 @@
|
|||
---
|
||||
name: ito-training
|
||||
description: Run an ML training job on a completed Itô compute booking through the canonical Itô backend. Use after ito-compute has booked GPU nodes and the user wants pre-training, fine-tuning, or RL on that metal. Chains off a booking record; ECC implements no training stack of its own.
|
||||
metadata:
|
||||
origin: ECC
|
||||
---
|
||||
|
||||
# Itô Training
|
||||
|
||||
Run training work on rented Itô metal by delegating to the canonical Itô compute
|
||||
backend (Layer 0.3). ECC does not implement a parallel training stack, trainer,
|
||||
or scheduler, and does no browser automation. This skill chains off a
|
||||
**completed booking** from `ito-compute`; it never books, reserves, or spends.
|
||||
|
||||
## Prerequisite
|
||||
|
||||
A completed booking from the `ito-compute` skill (booking id, node IPs, SSH,
|
||||
GPU SKU, node count, fabric) in harness memory. Without one, stop.
|
||||
|
||||
## Delegation
|
||||
|
||||
ECC calls the canonical backend through the `ecc ito` bridge; it never
|
||||
re-implements training. Authenticate once with `ecc ito login`, as
|
||||
`ito-compute` documents. Never put a key or token in arguments, files, logs, or
|
||||
chat.
|
||||
|
||||
```sh
|
||||
ecc ito train \
|
||||
--booking <booking-id> \
|
||||
--model-size <e.g. 8B> \
|
||||
--data <data-ref> \
|
||||
--target <capability> \
|
||||
--budget-usd <ceiling> \
|
||||
[--post-training sft|dpo|rlvr]
|
||||
```
|
||||
|
||||
## What the backend does (Layer 0.3)
|
||||
|
||||
The desk backend runs a staged, eval-gated pipeline; this skill reports stage
|
||||
gates and never overrides one:
|
||||
|
||||
1. Data prep — manifest, dedup, decontamination against the eval suite;
|
||||
150M-ladder decision job as the cheap pre-check for custom data.
|
||||
2. Parallelism and precision — selected from model size, node count, fabric;
|
||||
wasteful combinations refused.
|
||||
3. Checkpointing and fault tolerance — async DCP, torchft; detect < 10 min,
|
||||
resume < 15 min. Loss-spike restart is a proposed, human-gated action.
|
||||
4. Curriculum and eval gates — staged pretrain / mid-train / long-context /
|
||||
post-training, each with a fixed eval battery; a failed gate stops the run.
|
||||
5. Post-training — SFT → DPO → RLVR (GRPO with DAPO stability fixes),
|
||||
trainer/rollout separation with bounded staleness.
|
||||
|
||||
Emits desk telemetry (goodput, interruption rate, checkpoint bandwidth) so the
|
||||
desk prices training blocks honestly.
|
||||
|
||||
## Unavailable today
|
||||
|
||||
Not yet wired: the canonical CLI's `run` verb and the desk `training-run`
|
||||
backend are scaffolds. Until they land, this skill reports the missing
|
||||
capability and stops. Never substitute a local trainer or a purchase endpoint.
|
||||
Loading…
Reference in New Issue