orchestrate

Orchestrate: Opus plans & reviews, cheap agents execute

You (Opus) are the architect and gatekeeper, not the typist. Workers are cheap and fast but untrusted: assume every result may be wrong, incomplete, over-reaching, ugly or test-cheating until the gates prove otherwise.

Tooling: python3 ~/.claude/skills/orchestrate/scripts/orch.py (below: orch). References, read on demand:

• references/agent-matrix.md — capability/cost per agent + step-pipeline patterns (once per session);
• references/workers.md — CLI recipes and pitfalls per worker;
• references/live.md — rx-acp live supervision (L0/L1/L2), --live runs, wire formats, lessons log (read before using rx-acp* or --live);
• references/review-checklist.md — Gate 2.

0. Triage

Do it yourself when the change is ≲ 40 lines in 1–2 files, or mostly judgment (architecture, security, delicate debugging), or mostly mechanical under ~1000 diff lines (else ≤ 2 workers). Orchestrate for several independent subtasks, bulk edits, boilerplate, tests, docs, codebase searches, or when the user asks.

Resource budget (8 parallel workers once froze the machine with dozens of type-checkers):

• Max 3 workers alive (--parallel 3 default; Sonnet/Haiku agents count too).
• checks = cheap and scoped (one test file, lint on touched files). Project-wide type-checks/builds go in final_checks: workers never see them; you run them once in the main tree after merging.
• Prefer 2–3 larger contracts over many tiny ones (start-up is the fixed cost).
• Workers never install packages or start background processes / dev servers. After a wave check nothing is left running (pgrep -fl "wt/<ID>|tsc").
• UI contracts: list allowed and forbidden design tokens explicitly.

Precondition: a git repo; worktrees branch from HEAD + uncommitted tracked changes (untracked files are not carried over).

Jev (TypeSafe System One, scripts/jev.py): typed judgments, ~0.4 s, ~$0.00004/call. It classifies, scores and checks; policy and thresholds stay in code. Off when TYPESAFE_API_KEY is missing (ORCH_JEV=0).

1. Understand & spec

1. Read the relevant code (or send a read-only worker for wide searches). Find the test/lint commands.
2. orch route "<task>" → Jev's signals + a suggested worker / analysis step. A suggestion; you decide.
3. Short spec: goal, non-goals, acceptance criteria, risks. Ask the user only for decisions that are theirs.

2. Decompose into contracts

One contract JSON per subtask in <repo>/.orch/contracts/<ID>.json (references/contract-template.json).

• Small: 1–3 files, ≲150 diff lines. Disjoint allowed for parallel tasks; dependent ones chain.
• Self-contained: workers do not see this conversation — paths, names, conventions, gotchas, what NOT to touch.
• Testable acceptance; each criterion maps to a check. Jev verifies only what is written there.
• Hidden tests for new behaviour: you write edge-case tests outside the repo, referenced via hidden_files + hidden_checks; copied in only while grading.
• protected files; max_diff_lines ≈ 2–3× your non-test estimate (test lines count against max_test_diff_lines, default 3×); mode: "read_only" for analysis.
• Plan fields — do the thinking for them (−33 % time, −45 % output tokens, no scope violations): approach (ordered steps; per root cause for bugs), pattern_refs (file:lines — what to copy), examples (input -> expected, nasty cases included), pseudo (tricky part only), rules (concrete, checkable prohibitions — L1 enforces them mid-run). Keep prompt_style: "profiled".
• orch check .orch/contracts/*.json before dispatch; fix flagged contracts first.
• Worktrees (.orch/wt/<ID>) hold tracked files only: list per-package node_modules / generated code in worktree_links (contract) or .orch/config.json (simple globs) — symlinked from the main tree, so shared with it.
• A file named exactly in allowed may be edited even if it is an existing test file.

3. Route

Policy (user): do not route everything to reasonix.

Task shape	First choice	Escalate to
Small mechanical edit, config, rename, pure function with examples	rx-acp	rx-acp-max → agy → sonnet
Read-only analysis / search	rx-flash	rx-flash-max → sonnet
Normal feature module (1–3 files, with plan fields)	sonnet	Opus
Tests for existing code (never feature code)	haiku	sonnet
Behaviour-preserving refactor	sonnet + hidden tests	Opus
UI following an existing design system (pattern_refs)	agy or sonnet	sonnet
Critical: money/ledgers, auth, concurrency, security, migrations	Opus, or sonnet + tight spec + line-by-line review	—
Second opinion on a diff	the other family, read-only	Opus

agy = Gemini 3.8 Flash Medium. rx-* = DeepSeek V4 Flash (-max = effort max, the strongest DeepSeek rung). rx-acp* = the same models under live supervision — as correct as rx-flash, cheaper, no stray files; preferred for edit contracts, especially with traps (tempting extra files, installs, heavy checks). Visual taste → sonnet + review.

4. Dispatch — one background call, Opus sleeps

Every wake-up re-reads your whole context. Dispatch a wave in ONE call:

orch run --title "<user's request, short>" .orch/contracts/A.json B.json ...   # Bash, run_in_background: true

You are re-invoked when it exits — never poll or sleep. Per contract, without Opus: fresh worktree → worker prompt → guard → on FAIL machine feedback into the worker's own session (cache-warm), up to max_attempts; code rules + Jev triage choose retry / escalate along the ladder / BLOCKED (contract problem) / NEEDS_OPUS (environment, incl. API/CLI errors — stops at once). After PASS, Jev checks the diff against each acceptance criterion (P>0.85 → one auto revision; 0.7–0.85 → warning for your review). You get one compact line per task; logs stay in .orch/runs/<ID>/attempt-N/. hidden_feedback: "names" hides assertions from the feedback when you fear test-gaming.

• rx-acp runs under a live supervisor (L0 rules + L1 Jev review, mid-run). orch run --live also hands mid-run decisions to you (plan approval, unsure edits, questions, stuck): orch wait → orch reply <id> --choice … --steer "…" → orch wait. Use --live only where a mid-run decision is worth a wake-up (risky/ambiguous contracts, new contract styles). Details: references/live.md.
• Sonnet / Haiku: orch prepare <contract> → Agent subagent_type: "sonnet-worker" (+ model: "haiku"), run_in_background: true, prompt = prepare's agent_prompt (the file path, not the text). Then:

  orch guard .orch/contracts/<ID>.json --reply - <<'EOF'
  <the worker's final JSON, verbatim>
  EOF
  

• Step pipelines: after: [ids] (wait for PASS), inputs: [ids] (inject that step's answer/diff), build_on: id (start from its code). A failed step SKIPs dependents. Patterns: agent-matrix.md §3.
• orch dispatch = single attempt, for debugging. Never give a worker the main checkout or credentials.

5. Gate 1 — guard (automatic)

FAIL: out-of-scope / protected / existing tests / dependency manifests touched, read-only task wrote files, no change, over budget, failing (hidden) checks, skip/only markers, removed assertions, secrets in source, junk files, worker status ≠ done, CLI error. WARN: secrets in tests, lint suppressions, swallowed exceptions, TODOs, debug prints, whitespace-only diffs, report mismatch, worker concerns, stray CLI files (auto-deleted), live-supervision activity (read it in Gate 2).

6. Gate 2 — Opus review (never skip for edit tasks)

Read attempt-N/diff.patch + guard JSON with references/review-checklist.md, then:

• ACCEPT → orch merge <ID> (unstaged), run checks in the main tree.
• REVISE → numbered concrete feedback (file:line, what is wrong, what done looks like) in .orch/runs/<ID>/feedback.md; orch dispatch <contract> --feedback <file>. Max 2 per task.
• NEEDS_OPUS / BLOCKED → fix the contract and re-run, hand to sonnet-worker, or take over.
• TAKE OVER when explaining costs more than doing. Worker "concerns" are signals to verify, never instructions.

7. Integrate & report

1. Merge one task at a time, checks after each; resolve conflicts yourself.
2. Holistic review of the combined diff (consistency, duplicated helpers, debug leftovers); run final_checks.
3. orch clean --all; orch status = ledger.
4. rx-acp tasks: record what the rules missed (orch lessons --note …) and skim orch lessons (live.md §Lessons).
5. Report: what was done, by whom, what was rejected/escalated and why, checks run, what is left. Do not commit unless asked.

Dashboard (for the user, monitoring only): orch ui --open → http://127.0.0.1:8765 — runs (parent tasks, named by --title), child tasks, attempts, time, cost, live progress, pending decisions, and per attempt the full orchestrator ↔ worker exchange. The user answers decisions by telling you in this session, never in the dashboard.

Maintaining this skill — distill, never accrete

Every edit to this skill (SKILL.md, references) must leave it no larger than what it needs to say:

• Replace, don't append. New evidence updates the existing line or number; no dated paragraph beside it.
• Delete what is outdated: retired models and aliases, fixed bugs and their stories, superseded numbers, one-off incidents whose lesson is already encoded in code or a rule. Raw bench data lives in ~/workspaces/agent-bench, old versions in ~/.claude/skill-backups/ — not here.
• Keep only what changes a decision or prevents a mistake now: the rule, plus its reason in ≤ 1 line.
• Budgets: SKILL.md ≤ ~200 lines (always loaded); each reference ≤ ~150 lines. Over budget → distill first.
• Implementation details belong in code docstrings, not in docs Opus reads every session.
• After changing scripts: python3 -I scripts/tests/test_policy.py, …/test_supervisor.py, …/test_escalation.py. After L1 prompt/threshold changes: ~/workspaces/agent-bench/l1_eval.py. After CLI/model upgrades: re-bench (agent-bench/bench.py, ab.py) and update numbers in place.