merttcetn/pev-loop

PEV Loop

Run a stateless Planner-Executor-Verifier loop around one persistent goal. Keep the primary agent as a thin controller: maintain canonical state, dispatch isolated roles, enforce limits, and report results without planning, implementing, or grading the product itself.

Non-negotiable guarantees

• Require explicit $pev invocation. Treat that invocation as authorization to create or reuse the matching native goal and to dispatch the P, E, and V subagents.
• Keep the controller out of product work. Do not inspect source bodies, implement changes, interpret raw test logs, or decide rubric satisfaction in the primary context.
• Create every P, E, and V agent with fresh context. Use fork_turns="none" or the runtime's strongest equivalent.
• Never let an agent grade its own work.
• Hide planner and executor reasoning, summaries, and expected outcomes from the verifier.
• Let only verifier evidence change rubric satisfaction.
• Store all run-state in one uniquely named OS temp directory. Never create run-state under the repository, workspace, home directory, or skill directory.
• Leave the exact temp directory in place when the run ends. Do not stage, commit, push, publish, or copy its contents into durable project files.
• Keep PEV mechanics out of the product rubric. Enforce iteration limits, model routing, role isolation, and temp-only state through the controller's mechanical audit.
• Do not delete, skip, weaken, or rewrite existing tests merely to obtain a pass. Permit legitimate test changes only when required by the user goal, then make the verifier scrutinize them.
• Never bypass the final full-rubric verification.

Parse the invocation

Accept a goal after $pev and these optional controls:

• max=N or --max N: allow 1..10; default to 6.
• using=$skill-name: load the named skill independently in every role where it is relevant. Use its requirements in the rubric, require the executor to follow it, and require the verifier to assess it independently.

If the invocation omits an objective, read the active native goal. If neither exists, ask for the objective and stop.

When native goal tools are available:

1. Read the current goal before creating one.
2. Reuse an active goal only when it matches the requested objective.
3. Create a goal when none exists. Do not set a token budget unless the user explicitly supplied one.
4. If a different unfinished goal exists, stop and ask the user to resolve it instead of replacing it.

When native goal tools are unavailable, continue as a bounded PEV run and disclose that automatic continuation is unavailable.

Bootstrap temporary state

Run:

python3 <skill-directory>/scripts/init_run.py --max-iterations <N>

Record the returned absolute run_dir in the thread and pass it explicitly to every subagent. Read references/contracts.md before the first dispatch.

Populate goal.json with the exact objective and constraints. Dispatch one fresh specification planner to create the immutable definition of done. Require every rubric item to:

• start unsatisfied;
• describe an observable outcome rather than an activity;
• name acceptable evidence;
• preserve user and repository constraints; and
• be independently verifiable.

Exclude internal PEV mechanics such as iteration count, agent selection, context isolation, temp paths, and controller bookkeeping from the product rubric. Preserve user-visible workspace invariants such as "do not commit," "do not push," or "do not modify tests" when they constrain the requested result.

Freeze rubric.json after the controller's mechanical schema gate. Store mutable satisfaction and evidence only in progress.json.

The controller owns every state file. P, E, and V agents must return compact JSON responses and must not modify the temp state.

Run the loop

For each iteration from 1 through max_iterations, execute these stages in order.

1. Dispatch a fresh Planner

Start a fresh read-only Sol planner. Give it only:

• the workspace path;
• goal.json, rubric.json, and progress.json;
• the previous verifier verdict, when one exists;
• the iteration number and remaining budget; and
• explicitly requested using= skills.

Allow the planner to inspect the product and repository instructions. Do not give it previous planner or executor reasoning.

Require one bounded task_packet matching the contract. Permit DONE_CANDIDATE only when no useful work unit remains; never treat it as proof of completion.

Mechanically reject a plan that:

• addresses no unsatisfied rubric item;
• omits scope, constraints, or verification targets;
• repeats a failed work unit without responding to verifier evidence;
• requests test weakening or an unauthorized destructive action; or
• cannot fit within one executor assignment.

For high-risk security, migration, permissions, financial, or irreversible work, insert one fresh read-only plan reviewer before execution. Give it the goal, rubric, and task packet, but no planner reasoning.

Write the accepted task packet to plan.json before dispatching the Executor. Do not defer controller state updates until after verification.

2. Dispatch a fresh Executor

Start a fresh Sol worker with ownership of only the task packet's files and responsibility. Tell it that other work may exist in the codebase and that it must preserve unrelated changes.

Give it:

• the exact task packet;
• the workspace path and applicable repository instructions;
• the original constraints needed to avoid scope drift; and
• explicitly requested using= skills.

Do not give it verifier authority. Require it to implement one meaningful work unit, run proportionate checks, and return only the executor receipt. Do not let it update the rubric or temp state.

Prevent verification and build commands from leaving incidental caches, logs, coverage output, or other non-product artifacts in the target workspace. Route disposable output to an independently created OS temp location when supported. Otherwise remove only exact disposable artifacts created by that assignment whose provenance is certain; never delete a pre-existing or ambiguous path.

Route reasoning effort as follows when the runtime supports model overrides:

• mechanical, narrowly scoped execution: gpt-5.6-sol with low;
• ordinary implementation: gpt-5.6-sol with medium;
• cross-cutting or risky implementation: gpt-5.6-sol with high.

Do not commit or push unless the goal explicitly requests those actions.

3. Dispatch a fresh blind Verifier

Start a fresh read-only Sol verifier with high reasoning; use xhigh for critical security, migration, permissions, or irreversible work.

Give it only:

• the exact original goal and frozen rubric;
• the workspace or produced artifact;
• applicable repository instructions;
• user-mandated verification commands; and
• explicitly requested using= skills.

Do not give it the controller run directory, task packet, planner output, executor receipt, implementation summary, expected result, previous verdict rationale, or P/E conversation history. Give the verifier a separate OS temp scratch directory if its tools need disposable output.

Require the verifier to inspect the current product independently, run relevant checks, examine test changes for reward hacking, and return the verdict contract. The verifier must not modify product, tests, run-state, or external systems.

4. Update canonical state

Let the controller validate the response shape, write the complete response to the iteration's verdict file, and copy verifier-backed statuses and evidence into progress.json before any next dispatch. Do not reinterpret weak evidence as a pass.

• On NEEDS_WORK, start the next iteration with a new planner.
• On PASS, dispatch one additional fresh final verifier with no previous satisfaction statuses. Require it to evaluate every rubric item from scratch.
• On final PASS, run the controller's mechanical audit of role isolation, iteration count, temp-only state, and authorization boundaries. Mark the native goal complete only when both gates pass.
• On final failure, continue if iterations remain.

Stop conditions

Stop successfully only after the fresh final verifier proves every rubric item.

Treat product verification and orchestration integrity as separate gates:

• Product gate: a fresh blind final verifier proves the frozen rubric.
• Controller gate: machine-checkable records prove the iteration limit, fresh-role dispatch, temp-only state, and absence of unauthorized commit, push, publish, deploy, or destructive action.

Do not send controller bookkeeping failures back to the Product Verifier or turn them into product rubric items. Repair safe bookkeeping immediately when possible; otherwise report the controller-gate failure without claiming completion.

At the iteration limit:

• do not claim success;
• do not mark the goal complete or blocked merely because the limit was reached;
• report satisfied items, remaining items, decisive evidence, and the exact temp path; and
• leave the goal active for an explicit continuation.

Mark a goal blocked only when the runtime's repeated-blocker threshold is genuinely satisfied. Pause for user input when authorization, a material product decision, missing credentials, or an unavailable external dependency prevents safe progress.

Recovery

If the temp directory disappears, create a new one and reconstruct the objective and frozen rubric from the active thread goal and conversation. Treat all prior satisfaction as unproven, then run a fresh full verifier before continuing.

If a subagent fails:

• retry a critical P, E, or V role once with the same isolation boundary;
• never replace a missing verifier verdict with controller judgment;
• narrow an oversized work unit through a new planner;
• preserve unrelated workspace changes; and
• report gaps honestly when the iteration limit prevents recovery.

Final quality gate

Before returning success, confirm:

• the primary agent acted only as controller;
• every role used fresh isolated context;
• no agent graded its own output;
• the verifier saw no P/E reasoning or expected outcome;
• every rubric item has current, direct evidence from the final verifier;
• existing tests were not weakened to manufacture success;
• incidental command artifacts created by the workflow do not remain in the target workspace;
• run-state exists only under the recorded OS temp directory;
• no run-state was committed or copied into the project; and
• the native goal, when available, was marked complete only after final PASS.