fangdididi/trigger-rehearsal

Trigger Rehearsal

What this is

An agent that wakes itself up on events is only as safe as its trigger rules. The agent itself can be careful and still burn a day's budget, because the trouble usually sits one level up:

• a rule reacts to an event that its own work produces, so it fires again, and again;
• two rules feed each other through a second system (the agent comments, the comment syncs, the sync fires a webhook);
• a noisy source sends ten edits in two minutes and each one starts a fresh run;
• a cron line meant as "check now and then" fires 288 times a day;
• two rules both handle the same event and do the work twice;
• a rule that refunds, deletes or messages people runs without asking anyone.

None of these show up when you test the agent on one example. They show up when real traffic arrives. Trigger Rehearsal makes real traffic arrive early, on paper: you write the rules down in one neutral format, replay a day or a week of events through them, and read the timeline before anything is armed.

When to use it

• Before turning on any rule that lets an agent start work without a person typing a prompt.
• When adding a rule to an existing set (a new rule can close a loop that was open before).
• When a background agent ran far more often, or far less often, than expected.
• When choosing debounce windows, parallel limits and daily caps, and you want numbers instead of guesses.
• When a reviewer asks "what happens if this repo gets 50 issues in an hour?"

Not for: checking what the agent does inside a run (that is a prompt or skill question), or load-testing the event source itself.

The rule sheet

Everything is translated into rules.json. Keep the translation honest: one sheet rule per real rule, and write down the guards that really exist, not the ones you intend to add.

{
  "tz": "Asia/Shanghai",
  "run_s": 180,
  "external_sources": ["github"],
  "budget": {"runs_per_day": 40, "cost_per_run": 0.15, "cost_per_day": 6},
  "rules": [
    {
      "name": "triage-new-issue",
      "on": {"source": "github", "types": ["issues.opened", "issues.edited"]},
      "where": {"repo": "me/app"},
      "key": "subject",
      "debounce_s": 300,
      "parallel": 1,
      "max_per_day": 20,
      "unless_actor": ["my-bot"],
      "unless_label": "triaged",
      "emits": [{"source": "github", "types": ["issues.labeled"], "actor": "my-bot", "labels": ["triaged"]}],
      "sensitive": [],
      "asks": false
    },
    {"name": "morning-digest", "on": {"cron": "0 9 * * 1-5"}, "max_per_day": 1}
  ]
}

Field	Meaning
on.source + on.types	Which events start the rule. Leave types out to match every type from that source.
on.cron	A five-field cron line in the sheet's tz. A rule has either source or cron.
where	Event fields that must equal a value (or be one of a list).
key	Event field naming the subject (issue number, ticket id, URL). Used to merge repeats.
debounce_s	Wait this long after the last event for one subject, then run once.
parallel	How many runs of this rule may be active at once. Extra runs wait.
max_per_day	Hard ceiling per local day. Extra runs are dropped.
run_s	Typical run length, used for queueing. Falls back to the sheet value, then 120.
unless_actor, unless_label	Guards: skip events made by these actors, or carrying this label.
emits	Events the run itself produces (comments, labels, status changes, follow-up events). Include the actor and labels they will carry.
sensitive + asks	Actions that need a person (delete, spend, restart, send). asks: true means the run stops and asks first.
external_sources	Sources fed by the outside world. Anything else must be emitted by a rule.
budget	Optional runs per day, cost per run and cost per day used for the verdict.

Step by step

1. Collect the real rules. Read the actual config, not the description of it. Agent hub rules files, saved routines, CI workflow on: blocks, crontab lines, no-code automation screens. Note anything that writes back to a source another rule watches.
2. Translate to rules.json. Start from python3 scripts/rehearse.py new rehearsal/. For every rule, ask three questions and fill the fields: what wakes it, what does one run write back, and what stops it from waking on its own writes. If the answer to the third is "nothing", leave the guard out. The rehearsal is meant to catch that.
3. Lint. python3 scripts/rehearse.py lint rehearsal/rules.json. Fix every HOLD before replaying; REVIEW items can wait for the numbers.
4. Gather events. Best: export a real busy day from the source (webhook delivery log, issue timeline, helpdesk audit log) into events.jsonl, one JSON object per line with at least t, source, type, plus the fields your key and where use. If no history exists, write a short synthetic burst on purpose: ten edits in three minutes, two subjects at once, an event made by your own bot.
5. Replay over whole days. python3 scripts/rehearse.py replay rehearsal/rules.json rehearsal/events.jsonl --from 2026-11-02T00:00:00Z --to 2026-11-03T00:00:00Z --md rehearsal/report.md. Pass --from/--to so the per-day numbers are not inflated by a short window.
6. Read the timeline, not just the verdict. Look for runs on the same subject minutes apart (missing or short debounce), chains with growing depth (loops), long waits (parallel too low), drops (cap too low for real work, or a sign the rule fires on noise).
7. Adjust and repeat. Change one knob at a time and replay the same events, so the difference is clearly from that change.
8. Decide. ARM only when the last replay is ARM on a realistic day. Keep the final rules.json, the events file and report.md together as the record of why the rules were switched on.

Verdicts

• ARM: no HOLD or REVIEW findings. Safe to switch on as written.
• TRIM: something wastes runs or can skip work (no key, no cap, overlap, cap dropped real events, near budget). Switch on only after trimming, or with a written reason.
• HOLD: do not switch on. Self-loop, unguarded cycle, a chain that ran past depth 5, a sensitive action without asking, over budget, or a broken cron line.

The exit code follows the verdict (0, 1, 2; 3 means bad input), so a rehearsal can gate a config change in CI.

Example

A support team wants an agent to answer helpdesk tickets. Their sheet (in examples/loopy/) has an answer rule that posts a reply, which updates the ticket, which matches the same rule. A replay of one ordinary day gives:

HOLD   self-loop            answer-ticket: rule's own output matches its own trigger with no unless_actor/unless_label guard
HOLD   runaway              answer-ticket: chain kept re-firing past depth 5 (3 cuts)
HOLD   sensitive-unasked    refund-check: declares sensitive actions ['refund'] but 'asks' is not true
HOLD   over-runs-budget     -: 100.0/day against a budget of 30/day
REVIEW cron-tight           inbox-poll: up to 720 fires/day
VERDICT HOLD

Three ticket updates turned into 18 answer runs. The fix is small: add "unless_actor": ["support-bot"] to the answer rule, set "asks": true on the refund rule, and replace the two-minute poll with a script that emits inbox.changed only when the inbox really changed (see examples/loopy/rules.fixed.json). The next replay of the same day drops from 100 runs to 4 and the verdict moves to TRIM. The one remaining finding is that answering and refund checking both see a refund ticket, which is intended here, so it is accepted with a one-line written reason next to the report.

Tips

• The single best loop guard is a condition that becomes false after the run: a label the run adds and the rule excludes, or a status the rule no longer matches.
• Cross-system loops are the easy ones to miss. If the agent writes to system A and system A syncs to B, list the synced event under emits with the actor it will show.
• Debounce adds delay by design. Pick the window from how long a person takes to finish a burst of edits, and keep it short where someone waits for an answer.
• Debounce is not deduplication of actions. For anything irreversible, the run itself should still check "already done?" before acting.
• A daily cap is a fuse, not a plan. If the replay of a normal day hits it, the rule is too broad.
• Polling every few minutes is usually cheaper as a tiny script that compares state and emits one event on change; the agent only wakes when there is something new.
• Re-run the rehearsal whenever a rule is added. Most loops are created by the second rule, not the first.