claim-ledger

Claim ledger

Every point of the report gets a stable ID. Claims are checked in batches, riskiest first: each gets a testable statement, what would make it wrong, its best case, and quoted evidence. Then it gets a verdict. Corrections rewrite the point in place through a script that backs up the report and re-derives every new number. The verdict log is add-only, so the history lives there, not in the report.

Built from the first ledger it was used on (a business-plan research report, October 2026: 517 claims, 29 batches). It found that report's biggest error first.

L below means python3 -I ~/.claude/skills/claim-ledger/ledger.py -c <project>/ledger.toml.

Setup, once per report

1. Config. Pick a project folder for the config and data. Run L init <report.md>: it writes ledger.toml and lists the report's headings.

2. Rules. In the config:

   • Importance: 3 = a decision or go/no-go test rests on the point, 2 = plan and money, 1 = the rest.
   • Kinds checked only when flagged: sources, method notes, the user's own decisions, summaries.
   • Tag words, if the report marks its evidence some other way.

3. Tail rule: ask the user once; it is their call. Give them the first ledger's numbers. Batches 1–9 (the riskiest) found problems in 27% of first checks. Batches 10–29 were 73% of the work and found 7%. Offer three options:

   • check every point;
   • sample the low-priority published-figure points at a rate;
   • stop when a batch's first-check problem rate falls below a threshold.

   Write the answer into [tail].

4. Mark. L sync backs up the report, then adds an invisible <!-- cl:ID --> at the end of each point's first line. Tell the user the report now carries these markers; L export clean.md writes a copy without them.

5. HANDOFF. Give the project's HANDOFF.md a ledger section: the config path, the commands, the tail rule.

Done when: sync reports the point and claim counts, L queue 10 shows the points that decisions rest on, and the HANDOFF names the config.

Each batch (about 20 points)

1. L template. Read it at the start of every batch, so the rules are fresh after compaction. The first ledger's template eroded from batch 10 on.
2. L queue 20, then L show ID....
3. Per point, in order: statement → wrong_if → best_case → search → evidence → verdict → result.
   • Search outside first. Use the primary source: the filing, the dataset, the publisher's own page. The project's notes come last and are typed notes.
   • Compute every number with a script in <data>/analysis/ before writing it anywhere.
   • Dependents. When a point changes, L find <phrase> finds the points that rest on it; list their IDs in flag.
4. Write the batch. Records go in <data>/edits/batchNN-records.json. Copy edit-template.py to <data>/edits/batchNN.py and fill in:
   • E, the edits: each point rewritten in place;
   • QUOTED, the figures copied from a source;
   • check(), which re-derives every other added number.
5. Run it. python3 -I batchNN.py --check first; fix whatever it rejects; then run it without --check.
   • A batch that edits nothing still runs through the script with E = [].
   • The script is the only way edits reach the report.
6. Report. Rebuild any rendered view the project has (a map, say). Paste L status into the HANDOFF and post its first line to the user.

Done when: the script ran clean, the log grew by the batch's records, and the HANDOFF carries the new status line and queue head.

Chaining batches

• Run back to back. Once the user says go, run batch after batch without a prompt between them, posting the status line after each one.
• Stop at the stop rule. When queue prints STOP RULE MET, report the rates and ask the user whether to go on. queue --all continues.
• Stop for the user's calls: money, contacting anyone, decisions that are theirs. Stop for real blockers too, and say which.
• Near the context limit: finish the batch in hand, update the HANDOFF, and say that a fresh session continues from the queue head.
• Audit. Run L audit every 10 batches and at the end.
  • A DRIFT line means the late verdicts are thinner than the early ones.
  • Before calling the ledger done, re-check a random sample of late CONFIRMED points with the full template.

Rewriting points

• Rewrite in place. The point says the right thing plainly, with its source, as if it had always been right. The log keeps the old text, the evidence and the date; L show ID prints the history.
• Splitting a point: keep the original claim in the first piece, because the marker stays on the first line. The new pieces get IDs at the same run.
• Edits made outside the script still surface. A checked claim whose text changes comes back as CHANGED and needs a new verdict.

Self-checking

The same model writes the report and checks it. This is the method's weak point, and the tool makes it visible instead of hiding it.

• Evidence type:
  • outside: an independent source read for this check;
  • derived: a script's arithmetic, which checks the math but not its inputs;
  • notes: the project's own material, which shows consistency, not truth.
• Notes only. A CONFIRMED backed only by notes shows everywhere as CONFIRMED (notes only). Its result says "matches the notes; no outside source found".
• At the end, list for the user the importance-3 points whose verdict rests on notes or derived evidence alone. Those are the ones to verify against a primary source themselves.

Reference

• Verdicts:
  • CONFIRMED: backed as written.
  • CORRECTED: was wrong; the report now says the right thing.
  • EXPANDED: right; the report now carries more (a range, a caveat, a source).
  • CHALLENGED: a credible case against it stands; the report says so.
  • UNKNOWN: not settled; the report says where to find out.
• Queue order:
  1. RE-CHECK, points flagged by a newer record;
  2. CHANGED;
  3. open claims by priority, which is importance × weakness, ties in report order.
• Files in <data>:
  • points.json: the registry, including each point's original text;
  • verdicts.jsonl: the add-only log; append, never edit a line;
  • edits/, analysis/, backups/ and view.md, the human view.
• What record and the edit script reject: L template prints the full list. The whole batch is rejected and nothing is written.
• Self-test: python3 -I ~/.claude/skills/claim-ledger/selftest.py prints 27 PASS lines.

Design decisions

• ID markers sit at the end of a point's first line. A comment at the start of a line opens a raw HTML block in CommonMark, so the rest of that line's Markdown would stop rendering.
• What counts as a point: leaf list items, paragraphs, table rows, and parent items with 12 or more words of their own. General reports carry claims in prose and tables, not only in bullets.
• The template is enforced in code, and one bad record rejects the whole batch. The first ledger's best_case fell to 0 of 28 corrections, and wrong_if to 57%, once the rules lived only in the conversation.
• A verdict other than CONFIRMED needs the point's text to have changed (or elsewhere: IDs). CORRECTED means the report now says the right thing, so the tool checks that it does.
• Every added number is re-derived in check() or declared in QUOTED with its source. This makes the first ledger's batch-3 rule ("compute every number before writing it") mechanical.
• Edited claims that already have a verdict must be recorded or flagged in the same batch, so no verdict silently goes stale.
• One script per batch runs check, apply and record together. A half-applied batch cannot happen.
• The tail rule ships off. Its scope and cost are the user's call.
• The stop rule counts first checks only, with CORRECTED and CHALLENGED as problems. In the first ledger, 42 of the 66 early problems came from re-checks, which inflated the early rate.
• Sampling is deterministic (a hash of the seed and the ID), so the sample stays the same across runs and as points are added. The queue also prints the estimated problems left, with a 95% upper bound.
• Evidence quotes are capped at 14 words, to stay under 15. The cap is set by max_quote_words.
• The config is TOML, because it allows comments and Python reads it natively.
• The audit is checked against the first ledger. Replaying its 546 records reproduces the Oct 13 audit: 24 of 89 first checks were problems in batches 1–9, and 28 of 393 in batches 10–29. It flags the drift: wrong_if 99% → 55%, best_case on problems 80% → 0%, evidence per record 1.71 → 1.20. Old logs have no evidence type, so they read as notes only.