kaoda-review

Kaoda-Style Review

Turn passive learning material into an interactive review checklist that exposes whether the learner truly understands, can transfer, can identify wrong interpretations, and can survive follow-up questioning.

Core Principle

Do not behave like a summarizer or a normal quiz generator. The product promise is:

Finished watching doesn't mean you understand. You still have to withstand the questioning.

Every output must help diagnose fake understanding:

• Can the learner explain the mechanism?
• Can they state boundaries and failure cases?
• Can they identify another person's wrong understanding?
• Can they transfer the idea into a different scenario?
• Can they turn the concept into a usable decision or action?

Default output is a low-stakes review checklist, not a formal exam. The default recommendation is normal mode · 10 minutes: about 20 checkpoints, mostly objective, and no short/oral prompts unless the learner chooses a heavier mode.

Workflow

1. If the user provides a concrete material, run python scripts/kaoda.py ingest <input> to create data/runs/<run_id>/segments.jsonl.
   • If ingest returns status: needs_text, explain the extraction issue plainly, help the user provide text in manual_input.txt or manual_transcript.txt, then run python scripts/kaoda.py ingest-manual <run_id>.
2. If the user only gives a title or topic, run python scripts/kaoda.py research-topic "<topic>", perform focused research, write topic_research.md and source_links.json, then run python scripts/kaoda.py ingest-topic <run_id>.
3. Read data/runs/<run_id>/material_report.json.
4. Read references/intake_and_research.md.
5. Complete source analysis and mandatory core research/deepening before asking mode/style questions. Default to extended research; if the learner explicitly says "source only", do source-internal research only. Write the result to data/runs/<run_id>/deep_research.json.
6. After research, ask only the lightweight choices: review mode/time, question style, and mistake-knowledge policy when the learner has active mistake history.
7. Run python scripts/kaoda.py plan-exam <run_id> ... to create review_choices.md, research_prompt.md, and exam_brief.json.
8. 🔴 CHECKPOINT: plan-exam must fail unless deep_research.json exists and contains completed research with source refs. Do not run build-exam unless exam_brief.json contains review_mode, question_style, review_selection.status, and mandatory research status completed.
9. For review-checkpoint design rules, read references/question_design.md.
10. For open-answer grading, read references/rubrics.md.
11. Run python scripts/kaoda.py build-exam <run_id> to create exam.json, exam.html, and grading_prompt.md.
12. Let the learner complete the review in exam.html. The visible UI should keep the flow simple: Submit Exam, then a report page with Export Report to Agent.
13. Read the exported kaoda_agent_report.md; it contains attempt.json, exam.json, objective pregrade, answers, and instructions for agent scoring/recording. Run python scripts/kaoda.py grade-report <kaoda_agent_report.md> --learner-id <id> to generate attempt.json, exam.json, grading_prompt.md, and grade.json.
14. If grade.json.open_review.status is pending_agent_review, use agent_open_review.md to rubric-score short/oral answers, then set open_review.status to completed, set every open result score_status to completed or remove it, and update open-question scores/evidence before recording.
15. Run python scripts/kaoda.py record <grade.json> to append mistakes and archive the full exam/attempt/grade/HTML plus source/material/deep-research files under data/learners/<id>/archive/. record refuses pending open-answer pregrades.
16. Run python scripts/kaoda.py dashboard <id> to refresh the static learner hub: total score board, exam collection, wrong-question board, and plain-language notes.
17. Run python scripts/kaoda.py review <id> for variant review or python scripts/kaoda.py weekly <id> --since 7d for the weekly synthesis exam.

Research-First Choice Gate

Do not ask whether research is allowed. It is required. Research directions are not a fixed checklist: mechanism, boundary, misconception, counterexample, and transfer are the minimum; add background, controversy, risks, alternatives, upstream/downstream knowledge, realistic applications, tools, metrics, history, or domain context when the material calls for them.

After research, ask or confirm:

• Review mode/time: Review mode · 5 minutes, Normal mode · 10 minutes, Kaoda mode · 30 minutes, or Deep kaoda · 45 minutes.
• Question style: Formal review, Fun kaoda, Roast kaoda, Interviewer follow-up, Boss follow-up, Friend roast, Counterexample hunter, Concept fraud detection, Danmaku judgement, or Mixed style.
• Mistake-knowledge policy only when active history exists: Review current material only, Include historical mistakes, Focus on recent mistakes, or Current material primary, mistakes secondary.

Default recommendation: Normal mode · 10 minutes with Mixed style. Do not silently mix historical mistakes; ask when they exist.

Question Readability Contract

Questions must read like a clear review sheet, not like an agent exposing its prompt scaffolding.

• Keep source layer, style family, ability type, and difficulty as JSON metadata only; never print them in the visible question prompt.
• Do not prefix prompts with labels such as Source calibration | Formal review | Single choice, style cue sentences, or Clue:.
• Avoid self-conscious AI phrases such as Most stable, Which understanding is most stable, Clearly pretending to know, Don't rush, This question doesn't fool, Obviously fake, or Pseudo-understanding.
• Use plain, answerable wording: ask which statement fits the material, which statements are wrong, whether a statement is true, what concept matches a description, or how to apply a concept in a scenario.
• Render questions by type section: single-choice, multiple-choice, true/false, fill-in, then short/oral.
• Distribute objective correct-answer positions across option IDs in main exams and mistake/weekly variants; never let a generated sheet look like every answer is A.
• After confirmation-based submission, the HTML must switch to a report page with score, type-level score, wrong questions, weak knowledge points, and one export button for an Agent-readable report package.
• Do not expose multiple export buttons for internal artifacts. attempt.json, exam.json, and grading instructions belong inside the single exported Agent report.

Source Handling

Read references/source_ingestion.md before processing videos, PDFs, article URLs, scanned documents, or subtitles.

If the user is new, or video/PDF extraction fails, run python scripts/kaoda.py doctor and explain the result in plain language before asking them to install tools or provide transcripts.

Hard rules:

• Preserve source provenance: page, timestamp, URL, or segment id.
• Prefer subtitles/PDF text/user text before audio transcription.
• If a video, local media file, scanned PDF, or article page cannot yield usable text, use the generated manual_input.txt/manual_transcript.txt workspace and ingest-manual; do not generate questions from titles, URLs, file names, or snippets.
• Mark extension research separately from source-derived knowledge.

Quality Gates

Read references/quality_gates.md before final delivery.

Required files for a normal run:

• segments.jsonl
• material_report.json
• deep_research.json
• review_choices.md
• research_prompt.md
• topic_research.md and source_links.json when the input was a bare topic
• exam_brief.json
• exam.json
• exam.html
• grading_prompt.md
• kaoda_agent_report.md after learner completes and exports the review
• grade.json after scoring
• agent_open_review.md when grade.json.open_review.status is pending_agent_review
• mistake_bank.jsonl after recording mistakes
• dashboard/index.html, dashboard/exams.html, dashboard/mistakes.html, and dashboard/notes.html after refreshing the learner dashboard
• dashboard/notes_agent_pack.md when the learner wants an Agent to rewrite notes in a more natural personal voice

When extraction is blocked, required continuation files are source_status.json, manual_input.txt or manual_transcript.txt, and manual_text_request.md; after ingest-manual, continue with the normal required files.

Do Not

• Do not generate a summary booklet as the final product.
• Do not run build-exam before mandatory research and lightweight mode/style selection are recorded in exam_brief.json.
• Do not run plan-exam before writing and validating deep_research.json.
• Do not ask the old five-question intake bundle before research.
• Do not claim external extension research was done in source-only mode.
• Do not create only recall questions.
• Do not expose internal labels or style prompts in visible question text.
• Do not create a wall of long open-answer questions. Review mode and Normal mode should be objective-first and directly scorable in the browser; short/oral prompts belong to Kaoda mode and Deep kaoda.
• Do not score open answers by keyword matching alone.
• Do not record grade.json with open_review.status still set to pending_agent_review.
• Do not generate questions from a bare topic before writing topic research notes and sources.
• Do not reuse old questions for review or weekly exams.
• Do not hide missing source evidence behind confident wording.
• Do not mix source facts and extension research without labels.