lingyu9495-source/novel-deai-detector

AI-Writing Detection & Cleaning Workflow

Trigger Conditions

• After completing web novel text, need to self-check for AI traces before publishing (Fanqie/Qidian platform detection + reader perception)
• User requests "remove AI style" or "AI style detection"
• To confirm that any AI-generated text is "undetectable as AI" before delivery

I. Detection Tools

python ai_check.py <file_path> [deepseek|doubao]

• deepseek (default): Free and fast, detailed output (AI-score%/risk level/suspected sentences with reasons/modification suggestions) — used for initial screening + locating
• doubao: Doubao Seed 2.1 Pro, pay-per-use, stable output — used for final review
• Score: AI-like 0-100%; perplexity (whether word choice is too standard and smooth) + burstiness (whether rhythm is too flat)
• Risk thresholds: >60% = high risk (platform rejection/downranking) / 30-60% = medium risk (needs cleaning) / <30% = safe

II. ⚠️ Randomness Pitfall (Tested 2026-08-10, Most Important)

Same text, same model, continuous detection yields highly volatile results—single results are not trustworthy!

Real test case ("Golden Three Chapters" of Ask the Bones):

File	First Test	Retest	Doubao Review
Chapter 1 Draft	78%	—	After cleaning: 22%
Chapter 2 Cleaned	78%	18%	15%
Chapter 3 Cleaned	78%	25%	20%

Judgment Rules (to prevent misjudgment):

1. Single test >60% do not immediately finalize — immediately rerun the same script once
2. Final judgment based on Doubao Seed review (most stable in testing)
3. DeepSeek for initial screening + locating suspicious sentences; Doubao for final confirmation
4. Same file retest <30% and Doubao also <30% → deemed safe
5. After cleaning, always run dual-model retest; do not trust a single result

III. Cleaning Process (What to do when exceeding thresholds)

1. Simulate Human Deep Rewrite (most effective): Rewrite using a concrete character's identity ("Old Knife" style) — see the multi-layer-simulated-human-writing skill
   • Real result: Chapter 1 dropped from 78% to 22% dramatically
2. Deep rewrite principles: Short fragments, life noise (crowd scolding, phone ring, cigarette smell), delete phrases like "suddenly realized/couldn't help but/seemed", interrupt dialogues with interruptions, add "unnecessary" details, embed cliffhangers in actions rather than neat reversals
3. After cleaning, must run dual-model retest (DeepSeek initial + Doubao confirmation) — deliver only if <30%

IV. External Tool Research Conclusions (2026-08-10)

• GPTZero/ZeroGPT are accessible but have limited effectiveness on Chinese text and limited free quotas
• Optimal solution = custom model matrix (DeepSeek free main inspection + Doubao Seed review): better results, free batch processing, better fit for Chinese web novels

V. Common AI-Like Writing Features (Identified by DeepSeek Detector)

1. Typical "detail anchor" writing (stereotypical imagery like "old copper coins + green spots")
2. "Suddenly realized / suddenly glimpsed" as suspense transitions (neat sentence structure)
3. "Summary reversal" (symmetrical before-and-after, linear information release)
4. Dialogues that feel like tool characters exchanging info
5. Triplet parallelism / excessive em-dashes / closed-loop Q&A

IV-B. ⚠️ Doubao Review Timeout for Long Texts (Tested 2026-08-15)

Doubao mode detecting 20,000+ character full texts will timeout (Seed 2.1 Pro thinking mode processes word-by-word + full text too long; urllib timeout=240 directly gives TimeoutError):

• Solution: Segment detection — script splits into two halves (each ~16,000 characters), each runs with "segment 1/2" prompts, timeout ≥420, average the two AI scores
• DeepSeek main inspection has no such issue (fast); only Doubao review needs segmentation
• After segmentation, Doubao provides paragraph-level concrete evidence (quoting original text + reasons), more trustworthy than DeepSeek

IV-C. ⚠️ Doubao Over-Claims Repetition — Repetitions Must Be Script-Verified (Tested 2026-08-15)

Doubao review claimed "like a red-hot iron wire swallowed, appearing verbatim 4 times" — actually only 1 time in the full text; but "I know, days like this come fewer each day..." triplet parallel template did repeat 3 times (same structure, different words). Judgment rules:

1. Do not fully trust the specific repetition counts given by Doubao — always use script t.count(keyword) to verify before concluding; do not directly quote Doubao numbers
2. However, structural reuse accusations are usually true — "I know…even if…it is stolen" ×3 (day/second/night only changes words, not structure) is a real AI signal. When detecting, match based on de-nominalized structure (retain skeleton "I know…even if…stolen", ignore replaced concrete words); do not only do exact string matching
3. A count of 0 for exact string does not mean no repetition — the body may contain \n\n line breaks, so whole-paragraph matching can yield 0 hits. Normalize whitespace first (re.sub(r'\s+','',t)) then check
4. Lyrical parallelism template reuse = strong AI evidence: the same "I know ×3 → but I still want → even if ×3 → even if this X is stolen" structure appears 3 times in a thousand characters — a typical defect of AI long-text generation "calling verified sentence templates"

V-B. 4 Major Content-Level AI Traces (Tested in Lingqian Project 2026-08-14 · More Fatal Than Sentence Style)

When detecting, scan item by item; if any hit, mark as AI suspect:

1. Dense + Homogeneous Metaphors: consecutive similar modern metaphors within a few hundred characters ("like a timeout order / like a collection text / like a burst notification sound" in a row) = AI metaphor generator mode. Real humans use one or two occasionally, not in a row
2. Mechanical Short Sentence Rhythm: uniform template of "triplet short sentences — line break — dialogue — short sentence again" (AI loves independent short paragraph sentences; real humans break them, interjecting long sentences)
3. System Prompt Bracket Templates: square brackets + terminology (threshold/edit/enable/activate) = AI system-flow template words
4. Structured Emotion Label + Action Pairing: each emotion point paired with a signature action (Mother pushes / Ahe bites nails / Lingqian bites teeth — too neat = emotion checklist) Also: "not X but Y" pattern ("not a head, but a cloth" / "not crying, but fire") is typical AI rhetoric — 20+ occurrences densely exposed; keep only 2-3 most impactful per chapter, rewrite the rest directly

V-C. 5 Humanization Cleaning Instructions (Verified in Live Test: 92% AI Score → Safe Zone for Both Models)

After cleaning with these, real test results: DeepSeek 92% → 45% (borderline), Doubao review 10% (safe) — Dealing with dual-model disagreement: DeepSeek conservative, Doubao optimistic; take the midpoint; if borderline, run another cleaning round:

1. Break up metaphors: At most 1 modern life metaphor per 300 characters; replace the rest with sensory details (temperature/smell/light/sound/touch)
2. Disrupt short sentences: After 3 consecutive short sentences, insert one sentence ≥15 characters (psychological/action/environmental)
3. Wildize system prompts: Delete brackets and terminology; use concrete descriptions ("Something cracked inside my head, like a clay pot shattering on a well rim; from the fragments came a muffled voice: blood sacrifice, can summon one.")
4. Do not pair emotions: Leave blank / go off-topic / be ambiguous; do not precisely land every emotion point
5. Add real rough edges: Specific prices/person names/debts/verbal mistakes/environmental noise ("Ice sleeves, nine-ninety for two pairs" / "Aunt Wang owed three hundred, hanged herself") — life details AI cannot imitate are the most powerful defense against AI detection

V-D. ⚠️ Cleaning Side-Effect Pitfalls (Stepped into v3 and v5! Must Run Fixes)

Kimi style conversion / humanization cleaning can corrupt text formatting:

1. Lose Chinese curly quotes ("" → bare straight quotes "") — after cleaning, grep " and count; if straight quotes appear, they're broken
2. Introduce anachronistic words (historical text "bro" → Western Jin dynasty didn't have it!)
3. Missing punctuation at line end (tested 2026-08-14): long sentences split across lines lack punctuation at line end ("…embedded with soapberry's yellow\ncan't wash off") — add comma to narrative lines without end punctuation (skip dialogue/titles/blank lines) Fix script logic: Scan line by line, mark in_quote; alternate straight quote " to “/”; after fix, verify pairs (opens == closes) After cleaning, mandatory triple check: ① Quote fix (pair verification) ② Anachronistic term scan ③ Punctuation fix (add comma to line-end missing punctuation) ④ Mobile typesetting recheck (narrative sentences >40 chars split; dialogue long sentences keep — for breath flow when reading aloud) Line break judgment (Important): Humanization rhythm line breaks ("Can't wash off." alone = emphasis / echoing rhetoric "trembled so his teeth went sour\nsour enough to remind him of…") ≠ wrong splitting — AI reviewers (GLM/Doubao) may misjudge as "broken fragments"; retain emphasized breaks with punctuation; only fix those without

V-E. Audiobook Adaptation (User Mandated 2026-08-14: Dual-Quality for Reading + Listening)

Web novels on Fanqie/Ximalaya have many audiobook users. After AI detection, must also pass "listening feel" check:

1. Use "said" (说) not "yue" (曰) for dialogue markers — "yue" is written language; listeners' ears cannot distinguish who is speaking; "said" is recognizable by ear. If style conversion changes all dialogue to "yue" (ears can't tell) — must change back to "shuo/dao"
2. Split long narrative sentences into short ones: Narrative paragraphs >40 chars split into 2-3 short sentences (read in one breath, reciter doesn't gasp); dialogue long sentences keep (actors can pause for breath)
3. Homophonic ambiguity: In listening scenarios, similar-sounding words are easily misunderstood; avoid stacking written-style honorific modifiers (use "I / humble" etc.)
4. Verification method: After modification, listen through with TTS; places where you can't tell "who is speaking" = dialogue markers not written clearly

V-F. Platform-Level Detection: Quantify & Locate "Problem Chapters" + AI-Versus-AI Simulation (Tested 2026-09-05, Sending Soul Woman 3 Evaluations Terminated — Confirmed)

Key insight: Recommendation evaluation/root cause analysis looks at "published chapters", and online content ≠ local files (may have used an earlier draft for publishing) — diagnosis must first use CDP to crawl published full text chapter by chapter; cannot substitute with local files (Sending Soul Woman online chapters 1-55 full-text scrape script = published_links.json, open each modifychapter, save innerText — see fanqie-compliance-guard references/recommendation evaluation mechanism and termination practice). Quantified AI feature per-chapter script logic (analyze_ai_features.py, script in research/): each chapter calculate ① metaphor density = number of 像/如/仿佛/好似/如同/宛如 per 1000 chars ② CV of sentence length (split by 。！？) ③ number of em-dashes ④ chapter ending type (imagery/question/dialogue/action/other) → sort to generate problem list.

⚠️ Metaphor density must use strict-count — broad-count will double and inflate (2026-09-07 test correction, most important!): Regex 像|如|仿佛|… will mis-count characters in fixed words as metaphors: "画像" (portrait)'s "像", "如果/如何/比如/例如/假如/犹如/就算" (if/how/etc.)'s "如", etc. Sending Soul Woman chapter 40 has "画像" 16 times → broad count 34 occurrences (14.5/1000 chars) vs strict count 17 occurrences (7.2/1000 chars), a factor of two! The 2026-09-05 diagnosis of "chapter 40 at 14.5" was broad-count inflation; data for chapters 31/44 etc. equally suspicious and need review.

• Strict-count method: Before counting, first remove fixed words 画像|如果|如何|比如|例如|假如|如若|何如|就算|好像|好比|诸如|譬如 (re.sub(FIXED,'',t) before counting) — analyze_features.py outputs both counts simultaneously; for judging problem chapters and comparisons, always use strict count
• Metaphor list extraction (for rewriting use) must also first remove fixed words — this time, the list_similes script scanned by sentence for metaphor words, listing all 16 "画像" occurrences — wasting half the list Real test thresholds (Sending Soul Woman 55 chapters online, strict count; chapter 40 needs rejudgment):
• Metaphor density ≥7/1000 chars = problem chapter (human normal 3-6; strict count chapter 40 at 7.2 just over line; reduced version already at 4.7, passing); density ≤4.6 (chapter 1) deemed safe
• It's not "which volume" but "which chapters" — old chapters also have problems (31/40/44 blow up); good chapters (1/2/16/18/27 at 2-4.6) prove deep-rewrite method works, just some chapters not fully cleaned
• Sentence length CV <0.45 = rhythm too uniform (chapter 49 at 0.442); high proportion of isomorphic chapter endings = machine beat AI-Versus-AI simulation platform detection: Use Doubao seed-2-1-pro as detector; prompt requires giving AI generation probability % + original text evidence + conclusion from five dimensions: perplexity/burstiness/sentence style/metaphor density/machine beat. Real test: Chapter 1 judged 10% AI rate "high-quality human original" (high perplexity: West Canton regional + funeral industry specific details; metaphors fit character like "drumbeat on coffin board/mourning shoes like knives") — chapters that went through manual review + deep rewrite are actually clean; evaluation failure often due to individual problem chapters pulling the overall distribution up. Rewrite toolchain test (for fixing problem chapters, don't repeat mistakes):
• deepseek-v4-pro for "remove AI style/delete metaphor rewrite" success rate extremely low (7 chapters attempted, 2 succeeded, with truncation e.g., 2396→1732 characters, last sentence cut; all 5 segmented attempts returned empty)
• Kimi K3 with thinking off also returned empty for rewriting; older Doubao keys prone to timeout
• doubao-seed-2-1-pro (full quota Ark key) rewrites successfully: 187 seconds return; deletes "like" and adds folk details — main rewrite for density reduction = Doubao seed-2-1-pro Reduction rewrite prompt iron rules (full template in references/番茄AI检测量化定位与降密改写_20260905.md): Split into 3 segments each ~800 characters to avoid timeout; only delete about 1/3 of metaphors (priority: consecutive/cramped/cliché/natural to delete); strictly forbidden to change plot/dialogue/folk details/character names/paragraph order/first-person; word count control 85-100% of original; consistency gate = plot keywords (波罗堂/阿福/画像 etc.) occurrence count drop <20% to pass; output must not contain status cards/titles/explanations. Deletion examples: "voice like squeezed from throat" → "voice came out muffled"; "gaze like a brush sweeping over" → "gaze swept over"; "heartbeat like being gripped" → "heart skipped a beat". Evaluation of reduction effectiveness must use strict count — once misjudged "old method performed conservatively and ineffectively (chapter 40 only dropped from 14 to 12/1000 chars)", actually due to broad count; strict count: original 7.2 → reduced version 4.7 already within threshold; old method not useless, just only addressed the metaphor dimension. New method exploration (experimenting 2026-09-07): Beat-breaking OS rewrite — borrowing from Hahaha Great Ming's "author interjection" mechanism to reduce AI style (interjections break machine rhythm/increase perplexity), but in a safe form: first-person female protagonist inner-voice muttering/colloquial rough edges (not breaking immersion, not addressing the reader, not touching the red line of adding non-body text). Dual prompt = delete metaphors + naturally integrate 2-4 character OS pieces (like "I cursed in my heart, bad luck" / "I know the rules of this trade like the back of my hand" style). Full experiment design/scripts/comparison results in references/比喻密度口径修正与破节拍OS改写试验_20260907.md

VI. Integration with Writing Workflow

• Execute in the novel-master-workflow stage H5 / novel-writing-from-scratch stage E
• Batch process every 10 chapters → medium/high risk use simulated human deep rewrite → retest to pass before delivery

---

🙋 About the Author

Nine-Tier Brocade Carp e ｜ Packages every pitfall I've stepped into as a ready-to-run skill — no textbook writing. This skill is the version I use daily.

WeChat: ly5419495 (add with note "SkillHub", I prioritize) Official Account: First Five Agent (search on WeChat; reviews and methods are written there — no need to add WeChat to read)

Other tools I've made that run directly are on this shelf page (open in browser): https://skillpay.alipay.com/public/jiupinjinlie

If you get stuck or have other scenarios you want me to package into a skill, reach out via any of the methods above.