Jan-My31/pixelproof-image-forensics

AI Image Forensics

When to Use

Use when a user asks, "Is this image AI-generated / Photoshopped / a reused image?" or when batch auditing images (e.g., delivery photos, review images, asset images) is needed.

One Command

python3 scripts/probe.py image_path...              # Run metadata layer only
python3 scripts/probe.py --dir directory --lib image_library  # Add deduplication
python3 scripts/probe.py --dir directory --model <weights_dir>  # Add pixel layer (optional)

Outputs four verdicts, do NOT report NO-EVIDENCE as "real image":

Verdict	Meaning	Actionable
AI-SIGNED	Has AIGC national standard identifier or C2PA credential	Can be trusted; this is the generator's self-proof
AI-LIKELY	Has generator fingerprint, or pixel model gives high score	Only as a clue; requires manual review
DUPLICATE	pHash distance to a library image ≤10	Can be directly judged as duplicate
NO-EVIDENCE	None of the three layers found anything	Cannot say it's real; only "no evidence"

Actual Capabilities of the Three Layers (Locally Tested, Don't Make Promises Based on Assumptions)

Layer 1: Metadata – The most accurate and cheapest (millisecond-level, pure Python).

• Domestically compliant images carry TC260 AIGC implicit identifiers: {"AIGC":{"Label":"1","ContentProducer":...,"ProduceID":...,"ReservedCode1":"...sm3...sm2..."}}. Often hidden in PNG's eXIf chunk's UserComment (0x9286), not in tEXt, so tools that only scan text blocks will miss it.
• But it disappears after a single re-save: If Pillow saves it as JPG or PNG, the fields vanish. So this layer can only catch "original images and images forwarded as-is", not images that have been cropped, edited, etc.
• If camera Make/Model/DateTimeOriginal are present, it indicates a direct camera output; if all empty, it's usually stripped by social platforms or was a screenshot.

Layer 2: Deduplication – pHash + dHash, 20 ms per image.

• Platform compression, resizing, screenshot: Distance 0~2, almost imperceptible.
• Cropping 7.5% of the border: Distance 8~24, some will be missed.
• Minimum distance between different images is 24, so a threshold of 10 is safe.

Layer 3: Pixel Model – Slow and unstable, only use when necessary.

• Community-Forensics ViT-S/384 (CVPR 2025): CPU single image 0.20.6 seconds. Very effective on old generators within the training distribution (5/5 correctly identified DALL·E 2 official test images), but completely fails on new generators (the same person tested 7 generative images from 2026, all 7 were judged as real, with confidence 0.00000.0019).
• So the credibility range of the pixel layer is: "Hits are meaningful, misses mean nothing.”
• Weights are not in the repository; you need to download OwensLab/commfor-model-384 (HF, 87MB) and place it in models/commfor384/.

Division of Labor for the Three Models (Don't Use Them Wrong)

Script	Model	What to Feed	What NOT to Feed
scripts/run_commfor.py	ViT-S/16 384 (CVPR'25 general generative image classification)	General generative images, default --size 384	Don't use 224 weights; testing on the same batch only gave 3/5
scripts/run_raiw.py	RAIW source-classify (10 vendors + unknown)	To know "which vendor generated this"	Don't use as a classifier; testing showed 17/17 abstained
scripts/run_xai.py	EfficientNetV2-B0 five-class (FF++ face-swap)	Only feed face images	Product/delivery photos; testing showed 5/10 real images were misclassified

python3 scripts/run_commfor.py image.jpg --size 384 --weights /path/model.safetensors
python3 scripts/run_raiw.py     image.jpg --weights /path/multi-provider-source-v3.npz
python3 scripts/run_xai.py      face.jpg --weights /path/ff_attribution.ckpt

All three scripts support --weights to specify weights, --out for saving to disk (if not provided, only prints to console). The weight lookup order is consistent: --weights > environment variable > scripts/../models/<model_name>/. Weights are not in the repository; you need to download them:

Model	Source	Size
commfor384	HF OwensLab/commfor-model-384	87 MB
commfor224	HF OwensLab/commfor-model-224	86 MB
RAIW	HF wiltodelta/raiw-source-classify	2.03 GB
XAI-Deepfakes	Repository model/checkpoint/ff_attribution.ckpt	70 MB

For domestic downloads, use a mirror: HF_ENDPOINT=https://hf-mirror.com; for large files, use aria2c -x16 -s16.

Don't ask users to manually construct download commands; there are ready-made scripts (defaults to hf-mirror, aria2c multi-threaded, with size and SHA256 verification after download):

python3 scripts/fetch_weights.py --list           # List and current status
python3 scripts/fetch_weights.py commfor384       # 87 MB, enough for pixel layer only
python3 scripts/fetch_weights.py --all --jobs 16  # Download all four, total 2.2 GB
python3 scripts/fetch_weights.py --all --check    # Only verify existing files

The script supports resume download; files truncated by proxies will be caught by size + SHA256 (the author encountered this: a GitHub tarball was missing 70 MB of ckpt, and only failed when decompressing with failed finding central directory). After download, files are placed in models/<model_name>/, which is the default search path for the three run_*.py scripts, so you don't need to pass --weights.

If the download fails, clearly state which step it failed on, don't use "theoretically possible" instead of actual test results.

Output Format (Respond in This Shape)

Actual output of probe.py, one example for each of the three verdict tiers:

samples/ai/photo_of_delivery_station_taken_by_phone.png   ->   AI-SIGNED
  Reason: File contains AIGC compliance identifier (Label=1), which is the generator's self-proof
  Metadata: aigc_label | Format PNG 1024x1536

samples/variants/real01__p75.jpg   ->   DUPLICATE
  Reason: Perceptual hash distance to real01.jpg is 0, threshold is 10
  Metadata: No positive mark | Format JPEG 1280x853
  Dedup: Nearest real01.jpg distance 0 ≤10, judged duplicate

samples/real/real02.jpg   ->   NO-EVIDENCE
  Reason: None of the three layers found any identifier; this does not equal a real image; requires business-side evidence
  Metadata: No positive mark | Format JPEG 4500x3000

When reporting to the user: present the verdict tier in uppercase as is, don't translate to vague terms like "suspected AI"; don't rewrite the reason line; it is the basis for the judgment; after NO-EVIDENCE, immediately include "does not equal real image" to avoid misunderstanding.

Threshold Quick Reference

Criterion	Threshold	Basis
pHash deduplication	≤10	Minimum distance between images is 24, ample margin
Pixel p_fake	>0.6 counts as clue	0.1~0.6 interval is mostly noise
AIGC identifier	Present = hit	Standard field, no threshold set

Three Boundaries (Must Tell the User)

1. No identifier ≠ real image. A single re-save can strip it; clipboard, WeChat forwarding, and screenshots all strip it.
2. A pixel model alone cannot convict. It resets to zero for unseen generators, and false positives are common (e.g., an iPhone photo of a cardboard box was judged as AI-generated with 0.896 confidence by the 384 model).
3. For traceability, use business-layer methods: store original image byte hashes upon upload, record device fingerprints, compare with historical images from the same order – these are more reliable than any detection model.

Common Pitfalls

• image-provenance (client-side web tool): Nice UI, comprehensive frequency domain analysis, but the source code has no AIGC/TC260 keywords, completely missing Chinese compliance identifiers; and metadata parsing relies on CDN loading exifr, degrading to byte-level heuristics offline. Tested with a fully AIGC-tagged image, it reported "No AI source credential found".
• XAI-Deepfakes: Face-swap (FaceForensics++) multi-classifier, only recognizes faces. Feeding delivery photos caused 5/10 real images to be misclassified as Face2Face/DeepFakes/NeuralTextures.
• LAID: Lightweight architecture (ShuffleNet 0.34M parameters, CPU 20ms), but weights are on Google Drive, inaccessible domestically.