Doc-OCR Document Recognition + Structuring

What does Doc-OCR Document Recognition + Structuring do?

PDF / scans / images → editable text → structured data. PDFs with text layers are extracted directly; scanned documents are processed through automatic OCR (macOS Vision built-in, Chinese/English). Layout preservation (multi-column/complex layouts are reordered by reading order, not output as raw lines). After OCR, an LLM can extract invoice/contract fields or convert tables.

📁 Installation: hermes skills install jiawood2006/hermes-skills/skills/doc-ocr or follow Option 2 in the README → defaults to ~/.hermes/skills/utilities/doc-ocr/. The commands below are based on this path.

Trigger Conditions

User provides a PDF/image file and requests:

• "Extract text" / "Convert to text" / "OCR"
• Process scanned documents, contracts, invoices, book pages, screenshots
• "Extract invoice info" / "contract key points" / "convert to table" (structured)

Usage Steps

1. Single File → Text

python3 ~/.hermes/skills/utilities/doc-ocr/scripts/dococr.py contract.pdf
python3 ~/.hermes/skills/utilities/doc-ocr/scripts/dococr.py invoice.jpg

Output saved as <input_name>_ocr.txt.

2. Batch Directory (fault-tolerant per file, bad files won't interrupt)

python3 ~/.hermes/skills/utilities/doc-ocr/scripts/dococr.py ./scans/ -o all.txt
# Batch output summary: success N/M + failure list (individual bad files logged without breaking the batch)

3. Markdown Output / Force Layout Preservation for Multi-Column PDFs

python3 ~/.hermes/skills/utilities/doc-ocr/scripts/dococr.py book.pdf --md
# When text layer order is jumbled in a multi-column PDF, force Vision OCR layout preservation:
python3 ~/.hermes/skills/utilities/doc-ocr/scripts/dococr.py two-column-paper.pdf --md --force-ocr

4. Structured Extraction (Invoice/Contract/Table, requires LLM key)

# First OCR, then extract fields
python3 ~/.hermes/skills/utilities/doc-ocr/scripts/dococr.py invoice.jpg
python3 ~/.hermes/skills/utilities/doc-ocr/scripts/docstruct.py invoice_ocr.txt --type invoice

# Contract fields (Party A, Party B, Amount, Timeline, Penalty clauses)
python3 ~/.hermes/skills/utilities/doc-ocr/scripts/docstruct.py contract_ocr.txt --type contract

# Table → CSV
python3 ~/.hermes/skills/utilities/doc-ocr/scripts/docstruct.py table_ocr.txt --type table -o table.csv

• --type invoice: Invoice number / date / amount / tax / total with tax / buyer / seller / tax ID
• --type contract: Contract name / party A&B / signing date / amount / payment terms / timeline / penalty clauses
• --type table: Automatically identify row/column structure → CSV
• Output is human-friendly display + savable JSON/CSV (-o)

Dependencies (Install on first use)

pip3 install pymupdf pyobjc-framework-Vision
# Structured extraction requires an LLM key (environment variable LLM_API_KEY or ~/.deai_writer.conf)

Note: OCR depends on macOS Vision (macOS only). Linux requires installing tesseract or other engines.

Known Pitfalls

• Scan detection: PDFs with fewer than 20 characters of text layer automatically fall back to OCR; normal PDFs are extracted directly.
• Layout preservation: Vision OCR reorders text based on bounding box coordinates (top to bottom, left to right per line); two-column printed materials are processed as "left column then right column," suitable for papers/reports—fancy magazine layouts may still be jumbled—use --force-ocr + visual check.
• Handwriting: Vision works well on printed text and clear handwriting; sloppy handwriting is not guaranteed.
• Privacy selling point: Files are processed locally, not uploaded to third parties (structured extraction calls the LLM API; be cautious with sensitive documents).
• Missing fields: docstruct fills missing fields with null and does not fabricate them; poor OCR quality will result in fewer fields.
• Batch fault tolerance: Directory mode logs individual file failures in the summary without stopping the entire batch.

Quick Smoke Test

# Verify installation (works even without a document on hand)
python3 ~/.hermes/skills/utilities/doc-ocr/scripts/dococr.py --help
# Real test with any PDF/image
python3 ~/.hermes/skills/utilities/doc-ocr/scripts/dococr.py any-document.pdf --md
# Expect: output *_ocr.md with editable text