dsh-vision-toolkit 是做什么的？
A DeepSeek Harness plugin that helps text-only models perform better on visual tasks: intent-aware image Q&A, long-screenshot OCR, UI restoration, grounding, pixel diff, Artifacts, and Web UI.
Features
- Intent-aware image Q&A: Ask questions about images; the model detects intent and provides relevant answers.
- Long-screenshot OCR: Extract text from long screenshots (e.g., chat history, web pages).
- UI restoration: Convert UI screenshots into code or markup (e.g., HTML, React components).
- Grounding: Map textual references to specific regions within an image.
- Pixel diff: Compare two images and highlight differences.
- Artifacts: Generate and manage visual artifacts.
- Web UI: Interactive interface for testing and debugging.
Usage
To use this toolkit with DeepSeek Harness, follow these steps:
- Install the plugin in your DeepSeek Harness environment:
pip install dsh-vision-toolkit
- Configure the tools in your agent's tool list:
tools:
- name: image_qa
description: "Answer questions about an image"
input_schema:
type: object
properties:
image_url:
type: string
description: "URL or base64 encoded image"
question:
type: string
description: "The question about the image"
- name: long_screenshot_ocr
description: "Extract text from a long screenshot"
input_schema:
type: object
properties:
image_url:
type: string
description: "URL or base64 encoded screenshot"
- Invoke the tools from your agent logic:
# Example: Image QA
response = agent.call_tool("image_qa", {
"image_url": "https://example.com/screenshot.png",
"question": "What is the title of this page?"
})
print(response)
Supported Platforms
- DeepSeek Harness (native)
- Compatible with agents running on Claude, Cursor, Codex, and any other platform that supports Harness.
License
MIT License - See LICENSE for details.
Contributing
Contributions are welcome! Please open an issue or submit a pull request on GitHub.