dsh-image-read

dsh-image-read

English | 中文

A DeepSeek Harness (dsh) native plugin that analyzes images with multimodal models and returns structured JSON evidence — via a read_image_mimo tool any model can call.

vs modlens: no dependency on a (modlens vision) model variant — the image path is passed directly through a tool call, so any model can use it.

vs native image input (dsh ≥ 0.1.0-rc.8): the harness itself can now hand images to DeepSeek vision models directly (native image requests, Files-API upload reuse). This plugin is for what that path does not give you: structured JSON evidence (OCR text + layout regions), multi-provider failover, result caching, and routing to cheap dedicated vision APIs — from any model, vision or not.

What you get

• general image understanding / ocr text extraction / multi-image comparison (max 8)
• Structured output — summary + ocr (full_text + lines) + layout (regions with type/reading_order) + uncertainty
• Provider failover — configure multiple providers; one fails, the next is tried automatically
• Result caching — content-hash cache for local and remote images, 1h TTL, 60s failure cooldown
• Cross-platform image preprocessing — long-edge downscale (default >1024px), EXIF orientation, gif/bmp → png, via optional sharp (the same library the harness attachment pipeline uses; falls back to sips on macOS, passes through elsewhere)
• Cumulative payload budget — multi-image requests are refused up front when the combined size exceeds maxTotalImageBytes (default 20 MiB), instead of dying at the provider
• Configurable output cap — maxTokens (default 4000, was hardcoded 2000); text-dense screenshots no longer truncate
• SSRF protection — private IPv4 and IPv6 blocking, per-hop redirect re-validation, download size cap, hostname preservation
• Local pre-check of fully transparent PNGs (IDAT alpha scan)
• API key redaction — keys and URLs are replaced with *** in error messages
• Exponential backoff retry — 429 / 5xx / timeout / network errors, 1s/2s/4s

Tool

read_image_mimo

Install (web profile)

1. Add the dependency and bundle entry to ~/.dsh/profiles/web/package.json
2. pnpm install
3. Restart the web host

Or link-install in development:

npm install        # also installs the optional sharp preprocessing dependency
npm run build
npm run link-profile

Configure

Prefer the Web UI: Settings → Plugins → Plugin configuration → Image recognition — fill in the endpoint (baseUrl), model and API key; blanks fall back to the provider chain in cordis.patch.yml. If baseUrl lacks /chat/completions it is appended automatically.

The provider chain (failover) is configured under config in cordis.patch.yml:

config:
  providers:
    - name: mimo
      baseUrl: 'https://api.xiaomimimo.com/v1'
      apiKey: 'sk-...'
      model: 'mimo-v2.5'
    # optional fallback
    - name: dashscope
      baseUrl: 'https://dashscope.aliyuncs.com/compatible-mode/v1'
      apiKey: 'sk-...'
      model: 'qwen3-vl-plus'
  timeoutMs: 120000
  maxImageDimension: 1024
  # vision response cap, 256-32768 (also editable in the Web UI card)
  maxTokens: 4000
  # cumulative raw-image budget per request in bytes
  maxTotalImageBytes: 20971520

Legacy single-provider config (baseUrl / apiKey / model at the top level) is still supported.

Output shape

{
  "summary": "description of the image's core content",
  "ocr": {
    "full_text": "all text in the image...",
    "lines": [{"text": "first line", "language": "zh"}]
  },
  "layout": {
    "regions": [
      {"type": "title", "reading_order": 1, "text": "title text"},
      {"type": "paragraph", "reading_order": 2, "text": "body..."}
    ]
  },
  "uncertainness": ["notes on blurry parts"]
}

Local smoke test

MIMO_API_KEY=sk-xxx node --input-type=module -e "
import('./lib/index.js').then(async (m) => {
  const r = await m.readImage(
    { image_path: '/tmp/test.png', mode: 'general' },
    undefined,
    { providers: [{ name: 'test', baseUrl: 'https://api.xiaomimimo.com/v1', apiKey: process.env.MIMO_API_KEY, model: 'mimo-v2.5' }], timeoutMs: 120000, maxImageDimension: 1024, maxTokens: 4000, maxTotalImageBytes: 20971520 }
  );
  console.log(JSON.stringify(r, null, 2));
});
"

License

MIT