dsh-free-web-search

dsh-web-search-free

English | 中文

Keyless web search and fetch for DeepSeek Harness (dsh): two providers registered with ctx.web that need no API key.

• free (search): tries several public search endpoints in order and returns the first non-empty result set.
• free-http (fetch): ordinary HTTP retrieval with a browser User-Agent, following cross-origin redirects, blocking private hosts by default.

All three official search providers (deepseek-official, exa, perplexity) require a key, and web_search fails outright without one. This package replaces that route. Fetching was already keyless through the official dsh-web-fetch-http; free-http only adds the browser User-Agent and cross-origin redirects that search results need.

Install

Prerequisite: pnpm

dsh plugin always shells out to pnpm (through the .cmd shim on Windows). Without it you get pnpm not found on PATH. npm is not a substitute:

npm i -g pnpm     # or: corepack enable pnpm

Three ways to install

# 1. From npm (once published)
dsh plugin --profile web add dsh-web-search-free

# 2. From a local tgz (use this to move it between machines; npm pack includes the built lib/)
dsh plugin --profile web add ~/Downloads/dsh-web-search-free-0.1.0.tgz
dsh plugin --profile web add ./dsh-web-search-free-0.1.0.tgz     # relative specs anchor to the current directory

# 3. From a source directory (for iterating; run npm install && npm run build there first)
dsh plugin --profile web add /path/to/dsh_free_web_search

Profiles do not share plugins, so install into each one you use: dsh plugin --profile headless add ….

A tgz install is a snapshot: after changing the code, pack and add it again.

npm run build && npm pack     # produces dsh-web-search-free-<version>.tgz

The package declares dsh.bundle.patch, so installing it mounts one patch layer that points the web row's searchProvider at free and its fetchProvider at free-http. Restart dsh to pick it up.

Installing without pnpm

dsh plugin does three things: make sure the profile directory has a package.json, run the package manager there, and append any dsh.bundle.patch-declaring package to dsh.profile.bundles. The npm equivalent:

cd $DSH_HOME/profiles/web          # Windows: %USERPROFILE%\.dsh\profiles\web
npm i /path/to/dsh-web-search-free-0.1.0.tgz

Then add the package name to the end of dsh.profile.bundles in that same package.json:

{
  "dsh": {
    "profile": {
      "bundles": [
        "@deepseek-ai/dsh-base",
        "@deepseek-ai/dsh-web-app",
        "dsh-web-search-free"
      ]
    }
  }
}

The list is the patch-layer order and the last layer wins, so this entry must come after @deepseek-ai/dsh-web-app — otherwise that bundle sets the web row back to the official provider.

Confirming it took effect

The settings page does not show searchProvider: editable plugin cards cover only a few official plugins, and provider selection for dsh-web is composition config. Seeing web-search-free in the (read-only) plugin inventory only proves the plugin loaded. To check the composed value:

dsh --profile web --dump-config

That flag boots nothing and prints the composed plugin tree. Find - id: web; its config should read searchProvider: free and fetchProvider: free-http. If it still says deepseek-official, the bundle order above is wrong.

Then ask something that requires current information and inspect SUBTOOL: web_search in the Trajectory tab:

• Results with real source URLs — free served the search.
• WEB_PROVIDER_AMBIGUOUS — the pin did not apply, so two providers are usable at once.
• WEB_PROVIDER_ERROR listing bing: …; so360: … — this plugin ran, but every backend failed at that moment (rate limiting or network). Retry.

web_fetch on the Web surface

web_search works as soon as the plugin is installed. web_fetch needs one more step in the browser UI: the Web surface disables the host-plane tool-web row and mounts the tool inside a per-session agent preset instead, and a profile patch cannot reach a preset's rows.

1. Copy config/agent-presets/standard/agent.cordis.yml from the installation into $DSH_HOME/.agent-presets/standard-free-web/agent.cordis.yml.
2. Change fetch: false to fetch: true on its - id: tool-web row.
3. Select standard-free-web as the session (or default) preset in settings.

The new name is required: a user preset does not override a shipped one of the same name. TUI and headless sessions use the host-plane tool-web row, which this package's patch already sets to fetch: true.

Backends

id	Endpoint	Default	Notes
bing	www.bing.com/search?format=rss	yes	RSS is a published format, so it survives page redesigns; first choice
so360	www.so.com/s	yes	Destination comes from a[data-mdurl], so no per-result redirect unwrapping
ddg	html.duckduckgo.com/html/	no	Needs reachable DuckDuckGo; TLS failed on the development network, so it is unverified against the live endpoint
searxng	<searxngBaseURL>/search?format=json	no	Needs a self-hosted instance with the json format enabled

The order is the degradation order: a backend that throws, times out, or returns nothing hands off to the next. Only when all of them fail does the search throw WEB_PROVIDER_ERROR, whose message aggregates each backend's reason. Caller cancellation never degrades — it throws WEB_ABORTED immediately.

Configuration

Set under the config of the web-search-free row:

Field	Default	Meaning
backends	['bing','so360']	Backend order; an empty array means the default
region	wt-wt	DuckDuckGo-style region code, mapped to Bing's mkt/setLang (cn-zh → zh-CN)
timeoutMs	8000	Per-backend timeout
resultsPerBackend	10	Result count requested when a search carries no maxResults
searxngBaseURL	none	Required to enable searxng; an invalid value fails at load
userAgent	desktop Chrome UA	Shared by both providers
fetchProvider	true	Register free-http
fetchTimeoutMs	30000	Whole-fetch deadline
maxResponseBytes	5242880	Retained response bytes; the rest is dropped and truncated is set
maxRedirects	5	Redirect hops followed
maxUrlLength	2048	Accepted URL length
allowPrivateHosts	false	Allow fetching loopback, private-range and link-local hosts

Example — prefer Chinese results and add a self-hosted SearXNG as the last resort:

- id: web-search-free
  config:
    backends: [bing, so360, searxng]
    region: cn-zh
    searxngBaseURL: https://searx.example.internal

Error codes

The seam's vocabulary, surfaced in tool-result metadata:

• Search: WEB_PROVIDER_ERROR (every backend failed), WEB_ABORTED (caller cancelled).
• Fetch: WEB_INVALID_URL, WEB_BLOCKED_URL (embedded credentials or a private host), WEB_REDIRECT_BLOCKED, WEB_FETCH_TOO_LARGE, WEB_UNSUPPORTED_CONTENT_TYPE (binary content or an unknown charset), WEB_FETCH_TIMEOUT, WEB_ABORTED, WEB_PROVIDER_ERROR.
• A non-2xx response is not an error: the status code is returned as part of the result, per the seam's contract.

Known limitations

1. Less reliable than a paid API. These are public HTML and RSS endpoints: redesigns, rate limiting, CAPTCHAs, and regional blocks all take a backend out. Multiple backends lower the odds of a failed search; they cannot remove them. Mojeek was dropped after it returned a CAPTCHA page during development.
2. Compliance is the operator's responsibility. Bing's RSS copyright field restricts the results to personal, non-commercial use in an RSS aggregator, and scraping result pages may violate an engine's terms of service. Keep it to personal, low-frequency use.
3. No generated answer. Only sources[] is filled, never content: a free endpoint offers no trustworthy summary, and inventing one would poison the model's input.
4. publishedAt is sparse. Bing localizes pubDate per market into forms Date cannot parse, and the contract says omit rather than guess.
5. free-http blocks literal private addresses only. It rejects localhost, 127/8, 10/8, 172.16/12, 192.168/16, 169.254/16, 100.64/10, IPv6 loopback, ULA and link-local — but not a public name that resolves inward, which belongs to DNS-level filtering. Enabling allowPrivateHosts lets the model reach your internal network.
6. No proxy setting. Node's fetch ignores HTTPS_PROXY; configure a proxy at the system level if you need one.

Development

npm install
npm run typecheck
npm test          # offline unit tests, no network
npm run test:e2e  # live endpoint checks
npm run build     # tsc → lib/

Fixtures under tests/fixtures/ pin the current page structures, so a backend whose parsing breaks turns the tests red first.