semantic-search

dsh-semantic-search

Local semantic code search for DeepSeek Harness (dsh) — and any Node script.

中文文档：README.zh.md

sema builds a fragment-level index of a workspace: source is tokenized by a
language-aware tokenizer (camel/snake/kebab splitting, CJK n-grams), chunked with
symbol-aware boundaries (functions/classes stay intact), and embedded into fixed
dimension vectors — locally and dependency-free by default (feature-hashed
lexical TF-IDF), or via any OpenAI-compatible embedding endpoint. Queries run a
hybrid retrieval (vector cosine + BM25, fused with reciprocal-rank fusion) over
the index, so meaning-based search works even when exact terms don't match.

Ships as a dsh plugin bundle — the sema_search, sema_reindex and sema_stats
tools on the harness tool registry — plus a standalone sema CLI.

Highlights

• Works offline by default — the built-in lexical provider needs no network,
  no model download, no API key. Use it as a fast BM25-plus code search.
• Symbol-aware chunking — boundaries from a conservative per-language table
  (16 languages) keep functions/classes intact; a missed boundary degrades to a
  plain chunk rather than breaking.
• CJK-aware tokenizer — n-gram tokenization (default bigrams) aligns Chinese
  queries and documents without a segmentation library; full-width punctuation is
  folded, not treated as a hard break.
• Hybrid retrieval with RRF — vector cosine + BM25 channels are fused by
  reciprocal-rank fusion, so a document found by one channel still ranks.
• Graceful degradation — if a configured remote provider is unreachable, the
  index falls back to the local lexical provider (configurable via allowFallback).
• Incremental refresh + file watching — sema_reindex diffs by size+mtime,
  and an optional watcher keeps the index live.
• Persistence — the index is saved to <root>/.sema atomically (JSON metadata
  • binary vectors), with staleness detection when the provider/dimension changes.
• Deterministic — the same workspace and options produce the same index and
  the same ranked answers.

Supported languages

TypeScript, JavaScript, Python, Go, Rust, Java, Kotlin, Scala, C, C++, C#, Objective-C,
Ruby, PHP, Swift, Bash, plus common data/markup formats (JSON, YAML, TOML, Markdown,
HTML, XML...).

Installation

As a dsh bundle

The package declares "dsh": { "bundle": { "patch": "./cordis.patch.yml" } }. The
patch inserts one plugin row that mounts the bundle and registers
sema_search / sema_reindex / sema_stats on ctx.tools.

# from npm (name reserved; publish pending access setup)
npm install -g dsh-semantic-search

# straight from this repository
dsh plugin --profile demo add github:JohnXu22786/semantic-search

# or from a local checkout
dsh plugin --profile demo add /path/to/semantic-search

As a standalone CLI

npm install -g dsh-semantic-search   # or: npm run build && node bin/sema.mjs
sema --help

CLI usage

sema index                build the full index from the workspace
sema reindex [--full]     incremental refresh (or full rebuild with --full)
sema search <query...>    hybrid vector + BM25 search, prints top hits
sema stats [--json]       index health, provider, and sizing numbers

Global options:

--root <dir>          workspace root (default: current directory)
--data-dir <dir>      index storage directory (default: <root>/.sema)
--provider <kind>     embedding provider: lexical | openai (default: lexical)
--dim <n>             embedding dimension (lexical default: 4096; openai 0 = auto)
--base-url <url>      OpenAI-compatible embeddings endpoint base URL
--model <name>        embeddings model name (openai only)
--api-key <key>       API key (openai only; env: SEMA_EMBEDDING_API_KEY)
--top <n>             hits to print for search (default: 20)
--json                machine-readable output where supported
--help                show this help

CLI exit codes

• 0 — success (including a search with zero hits and a --version/--help call).
• 1 — a runtime failure (config error, build/index/search error).
• 2 — a usage error: unknown command, unknown flag, or a missing query.

Configuration

The plugin is configurable through the bundle row's config (see
cordis.patch.yml for an example), the CLI flags above, or defaults in code:

Development

npm ci
npm test          # build + run the node:test suite (76 tests)
npm run typecheck
npm run build     # tsc -> lib/

License

MIT — see LICENSE. © 2026 dsh-semantic-search contributors.