dsh-verdict-guard

verdict-guard

English | 中文

A native DeepSeek Harness plugin.
A turn that states a verdict does not close until the verdict carries something a reader can follow.

model:  "The filter does not work. Closing this direction."
guard:  held back — a verdict without evidence.
model:  (runs the check, quotes the output — or withdraws the claim)

Why

The most expensive thing an agent produces is not a wrong fix. It is a wrong
closing verdict: "it doesn't work", "no bug here", "not reproducible".

A false "it works" is caught by the next run — it leaves a trace.
A false "it doesn't work" leaves none. The direction is closed quietly, and
nobody re-opens a question that already has an answer. The two error classes
are not equally priced, so they should not carry the same standard of proof.

This plugin makes the asymmetry mechanical: at the turn's stop boundary it
reads what the model is about to close on, and what the turn actually did.

What counts as evidence

Signal	Where it comes from
a fenced block, an exit code, a test tally, an HTTP status	the answer's text
file.ts:88, a path, a log name, journalctl, pytest, …	the answer's text
a verifying tool result — bash, read, grep, test, …	the session log for that turn

The third row is why this is a plugin and not a shell hook. A text-only checker
sees what the model wrote. The session log says what actually ran.

That gives one check a hook cannot make:

The answer pastes a command transcript — a fenced block, 12 passed, 0 failed,
an exit code — while the turn called no tool at all. Nothing produced that
text except the model. It is held back with a sharper message than a plain
missing citation.

A bare file path is deliberately not enough to trigger that stricter case: a
path can honestly be recalled from earlier in the conversation, a transcript cannot.

Install

dsh plugin add dsh-plugin-verdict-guard

That is the whole install. The package declares a dsh.bundle manifest, so it
enters your profile as its own bundle layer with every option at its default —
no row to write by hand. Verify it composed into the tree:

dsh --profile headless --dump-config | grep -A 3 verdict-guard

To change a default, add a row to your profile's own patch layer
($DSH_HOME/profiles/<profile>/cordis.patch.yml), which is applied after every
bundle layer:

- id: verdict-guard
  config:
    locale: en
    requireToolEvidence: false
    maxInterventionsPerTurn: 2

Config

Option	Default	Meaning
locale	both	which verdict vocabularies participate — en, ru, both
requireToolEvidence	true	a verdict needs a verifying tool result, not merely any tool result — writing a file is not checking a claim
verifyingTools	bash/read/grep/test/…	tool names that count as verification
extraVerdictPatterns	—	regex sources appended to the built-in verdict vocabulary
extraEvidencePatterns	—	regex sources appended to the built-in evidence vocabulary
verdictPatterns	—	replace the built-in verdict vocabulary entirely
evidencePatterns	—	replace the built-in evidence vocabulary entirely
maxInterventionsPerTurn	1	how many times one turn may be held back
maxInterventionsPerSession	6	total holds per session
verbose	false	log every pass decision at debug level

An empty list is read as "not supplied", never as "match nothing" — so a
config that fills unset array options with [] cannot silently disarm the guard.

What this is not

It does not verify that a claim is true — no oracle, no acceptance criteria,
no contract. It checks one thing: whether the answer offers anything to check
at all. A wrong claim with a real command output beside it passes here and is
caught by the reader; a right claim with nothing beside it is held.

It also holds only once per turn by default. It is a speed bump on the way
out, not a gate you have to satisfy.

How it works

One listener on agent/turn-stopping, the harness's own stop boundary:

agent/turn-stopping  →  read the turn's closing assistant text
                     →  read the turn's tool/call + tool/result events
                     →  verdict? evidence?  →  agent.steer(reason)

Steering makes the machine observe pending input and run another step, which is
the documented way for a listener to object at that boundary. Nothing in the
agent loop is modified.

The guard caps itself. The harness has no loop guard here yet: the loop
carries a TODO(stop-loop-guard), and the Claude Code hook bridge reports
stop_hook_active: false unconditionally, so a listener that steers must limit
itself or it can hold a turn open forever. maxInterventionsPerTurn (default
1) is that limit — one hold, then the turn closes whatever the model says.

Compatibility

Built and tested against 0.1.0-rc.8.

⚠️ The latest dist-tag on the harness's own sub-packages is stale.
@deepseek-ai/dsh-tools, -llm, -session and friends publish their current
line under next (0.1.0-rc.8) while latest still points at 0.0.1-rc.1.
Installing them without an explicit range mixes two incompatible generations —
and some 0.0.1-rc.1 packages depend on @deepseek-ai/dsh-bash, which no
longer exists. Pin ^0.1.0-rc.8, or install with @next.

Development

npm install
npm test     # 24 tests: the detector, plus the plugin on a real agent loop
npm run build

The integration tests drive the real AgentLoop with a scripted model adapter,
so both outcomes — held and closed — are proven without a provider key.

Who wrote this

Built by Sagit Nabiev while running agents against production systems, where one
confident unsupported verdict costs a day. If you want something like this built
for your own harness or agent stack — a gate, a bridge, a plugin — write to
founder@perpreconcile.com.

License

MIT