mcp-agent-openjev

Local typed probabilistic decision service — Choice, Noul, Score with calibrated probabilities and abstention — powered by OpenJev (the tuned 27B Qwen3.5 decision model) over Bionic's Open Responses endpoint in LM Studio.

OpenJev is tuned so a single output position carries the decision: one score per option letter, then a calibration step. LM Studio's /v1/completions and /v1/chat/completions never return logprobs (and cap top_logprobs at 20), so this project drives the one surface that does — /v1/responses — with the two settings that make the readout exact: reasoning.effort: "none" (the model emits exactly one token, the answer letter) and undistorted logits (temperature: 1.0, top_p: 1.0, frequency_penalty: 0, presence_penalty: 0).

For whom: anyone with a local OpenJev GGUF in LM Studio who wants System-1 style typed decisions (routing, guardrails, intent detection, severity ratings) — and agents that want them as an MCP tool.

How it works

For a Choice over N options, the client:

1. labels the options A, B, C, … and builds the OpenJev prompt (State: / Question: / Options:);
2. sends it to POST /v1/responses with max_tokens=1, reasoning.effort=none, and an uncapped top_logprobs=64;
3. reads each letter's log-probability at the answer position;
4. temperature-scales the logits (softmax, T=0.85) for calibrated posterior probabilities;
5. abstains (value="UNKNOWN") when the winner's confidence falls below JEV_ABSTAIN_THRESHOLD (auto scales it to 1.25 / N), preserving the raw argmax in tentative_value.

Install

Requires uv and a running LM Studio with the OpenJev model loaded:

uv sync

Copy .env.example to .env and set JEV_MODEL to the OpenJev model key LM Studio serves (default openjev). Point JEV_BASE_URL at the LM Studio server (default http://127.0.0.1:1234).

Usage

One-shot CLI

uv run mcp-agent-openjev doctor

uv run mcp-agent-openjev choice "unauthorized login from an unknown IP" `
    -c billing -c tech_support -c security `
    --criteria "pick the handling department"

uv run mcp-agent-openjev noul "the request is urgent" "this is a support request"

uv run mcp-agent-openjev score "PII exposed in a public bucket for 3 days" `
    -t low -t medium -t high -t critical

Score tiers

A tier is either a label string or a dict with label (or value) and an optional score:

// plain labels - weight is the position in the list
["low", "medium", "high", "critical"]

// explicit form - 'score' must equal the tier's position
[{"label": "low", "score": 0}, {"label": "critical", "score": 3}]

score must match the tier's position because a tier's weight is its ordinal position - that is what makes expected_score an expected tier index, which is what callers compare against. A tier dict without label/value, a duplicate label, or a score that disagrees with the position is rejected (InvalidTierError, HTTP 400 on the service) rather than silently degraded to a positional label - a degraded tier is rendered into the prompt as 0. 0, so the model ends up ranking meaningless numbers. level_probabilities is always keyed by tier label.

The response also echoes the scale it used, so the score never has to be read against an assumed convention:

{
  "expected_score": 2.2056,
  "level_probabilities": {"low": 0.00005, "high": 0.783, "critical": 0.211, "UNKNOWN": 0.0002},
  "tier_weights": {"low": 0.0, "high": 2.0, "critical": 3.0, "UNKNOWN": 0.0},
  "confidence": 0.783,
  "abstained": false
}

tier_weights is ordered by tier, and includes UNKNOWN (weight 0.0) when abstention is enabled, so expected_score can be recomputed from level_probabilities alone.

HTTP service

uv run mcp-agent-openjev http --host 127.0.0.1 --port 8377
curl http://localhost:8377/health
curl -X POST http://localhost:8377/v1/decide/choice -H "Content-Type: application/json" -d '{
  "state": {"ticket": "unauthorized login from an unknown IP"},
  "candidates": ["billing", "tech_support", "security"],
  "criteria": {"billing": "payments", "security": "unauthorized access"}
}'

Endpoints: POST /v1/decide/choice, POST /v1/decide/noul, POST /v1/decide/score, GET /health. Choice and score accept optional model / temperature / abstain_threshold overrides per request.

MCP server

uv run mcp-agent-openjev serve              # stdio (default)
uv run mcp-agent-openjev serve --http       # streamable-http on :8030

Tools: decide_choice, decide_noul, decide_score, decision_status. Wire it into opencode's MCP block:

"mcp-agent-openjev": {
  "type": "local",
  "command": ["uv", "--project", "<path-to-mcp_agent_openjev>", "run", "python", "-m", "mcp_agent_openjev", "serve"],
  "environment": {}
}

Examples

uv run python examples/ticket_router.py
uv run python examples/severity_score.py

Decisions

Type	Purpose	Returns
Choice	categorical decision over candidates	value, probabilities, confidence, abstained, tentative_value
Noul	binary truth judgment	value, probability_true, confidence
Score	ordered tier evaluation	expected_score, level_probabilities, confidence

Configuration

Variable	Default	Meaning
JEV_BASE_URL	http://127.0.0.1:1234	LM Studio server URL
JEV_MODEL	openjev	Model key LM Studio serves
JEV_API_KEY	(empty)	Sent as Authorization: Bearer only when set; falls back to LM_STUDIO_API_KEY
JEV_TEMPERATURE	0.85	Softmax temperature scaling (OpenJev READOUT_T)
JEV_NOUL_T	1.829074	Noul sigmoid scale (OpenJev READOUT_NOUL_T)
JEV_NOUL_BIAS	0	Noul sigmoid bias (OpenJev READOUT_NOUL_BIAS)
JEV_PERMS	1	Option orders to average over (accuracy mode; costs latency)
JEV_ABSTAIN_THRESHOLD	auto	Abstention threshold; or auto (1.25 / N)
JEV_TIMEOUT	120	Backend request timeout (seconds)

The released helper's READOUT_T, READOUT_NOUL_T, READOUT_NOUL_BIAS and READOUT_PERMS variables are also honoured.

License

The adapter, service, CLI and MCP surface in this repository are MIT (see LICENSE). Two upstream licenses are carried alongside because the calibration mirrors their code and model:

• LICENSE-APACHE-2.0.txt — Apache 2.0. Covers the openjev-server code this calibration mirrors (helper/, serve/) and OpenJev's Apache-2.0 open base model, as requested by the OpenJev model repo.
• CC BY-NC 4.0 — the OpenJev weights. Free for research and other non-commercial use with attribution; commercial use requires a separate licence (open a discussion on the model repo). The terms travel with the GGUF, so they apply however the model is served.