lead_scoring

Example: lead qualification — a REAL run

this is a real, paid pipeline run, not a fixture. It was produced by actually calling Anthropic and Tavily — so the numbers, citations, and risk verdict are real model output over real web search results (and carry all the caveats that implies).

Date	2026-07-25
Subject	Stripe (as a hypothetical inbound B2B lead)
LLM	anthropic/claude-opus-5
Retrieval	Tavily web search
Cost	~$0.616 (64,074 tokens, 12 LLM calls, 114s — as reported by the run)
Grounding	70 of 77 claims (~91%) cite a retrieved source; the rest are flagged [UNCITED]
Config	config.yaml — the config used, plus request_timeout_seconds, added afterwards (a client-side wait; it can't change output)

What's here

• output.json — the final risk_score: an overall score + level, one factor per expert, and a verdict. This is what a CRM/automation would consume.
• research_log.json — the grounded evidence trail behind that verdict: every expert's findings broken into individual claims, each carrying the source URL(s) that support it. This is the audit trail the score is built from.
• run_record.json / trace.jsonl — the run's own accounting: per-call tokens and cost, phase timings, and every retrieval query issued.

The verdict it reached

42/100, medium risk — "a high-value prospect worth pursuing, but not yet a qualified lead."

What makes that interesting is which dimension scored badly. The two high factors are not about Stripe at all — they're about the absence of evidence:

"Buying intent is entirely unverified: no source documents a discovery call, form fill, demo request…" — and product fit "cannot be scored because Stripe's stated needs are undocumented."

Meanwhile ability-to-pay came back low risk on hard cited numbers. A naive scorer handed a famous company would have returned "great lead." This panel separated is this company solvent from is this actually a lead, and refused to score the half it had no evidence for.

What this run actually demonstrates

• Real, sourced findings. The March 2026 FTC warning letters to the CEOs of Stripe, PayPal and others cite ftc.gov; the 2025 annual-letter figures ($1.9T payment volume, $6.8B revenue, ~33% growth, ~47% FCF margin) cite stripe.com; the trust-bank-charter and $1.1B Bridge acquisition items cite bankingdive.com and chargeflow.io. 24 distinct sources across the run — you can click through and check them.
• Grounding that refuses to over-claim. 7 claims carry no citation and are surfaced as [UNCITED] rather than dressed up with one. The pipeline only credits sources actually retrieved for that question, so a model asserting something from its own training data gets flagged, not laundered.
• Uncertainty reported as a finding. One financial-stability factor is a data-quality caveat — no audited financials, credit ratings, or liquidity disclosures exist for a private company — scored and cited like any other factor rather than silently omitted.
• Cost visibility. The run self-reported est_cost=$0.6163 against the config's max_cost_usd: 1.00 hard stop.

Reproduce it

# needs real ANTHROPIC_API_KEY and TAVILY_API_KEY in .env
uv run panelist run examples/lead_scoring/config.yaml

Expect roughly $0.60 and two minutes on claude-opus-5. The earlier version of this example ran on a cheap web-connected model for ~$0.02; Opus 5 costs ~25× that and buys markedly better grounding (91% of claims cited, up from 71%) plus the reasoning that produced the absence-of-evidence read above. Swap llm.model to claude-haiku-4-5 for a cheaper run.