One command to benchmark AI guardrails and coding agents across safety, security, jailbreak, prompt-injection, and secure-code tasks.
-
Updated
Jun 26, 2026 - Python
#
eval-harness
Here are 38 public repositories matching this topic...
BYOA Agent harness that ensures alignment end-to-end – with built-in eval system.
bdd specs byoa evals agent-orchestration agent-workflow agent-harness eval-harness harness-engineering
-
Updated
Jun 28, 2026 - TypeScript
Local Codex MCP harness: contracts, persistent RAG memory, raw traces, verification records, governance policy and PASS/FLAG/BLOCK audits, observability reports, harness profiles, eval runs, Meta-Harness-lite promotion records, natural-language harness specs, MCP resources/prompts, multi-client installer, and completion gates.
mcp observability governance codex ai-agents rag model-context-protocol agent-ops codex-cli agent-harness eval-harness skills-sh harness-engineering
-
Updated
May 12, 2026 - JavaScript
Evaluate AI agents with Unix-style pipeline commands. Schema-driven adapters for any CLI agent, trajectory capture, pass@k metrics, and multi-run comparison.
cli typescript grader ai-agents bun jsonl llm-evaluation agent-evaluation unix-pipeline agent-comparison trajectory-capture eval-harness pass-at-k headless-adapter
-
Updated
Jun 17, 2026 - TypeScript
Deterministic synthetic two-party conversation corpus generator for testing AI scoring systems.
-
Updated
Jun 24, 2026 - Python
LLM-powered clinical extraction + structured evals. Prompt strategies, hallucination detection, and per-field F1 scoring.
-
Updated
May 1, 2026 - TypeScript
A from-scratch hybrid rule + LLM-as-judge eval harness: pass-rate, judge-vs-rule Cohen's kappa, latency/cost, HTML report, CI gate. Runs offline.
-
Updated
Jun 17, 2026 - Python
Production-minded LLM eval harness for safety, reliability, cost, and latency analysis.
python evaluation observability model-evaluation ai-safety red-teaming llm-evals eval-harness safety-evals
-
Updated
May 25, 2026 - Python
Prototype adapter for CLI agents to play Crusader Kings III through a constrained, auditable CK3 mod bridge.
-
Updated
Jun 10, 2026 - Python
Verification-native local coding agent runtime with eval gates, memory, subagents, and model profiles.
nodejs cli benchmark automation typescript ai verification multi-agent openai developer-tools agents local-first ai-agent llm anthropic agent-evals claude-code coding-agent subagents eval-harness
-
Updated
May 16, 2026 - TypeScript
Codex-native OZM skill pack for AI coding agent governance, agentic coding loops, claim ceilings, and AGENTS.md-aware workflows.
mcp multi-agent codex prompt-injection skill-pack claim-verification llm-evals agent-orchestration agent-memory ai-coding-agent coding-agent agentic-coding agents-md agent-governance context-engineering software-engineering-agents eval-harness ai-agent-governance codex-native tool-protocol
-
Updated
May 30, 2026 - Python
She who catches hubris in agents — a Python evaluation harness for agentic failure modes.
-
Updated
Jun 27, 2026 - Python
Document processing pipeline engine — adapters, contracts, domain packs, eval harnesses
python pipeline document-processing rag pydantic air-gapped document-ai contract-first llm eval-harness
-
Updated
Jun 28, 2026 - Python
Open Agentsia Labs benchmark harness for model runners, multi-turn evals, reproducible rubric scoring, proof bundles, and RunRecord output.
benchmark adtech reproducibility llm-evaluation eval-harness frontier-ai proof-bundles open-benchmark assay-adtech assay-harness runrecord multi-turn-evals iab-tech-lab
-
Updated
Jun 27, 2026 - TypeScript
A supervisor-driven multi-agent system where a central orchestrator decomposes tasks, delegates to specialized worker agents, and synthesizes final outputs. Designed for controllability, observability, and production workflows.
python multi-agent-systems local-first streamlit ollama structured-outputs agent-orchestration eval-harness
-
Updated
Jun 17, 2026 - Python
YAML-driven evaluation harness for WhatsApp RAG bots
-
Updated
Apr 29, 2026 - Python
Open evaluation harness for mental health LLM responses. 5 clinically-grounded rubrics, LLM-as-judge with bias controls, crisis-detection routing to 988 protocols.
psychology cbt ai-safety conversational-ai clinical-ai cohen-kappa ollama llm-evaluation llm-as-judge mental-health-ai ai-eval inter-rater-reliability eval-harness lifeline-988 open-source-eval
-
Updated
May 29, 2026 - Python
Tolerant apply_patch for LLM-generated diffs, plus an eval harness for code agents.
-
Updated
May 6, 2026 - Python
Production-style LLM evaluation harness for structured clinical extraction — compares prompt strategies across accuracy, cost, and hallucination.
nlp json-schema nextjs model-evaluation hono structured-output few-shot-learning ai-evaluation prompt-engineering anthropic llm-evaluation hallucination-detection llm-reliability eval-harness prompt-comparison
-
Updated
May 1, 2026 - TypeScript
Can Computer-Use Agents manage buy-side procurement operations? A benchmark across live e-commerce, with multi-model adapters (Northstar, OpenAI, Claude, Gemini), Kernel-hosted browsers, and Harbor/ATIF-v1.6 trajectory export.
benchmark kernel procurement cua lightcone computer-use-agent eval-harness tzafon-northstar openai-computer-use
-
Updated
May 10, 2026 - Python
Improve this page
Add a description, image, and links to the eval-harness topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the eval-harness topic, visit your repo's landing page and select "manage topics."