prompt-foo

Promptfoo — open-source CLI and library for evaluating and red-teaming LLM applications, agents, and RAG pipelines. MIT-licensed, 21k+ stars, now part of OpenAI.

• Website: https://www.promptfoo.dev
• Source: https://github.com/promptfoo/promptfoo
• Docs: https://www.promptfoo.dev/docs/intro/
• CLI reference: https://www.promptfoo.dev/docs/usage/command-line/
• Red-team quickstart: https://www.promptfoo.dev/docs/red-team/quickstart/
• Pricing: https://www.promptfoo.dev/pricing/

What's profiled

Open-source project (Tier 2: CLI + library + hosted enterprise tier). No standalone REST API documentation surface, so no openapi/ artifacts are generated. The profile captures CLI commands, providers, integrations, plugins, strategies, framework alignment, and packaging (npm / PyPI / Homebrew / Helm) in apis.yml.

Related repos in the promptfoo org

• promptfoo/promptfoo — core CLI and library (TypeScript)
• promptfoo/promptfoo-python — Python wrapper for the CLI
• promptfoo/promptfoo-action — GitHub Action for PR-level eval gating
• promptfoo/modelaudit — ML model security scanner
• promptfoo/mcp-agent-provider — custom provider for testing MCP servers
• promptfoo/evil-mcp-server — adversarial MCP target for red-team testing
• promptfoo/example-app, promptfoo/demo-app — reference apps
• promptfoo/js-rouge — ROUGE summarization metric implementation