Probity

Probity forces AI coding agents to follow your rules. It hooks into your agent and checks every file write and shell command before it happens. When an action breaks a rule, Probity blocks it and tells the agent why.

You can use it to enforce Test-Driven Development with the built-in rule, block destructive commands, or keep unwanted patterns out of your code. Writing your own rules takes a few lines of TypeScript, and one config works across most coding agents.

Probity is featured on the Modern Software Engineering channel, where Emily Bache and Nizar Selander discuss what TDD looks like when an agent writes the code. A separate Q&A with factor10's Jimmy Nilsson covers using Probity on real projects, and links the full kata run.

How it works

When a rule is broken, the agent sees a reason and a path forward:

Probity: you're adding production code before a failing test has been
observed.

The next TDD-legal step is to add one focused test in src/cart.test.ts
and run it to a clean assertion failure before implementing only the
minimum code to pass it.

The agent corrects course and continues.

Rules can be deterministic, matching commands or file content by string or regex, or AI-validated using official SDKs. Both kinds can read recent session activity, so actions are judged in context.

Quick start

npm install -D @nizos/probity

Create probity.config.ts at your project root:

import { defineConfig, enforceTdd } from '@nizos/probity'

export default defineConfig({
  rules: [
    {
      files: ['src/**', 'test/**'],
      rules: [enforceTdd()],
    },
  ],
})

Then wire it into your agent. One-time setup per agent.

Built-in rules

• enforceTdd(): failing test first, minimal implementation, refactor on green
• forbidCommandPattern(): block shell commands by pattern
• requireCommand(): require a prior command (e.g. tests before commit)
• forbidContentPattern(): block writes containing a pattern
• enforceFilenameCasing(): enforce a filename casing style

FAQ

Does it work with my agent? Probity currently works with Claude Code, Codex, and GitHub Copilot CLI, with more coming.

Does it work with my language? Probity reads each agent's session transcript directly, so there are no per-framework reporters to install. It works with any language and test runner that your agent can work with.

Does Probity need its own API key or subscription? No. AI-validated rules use each vendor's official SDK and reuse whatever authentication your agent already has, so Probity doesn't require its own access or billing.

Does this cost extra tokens? AI-validated rules add a turn each time they check an action, so yes, some. Working in small TDD steps adds turns of its own, with or without Probity. Pattern-based rules add none.

Does enforcing TDD with an agent guarantee good design? Not on its own. Probity keeps the agent on track: a failing test before the code, the minimum to pass it, refactoring on green. That removes a lot of handholding and prompt fatigue. The shape the implementation and the tests take is still yours to steer.

I'm already using TDD Guard. Should I switch? Yes. Probity handles refactors and multi-step edits more reliably, is safe with parallel sessions, and supports more agents. See Migrating from TDD Guard.

Documentation

• Setup: wire Probity into your agent
• Configuration: config file shape, path scoping, and custom rules
• Rules: built-in rules and their options
• Migrating from TDD Guard: steps and customization mapping

Contributing

Contributions are welcome. See the contributing guidelines to get started.

License

MIT