Skip to content

Define an evidence-backed agent capability manifest #64

Description

@szmyty

Outcome

Define a versioned, machine-readable manifest for agent-facing capabilities and compatibility so “works with agent X” becomes a set of bounded, independently testable statements rather than a single undocumented badge.

This issue supplies the capability-contract portion of the broader agent catalog and cost-control work in #41.

Contract dimensions

Represent, where applicable:

  • agent/runtime identity and version;
  • launch, installation, and configuration mechanism;
  • model-provider and API protocols;
  • tool calling and MCP transports;
  • streaming and structured-output behavior;
  • context mutation and protected-context boundaries;
  • retrieval and external knowledge availability;
  • persistent-memory behavior and storage boundary;
  • multi-agent delegation/handoff behavior;
  • observability, trace, and audit endpoints;
  • cancellation, retry, timeout, and failure semantics;
  • supported OS, architecture, container, and runtime versions;
  • security, permission, secret, and network requirements;
  • known partial, blocked, experimental, or unsupported paths.

State model

The schema must distinguish at least:

  • implemented — code exists;
  • enabled — configured for a specific integration;
  • tested — exercised by identified evidence;
  • supported — within the maintained compatibility promise;
  • experimental;
  • partial;
  • blocked;
  • unsupported;
  • unknown;
  • not-applicable.

Absence must remain unknown, never implicitly false or supported.

Evidence linkage

Every positive compatibility cell must link to evidence conforming to egohygiene/hygiene#30, including represented commit/version, environment, test/workflow, timestamp, raw artifact reference, and freshness.

Compose rather than duplicate:

Pilot

Represent one real agent integration end to end, including at least one supported path, one partial or blocked path, and one unknown capability. Validate both the manifest and its linked evidence.

Acceptance criteria

  • A versioned agent-capability schema and controlled vocabulary exist.
  • Implemented, enabled, tested, supported, blocked, unsupported, unknown, and not-applicable remain distinct.
  • Major launch, protocol, tool, context, retrieval, memory, handoff, observability, failure, platform, and security dimensions are representable.
  • Every positive tested/supported assertion links to claim evidence.
  • The schema supports agent/runtime versions and capability-specific freshness.
  • One real integration validates end to end with supported, partial/blocked, and unknown examples.
  • Human-readable compatibility documentation is generated from or directly backed by the manifest.
  • Agent-catalog, Realm, Relay, Observatory, and consumer ownership boundaries are documented.

Non-goals

  • Declaring support from package presence or source compilation alone.
  • Encoding provider credentials.
  • Treating all agents as behaviorally equivalent.
  • Replacing provider-native configuration or Realm environment manifests.

Source

Captured from the Headroom crossover review as a separable child contract under Aether #41.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions