Skip to content

[BREAKING] [FIX]: Evaluate attack verdicts over final traces - #150

Merged
Spencer Schoenberg (spencrr) merged 10 commits into
microsoft:mainfrom
spencrr:dev/spencrr/trace-xpia-stopping
Nov 2, 2026
Merged

[BREAKING] [FIX]: Evaluate attack verdicts over final traces#150
Spencer Schoenberg (spencrr) merged 10 commits into
microsoft:mainfrom
spencrr:dev/spencrr/trace-xpia-stopping

Conversation

@spencrr

@spencrr Spencer Schoenberg (spencrr) commented Aug 4, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Description

XPIA now evaluates the attack objective once over the final trace, separately from online stopping. stop_when accepts an evaluator, None, or the default StopWhen.AUTO. StopWhen.AUTO reuses the verdict evaluator as the stop condition only when detection stays true as turns are appended: ToolCalled, SideEffectOccurred, and ResponseContains with ResponseScope.ANY_TURN. StopWhen is exported from rampart.attacks and rampart.

When the stop and verdict evaluators are the same object, the latest online evaluation is reused if it already covers the final trace, so no duplicate judge call is made. Final verdict evidence is stored on Result.final_trace_evaluation, trace completion uses TraceEndReason, and online stop evidence uses EvaluationPurpose.STOP_CHECK. Final evaluation runs inside the active session and injection stack. Observability downgrades are recorded without mutating response metadata, and cleanup failures discard otherwise successful verdict evidence and return ERROR.

The list-based resolve_as_attack and per-turn evaluate_turn_async helpers are removed because no built-in strategy uses them. The trace-contract declaration records a compatible change against main, which now includes the probe cadence change from #149.

Breaking changes

  • XPIA verdicts are computed once over the final trace. Single-trigger attacks with deterministic evaluators keep their verdicts.
  • With the default StopWhen.AUTO, other evaluators, including LLM judges, no longer stop early. They are called once on the final trace, and adaptive drivers can run up to max_turns. Pass the same evaluator as stop_when to restore per-turn early stopping without a duplicate final call.
  • With the default, stochastic evaluators are sampled once per run instead of once per turn, so trial pass rates can shift.
  • Auto-stopped attacks expose online evaluator feedback to adaptive drivers.
  • Removed helpers: replace resolve_as_attack(eval_results=...) with resolve_attack_verdict(evaluation=...), and replace evaluate_turn_async with run_trace_async plus evaluate_final_trace_async.

Checklist

  • pre-commit run --all-files passes
  • Tests added or updated for changes — automatic, explicit, and disabled stopping; StopWhen values; exact evaluator call counts; stable-evaluator classification and composition; cleanup ordering; zero and max turns; observability; metadata isolation; and summaries
  • Documentation updated

@azure-pipelines

azure-pipelines Bot commented Aug 4, 2026

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@spencrr
Spencer Schoenberg (spencrr) force-pushed the dev/spencrr/trace-xpia-stopping branch from d80c55b to 84d0197 Compare August 8, 2026 02:31
@spencrr
Spencer Schoenberg (spencrr) force-pushed the dev/spencrr/trace-xpia-stopping branch 2 times, most recently from cdcad35 to bedbf6b Compare August 27, 2026 17:30
@spencrr
Spencer Schoenberg (spencrr) force-pushed the dev/spencrr/trace-xpia-stopping branch 2 times, most recently from 3051684 to 4074ae9 Compare September 8, 2026 22:46
@spencrr
Spencer Schoenberg (spencrr) force-pushed the dev/spencrr/trace-xpia-stopping branch 4 times, most recently from 3ee9e25 to bcc3fd6 Compare October 1, 2026 00:59
@spencrr Spencer Schoenberg (spencrr) mentioned this pull request Oct 1, 2026
Merged
3 tasks done
@spencrr Spencer Schoenberg (spencrr) changed the title [FIX]: Evaluate attack verdicts over final traces [FIX] [BREAKING]: Evaluate attack verdicts over final traces Oct 1, 2026
@spencrr
Spencer Schoenberg (spencrr) force-pushed the dev/spencrr/trace-xpia-stopping branch from bcc3fd6 to bb58b64 Compare October 8, 2026 02:14
@spencrr Spencer Schoenberg (spencrr) changed the title [FIX] [BREAKING]: Evaluate attack verdicts over final traces [BREAKING] [FIX]: Evaluate attack verdicts over final traces Oct 8, 2026
Spencer Schoenberg (spencrr) added a commit that referenced this pull request Oct 9, 2026
@spencrr
[REFACTOR]: Add shared linear trace runner (#148)
66c6ca1 
<!-- This repository loosely follows the "Conventional Commits"
specification for commit messages. See
https://www.conventionalcommits.org/ for more information. -->
<!-- Squash-merge commit messages must match:
^\[(FEAT|FIX|REFACTOR|STYLE|TEST|DOCS|CI|MAINT|META|REVERT)\](
\[BREAKING\])?:\s.+\(#\d+\) -->
<!-- GitHub appends (#N) automatically during squash-merge; just use the
[TAG]: description format for your PR title. -->
<!-- If your PR is not yet ready for review, please mark it as [DRAFT].
-->

## Description

<!-- What does this PR do? Provide a brief summary of the changes. -->
<!-- Mention any relevant issues or pull requests with #<issue_number>
-->
<!-- Tag any relevant reviewers or teams using @<username> -->

Adds the linear trace runner shared by attack and probe strategies.
`run_trace_async` drives a conversation with an optional online
`stop_when` check, keeps separate raw and annotated turn histories, and
records `TraceEndReason`. `evaluate_final_trace_async` evaluates the
final trace once. It reuses the latest online evaluation only when the
evaluator, raw turns, manifest, and observability level are all
identical.

The runner does not own session lifetime, polarity, cleanup, or
exception conversion; those remain strategy and `BaseExecution`
responsibilities. Driver history receives a shallow copy, evaluator
contexts contain annotation-free turns, and reused evaluator evidence is
copied defensively.

`EvaluationRecord`, `TraceRun`, `run_trace_async`, and
`evaluate_final_trace_async` are exported from `rampart.core` for custom
strategies and documented in the API reference. Built-in probes adopt
them in #149 and XPIA in #150.

## Breaking changes
<!-- If none, write "None". If breaking, describe the impact and
migration path. -->

None. This PR adds APIs and does not change existing strategies.

## Checklist

- [x] `pre-commit run --all-files` passes
- [x] Tests added or updated for changes <!-- Please describe what tests
were added or updated --> — termination reasons, raw and annotated
histories, stop behavior, exact-context reuse, changed observability and
manifest, optional-evidence copying, post-run mutation, exceptions, and
turn budgets
- [x] Documentation updated
@spencrr
Spencer Schoenberg (spencrr) force-pushed the dev/spencrr/trace-xpia-stopping branch from bb58b64 to 8bbadc9 Compare October 9, 2026 00:48
@nina-msft Nina Chikanov (nina-msft) mentioned this pull request Oct 22, 2026
Merged
3 tasks done
@spencrr
Spencer Schoenberg (spencrr) marked this pull request as ready for review October 26, 2026 22:44
nina-msft
Nina Chikanov (nina-msft) reviewed Oct 28, 2026
View reviewed changes
Comment thread docs/attacks/xpia.md Resolved
Comment thread schemas/trace-compatibility.json Resolved
@spencrr
[FIX]: Evaluate attack verdicts over final traces
a4d451c
@spencrr
[BREAKING]: Retire aggregate attack and per-turn evaluation helpers
62bd3cd 
Remove the old reducers and helper now that both built-in strategies evaluate terminal traces. Keep explicit-scope stopping classification, provenance and extension documentation aligned with the shared runner.
@spencrr
[REFACTOR]: Use final-trace naming in XPIA
0621c25 
Call evaluate_final_trace_async and describe XPIA verdict evidence as final-trace evaluation in code, tests, and extension guidance.
@spencrr
[DOCS]: Add XPIA verdict upgrade note
469360c 
Explain the per-turn to final-trace verdict change, its single-trigger blast radius, automatic stopping defaults, LLM-judge cost trade-off, trial sampling impact, and replacements for removed helpers.
@spencrr
[REFACTOR]: Replace the auto stop literal with StopWhen
a8870ff 
Type Attacks.xpia(stop_when=...) as Evaluator | StopWhen | None and default to StopWhen.AUTO, following the enum-over-Literal standard. StopWhen is a string enum, so its equal string value remains accepted at runtime. Export it from rampart.attacks and the top-level package.
@spencrr
[DOCS]: Describe current attack cadence without an upgrade note
03c31af 
Remove the XPIA upgrade note in line with the no-deprecation policy, keeping the stop_when reuse tip in the driver-budget note, and drop the pre-migration attack wording from the lifecycle overview.
@spencrr
[DOCS]: Finish final-trace guidance for attacks
ee3508d 
Point the packaged onboarding skill at resolve_attack_verdict now that resolve_as_attack is gone, and describe XPIA stopping accurately. Document the observability_adjusted metadata key, note that StopWhen.AUTO applies to built-in evaluators only, cover XPIA in the compatibility rationale, and correct the phase count and diagram label.
@spencrr
[MAINT]: Rebase the trace-contract decision onto the probe change
6e99828 
Reference the fingerprint merged with the probe cadence change and narrow the rationale to the XPIA changes this PR makes.
@spencrr
[DOCS]: Qualify scope guidance for automatic stopping
799eea5 
ResponseScope still only quantifies the turns an evaluator sees, but under XPIA StopWhen.AUTO it also decides eligibility for automatic stopping, since only ANY_TURN detection is stable as the trace grows. Say so in the docs and docstrings, and state that TranscriptScope never affects stopping. Correct the compatibility rationale: the xdist envelope is unchanged by this PR, so it rejects older transport versions rather than older XPIA semantics; mixed installations remain unsupported. Add tests that non-ANY_TURN scopes run the whole sequence and that subclasses of built-in evaluators are not auto-stop eligible.
nina-msft
Nina Chikanov (nina-msft) previously approved these changes Oct 30, 2026
View reviewed changes

@nina-msft Nina Chikanov (nina-msft) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. Two non-blocking documentation nits below; no code-level blockers found.

Comment thread docs/attacks/xpia.md Resolved
Comment thread docs/usage/authoring-tests.md Resolved
@spencrr
[DOCS]: Cover negated scopes and explicit judges in stopping guidance
493bdf9 
Negation swaps the stability flags, so a negated ALL_TURNS ResponseContains is also eligible for StopWhen.AUTO; the previous wording said only ANY_TURN qualified, contradicting the worked example above it. Say that TranscriptScope only affects stopping when a judge is passed explicitly as stop_when. Add an end-to-end test for the negated case.
nina-msft
Nina Chikanov (nina-msft) approved these changes Nov 2, 2026
View reviewed changes

@nina-msft Nina Chikanov (nina-msft) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀 ✨

@spencrr
Spencer Schoenberg (spencrr) merged commit bae38ad into microsoft:main Nov 2, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nina-msft Nina Chikanov (nina-msft) nina-msft approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@spencrr @nina-msft