Skip to content

fix(instrumentation): guarantee span ends when Anthropic stream manager exit raises - #757

Open
Aftabbs wants to merge 1 commit into
JudgmentLabs:mainfrom
Aftabbs:fix/stream-manager-span-leak
Open

fix(instrumentation): guarantee span ends when Anthropic stream manager exit raises#757
Aftabbs wants to merge 1 commit into
JudgmentLabs:mainfrom
Aftabbs:fix/stream-manager-span-leak

Conversation

@Aftabbs

@Aftabbs Aftabbs commented Jul 18, 2026

Copy link
Copy Markdown

Summary

When MessageStreamManager.__exit__ or AsyncMessageStreamManager.__aexit__ raises an exception, post_hook_exit_impl / post_hook_aexit_impl was never called, so span.end() was skipped and the OTel span leaked. Every subsequent request then ran under a stale unclosed parent span.

Root Cause / Motivation

In both WrappedMessageStreamManager.__exit__ and WrappedAsyncMessageStreamManager.__aexit__, the telemetry cleanup was called sequentially after the underlying manager's exit:

# before (sync)
def __exit__(self, exc_type, exc_val, exc_tb):
    self._manager.__exit__(exc_type, exc_val, exc_tb)  # if this raises...
    post_hook_exit_impl()                               # ...this never runs → span leaks

When the underlying stream manager raises (e.g., network timeout, server error during stream teardown), Python propagates that exception before post_hook_exit_impl() is reached, leaving the OTel span permanently open.

Changes

File What changed
src/judgeval/instrumentation/llm/llm_anthropic/messages_stream.py Wrapped _manager.__exit__ and _manager.__aexit__ calls in try/finally so span cleanup always runs; added clarifying comment to the intentional bare except in token-extraction
src/tests/instrumentation/llm/anthropic/test_messages_stream.py New test file: 5 tests covering span creation, token recording, and the span-leak fix for both sync and async paths

Testing

  • Existing tests pass (7 / 7 in test_messages.py)
  • New test: test_span_ends_even_when_manager_exit_raises — verifies span is exported even when __exit__ raises (was asserting False before fix)
  • New test: test_span_ends_even_when_manager_aexit_raises — same for async path
  • New tests: test_span_is_created and test_span_records_token_usage for both wrappers
  • All 12 tests pass

Impact

Users who experience stream teardown errors (network issues, server-side errors during streaming) previously accumulated leaked OTel spans that were never closed. This caused traces to appear as perpetually-in-progress in Judgment and could inflate trace timing metrics.


Open in Devin Review

…er exit raises

If MessageStreamManager.__exit__ or AsyncMessageStreamManager.__aexit__
raised an exception, post_hook_exit_impl / post_hook_aexit_impl was
never called, so span.end() was skipped and the OTel span leaked.

Wrap the manager exit call in try/finally so cleanup always runs.
Also annotate the bare except in the token-extraction block to make
clear it is intentional — metadata is best-effort and the span ends
unconditionally via the new finally path.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant