[Microsoft.Extensions.AI.OpenAI] Final image is never yielded when streaming a response that uses HostedImageGenerationTool

[marcominerva]

Description

When using HostedImageGenerationTool with the OpenAI/Azure OpenAI Responses API through GetStreamingResponseAsync (or AIAgent.RunStreamingAsync), the final generated image is never surfaced. Only the partial images requested via ImageGenerationOptions.StreamingCount are emitted as ImageGenerationToolResultContent.

The cause is in OpenAIResponsesChatClient.GetStreamingResponseAsync. Partial images are converted correctly:

extensions/src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAIResponsesChatClient.cs

Lines 494 to 496 in 2a8df4c

	case StreamingResponseImageGenerationCallPartialImageUpdate streamingImageGenUpdate:
	yield return CreateUpdate(GetImageGenerationResult(streamingImageGenUpdate, options));
	break;

But when the image generation item completes (response.output_item.done), it is grouped with content types whose deltas already carried the full payload, and an empty update is yielded:

extensions/src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAIResponsesChatClient.cs

Lines 608 to 615 in 2a8df4c

	// For ResponseItems where we've already yielded partial deltas for the whole content,
	// we still want to yield an update, but we don't want it to include the ResponseItem
	// as the RawRepresentation, since if it did, when roundtripping we'd end up sending
	// the same content twice (first from the deltas, then from the raw response item).
	// Just yield an update without AIContent for the ResponseItem.
	case MessageResponseItem or ReasoningResponseItem or ImageGenerationCallResponseItem:
	yield return CreateUpdate();
	break;

That assumption does not hold for ImageGenerationCallResponseItem: partial images are not deltas of the final image, they are lower-quality intermediate renders. As a result ImageGenerationCallResponseItem.ImageResultBytes is never converted into an ImageGenerationToolResultContent, and the last image a consumer receives is the last partial one.

The non-streaming path is correct, since it goes through AddImageGenerationContents, which uses outputItem.ImageResultBytes:

extensions/src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAIResponsesChatClient.cs

Lines 1968 to 1980 in 2a8df4c

	private static void AddImageGenerationContents(ImageGenerationCallResponseItem outputItem, CreateResponseOptions? options, IList<AIContent> contents)
	{
	var imageGenTool = options?.Tools.OfType<ImageGenerationTool>().FirstOrDefault();
	string outputFormat = imageGenTool?.OutputFileFormat?.ToString() ?? "png";
	contents.Add(new ImageGenerationToolCallContent(outputItem.Id));
	contents.Add(new ImageGenerationToolResultContent(outputItem.Id)
	{
	RawRepresentation = outputItem,
	Outputs = [new DataContent(outputItem.ImageResultBytes, $"image/{outputFormat}")]
	});
	}

Note that when StreamingCount is not set, streaming yields no image at all.

Reproduction Steps

IChatClient chatClient = new OpenAIClient(new ApiKeyCredential(apiKey), options)
    .GetResponsesClient()
    .AsIChatClient("gpt-5.5");

ChatOptions chatOptions = new()
{
    Tools =
    [
        new HostedImageGenerationTool
        {
            Options = new()
            {
                ModelId = "gpt-image-2",
                Count = 1,
                ImageSize = new(1536, 1024),
                MediaType = MediaTypeNames.Image.Png,
                StreamingCount = 3
            }
        }
    ]
};

var images = new List<byte[]>();

await foreach (var update in chatClient.GetStreamingResponseAsync(
    "Create a picture of a mountain landscape with a tree in the foreground, a cow and a wooden house overlooking a lake.",
    chatOptions))
{
    images.AddRange(update.Contents
        .OfType<ImageGenerationToolResultContent>()
        .Where(c => c.Outputs is not null)
        .SelectMany(c => c.Outputs!.OfType<DataContent>())
        .Select(c => c.Data.ToArray()));
}

Console.WriteLine(images.Count); // 3 (only the partials), and images[^1] is partial #3, not the final image

Running the same prompt with GetResponseAsync (non-streaming) returns the correct, final image.

Expected behavior

When response.output_item.done carries an ImageGenerationCallResponseItem, the streaming path should yield an ImageGenerationToolResultContent containing ImageResultBytes, consistent with the non-streaming path (AddImageGenerationContents). Consumers of the streaming API should receive the final image, regardless of whether StreamingCount was set.

Actual behavior

An empty ChatResponseUpdate is yielded for the completed image generation item and the final image bytes are silently dropped. Consumers end up displaying the last partial image, which is visibly lower quality / different from the final result. When StreamingCount is not configured, no image is produced at all while streaming.

Regression?

Unknown — it appears to have been present since HostedImageGenerationTool streaming support was added.

Known Workarounds

Read the final image from the raw representation of the update, since CreateUpdate() still sets RawRepresentation = streamingUpdate:

static (byte[] Content, string ContentType)? GetFinalImage(ChatResponseUpdate update)
{
    if (update.RawRepresentation is StreamingResponseOutputItemDoneUpdate
        {
            Item: ImageGenerationCallResponseItem { ImageResultBytes: { } bytes }
        })
    {
        return (bytes.ToArray(), MediaTypeNames.Image.Png);
    }

    return null;
}

When going through Microsoft.Agents.AI, the ChatResponseUpdate must first be unwrapped from AgentResponseUpdate.RawRepresentation.

Alternatively, use the non-streaming GetResponseAsync, giving up streaming entirely.

Configuration

• Microsoft.Extensions.AI.OpenAI 10.10.0 (via Microsoft.Agents.AI.OpenAI 1.21.0), OpenAI 2.13.0
• .NET 10 (SDK 10.0.401), C# 14
• Windows 11 x64
• Azure OpenAI Responses API, gpt-5.5 + gpt-image-2

Other information

Suggested fix: remove ImageGenerationCallResponseItem from the case MessageResponseItem or ReasoningResponseItem or ImageGenerationCallResponseItem: group and give it a dedicated case that yields the final result, e.g.:

case ImageGenerationCallResponseItem imageGenItem:
    yield return CreateUpdate(new ImageGenerationToolResultContent(imageGenItem.Id)
    {
        RawRepresentation = imageGenItem,
        Outputs = [new DataContent(imageGenItem.ImageResultBytes, $"image/{outputFormat}")]
    });
    break;

It may also be worth marking partial images explicitly (they currently only carry PartialImageIndex inside DataContent.AdditionalProperties), so consumers can distinguish a partial render from the final image without relying on arrival order.

[iamAdarshh]

I'd like to take this. The plan is to give ImageGenerationCallResponseItem its own case in the streaming path so it yields the final image, plus tests. This would also fix microsoft/agent-framework#8674.

@rogerbarreto