.Net: Bug: OpenAiResponseAgent chat message images removed by SK

[shravan2x]

Describe the bug
It appears that SK currently removes any ImageContent in a user message before it's sent to OpenAI. The offending code seems to be this ToResponseItem call. The implementation of this method only retains the main chat string and ignores all items.

Pseudocode:

ChatMessageContent userMessage = new ChatMessageContent(AuthorRole.User, "describe the image");
userMessage.Items.Add(new ImageContent(imageUri));
await foreach (AgentResponseItem<ChatMessageContent> response in agent.InvokeAsync(userMessage))
{
 // ...
}

Platform

• Language: C#
• Source: NuGet package version 1.61.0
• AI model: o3-pro

[justinbarias]

+1 to this bug. It doesnt process any KernelContent that doesn't have an message.Content (string).
So all other Kernel Content that isnt TextContent (Binary, Image, etc) are all ignored

[JP-droidic]

Can someone have a look at this issue please, this is a big showstopper for moving responses API to production