AIAgent and AgentSession persistence

[AdriAJ1]

Hello,
I currently have a .NET 8 project using Semantic Kernel, and I am creating a new .NET 10 project with the Agent Framework library.

The project is basically a REST API where you can make requests to an agent, including typical features such as memory management, tools, and dynamic instructions.

I am currently testing and learning how the library works, and I came across a question that is more about “good practices.”

Between each request, I store and recover the session by serializing and deserializing it using agent.SerializeSessionAsync(session) and agent.DeserializeSessionAsync(session).

My question is about the AIAgent object (which you need to serialize and deserialize the memory).
Coming from Semantic Kernel, the Kernel was a lightweight object that was recommended to be recreated on each request.
Does AIAgent follow the same principle, or is it better to keep it in memory between executions?

I might need to change some parameters between requests, but ChatClientAgentRunOptions is enough for that.
With this in mind, I have two approaches:

1. Maintain my current code and create an AIAgent object for each request.
2. Store the AIAgent in memory and reuse it until the session is closed, then dispose of it.

Thanks in advance

My current test code look like this:

    public async Task<string> GetAnswerAsync(JsonElement sessionJson, string text)
    {
        // ¿Create a new agent each request? but mantaining session (stored in memory)
        var agent = CreateAgent();
        var session = await agent.DeserializeSessionAsync(sessionJson);

        // Dinamically generating instructions and configuration in each request
        var instructions = $"You are a helpful assistant called Mike. The current time is {DateTime.Now}";

        // In the future there will be more fields and may change in each request
        var additionalProperties = new AdditionalPropertiesDictionary();
        additionalProperties.Add("agentId", "exampleGUID");

        // Example, plugins may change in each request
        var toolsPlugin = new TransferPlugin();

        var agentOptions = new ChatClientAgentRunOptions
        {
            ChatOptions = new()
            {
                Instructions = instructions,
                MaxOutputTokens = 750,
                Temperature = 0.7f,
                TopP = 1.0f,
                FrequencyPenalty = 0.0f,
                PresencePenalty = 0.0f,
                ToolMode = ChatToolMode.Auto,
                Tools = [.. toolsPlugin.AsAITools()],
                AdditionalProperties = additionalProperties
            }
        };

        // Retrieve response
        var response = await agent.RunAsync(message: text, session: session, options: agentOptions);

        _log.LogInformation("[agsv] Response: " + response.ToString());

        // TODO - Store sessionJson back in memory
        // var newSession = await agent.SerializeSessionAsync(session);

        return response.Text;
    }

    public AIAgent CreateAgent()
    {
        var testEndpoint = "XXXX";
        var testKey = "XXXX";
        var deployment = "gpt-4o-mini";

        // Custom HTTP client with custom configuration
        PipelineTransport transport = new HttpClientPipelineTransport(_client);
        var options = new AzureOpenAIClientOptions { Transport = transport };

        AzureOpenAIClient client = new AzureOpenAIClient(
            new Uri(testEndpoint),
            new ApiKeyCredential(testKey),
            options);

        var chatClient = client.GetChatClient(deployment).AsIChatClient();

        // Limit memory messages
        var truncation = new TruncationCompactionStrategy(CompactionTriggers.GroupsExceed(20), minimumPreservedGroups: 20);

        var chatAgent = chatClient
            .AsBuilder()
                .UseAIContextProviders(new CompactionProvider(truncation))
                .Use(getResponseFunc: LoggingChatMiddleware, getStreamingResponseFunc: null)
            .BuildAIAgent();

        var agent = chatAgent
            .AsBuilder()
                .Use(LoggingToolMiddleware)
            .Build();

        return agent;
    }

[moonbox3]

@westey-m would you be able to help here, please?

[westey-m]

Hi @AdriAJ1, AIAgent objects are designed to be stateless objects, and can typically live for the length of the application lifetime. You can even add them to a DI container for the lifetime of the application if needed.

That said, re-creating an agent would work, and we don't typically do any expensive work on creation. That said, if you were building a high scale service where you serve many requests, it's good to avoid recreating objects on each request, due to the amount of new allocations.

So if you want to be super efficient, I'd got for option 2.