Agentic AI with Semantic Kernel

Note

This repository compliments the following YouTube video: Build an AI Agent that Controls Your App UI

This sample demonstrates how to build a multi-agent AI system using Semantic Kernel. The system includes three agents:

• Planner – creates a plan based on the user's input.
• Reviewer – reviews the plan and provides feedback to the planner.
• Executor – carries out the plan step by step when prompted.

The solution follows a human-in-the-loop approach: before executing the plan, agents present it to the user for step-by-step approval.

How to Run the Project

You can run the agents using either a local model (via Ollama) or OpenAI's cloud models.

Run with Ollama

To use a local Ollama-based model, configure the Ollama chat completion in the Init method`:

builder.AddOllamaChatCompletion(modelId: "llama3.1:8b", endpoint: new Uri("http://localhost:11434/"));

You’ll also need to install Ollama and run the following command in the command prompt:

ollama run llama3.1:8b This command will download and launch the LLaMA model locally.

Note

The llama3.1:8b model used in this project may not solve all tasks correctly. For real-world production apps, you may need to use more advanced models hosted on your server.

Run with OpenAI

To use OpenAI instead of a local model, configure the OpenAI chat completion in the Init method

builder.AddOpenAIChatCompletion("gpt-4.1-mini", "[YOUR OPENAI API KEY]");

You can create an API key on the OpenAI API keys page. If you weren’t granted free credits during registration, you may need to top up your balance with a minimum payment (usually around $5).

Implementation Details

Below are the key steps we follow to build the system:

1. Initialize the Semantic Kernel (build the kernel and register services):

var builder = Kernel.CreateBuilder();
builder.AddOllamaChatCompletion(modelId: "llama3.1:8b", endpoint: new Uri("http://localhost:11434/"));
builder.Services.AddKeyedSingleton(PlannerAgentStep.AgentServiceKey,

(AgentService.cs: Init)

2. Create agents using the ChatCompletionAgent class:

ChatCompletionAgent CreateAgent(string name, string instructions, Kernel kernel, IEnumerable<KernelPlugin> plugins = null, PromptExecutionSettings promptSettings = null) {
//...
return new() { Name = name,
               Instructions = instructions,
               Kernel = kernel,
               Arguments = new KernelArguments(promptSettings)
};}

(AgentService.cs: CreateAgent)

3. Add plugins (tools) to the Executor agent:

KernelPlugin customersPlugin = KernelPluginFactory.CreateFromObject(pluginsSourceObject);
//...
kernel.Plugins.AddRange(plugins);
};}

(AgentService.cs: Init)

4. Define process steps for each agent:

public class PlannerAgentStep : KernelProcessStep {
    [KernelFunction]
    public async Task CreatePlan(Kernel kernel, KernelProcessStepContext context, string taskDescription) {
        //...
    }

    [KernelFunction]
    public async Task RefinePlan(Kernel kernel, KernelProcessStepContext context, ReviewResult reviewResult) {
        //...
    }
}
public class ReviewerAgentStep : KernelProcessStep {

    [KernelFunction]
    public async Task ReviewPlanAsync(Kernel kernel, KernelProcessStepContext context, Plan plan) {
        //...
    }
}


public class ExecutorAgentStep : KernelProcessStep {

    [KernelFunction]
    public async Task ExecuteStep(Kernel kernel, KernelProcessStepContext context, PlannedStepFlow stepFlow) {
        //...
    }
}

(AiProcessSteps.cs)

5. Define the process flow (i.e., how data is transferred between steps).
   For example, the planner sends the plan to the reviewer, who then sends feedback back to the planner for refinement:

  ProcessBuilder processBuilder = new("Planning");
  var plannerStep = processBuilder.AddStepFromType<PlannerAgentStep>();
  var reviewerStep = processBuilder.AddStepFromType<ReviewerAgentStep>();
  //...

  processBuilder
      .OnInputEvent(StepEvents.StartProcess)
      .SendEventTo(new ProcessFunctionTargetBuilder(plannerStep, functionName: nameof(PlannerAgentStep.CreatePlan), parameterName: "taskDescription"));
  
  plannerStep
      .OnEvent(StepEvents.PlanPrepared)
      .SendEventTo(new ProcessFunctionTargetBuilder(reviewerStep, parameterName: "plan"));
  
  reviewerStep
      .OnEvent(StepEvents.PlanRejected)
      .SendEventTo(new ProcessFunctionTargetBuilder(plannerStep, functionName: nameof(PlannerAgentStep.RefinePlan), parameterName: "reviewResult"));
  
  process = processBuilder.Build();
//...
return new() { Name = name,
               Instructions = instructions,
               Kernel = kernel,
               Arguments = new KernelArguments(promptSettings)
};}

(AgentService.cs: InitProcess)

6. Implement human-in-the-loop support with an external client:

• Add a user proxy step

      var userProxyStep = processBuilder.AddProxyStep("UserProxy", [
          StepEvents.PlanPreparedExternal,
          StepEvents.PlanApprovedExternal]);
  };

  (AgentService.cs: InitProcess)

• Emit an external event:

      reviewerStep
          .OnEvent(StepEvents.PlanApproved)
          .EmitExternalEvent(userProxyStep, StepEvents.PlanApprovedExternal);
  };

  (AgentService.cs: InitProcess)

• Create a client message channel:

      public class ExternalClient(Func<string, KernelProcessProxyMessage, Task> actionCallback)
          : IExternalKernelProcessMessageChannel {
  
        Func<string, KernelProcessProxyMessage, Task> actionCallback = actionCallback;
        public Task EmitExternalEventAsync(string externalTopicEvent, KernelProcessProxyMessage message) => actionCallback(externalTopicEvent, message);  
        public ValueTask Initialize() => ValueTask.CompletedTask;
        public ValueTask Uninitialize() => ValueTask.CompletedTask;
      }

  (AgentService.cs: ExternalClient)

• Pass the message channel to the process:

        await Task.Run(async () =>
          await process.StartAsync(kernel, new KernelProcessEvent {
              Id = StepEvents.StartProcess,
              Data = userTask
          },
      externalMessageChannel: new ExternalClient(actionCallback)));

  (AgentService.cs: StartNewTaskProcessAsync)