Agent Configuration

Agent Configuration

Each agent is stored in the database with its own complete configuration. Settings not defined on the agent fall back to the global defaults in appsettings.json / environment variables.

---

Creating an Agent

Via the Discover Page (Recommended)

The Discover page connects to your Power Platform tenant and lets you browse all environments and Copilot Studio agents you have access to. Click Import to library on any agent to create it automatically with the correct Bot ID pre-filled.

Via the Web UI

1. Navigate to Agents in the sidebar
2. Click New Agent
3. Fill in all required fields (see table below)
4. Click Save

Via the Setup Wizard

The Setup Wizard guides you through creating your first agent as Step 1.

---

Agent Fields

Identity

Field	Required	Description
Name	Yes	Human-readable name, e.g. "Production — Customer FAQ"
Environment	Yes	dev / staging / production
Tags	No	Free-form tags for filtering

Direct Line

Field	Required	Description
Direct Line Secret	Yes	Web Channel security secret from Copilot Studio
Bot ID	Yes	Copilot Studio agent identifier (from the agent URI)
Use WebSocket	No	true (default) = WebSocket streaming; false = HTTP polling
Reply Timeout (seconds)	No	Max wait time for a bot reply (default: global setting)
Max Retries	No	Number of retry attempts on transient Direct Line failures (0–10, default: 3)
Backoff Seconds	No	Seconds to wait between retry attempts (1–60, default: 5)

Where to find Direct Line credentials:

1. Open your agent in Copilot Studio
2. Go to Settings → Security → Web channel security
3. Copy Secret 1 (or Secret 2 for rotation)
4. The Bot ID is visible in the Web channel URI

Judge Evaluation (per-agent overrides)

If any of these are left blank, the global judge configuration is used.

Field	Description
Judge Endpoint	Override the AI judge endpoint for this agent
Judge API Key	Override the API key
Judge Model	Override the model deployment name
Judge Temperature	Override LLM temperature (0–2)
Pass Threshold	Minimum score (0–1) for a test to pass (default: 0.70)

Question Generation (per-agent overrides)

If left blank, the global question generation configuration is used.

Field	Description
QGen Endpoint	Override the Azure OpenAI endpoint for question generation
QGen API Key	Override the API key for question generation
QGen Model	Override the model deployment name
QGen System Prompt	Override the system prompt used when generating questions for this agent. Leave empty to use the global system prompt (Settings → AI Question Gen) or the built-in default. Click Load default to edit to start from the built-in prompt.

---

Configuration Resolution Order

AgentConfigurationService resolves settings in this priority order:

Per-Agent setting (Agent entity)
    │  (if set)
    └── Global DB setting (GlobalQuestionGenerationSetting / global judge config)
            │  (if set)
            └── Built-in hardcoded default

This applies to both judge and question-generation configuration, including system prompts.

This allows you to:

• Share a single AI Foundry endpoint across all agents
• Override the judge model only for agents where a different model is needed
• Set different pass thresholds per environment (e.g., higher for production)
• Customise the question-generation prompt globally in Settings, then fine-tune it further per agent

---

Managing Agents

Edit an Agent

Agents → click the agent name → Edit → update fields → Save

Delete an Agent

Agents → click the agent name → Delete

Deleting an agent removes it from all suite associations. Existing run history (which references the agent by ID and stores a config snapshot) is preserved for historical reporting.

Deactivate an Agent

Set IsActive = false to hide an agent from selection without deleting its history. Use Edit → toggle the active flag.

---

Multiple Agents — Example Setup

Agent 1: "Production — Customer FAQ"
  Environment: production
  Direct Line Secret: <prod-secret>
  Bot ID: <prod-bot-id>
  Pass Threshold: 0.80   ← higher bar for production

Agent 2: "Staging — Customer FAQ"
  Environment: staging
  Direct Line Secret: <staging-secret>
  Bot ID: <staging-bot-id>
  Pass Threshold: 0.70   ← standard bar

Agent 3: "Dev — Customer FAQ"
  Environment: dev
  Direct Line Secret: <dev-secret>
  Bot ID: <dev-bot-id>
  Pass Threshold: 0.60   ← lower bar for work-in-progress

All three can be associated with the same test suite. Running the suite executes against all three in parallel.

---

Rotating Web Channel Secrets

Copilot Studio provides Secret 1 and Secret 2 simultaneously for zero-downtime rotation:

1. Generate Secret 2 in Copilot Studio (or regenerate it)
2. Update the agent's Web Channel Secret to Secret 2
3. Test the connection
4. Revoke Secret 1 in Copilot Studio