Skip to content

Deterministic Grafana preflight tool (fallback for prompt-driven validation) #38

Description

@tallpsmith

The Why

Issue #10 implements Grafana datasource validation as a prompt-driven workflow — Claude calls mcp-grafana tools and compares URLs. This works but is non-deterministic: it relies on Claude correctly executing a multi-step comparison. If this proves unreliable in practice (wrong datasource selected, URL comparison flubbed, etc.), we need a deterministic fallback.

Depends on: #10 (prompt-driven preflight must be tried first)


Context from #10 Design

The #10 design chose "Option A: Prompt-driven preflight" over "Option B: Preflight tool with user-supplied datasource config" because it's simpler and requires no extra configuration. The design explicitly noted:

"If prompt-driven validation proves unreliable, fall back to a deterministic pcp_grafana_preflight tool with user-supplied config."

This issue is that fallback.


Proposed Feature

A new pmmcp tool: pcp_grafana_preflight

Configuration:

  • PMMCP_GRAFANA_DATASOURCE_UID — the expected Grafana datasource UID (user pre-configures this)

Behaviour:

  1. Claude calls mcp-grafana.get_datasource(uid) and passes the result to pcp_grafana_preflight
  2. The tool extracts the datasource's pmproxy URL from the response
  3. Compares it deterministically with pmmcp's configured PMPROXY_URL
  4. Returns a structured result: { "valid": true/false, "datasource_uid": "...", "pmproxy_url_match": true/false, "details": "..." }
  5. Caches the result for the session

Advantages over prompt-driven:

  • URL comparison is deterministic Python code, not LLM interpretation
  • Structured return value — prompts can branch on valid: true/false
  • Cacheable in server state (not just conversation context)

When to Build This

Don't build this proactively. Only implement if:

  • Users report that prompt-driven preflight is matching wrong datasources
  • URL comparison is failing (e.g. http://pmproxy:44322 vs http://localhost:44322 — same host, different names)
  • Session context loss causes repeated unnecessary preflight checks

The URL aliasing case (same pmproxy, different hostnames) is actually the most likely failure mode — worth watching for.


Open Questions

  • Should the tool accept the raw datasource JSON from mcp-grafana, or just the URL string?
  • How to handle URL aliasing (localhost vs hostname vs IP)? Normalize URLs before comparing?
  • Should this replace the prompt-driven flow entirely, or augment it (prompt discovers, tool validates)?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions