Skip to content

feat: Databricks GPT-5.x endpoints unblock + re-run eval Measurement A #81

Description

@deanban

Problem
On workspace dbc-6395977d-5b1c all 5 databricks-gpt-5-* endpoints (and their short aliases like 54mini) return 403 PERMISSION_DENIED: rate limit of 0, verified against both the native serving-endpoints surface and the MLflow AI Gateway surface (https://7474647691394693.ai-gateway.cloud.databricks.com/mlflow/v1) with a Gateway-scoped PAT. This is a workspace-wide Databricks-side throttle, not token-scoped. Blocks Measurement A of the native Databricks Mosaic AI provider eval.

Proposed solution
File a Databricks support ticket (subject + workspace ID + error payload are documented in docs/runbooks/databricks-mosaic-provider.md under "GPT-5.x is blocked — support-ticket playbook"). Once the throttle lifts, re-run sema eval run on showcase/cbioportal_to_omop/slices/dev_slice_poc.yaml with LLM_MODEL=databricks-gpt-5-4-mini as a third condition and append results to eval-runs/databricks-mosaic-comparison-2026-04-22.md.

Alternatives considered
Stay on llama-3-3-70b via AI Gateway (works but ~2x slower than the OpenRouter baseline; measured in the 2026-04-22 eval report).

Reference: tasks.md §13.1; docs/runbooks/databricks-mosaic-provider.md.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions