[WIP] Fix missing OPENAI_API_KEY in GenAIScript smoke test

[Copilot]

✅ Fix GenAIScript OPENAI_API_KEY Missing Issue

Problem

The GenAIScript smoke test workflow was failing at run #18722204018 with error OPENAI_API_KEY missing because the workflow uses the openai:gpt-4.1 model but doesn't validate or provide the required API key.

Solution Implemented

• Investigate the issue and understand the workflow structure
• Add OPENAI_API_KEY validation step to shared/genaiscript.md (following Codex pattern)
• Pass OPENAI_API_KEY environment variable to GenAIScript execution steps
• Recompile workflows to generate updated .lock.yml files
• Update documentation with OPENAI_API_KEY requirement
• Test the changes - all unit tests pass ✅

Changes Made

1. Modified .github/workflows/shared/genaiscript.md:

• ✅ Added "Validate OPENAI_API_KEY secret" step that runs before GenAIScript installation
• ✅ Validation step checks if OPENAI_API_KEY secret is configured
• ✅ Provides clear error message with repository settings instructions
• ✅ Added OPENAI_API_KEY to environment in "Run GenAIScript" step
• ✅ Updated documentation to highlight OPENAI_API_KEY requirement

2. Recompiled workflow (smoke-genaiscript.lock.yml):

• ✅ Validation step added to both agent and detection jobs
• ✅ OPENAI_API_KEY environment variable configured in all GenAIScript steps
• ✅ Workflow will fail fast with helpful error if secret is missing

Validation Behavior

Before Fix:

Error: OPENAI_API_KEY missing
    at parseTokenFromEnv (env.js:216:19)

After Fix:

Error: OPENAI_API_KEY secret is not set
The GenAIScript engine with openai:gpt-4.1 model requires OPENAI_API_KEY secret to be configured.
Please configure this secret in your repository settings.
Documentation: https://githubnext.github.io/gh-aw/reference/engines/

The workflow will now:

• ✅ Fail fast with a clear, actionable error message
• ✅ Provide instructions on how to configure the secret
• ✅ Only proceed to run GenAIScript if the secret is available
• ✅ Prevent cryptic errors from GenAIScript core

Testing

• ✅ All unit tests pass (make test-unit)
• ✅ Workflow compiles successfully without errors
• ✅ Code formatting applied (make fmt)
• ✅ No breaking changes introduced

Files Changed

.github/workflows/shared/genaiscript.md      | +16 lines (validation + env + docs)
.github/workflows/smoke-genaiscript.lock.yml | +40 lines (compiled output)

References

• Resolves issue #18722204018
• Pattern based on existing Codex smoke test validation
• Error signature: OPENAI_API_KEY_MISSING_GENAISCRIPT

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[smoke-detector] 🔍 Smoke Test Investigation - GenAIScript OPENAI_API_KEY Missing</issue_title>
<issue_description># 🔍 Smoke Test Investigation - Run githubnext/gh-aw#18722204018

Summary

The GenAIScript smoke test failed due to a missing OPENAI_API_KEY environment variable. The workflow is configured to use the openai:gpt-4.1 model, but the required API key is not provided in the workflow configuration, causing immediate failure during model resolution.

Failure Details

• Run: #18722204018
• Commit: 5d9b9b9 - "Add changeset for reporting instructions feature"
• Trigger: workflow_dispatch
• Duration: 1.8 minutes
• Workflow: Smoke GenAIScript

Root Cause Analysis

Primary Failure: Missing API Key

The GenAIScript engine attempted to resolve the model openai:gpt-4.1 but failed because the OPENAI_API_KEY environment variable is not configured:

Error: OPENAI_API_KEY missing
    at parseTokenFromEnv ((redacted))

This error occurred during the model resolution phase, before any actual AI work could begin.

Secondary Failure: Missing Output File

The create_issue job failed as a cascading failure because it expected an output file from the agent job:

Error: ENOENT: no such file or directory, open '/tmp/gh-aw/safe-outputs/agent_output.json'

Since the agent job failed early in execution, it never created the expected output file.

Tertiary Error: Unhandled Exception in GenAIScript

Additionally, GenAIScript itself threw an unhandled exception when trying to set outputs:

TypeError: Cannot read properties of undefined (reading 'text')
    at githubActionSetOutputs ((redacted))

This indicates that GenAIScript's error handling doesn't gracefully handle failed runs.

Failed Jobs and Errors

Job: agent

• Status: Failed (exit code 255)
• Duration: 1.1 minutes
• Primary Error: OPENAI_API_KEY missing
• Location: env.js:216:19 in genaiscript core

Job: create_issue

• Status: Failed
• Duration: 5 seconds
• Primary Error: ENOENT: no such file or directory
• Root Cause: Dependency on failed agent job

Investigation Findings

Configuration Issue

The GenAIScript smoke test workflow has a model configured in the prompt file:

script({
  model: 'openai:gpt-4.1',
  system: [],
  'system-safety': false,
})

However, the workflow does not provide the necessary OPENAI_API_KEY environment variable that this model requires.

Error Handling Gap

GenAIScript's GitHub Action integration has insufficient error handling - when a run fails, it attempts to set outputs but encounters undefined values, causing a secondary error that masks the primary issue.

Recommended Actions

Immediate Fixes

• Option A: Add OPENAI_API_KEY to GitHub repository secrets and configure the Smoke GenAIScript workflow to use it
• Option B: Change the model configuration to use a properly configured provider (e.g., GitHub Models with github:gpt-4o, or Azure OpenAI if configured)
• Option C: Add conditional execution to skip GenAIScript smoke test if OPENAI_API_KEY is not available

Code Improvements

• Fix GenAIScript's githubActionSetOutputs function to handle failed runs gracefully (add null checks)
• Improve error messages in GenAIScript to suggest configuration steps when API keys are missing
• Add a validation step at workflow start to check for required secrets before running tests

Documentation

• Document required environment variables for each smoke test workflow
• Add troubleshooting guide for common configuration errors
• Update GenAIScript smoke test README with setup instructions

Prevention Strategies

1. Pre-flight Validation: Add a validation step in workflows to verify required environment variables are set before executing agent tasks
2. Better Error Messages: Enhance GenAIScript error handling to provide actionable guidance when configuration is missing
3. Workflow Guards: Use GitHub Actions conditionals to skip tests when required secrets are unavailable
4. Smoke Test Matrix: Consider testing multiple configurations (with/without various API keys) to ensure graceful degradation

Historical Context

This is the first recorded instance of this specific error pattern in the investigation database. The error signature OPENAI_API_KEY_MISSING_GENAISCRIPT has been documented for future pattern matching.

Pattern Severity: High
Is Flaky: No
External Dependency: Yes (OpenAI API)

---

Investigation completed by Smoke Detector 🔍
Pattern stored: /tmp/gh-aw/cache-memory/patterns/openai_api_key_missing.json
Investigation record: `/tmp/gh-aw/cache-memor...

Fixes #2142

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[github-actions]

Agentic Changeset Generator triggered by this pull request.