infraskill - runbook like skill setup and execution framework v0

[priyanshujain]

Feature: Skill Creation and Management System

Overview

Implement a skill creation system where users can define infrastructure automation workflows as GitHub Actions YAML files through the InfraGPT UI. Skills are atomic infrastructure operations that can be chained together by the AI orchestrator to complete complex tasks.

User Flow

1. Skill Creation

User → InfraGPT UI → Skill Builder → GitHub PR → Human Approval → Active Skill

Detailed Flow:

1. User navigates to "Create Skill" page in InfraGPT UI
2. User fills in skill metadata (name, description, category)
3. User defines inputs and outputs for the skill
4. User either:
   • Writes GitHub Actions YAML directly in editor
   • Uses "Add Command" helper to append commands
   • Uses template/starter YAML and modifies
5. UI shows real-time YAML preview with syntax highlighting
6. User validates YAML (actionlint integration)
7. User clicks "Create Skill" → Creates GitHub PR + DB entry
8. User reviews PR on GitHub and approves/merges
9. InfraGPT detects PR merge → Marks skill as active
10. Skill is now available for task orchestration

2. Skill Management

• Pending Skills Dashboard: Shows skills waiting for PR approval with links
• Active Skills Library: Browse and search available skills
• Skill Execution History: View past executions and logs

3. Task Execution (Future)

• InfraGPT analyzes user request
• Selects sequence of skills needed
• Executes skills sequentially via GitHub Actions
• Reports results back to user

Technical Architecture

Database Schema

CREATE TABLE skills (
  id SERIAL PRIMARY KEY,
  name VARCHAR UNIQUE NOT NULL,
  description TEXT,
  category VARCHAR,
  
  -- GitHub integration
  github_path VARCHAR,
  pr_number INTEGER,
  pr_url VARCHAR,
  commit_hash VARCHAR,
  
  -- Skill definition
  inputs_schema JSONB,
  outputs_schema JSONB,
  yaml_content TEXT,              -- Full GitHub Actions YAML
  
  -- Status tracking
  status VARCHAR DEFAULT 'pending_approval',
  validation_errors TEXT,
  created_at TIMESTAMP DEFAULT NOW(),
  approved_at TIMESTAMP
);

CREATE TABLE skill_executions (
  execution_id VARCHAR PRIMARY KEY,
  skill_id INTEGER REFERENCES skills(id),
  task_id VARCHAR,
  status VARCHAR DEFAULT 'running',
  inputs JSONB,
  outputs JSONB,
  github_run_id BIGINT,
  logs_url VARCHAR,
  started_at TIMESTAMP DEFAULT NOW(),
  completed_at TIMESTAMP
);

CREATE TABLE tasks (
  id VARCHAR PRIMARY KEY,
  description TEXT,
  user_request TEXT,
  skills_sequence VARCHAR[],
  current_skill_index INTEGER DEFAULT 0,
  status VARCHAR DEFAULT 'running',
  created_at TIMESTAMP DEFAULT NOW(),
  completed_at TIMESTAMP
);

API Endpoints

POST /api/skills/create
GET  /api/skills?status=pending_approval
GET  /api/skills?status=active
POST /api/skills/{id}/validate
GET  /api/skills/{id}
PUT  /api/skills/{id}

POST /api/skills/callback          # GitHub Actions callback
POST /api/skills/{name}/execute    # Trigger skill execution

GitHub Actions YAML Template

name: "{{SKILL_NAME}}"

on:
  workflow_dispatch:
    inputs:
      execution_id:
        description: 'Execution ID for tracking'
        required: true
        type: string
      {{DYNAMIC_INPUTS}}
      callback_url:
        description: 'InfraGPT callback URL'
        required: true
        type: string

jobs:
  execute:
    runs-on: ubuntu-latest
    steps:
      - name: "Execute skill"
        id: execute
        run: |
          {{USER_COMMANDS}}
          
      - name: "Report back to InfraGPT"
        if: always()
        run: |
          curl -X POST ${{ github.event.inputs.callback_url }} \
            -H "Content-Type: application/json" \
            -d '{
              "execution_id": "${{ github.event.inputs.execution_id }}",
              "status": "${{ job.status }}",
              "outputs": {{DYNAMIC_OUTPUTS}},
              "logs_url": "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
            }'

UI Components

1. Skill Builder Page

<SkillBuilder>
  <SkillMetadata 
    name="" 
    description="" 
    category="" 
  />
  
  <SkillInputsOutputs 
    inputs={[]} 
    outputs={[]} 
  />
  
  <YAMLEditor 
    value={yamlContent}
    onChange={setYamlContent}
    language="yaml"
    theme="github"
  />
  
  <CommandHelper>
    <Button onClick={openAddCommandModal}>
      ➕ Add Command
    </Button>
  </CommandHelper>
  
  <ValidationPanel 
    errors={validationErrors}
    isValid={isValidYAML}
  />
  
  <ActionBar>
    <Button onClick={validateYAML}>Validate</Button>
    <Button onClick={createSkill} disabled={!isValidYAML}>
      Create Skill & PR
    </Button>
  </ActionBar>
</SkillBuilder>

2. Add Command Modal

<AddCommandModal>
  <Input 
    placeholder="Enter shell command..."
    value={newCommand}
    onChange={setNewCommand}
  />
  
  <VariableHelper>
    Available variables:
    {inputs.map(input => 
      <Tag onClick={() => insertVariable(input.name)}>
        ${{ github.event.inputs.{input.name} }}
      </Tag>
    )}
  </VariableHelper>
  
  <Button onClick={appendCommand}>Add to YAML</Button>
</AddCommandModal>

3. Pending Skills Dashboard

<PendingSkillsPage>
  {pendingSkills.map(skill => (
    <SkillCard key={skill.id}>
      <SkillHeader name={skill.name} description={skill.description} />
      <YAMLPreview content={skill.yaml_content} />
      <ActionBar>
        <Link href={skill.pr_url} target="_blank">
          📋 Review PR #{skill.pr_number}
        </Link>
        <StatusBadge status={skill.status} />
      </ActionBar>
    </SkillCard>
  ))}
</PendingSkillsPage>

Action Items

Phase 1: Core Skill Creation (Week 1-2)

• Backend API Development

  • Database schema setup
  • Skill CRUD API endpoints
  • GitHub API integration for PR creation
  • Actionlint validation integration

• Frontend UI Development

  • Skill Builder page with YAML editor
  • Input/Output schema builder
  • Real-time YAML validation
  • Add Command modal/helper

Phase 2: GitHub Integration (Week 3)

• GitHub Workflow

  • PR creation with generated YAML
  • Webhook handling for PR status changes
  • Skill activation on PR merge
  • Error handling for failed PRs

• UI Dashboard

  • Pending skills dashboard
  • Active skills library
  • Skill search and filtering

Phase 3: Skill Execution (Week 4-5)

• Execution Engine

  • GitHub Actions triggering via API
  • Callback handler for execution results
  • Execution status tracking
  • Error handling and retry logic

• Task Orchestration

  • Sequential skill execution
  • Task status management
  • Execution history and logs

Phase 4: Polish & Testing (Week 6)

• Quality & Testing

  • Unit tests for API endpoints
  • Integration tests for GitHub workflow
  • UI testing for skill creation flow
  • Error handling edge cases

• Documentation

  • API documentation
  • User guide for skill creation
  • Developer setup instructions

Acceptance Criteria

Skill Creation

• User can create a skill with name, description, inputs, outputs
• User can write/edit GitHub Actions YAML directly
• User can use "Add Command" helper to append commands
• YAML is validated with actionlint before PR creation
• Successful skill creation generates GitHub PR
• PR approval automatically activates the skill

Skill Management

• Pending skills are displayed with PR links
• Active skills are searchable and browsable
• Skill execution history is viewable
• Validation errors are clearly displayed

Integration

• GitHub Actions execute successfully with proper inputs
• Execution results are posted back to InfraGPT
• Failed skills don't break task execution
• All executions are logged and traceable

Dependencies

• GitHub API access for PR creation and Actions triggering
• Actionlint binary for YAML validation
• PostgreSQL for data persistence
• React/Next.js for frontend UI
• GitHub Actions runners (user-managed)

Notes

• Skills are atomic operations that should be independent
• All skill execution happens via GitHub Actions (self-hosted or GitHub-hosted)
• Human approval required for all skills via GitHub PR process
• No automatic retry on skill failure - tasks fail fast
• Skills are versioned by Git commit hash for reproducibility

[priyanshujain]

Existing runbook platforms

• https://docs.aws.amazon.com/systems-manager/

[priyanshujain]

Prompts:

1.

We are building a skill creation system where users can define infrastructure automation workflows as GitHub Actions YAML files through the InfraGPT UI. "Skills" represent atomic infrastructure tasks that can be chained together by the AI orchestrator to accomplish complex tasks.

A Skill constitutes a group of operations in order.

For now, we’re focusing on generating a YAML configuration for each skill that includes a list of operations to execute in sequence.

Each operation can be one of the following (but not limited to):

A gcloud or kubectl command

Running a Python script

A Git operation (e.g., commit, clone)

SSH into a VM and execute commands

Calling an external API

Executing an SQL query on a PostgreSQL database

Each operation type may require its own setup or configuration. For example:

gcloud commands require a configured Google Cloud project

kubectl commands need a valid kubeconfig

SQL queries require database connection setup

We want your input on how we should define these operations within the YAML file for each skill. Think about what this YAML structure should look like, including any necessary configuration or setup required for each operation type.

Please refer to the App Skills page for additional context.

Our goal is to make it as easy as possible for users to create and edit these operations within a skill.