Skip to content

Improve skill generation to check for provider SDK webhook verification support #31

@leggetter

Description

@leggetter

Problem

When generating new provider skills, the current process doesn't systematically check whether the provider's official SDK supports webhook signature verification. This has led to generated skills that use manual verification when the provider actually recommends using their SDK.

Example: The initial Paddle skill generations used manual verification, but feedback from Paddle employees confirmed that their SDK (@paddle/paddle-node-sdk, paddle-billing for Python) provides verification and is their recommended approach.

Current State

The AGENTS.md guidance currently says:

Prefer manual signature verification — SDK methods can have undocumented parameter names or change between versions; manual verification is more reliable and educational

While this reasoning is valid for some cases, it doesn't account for providers who:

  1. Actively recommend their SDK for verification
  2. Have well-documented, stable verification APIs
  3. Handle edge cases (timestamp tolerance, secret rotation) that manual implementations might miss

Proposed Solution

1. Update providers.yaml to track SDK verification support

Add a new field to each provider entry:

providers:
  - name: paddle
    displayName: Paddle
    docs:
      webhooks: https://developer.paddle.com/webhooks/overview
      verification: https://developer.paddle.com/webhooks/signature-verification
    sdk:
      verification: true  # Provider SDK supports webhook verification
      recommended: true   # Provider recommends SDK over manual verification
      packages:
        node: "@paddle/paddle-node-sdk"
        python: "paddle-billing"
      docs: https://developer.paddle.com/api-reference/overview
    notes: >
      Payment and subscription platform. Uses Paddle-Signature header...

2. Update generation prompts

Update scripts/skill-generator/prompts/generate-skill.md to:

  1. Check if the provider has an official SDK with verification support
  2. If SDK verification is recommended, make SDK the primary approach
  3. Still include manual verification as an alternative (for learning, SDK-free deployments, etc.)

3. Update AGENTS.md guidance

Change from "prefer manual" to a more nuanced approach:

  • If provider recommends SDK verification → use SDK as primary, show manual as alternative
  • If provider has no SDK or SDK doesn't support verification → use manual verification
  • If SDK verification is poorly documented/unstable → prefer manual with SDK as optional

4. Update skill structure

For skills where SDK is recommended:

### Signature Verification (SDK — Recommended)

{Provider} recommends using their official SDK for webhook verification...

### Manual Verification (Alternative)

For environments where you can't use the SDK...

Benefits

  1. Better alignment with provider recommendations — Skills will follow what providers actually recommend
  2. Improved reliability — SDKs often handle edge cases (timestamp tolerance, secret rotation, etc.)
  3. Trust from provider communities — Provider DevRel teams will be more likely to recommend skills that follow their guidance
  4. Still educational — Manual verification examples remain for learning purposes

Tasks

  • Update providers.yaml schema to include SDK verification fields
  • Audit existing providers for SDK verification support
  • Update scripts/skill-generator/prompts/generate-skill.md to incorporate SDK checking
  • Update AGENTS.md to reflect nuanced guidance
  • Update existing skills where SDK verification is recommended (Paddle, etc.)

Related

  • Paddle skill feedback from provider employees recommending SDK approach

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions