-
Notifications
You must be signed in to change notification settings - Fork 3
Description
Problem
When generating new provider skills, the current process doesn't systematically check whether the provider's official SDK supports webhook signature verification. This has led to generated skills that use manual verification when the provider actually recommends using their SDK.
Example: The initial Paddle skill generations used manual verification, but feedback from Paddle employees confirmed that their SDK (@paddle/paddle-node-sdk, paddle-billing for Python) provides verification and is their recommended approach.
Current State
The AGENTS.md guidance currently says:
Prefer manual signature verification — SDK methods can have undocumented parameter names or change between versions; manual verification is more reliable and educational
While this reasoning is valid for some cases, it doesn't account for providers who:
- Actively recommend their SDK for verification
- Have well-documented, stable verification APIs
- Handle edge cases (timestamp tolerance, secret rotation) that manual implementations might miss
Proposed Solution
1. Update providers.yaml to track SDK verification support
Add a new field to each provider entry:
providers:
- name: paddle
displayName: Paddle
docs:
webhooks: https://developer.paddle.com/webhooks/overview
verification: https://developer.paddle.com/webhooks/signature-verification
sdk:
verification: true # Provider SDK supports webhook verification
recommended: true # Provider recommends SDK over manual verification
packages:
node: "@paddle/paddle-node-sdk"
python: "paddle-billing"
docs: https://developer.paddle.com/api-reference/overview
notes: >
Payment and subscription platform. Uses Paddle-Signature header...2. Update generation prompts
Update scripts/skill-generator/prompts/generate-skill.md to:
- Check if the provider has an official SDK with verification support
- If SDK verification is recommended, make SDK the primary approach
- Still include manual verification as an alternative (for learning, SDK-free deployments, etc.)
3. Update AGENTS.md guidance
Change from "prefer manual" to a more nuanced approach:
- If provider recommends SDK verification → use SDK as primary, show manual as alternative
- If provider has no SDK or SDK doesn't support verification → use manual verification
- If SDK verification is poorly documented/unstable → prefer manual with SDK as optional
4. Update skill structure
For skills where SDK is recommended:
### Signature Verification (SDK — Recommended)
{Provider} recommends using their official SDK for webhook verification...
### Manual Verification (Alternative)
For environments where you can't use the SDK...Benefits
- Better alignment with provider recommendations — Skills will follow what providers actually recommend
- Improved reliability — SDKs often handle edge cases (timestamp tolerance, secret rotation, etc.)
- Trust from provider communities — Provider DevRel teams will be more likely to recommend skills that follow their guidance
- Still educational — Manual verification examples remain for learning purposes
Tasks
- Update
providers.yamlschema to include SDK verification fields - Audit existing providers for SDK verification support
- Update
scripts/skill-generator/prompts/generate-skill.mdto incorporate SDK checking - Update
AGENTS.mdto reflect nuanced guidance - Update existing skills where SDK verification is recommended (Paddle, etc.)
Related
- Paddle skill feedback from provider employees recommending SDK approach