[pull] main from regenrek:main

packages/sdk/README.md

@@ -184,6 +184,89 @@ Processes a prompt template file with variable substitution.
 
 Variables are replaced using `{{VARIABLE_NAME}}` syntax.
 
+## GitHub API Token Setup
+
+The SDK can access private GitHub repositories and increase API rate limits by using a GitHub personal access token. Here's how to generate and use one:
+
+### Step 1: Generate a GitHub Personal Access Token
+
+1. **Sign in to GitHub** and navigate to your account settings
+2. **Go to Developer settings** → Personal access tokens → Tokens (classic)
+   - Or directly visit: https://github.com/settings/tokens
+3. **Click "Generate new token"** → "Generate new token (classic)"
+4. **Configure your token:**
+   - **Note**: Give it a descriptive name (e.g., "codefetch-sdk")
+   - **Expiration**: Choose an appropriate expiration time
+   - **Scopes**: Select the following permissions:
+     - `repo` (Full control of private repositories) - if you need to access private repos
+     - `public_repo` (Access public repositories) - for public repos only
+5. **Generate the token** and copy it immediately (you won't be able to see it again)
+
+### Step 2: Use the Token in Your Code
+
+#### Option A: Environment Variable (Recommended)
+```bash
+# Set in your shell
+export GITHUB_TOKEN="ghp_your_token_here"
+
+# Or in a .env file
+GITHUB_TOKEN=ghp_your_token_here
+```
+
+The SDK will automatically use the `GITHUB_TOKEN` environment variable:
+```typescript
+// No need to pass token - it reads from process.env.GITHUB_TOKEN
+const result = await fetch({
+  source: "https://github.com/owner/private-repo",
+});
+```
+
+#### Option B: Pass Directly in Code
+```typescript
+const result = await fetch({
+  source: "https://github.com/owner/private-repo",
+  githubToken: "ghp_your_token_here", // Only for web fetch
+});
+
+// Or with the low-level API
+const client = new GitHubApiClient(owner, repo, logger, {
+  token: "ghp_your_token_here",
+});
+```
+
+### Step 3: For Cloudflare Workers
+
+Store your token as a secret:
+```bash
+wrangler secret put GITHUB_TOKEN
+```
+
+Then use it in your worker:
+```typescript
+export interface Env {
+  GITHUB_TOKEN: string;
+}
+
+const result = await fetchFromWeb("https://github.com/owner/repo", {
+  githubToken: env.GITHUB_TOKEN,
+});
+```
+
+### Benefits of Using a Token
+
+- **Access private repositories**: Required for non-public repos
+- **Increased rate limits**: From 60 to 5,000 requests per hour
+- **Avoid rate limiting**: Essential for fetching large repositories
+- **Consistent access**: No interruptions from hitting rate limits
+
+### Security Best Practices
+
+1. **Never commit tokens to version control**
+2. **Use environment variables or secrets management**
+3. **Limit token scope to minimum required permissions**
+4. **Rotate tokens regularly**
+5. **Use different tokens for different environments**
+
 ## Advanced Features
 
 ### Token Limiting Strategies