Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
0d0c747
feat: add copy-discussions shell script
joshjohanning Oct 1, 2025
669e7a8
feat: add copy-discussions node script
joshjohanning Oct 1, 2025
684a95a
feat: add functionality to close discussions in target repository if …
joshjohanning Oct 1, 2025
8776761
feat: add functionality to mark discussion comments as answers during…
joshjohanning Oct 1, 2025
a78ebd6
Merge branch 'main' into copy-discussions
joshjohanning Oct 1, 2025
6089e5d
feat: add support for copying poll data and visual representation in …
joshjohanning Oct 1, 2025
b280310
fix: incomplete string escaping
joshjohanning Oct 1, 2025
ee1e0cc
docs: update README to include detailed usage and features for copy-d…
joshjohanning Oct 1, 2025
2df58c9
fix: ensure linting commands do not fail the workflow by adding '|| t…
joshjohanning Oct 1, 2025
b3e800f
Revert "docs: update README to include detailed usage and features fo…
joshjohanning Oct 1, 2025
eb3f21a
docs: enhance README for copy-discussions.js with usage instructions …
joshjohanning Oct 1, 2025
ae3e30b
feat: enhance discussion copying by preserving reactions, locked stat…
joshjohanning Oct 1, 2025
118b323
Revert "fix: ensure linting commands do not fail the workflow by addi…
joshjohanning Oct 1, 2025
2024175
Merge branch 'main' into copy-discussions
joshjohanning Oct 1, 2025
afa4f9c
fix: correct formatting of metadata sections in discussions and comments
joshjohanning Oct 1, 2025
e0662b6
fix: update script documentation to clarify copied discussion elements
joshjohanning Oct 1, 2025
02cd024
feat: add copy-discussions.sh script for transferring discussions bet…
joshjohanning Oct 1, 2025
ac7e53c
Revert "feat: add copy-discussions.sh script for transferring discuss…
joshjohanning Oct 1, 2025
fbf843f
refactor: remove copy-discussions.sh script for transferring discussi…
joshjohanning Oct 1, 2025
8335272
fix: enhance help message for copy-discussions.js script
joshjohanning Oct 1, 2025
5610b6b
fix: improve help message and clarify environment variable usage in c…
joshjohanning Oct 1, 2025
e5b6475
feat: configurable rate limit
joshjohanning Oct 1, 2025
02b7fa7
fix: reduce rate limit sleep duration and improve discussion processi…
joshjohanning Oct 1, 2025
95187df
feat: add support for resuming discussion copying with --start-from o…
joshjohanning Oct 2, 2025
8bbdf32
fix: implement retry logic for discussion creation to handle rate limits
joshjohanning Oct 2, 2025
72c220b
docs: update documentation to clarify secondary rate limit guidelines…
joshjohanning Oct 2, 2025
6ea6480
feat: enhance rate limit handling with Octokit throttling and remove …
joshjohanning Oct 2, 2025
84e340c
fix: adjust discussion processing delay to comply with GitHub's rate …
joshjohanning Oct 2, 2025
6689cf3
feat: track primary and secondary rate limit hits for improved monito…
joshjohanning Oct 2, 2025
39d34ea
refactor(scripts)!: reorganize and rename copy-discussions to migrate…
joshjohanning Oct 2, 2025
8ec4d52
docs: update README to clarify npm installation steps and environment…
joshjohanning Oct 2, 2025
09b2928
docs: update README to enhance GitHub App usage instructions and clar…
joshjohanning Oct 2, 2025
6963384
docs: update README and script to enhance rate limit handling and ret…
joshjohanning Oct 2, 2025
726d14c
fix: update script usage examples to reflect correct script name
joshjohanning Oct 6, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
**/.DS_Store
*.pem
*.json
!/scripts/*/package*.json
node_modules*
test*.js
test*.sh
Expand Down
4 changes: 4 additions & 0 deletions scripts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,10 @@ My use case is to use this list to determine who needs to be added to a organiza
1. Run: `./new-users-to-add-to-project.sh <org> <repo> <file>`
2. Don't delete the `<file>` as it functions as your user database

## migrate-discussions

See: [migrate-discussions](./migrate-discussions/README.md)

## migrate-docker-containers-between-github-instances.sh

Migrate Docker Containers in GitHub Packages (GitHub Container Registry) from one GitHub organization to another.
Expand Down
170 changes: 170 additions & 0 deletions scripts/migrate-discussions/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
# migrate-discussions.js

Migrate GitHub Discussions between repositories, including categories, labels, comments, and replies. This script can migrate discussions across different GitHub instances and enterprises with comprehensive rate limit handling and resume capabilities.

## Prerequisites

- `SOURCE_TOKEN` environment variable with GitHub PAT that has `repo` scope and read access to source repository discussions
- Alternatively, use a GitHub App token (recommended for better rate limits and security)
- `TARGET_TOKEN` environment variable with GitHub PAT that has `repo` scope and write access to target repository discussions
- Alternatively, use a GitHub App token (recommended for better rate limits, security, and authorship) (✨ **recommended for target token!**)
- Dependencies installed via `npm i`
- Both source and target repositories must have GitHub Discussions enabled

### Using a GitHub App (Recommended)

GitHub Apps provide better rate limits and security compared to personal access tokens. To use a GitHub App:

1. Create or use an existing GitHub App with `repo` permissions
2. Install the app on the source and/or target repositories
3. Generate a token using the GitHub CLI and [`gh-token`](https://github.com/Link-/gh-token) extension:

```bash
export SOURCE_TOKEN=$(gh token generate --app-id YOUR_SOURCE_APP_ID --installation-id YOUR_SOURCE_INSTALLATION_ID --key /path/to/source/private-key.pem --token-only)
export TARGET_TOKEN=$(gh token generate --app-id YOUR_TARGET_APP_ID --installation-id YOUR_TARGET_INSTALLATION_ID --key /path/to/target/private-key.pem --token-only)
```

## Script usage

Basic usage:

```bash
export SOURCE_TOKEN=ghp_abc
export TARGET_TOKEN=ghp_xyz
# export SOURCE_TOKEN=$(gh token generate --app-id YOUR_SOURCE_APP_ID --installation-id YOUR_SOURCE_INSTALLATION_ID --key /path/to/source/private-key.pem --token-only)
# export TARGET_TOKEN=$(gh token generate --app-id YOUR_TARGET_APP_ID --installation-id YOUR_TARGET_INSTALLATION_ID --key /path/to/target/private-key.pem --token-only)
# export SOURCE_API_URL= # if GHES
# export TARGET_API_URL= # if GHES/ghe.com
cd ./scripts/migrate-discussions
npm i
node ./migrate-discussions.js source-org source-repo target-org target-repo
```

Resume from a specific discussion number (useful if interrupted):

```bash
node ./migrate-discussions.js source-org source-repo target-org target-repo --start-from 50
```

## Optional environment variables

- `SOURCE_API_URL` - API endpoint for source (defaults to `https://api.github.com`)
- `TARGET_API_URL` - API endpoint for target (defaults to `https://api.github.com`)

Example with GitHub Enterprise Server:

```bash
export SOURCE_API_URL=https://github.mycompany.com/api/v3
export TARGET_API_URL=https://api.github.com
export SOURCE_TOKEN=ghp_abc
export TARGET_TOKEN=ghp_xyz
node ./migrate-discussions.js source-org source-repo target-org target-repo
```

## Features

### Content Migration

- Automatically creates missing discussion categories in the target repository
- Creates labels in the target repository if they don't exist
- Copies all comments and threaded replies with proper attribution
- Copies poll results as static snapshots (with table and optional Mermaid chart)
- Preserves reaction counts on discussions, comments, and replies
- Maintains locked status of discussions
- Indicates pinned discussions with a visual indicator
- Marks answered discussions and preserves the accepted answer

### Rate limiting and reliability

- **Automatic rate limit handling** with Octokit's built-in throttling plugin
- **Intelligent retry logic** with configurable retries for both rate-limit and non-rate-limit errors
- **GitHub-recommended delays** - 3 seconds between discussions/comments to stay under secondary rate limits
- **Resume capability** - Use `--start-from <number>` to resume from a specific discussion if interrupted
- **Rate limit tracking** - Summary shows how many times primary and secondary rate limits were hit

### User experience

- Colored console output with timestamps for better visibility
- Comprehensive summary statistics at completion
- Detailed progress logging for each discussion, comment, and reply

## Configuration options

Edit these constants at the top of the script:

- `INCLUDE_POLL_MERMAID_CHART` - Set to `false` to disable Mermaid pie charts for polls (default: `true`)
- `RATE_LIMIT_SLEEP_SECONDS` - Sleep duration between API calls (default: `0.5` seconds)
- `DISCUSSION_PROCESSING_DELAY_SECONDS` - Delay between processing discussions/comments (default: `3` seconds)
- `MAX_RETRIES` - Maximum retries for both rate-limit and non-rate-limit errors (default: `15`)

## Summary output

After completion, the script displays comprehensive statistics:

- Total discussions found and created
- Discussions skipped (when using `--start-from`)
- Total comments found and copied
- **Primary rate limits hit** - How many times the script hit GitHub's primary rate limit
- **Secondary rate limits hit** - How many times the script hit GitHub's secondary rate limit
- List of missing categories that need manual creation

### Example summary output

```text
[2025-10-02 19:38:44] ============================================================
[2025-10-02 19:38:44] Discussion copy completed!
[2025-10-02 19:38:44] Total discussions found: 10
[2025-10-02 19:38:44] Discussions created: 10
[2025-10-02 19:38:44] Discussions skipped: 0
[2025-10-02 19:38:44] Total comments found: 9
[2025-10-02 19:38:44] Comments copied: 9
[2025-10-02 19:38:44] Primary rate limits hit: 0
[2025-10-02 19:38:44] Secondary rate limits hit: 0
[2025-10-02 19:38:44] WARNING:
The following categories were missing and need to be created manually:
[2025-10-02 19:38:44] WARNING: - Blog posts!
[2025-10-02 19:38:44] WARNING:
[2025-10-02 19:38:44] WARNING: To create categories manually:
[2025-10-02 19:38:44] WARNING: 1. Go to https://github.com/joshjohanning-emu/discussions-test/discussions
[2025-10-02 19:38:44] WARNING: 2. Click 'New discussion'
[2025-10-02 19:38:44] WARNING: 3. Look for category management options
[2025-10-02 19:38:44] WARNING: 4. Create the missing categories with appropriate names and descriptions
[2025-10-02 19:38:44]
All done! ✨
```

## Notes

### Category handling

- If a category doesn't exist in the target repository, discussions will be created in the "General" category as a fallback
- Missing categories are tracked and reported at the end of the script

### Content preservation

- The script preserves discussion metadata by adding attribution text to the body and comments
- Poll results are copied as static snapshots - voting is not available in copied discussions
- Reactions are copied as read-only summaries (users cannot add new reactions to the migrated content)
- Attachments (images and files) will not copy over and require manual handling

### Discussion states

- Locked discussions will be locked in the target repository
- Closed discussions will be closed in the target repository
- Answered discussions will have the same comment marked as the answer
- Pinned status is indicated in the discussion body (GitHub API doesn't allow pinning via GraphQL)

### Rate limiting

- GitHub limits content-generating requests to avoid abuse
- No more than 80 content-generating requests per minute
- No more than 500 content-generating requests per hour
- The script stays under 1 discussion or comment created every 3 seconds (GitHub's recommendation)
- Automatic retry with wait times from GitHub's `retry-after` headers
- If rate limits are consistently hit, the script will retry up to 15 times before failing

### Resume capability

- Use `--start-from <number>` to skip discussions before a specific discussion number
- Useful for resuming after interruptions or failures
- Discussion numbers are the user-friendly numbers (e.g., #50), not GraphQL IDs
Loading