Skip to content

Repository files navigation

YouTube Subscription Extractor

Python YouTube Data API Platform License

Collect the YouTube subscriptions from all your Google accounts, combine them into one clean list, and — if you want — subscribe a brand-new account to every channel on that list.

Imagine you have five or six Google accounts, each subscribed to different YouTube channels, collected over many years. One day you decide: "I want all of these in one place." This project is the complete toolkit for exactly that journey. It is a small, friendly set of command-line programs — no servers, no databases, nothing leaves your computer except the official calls to YouTube itself.

This README is the single source of truth for the project. Everything you need — setup, configuration, daily use, architecture, troubleshooting, and contribution rules — lives in this one file.


📋 Table of Contents


The Story in One Picture

The project is three small programs that work as a pipeline. You can stop after step 1 or 2 — step 3 is optional.

flowchart LR
    A[("youtube_accounts.csv<br>your list of accounts")] --> B["Step 1 · Extract<br>run.bat / run.sh"]
    B --> C[("output/channels_&lt;account&gt;.csv<br>one file per account")]
    C --> D["Step 2 · Merge<br>merge.bat / merge.sh"]
    D --> E[("output/merged_channels.csv<br>every channel, listed once")]
    E --> F["Step 3 · Subscribe<br>subscribe.bat / subscribe.sh"]
    F --> G["Target account from .env<br>now follows every channel"]

    style B fill:#e8f0fe,stroke:#4285f4,color:#174ea6
    style D fill:#e6f4ea,stroke:#34a853,color:#0d652d
    style F fill:#fce8e6,stroke:#ea4335,color:#a50e0e
Loading
  1. Extract — signs in to each Google account in your CSV list (through a normal browser login) and downloads its full YouTube subscription list.
  2. Merge — combines all the per-account files into one consolidated list, removes duplicate channels, and records which accounts subscribe to each channel.
  3. Subscribe — signs in as one target account (set in .env) and subscribes it to every channel in the merged list, skipping channels it already follows.

Features

  • Multi-account extraction — process any number of Google accounts in a single run, listed in one CSV file.
  • Full channel details — channel ID, name, topic categories (Music, Knowledge, Gaming, …), and a direct link for every subscription.
  • Automatic pagination — hundreds of subscriptions per account are handled without any extra steps.
  • Smart merging with de-duplication — one consolidated list where every channel appears exactly once, with a Subscribed By column showing which of your accounts follow it.
  • Bulk subscribe — point a target account at the merged list and let the program subscribe it to everything, safely and resumably.
  • Secure OAuth 2.0 sign-in — the official Google login in your own browser, including your normal 2-step verification. Passwords are never seen or stored by this project.
  • Remembered logins — after the first sign-in per account, tokens are stored locally and reused; later runs are hands-free.
  • Self-healing authentication — if a saved token expires and cannot be refreshed, the program automatically falls back to a fresh browser sign-in instead of failing.
  • .env-based configuration — all paths and the subscribe target live in one small config file; no questions asked at run time.
  • Cross-platform.bat launchers for Windows, .sh launchers for macOS and Linux, plain Python underneath.

Architecture Overview

The code follows a simple rule: one module, one job. Launcher scripts read your .env settings and call the right Python entry point; the entry points share three helper modules.

flowchart TD
    ENV[".env<br>paths and target account"] --> RB & MB & SB

    subgraph Launchers["Launchers (Windows .bat / Unix .sh)"]
        RB["run"]
        MB["merge"]
        SB["subscribe"]
    end

    RB --> YE["src/youtube_extractor.py<br>extraction entry point"]
    MB --> MC["src/merge_channels.py<br>merge entry point"]
    SB --> SC["src/subscribe_channels.py<br>subscribe entry point"]

    YE --> AUTH["src/auth.py<br>OAuth 2.0 sign-in,<br>token storage and refresh"]
    YE --> API["src/youtube_api.py<br>subscription fetching,<br>channel categories"]
    YE --> CSVH["src/csv_handler.py<br>CSV reading and export"]
    SC --> AUTH
    SC --> MC

    AUTH --> SECRETS[("secret/<br>credentials_config.json<br>token_*.pickle")]
    API --> YT["YouTube Data API v3"]
    SC --> YT
    CSVH --> OUT[("output/<br>channels_*.csv<br>merged_channels.csv")]
    MC --> OUT

    style YT fill:#fce8e6,stroke:#ea4335,color:#a50e0e
    style SECRETS fill:#fef7e0,stroke:#f9ab00,color:#7c4a03
Loading
Module Responsibility
src/youtube_extractor.py Orchestrates extraction: load config → read accounts → authenticate → fetch → export
src/merge_channels.py Reads all channels_*.csv files, de-duplicates by Channel ID, writes the merged file
src/subscribe_channels.py Reads the merged file and subscribes the target account to each channel
src/auth.py Everything OAuth: client selection per account, browser flow, token save/refresh, helpful error messages
src/youtube_api.py YouTube Data API calls: subscription pages, channel topic categories
src/csv_handler.py Input CSV parsing (with header detection) and per-account CSV export

Prerequisites

Before you begin, you need:

  • Python 3.7 or higher — check with python --version; download from https://www.python.org/downloads/ if needed.
  • An internet connection — the programs talk to the official YouTube Data API.
  • The Google accounts you want to read from — you must be able to log in to each one in a browser (including its 2-step verification).
  • A free Google Cloud project for the OAuth credentials — created once, explained step by step below. No payment is required; the API's free daily quota is enough.

Installation

  1. Download or clone this project onto your computer.

  2. Open a terminal in the project folder

    • Windows: Shift + Right-click in the folder → "Open PowerShell window here"
    • macOS/Linux: Right-click → "Open Terminal here"
  3. Install the three required libraries:

    pip install -r requirements.txt

    That installs google-api-python-client, google-auth-oauthlib, and google-auth-httplib2 — the official Google libraries. There are no other dependencies.

    💡 The launcher scripts (run.bat, run.sh, subscribe.bat, subscribe.sh) check for these libraries and install them automatically if missing.


One-Time Google Cloud Setup

This is the only technical part, and you do it once. In plain words: Google will not let any program read your YouTube data unless you first create your own "app registration" (an OAuth client) in Google Cloud and explicitly approve the accounts allowed to use it.

Step 1 — Create a project and enable the API

  1. Go to https://console.cloud.google.com/ and sign in.
  2. Click the project selector at the top → New project → give it any name → Create.
  3. In the menu, open APIs & Services → Library, search for "YouTube Data API v3", open it, and click Enable.

Step 2 — Configure the consent screen and add Test Users

  1. In the menu, open Google Auth Platform (older UI: APIs & Services → OAuth consent screen).
  2. Under Branding, set an app name and your support/developer email. Choose user type External.
  3. Open Audience and find the Test users section. Click + Add users and add every email address you plan to sign in with through this project. Save.

⚠️ This is the step people miss. While the app's publishing status is "Testing", Google blocks any account that is not on the Test users list with the error "Access blocked … Error 403: access_denied"before it even asks for a password. 2-step verification on your phone cannot help, because the sign-in never gets that far.

Step 3 — Add the data scopes

  1. Open Data access and click Add or remove scopes.
  2. For extracting subscriptions, add: .../auth/youtube.readonly (View your YouTube account).
  3. For the optional subscribe program, also add: .../auth/youtube (Manage your YouTube account) — subscribing is a write operation and needs this broader permission.

Step 4 — Create the Desktop OAuth client

  1. Open Clients (older UI: APIs & Services → Credentials) → Create client.
  2. Application type: Desktop app. Give it a name → Create.
  3. Copy the Client ID and Client secret (or download the JSON). You will paste these into the project's config file next.

💡 One project or many? You can use a single Google Cloud project for all your accounts (add every email as a Test user on it), or create one project per account. This application supports both — the config file maps each email to the OAuth client it should use.

Optional but recommended — Publish the app

On the Audience page, click Publish app to switch from "Testing" to "In production". You will see a one-time "Google hasn't verified this app" warning at sign-in (click Advanced → continue; it is your own app), and in exchange:

  • the Test users list is no longer needed, and
  • saved logins stop expiring every 7 days (a Testing-mode restriction), so re-authentication becomes rare.

Configuration

Three small files control everything. All of them stay on your computer and are excluded from git.

1. .env — paths and the subscribe target

Copy .env.example to .env and adjust:

# Path to the CSV file with the list of account email IDs
ACCOUNTS_FILE=youtube_accounts.csv

# Folder where per-account subscription files (channels_*.csv) are written
OUTPUT_DIR=output

# Consolidated, de-duplicated file created by merge.bat / merge.sh
MERGED_FILE=output/merged_channels.csv

# Target account that subscribe.bat / subscribe.sh will subscribe
# to every channel in the merged list
SUBSCRIBE_ACCOUNT=your.account@gmail.com

Relative or absolute paths both work. With .env in place, the programs run without asking any questions.

2. youtube_accounts.csv — the accounts to read from

One email per line. A header line like username or email is detected and skipped automatically.

username
first.account@gmail.com
second.account@gmail.com

3. secret/credentials_config.json — OAuth clients and the account map

This file holds the OAuth client(s) you created in Google Cloud, plus a map that tells the program which client each email uses:

{
    "my_youtube_client": {
        "client_id": "YOUR_CLIENT_ID.apps.googleusercontent.com",
        "project_id": "your-project-id",
        "auth_uri": "https://accounts.google.com/o/oauth2/auth",
        "token_uri": "https://oauth2.googleapis.com/token",
        "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
        "client_secret": "YOUR_CLIENT_SECRET",
        "redirect_uris": ["http://localhost"]
    },
    "account_to_client_mapping": {
        "first.account@gmail.com": "my_youtube_client",
        "second.account@gmail.com": "my_youtube_client"
    }
}

Rules the program enforces:

  • Every email must be mapped. An unmapped email stops with a clear error instead of silently using the wrong client (which would cause a confusing "Access blocked" screen from the wrong project).
  • As an escape hatch, a client named default_client (if you add one) is used for any unmapped email.
  • The SUBSCRIBE_ACCOUNT from .env must be mapped here too, and its client's project must have the Manage your YouTube account scope.

Environment Variables

All variables live in the .env file at the project root. The launcher scripts load it and pass values on; the Python entry points also read it directly, so python src/... works standalone too. Real environment variables (set in your shell) take precedence over .env values in the Python programs.

Variable Used by Meaning Default if missing
ACCOUNTS_FILE run Path to the CSV of account emails The launcher asks you to type a path
OUTPUT_DIR run, merge Folder for the per-account CSV files output
MERGED_FILE merge, subscribe Path of the consolidated CSV output/merged_channels.csv
SUBSCRIBE_ACCOUNT subscribe Email that will be subscribed to every merged channel None — subscribe refuses to run without it

How to Run — The Three Programs

Each program has a Windows launcher (.bat, just double-click) and a macOS/Linux launcher (.sh). All of them read their settings from .env.

Step 1 · Extract subscriptions

Platform Command
Windows .\run.bat (or double-click it)
macOS / Linux ./run.sh
Direct Python python src/youtube_extractor.py youtube_accounts.csv

What you will see, account by account:

  1. If a saved login exists and still works → subscriptions download immediately, no questions.
  2. Otherwise a browser window opens. Sign in with that account (your usual password and 2-step verification), click through the "Google hasn't verified this app" warning if the app is unpublished (Advanced → continue), and click Allow.
  3. The program saves the login token in secret/ and writes output/channels_<account>.csv.

💡 Keep the terminal open while the browser is up — the sign-in hands control back to the program through a local address (http://localhost), so the browser must be on the same machine.

Step 2 · Merge into one list

Platform Command
Windows .\merge.bat
macOS / Linux ./merge.sh
Direct Python python src/merge_channels.py [output_dir] [merged_file]

No sign-in needed — this is pure file work. Every channels_*.csv in the output folder is combined; duplicates are removed by Channel ID; the result is sorted by channel name and written to MERGED_FILE. Channels followed by several accounts get them all listed in the Subscribed By column, so no information is lost.

Step 3 · Subscribe a target account (optional)

Platform Command
Windows .\subscribe.bat
macOS / Linux ./subscribe.sh
Direct Python python src/subscribe_channels.py [email] [merged_file]
Preview only python src/subscribe_channels.py --dry-run

The program signs in as SUBSCRIBE_ACCOUNT (asking for the broader Manage your YouTube account permission — that token is stored separately as token_<email>_manage.pickle), checks which channels the account already follows, and subscribes it to every remaining channel in the merged list.

sequenceDiagram
    autonumber
    participant You
    participant Program as subscribe program
    participant YouTube as YouTube Data API

    You->>Program: start subscribe.bat / subscribe.sh
    Program->>YouTube: sign in as SUBSCRIBE_ACCOUNT (manage scope)
    Program->>YouTube: list current subscriptions
    loop every channel in merged_channels.csv
        alt already subscribed
            Program->>Program: skip (no quota used)
        else new channel
            Program->>YouTube: subscribe (costs 50 quota units)
        end
    end
    Program-->>You: summary — subscribed / skipped / failed
    Note over Program: if the daily quota runs out, it stops safely.<br>Run it again tomorrow — it continues where it left off.
Loading

⚠️ Quota planning: each new subscription costs 50 API units and a Google Cloud project has 10,000 free units per day — roughly 200 new subscriptions per day. The program detects the quota limit, stops cleanly, and tells you how many channels are left. Because already-subscribed channels are skipped, simply re-running it the next day finishes the job.


How Authentication Works

You never type a password into this project. Sign-in happens on Google's own pages in your browser; the project only receives a token — a revocable permission slip scoped to YouTube data.

flowchart TD
    S["Authenticate an account"] --> T{"Saved token in secret/ ?"}
    T -- "yes, still valid" --> OK["✔ Signed in silently"]
    T -- "expired, refreshable" --> R{"Refresh succeeds?"}
    R -- yes --> OK
    R -- "no (revoked or 7-day testing expiry)" --> B
    T -- "no token yet" --> B["Open browser for Google sign-in"]
    B --> L["You enter password + 2-step verification<br>and approve the requested access"]
    L --> SAVE["Token saved to secret/token_&lt;email&gt;.pickle"]
    SAVE --> OK

    style OK fill:#e6f4ea,stroke:#34a853,color:#0d652d
Loading

Key facts:

  • Which OAuth client is used for an email is decided by account_to_client_mapping in secret/credentials_config.json.
  • Tokens are per account and per permission level. Extraction tokens (token_<email>.pickle, read-only) and subscribe tokens (token_<email>_manage.pickle, manage) live side by side in secret/.
  • Google blocks the app before the login when the account is not a Test user of the OAuth project (in Testing mode). That is the "Access blocked … 403: access_denied" screen — fix it in Google Cloud, not on your phone.
  • Testing-mode logins expire after 7 days. The program then falls back to a fresh browser sign-in automatically. Publishing the app to production removes this expiry.

Understanding the Output Files

output/channels_<account>.csv — one per extracted account

Column Meaning
Channel ID YouTube's unique ID for the channel (starts with UC…)
Channel Name The channel's display name
Category Topic categories from YouTube (e.g. Music, Knowledge), up to three, or General
Type Always Channel
Channel Link Direct URL to the channel
New to List Reserved marker, currently always Yes

output/merged_channels.csv — the consolidated list

All columns above, plus:

Column Meaning
Subscribed By Which of your accounts follow this channel, separated by ; (e.g. alice; bob)

Every channel appears exactly once, sorted by name. The file is rebuilt from scratch on every merge, so it is always in sync with the per-account files.


Project Structure

youtube-subscription-extractor/
│
├── src/                           # All Python source code
│   ├── youtube_extractor.py       # Entry point 1: extract subscriptions
│   ├── merge_channels.py          # Entry point 2: merge + de-duplicate
│   ├── subscribe_channels.py      # Entry point 3: bulk subscribe
│   ├── auth.py                    # OAuth 2.0 sign-in and token handling
│   ├── youtube_api.py             # YouTube Data API calls
│   └── csv_handler.py             # CSV reading and writing
│
├── secret/                        # 🔒 Private — never committed
│   ├── credentials_config.json    # OAuth clients + account-to-client map
│   ├── token_<email>.pickle       # Saved read-only logins
│   └── token_<email>_manage.pickle  # Saved manage-scope logins (subscribe)
│
├── output/                        # 🔒 Generated data — never committed
│   ├── channels_<account>.csv     # One file per extracted account
│   └── merged_channels.csv        # Consolidated, de-duplicated list
│
├── .github/workflows/             # CI pipelines (tests, security, docs)
├── .env                           # 🔒 Your local configuration
├── .env.example                   # Template for .env
├── youtube_accounts.csv           # 🔒 Your account list
├── youtube_accounts.example.csv   # Template for the account list
├── requirements.txt               # The three Google API libraries
├── run.bat / run.sh               # Launcher: extract
├── merge.bat / merge.sh           # Launcher: merge
├── subscribe.bat / subscribe.sh   # Launcher: subscribe
├── LICENSE                        # MIT license
└── README.md                      # This file — the only documentation

Files marked 🔒 are listed in .gitignore and never leave your machine.


Coding Conventions and Standards

Keep contributions consistent with the existing style:

  • One module, one job. Entry points orchestrate; helpers (auth, youtube_api, csv_handler) do the work. New user-facing features also get a launcher pair (.bat + .sh).
  • Docstrings everywhere. Every module and function has a docstring with Args: / Returns: / Raises: sections, and type hints on signatures.
  • Friendly console output. Status lines use the established symbols ( success, skipped, ⚠️ warning, error) and always tell the user what to do next when something fails.
  • Windows-safe printing. Entry points call sys.stdout.reconfigure(errors='replace') so legacy consoles degrade to ? instead of crashing on symbols.
  • UTF-8 for all file I/O, newline='' for CSV writers.
  • No new dependencies beyond the three Google libraries without a strong reason.
  • Never handle secrets casually — temporary credential files go to the system temp directory, tokens stay inside secret/.
  • Lint targets (matching CI): flake8 with max line length 127 and max complexity 10.

Testing

There is no automated unit-test suite yet (a welcome contribution!). Today, verification is done like this:

  1. Syntax-check all sources (exactly what CI runs):

    python -m py_compile src/*.py
  2. Dry-run the subscriber — parses .env, the config, and the merged file without calling YouTube:

    python src/subscribe_channels.py --dry-run
  3. Small-scale live test — put a single account in a test CSV, run the extractor against it, then merge, and inspect the CSVs in output/.

  4. Lint and security scan (optional, mirrors CI):

    pip install flake8 bandit
    flake8 src/ --max-line-length=127 --max-complexity=10
    bandit -r src/ -ll

CI/CD

Three GitHub Actions workflows run on every push and pull request to main (see .github/workflows/):

Workflow What it does
tests.yml Compiles all sources (py_compile) on Windows, macOS, and Linux across Python 3.7–3.11; runs flake8 and a bandit scan; verifies the folder structure and that no credentials file sits in the repository root
security.yml Runs bandit and dependency vulnerability checks (also weekly on a schedule) and validates that no secrets are committed
documentation.yml Verifies README.md and LICENSE exist, the Markdown is well-formed, and the required README sections are present

There is no deployment step — this is a local tool, not a hosted service. "Releasing" means merging to main; users pull the repository and run it directly.


Troubleshooting

🔴 "Access blocked: … has not completed the Google verification process — Error 403: access_denied"

The signed-in account is not a Test user of the OAuth project it is mapped to (and the app is in Testing mode). Fix: Google Cloud Console → select the right project → Google Auth Platform → AudienceTest users+ Add users → add that email → Save → run again. Or click Publish app on the same page to stop needing test users at all. Tip: the app name shown on the error page tells you which project's consent screen you hit — check that it matches the project configured for that email in credentials_config.json.

🔴 "No OAuth client is mapped to account: …"

The email is missing from account_to_client_mapping in secret/credentials_config.json. Add a line mapping it to one of your configured clients.

🔴 "invalid_grant: Bad Request" during token refresh

The saved token is dead — most often the 7-day Testing-mode expiry. The program automatically falls back to a browser sign-in; if it keeps happening, publish the app to production (see the setup section).

🔴 "invalid_client" at sign-in

The client secret in credentials_config.json no longer matches Google Cloud (for example, the secret was rotated). Copy the current secret from Clients → your client into the config file.

🔴 "quotaExceeded" while subscribing

You used up the project's 10,000 free daily units (about 200 new subscriptions). The program stops cleanly and reports how many channels remain. Re-run it after the quota resets (midnight Pacific time) — completed channels are skipped automatically.

🟡 The browser doesn't open automatically

Copy the URL printed in the console into a browser on the same machine and finish the sign-in there. Keep the terminal running.

🟡 "No module named 'google'"

Dependencies missing: pip install -r requirements.txt (the launchers normally do this for you).

🟡 Status symbols print as ? in the terminal

Cosmetic only — your Windows console uses a legacy code page. The CSV files themselves are correct UTF-8.

🟡 A stale login misbehaves

Delete that account's token_*.pickle file in secret/ and run again — you will get a fresh sign-in.


Known Limitations

  • Sign-in is interactive by design. Each account's first login (and any login after a token dies) needs a human in the browser — password, 2-step verification, consent. There is no headless or fully unattended mode, and automating Google's login is not supported.
  • Subscribing is quota-bound: about 200 new subscriptions per project per day (50 units each from a 10,000-unit free quota). Large merged lists take multiple days; the program resumes automatically.
  • Testing-mode apps expire logins every 7 days and only allow listed test users (maximum 100). Publishing the app removes both limits, at the cost of a one-time "unverified app" warning per sign-in.
  • CSV only. No OPML or JSON export yet.
  • No unsubscribe or sync-back. The subscriber only adds subscriptions; it never removes any.
  • New to List is a static column (always Yes) — cross-run change tracking is not implemented.

Security

  • Your passwords are never seen or stored. Sign-in happens on Google's pages; the project stores only OAuth tokens, locally, in secret/.
  • Everything sensitive is git-ignored: secret/, output/, .env, and youtube_accounts.csv never leave your machine.
  • Least privilege by default: extraction uses the read-only YouTube scope; only the optional subscribe program asks for the manage scope, under a separate token.
  • Temporary credential files are written to the operating system's temp folder (never into the project) and deleted immediately after use.
  • Revoke at any time: remove the app's access at https://myaccount.google.com/permissions, or delete the token files in secret/.
  • CI guards: the security workflow scans the code (bandit) and dependencies, and fails if a credentials file appears where it should not.

If you discover a security issue, please open a GitHub issue without including secrets or tokens, and describe the problem and reproduction steps.


Contributing

Contributions are welcome — this project intentionally stays small and readable.

  1. Fork and branch from main (for example feature/opml-export or fix/quota-message).
  2. Follow the conventions in Coding Conventions and Standards.
  3. Verify before you push: python -m py_compile src/*.py, a real or dry run of the affected program, and flake8 (CI runs all of these).
  4. Keep secrets out — never commit anything from secret/, output/, .env, or real account emails.
  5. Update this README in the same pull request when behavior, configuration, or workflows change. It is the only documentation file — there is nothing else to update.
  6. Open a pull request describing what changed, why, and how you tested it (platforms, single vs. multiple accounts).

Good first contributions: a pytest suite, OPML/JSON export, real change-tracking for New to List.


FAQ

Q: Does this cost money? No. The YouTube Data API's free quota (10,000 units/day per project) is plenty for extraction, which costs roughly 1 unit per channel plus 1 per page of 50 subscriptions. Only bulk subscribing is expensive (50 units each) — that is a time limit, not a bill.

Q: Can it modify my YouTube accounts? The extractor cannot — it authenticates read-only. The optional subscribe program can do exactly one write action: add subscriptions to the target account you configure. Nothing is ever deleted.

Q: Why do I have to log in through a browser? Can't it just use my phone's 2-step verification? Google requires the full interactive sign-in the first time — password, then your normal 2-step verification (the phone prompt or SMS happens right there in the flow). After that, the saved token keeps future runs hands-free.

Q: Can I extract someone else's subscriptions? Only if they sign in themselves during the run and their email is a test user of your OAuth project (or the app is published). Consent is built into the flow.

Q: What if an account has hundreds of subscriptions? Fine — results are fetched in pages of 50 automatically until done.

Q: Several of my accounts follow the same channels. What happens when I merge? Each channel appears once in the merged file, and the Subscribed By column lists every account that follows it.

Q: It worked for months and then asked me to log in again. Normal — tokens die eventually (always after 7 days while the app is in Testing mode). The program simply sends you through the browser sign-in once more.


License

This project is released under the MIT License — see the LICENSE file. In short: use, modify, and share freely; no warranty.


Happy extracting! 🚀