gsearch

Scrape the rich content of a Google Search results page from the command line. No API key, no SerpApi bill: gsearch drives a real Chrome through patchright, waits for the page to render, and pulls every structured block Google shows: knowledge panels, sports tables, weather, translation, the dictionary, the calculator, stock and crypto quotes, featured snippets, People Also Ask, AI Overviews, and the organic results underneath.

gsearch search "epl"

──────────────── Google Search - 'epl' ────────────────
  response: 2143ms  |  features: Sports Standings, Knowledge Panel
  organic: 11  |  paa: 4  |  stories: 3

Standings:  Premier League
  #   Team            P    W   D   L   Pts
  1   Liverpool       29   21  6   2   69
  ...

Full documentation: gsearch.tamnd.com.

Why

The data Google puts above the ten blue links, the answer box, the score table, the weather card, the panel on the right, is the most useful part of a search and the hardest to get at. The paid SERP APIs cover it, but you pay per query and trust their parse. gsearch renders the same page you would see in a browser and reads the live DOM, so you get the rich blocks and the organic results in one structured object, on your own machine, for free.

It is one tool with a small command surface, JSON and Markdown output that composes, and a DuckDB store when you want to keep what you pull.

Install

gsearch is a Python package. The fastest way to run it is with uv:

uv tool install gsearch
gsearch search "weather london"

Or from source:

git clone https://github.com/tamnd/gsearch
cd gsearch
make sync          # create .venv and install
make browser       # download the Chromium build patchright drives (once)
uv run gsearch search "epl"

The one runtime requirement beyond the Python deps is a Chrome/Chromium build for patchright to drive. make browser (or uv run patchright install chromium) installs a private copy; if you already have Google Chrome, gsearch uses it via the chrome channel.

Quick start

gsearch search "epl"                        # all rich blocks + organic results
gsearch search "12.5 km in miles"           # the unit converter card
gsearch search "define ephemeral"           # the dictionary entry
gsearch search "AAPL stock" --json          # structured JSON to stdout
gsearch search "python tutorial" -v videos  # the videos vertical
gsearch search "running shoes" -v shopping  # the shopping vertical
gsearch search "real madrid" --save         # also persist to DuckDB

Every run writes a JSON and a Markdown file under ~/data/gsearch/ by default (turn that off with --no-export). Add --no-headless to watch the Chrome window, useful the first time, when Google may show a consent or CAPTCHA page you solve once and the persistent profile remembers.

How it works

1. Launch Google Chrome through patchright, which patches the CDP-level tells that mark automated browsers.
2. Use a persistent profile (~/.cache/gsearch-profile) so a returning-user session avoids the CAPTCHA after the first solve.
3. Navigate to the ordinary https://www.google.com/search URL for the query and vertical, wait for the results to render.
4. Run extract.js against the live DOM. That single script is the heart of the tool: ~2,900 lines of resilient selectors that turn each Google block into a clean object.
5. Hand the result back to Python for rendering, JSON/Markdown export, and optional DuckDB storage.

Because it reads the rendered page rather than a private API, there are no keys, no quotas, and the output tracks what a human actually sees.

Commands

Command	What it does
search	Fetch all rich content for a query (the main command)
info	Show DuckDB statistics and a feature breakdown
dump	Print recent searches stored in the database
export	Re-export recent stored results to JSON + Markdown
capture	Save the raw AJAX/XHR responses Google makes while loading a SERP

Run gsearch <command> --help for the full flag list on any command.

Verticals

search -v/--vertical switches the tab. Each vertical has its own extractor and its own output shape:

Vertical	Returns
web (default)	Rich feature blocks + organic results
images	Image cards with source, dimensions, and thumbnails
videos	Video results with channel, duration, views
news	Articles and top-stories clusters
shopping	Products with price, store, rating, delivery
books	Books with author, year, rating, publisher

gsearch search "cats" -v images
gsearch search "climate change" -v news
gsearch search "python programming" -v books

Output

By default search prints a readable summary, and also writes two files:

~/data/gsearch/json/2026/06/13/22-41-epl.json
~/data/gsearch/markdown/2026/06/13/22-41-epl.md

• --json prints the full structured object to stdout (and is the format to pipe into jq or another program).
• --no-export suppresses the file output.
• --save additionally upserts the result into DuckDB, keyed by query.

The JSON object carries detected_features, a features map (one entry per block kind), organic_results, people_also_ask, related_searches, top_stories, local_pack, and, for non-web verticals, vertical_results.

gsearch search "AAPL stock" --json | jq '.features.stock'

Storing and revisiting results

With --save, each result lands in a DuckDB database (~/data/gsearch/gsearch.duckdb, or --db / GSEARCH_DB). The store keeps the full blobs plus boolean flags for quick filtering, so you can see what kinds of pages you have collected and re-export them later:

gsearch search "real madrid" --save
gsearch search "weather tokyo" --save
gsearch info                 # feature breakdown across everything stored
gsearch dump -n 5            # the five most recent, rendered
gsearch export -n 20         # rewrite the latest 20 to json + markdown

Capturing Google's internal API

capture records the raw batchexecute / /_/search/ JSON responses Google fires while the page loads, one file per response, for offline analysis:

gsearch capture "epl" --no-headless --out /tmp/epl-capture

This is a research aid for understanding Google's internal structure; everyday extraction does not need it.

Configuration

gsearch keeps its state under one tree, ~/data/gsearch by default: the exported JSON and Markdown, and the DuckDB file. A few knobs:

Flag / env	Meaning
--db, GSEARCH_DB	DuckDB path (default ~/data/gsearch/gsearch.duckdb)
--profile-dir	Chrome persistent profile (default ~/.cache/gsearch-profile)
--no-profile	Use a fresh ephemeral profile instead
--headless/--no-headless	Show or hide the Chrome window (default headless)
--lang, --country	Search language (hl) and country (gl)
--pages, -p	Follow "Next" up to N result pages
--timeout	Browser wait timeout in seconds

A note on use

gsearch automates a browser against Google Search. It is meant for personal, low-volume research and learning. Respect Google's terms of service, keep the rate gentle, and do not point it at anything you are not allowed to scrape. If the IP gets rate-limited you will see a CAPTCHA page; run once with --no-headless, solve it, and the persistent profile carries the session forward.

Development

make sync     # create .venv, install project + dev deps
make browser  # install the Chromium build (once)
make test     # run the offline test suite
make lint     # ruff
make build    # wheel + sdist into dist/

The package is small and layered. cli.py is the Typer command tree and the terminal renderer. browser.py drives patchright and owns the CAPTCHA-recovery loop. extract.js is the DOM extractor that runs inside the page. exporter.py writes the JSON and Markdown files. store.py is the DuckDB layer.

License

Apache 2.0.

This project is an independent client. It is not affiliated with, endorsed by, or sponsored by Google. "Google" is a trademark of Google LLC.