myrient-scrape

A Python3 CLI tool to scrape, list, and sync retro video game files from Myrient for ArkOS retro gaming handheld devices.

About

Myrient is a video game preservation website hosting over 330 TB of downloadable game collections. It provides free, unrestricted downloads without registration, ads, or paywalls, with fast speeds and download manager support.

ArkOS is a custom Linux distribution for retro gaming handhelds. This tool's console folder structure aligns with ArkOS's EASYROMS naming conventions.

Features

• Scrape: Parse Myrient's folder structure and generate markdown checklists of available game files organized by console
• Track: Mark desired files with checkboxes in console-specific markdown files
• Search: Interactive search across all consoles to find and mark games
• Sync: Automatically download checked files to organized directories
• Rename: Specify custom filenames for downloaded files
• Best-of Lists: Auto-mark curated collections of highly-rated classic games
• Version Filtering: Smart filtering to show preferred regions (World/Europe/USA)
• Minimal Dependencies: Built with Python3 standard library + requests and pick

Quick Start

Scrape Myrient

# Full scrape of entire archive
uv run scrape

# Targeted scrape of specific folder
uv run scrape /No-Intro/Nintendo%20-%20Nintendo%20DS%20%28Decrypted%29/

Mark Files for Download

Edit the generated markdown files in ./lists/ and check the boxes for files you want:

- [x] game-i-want.zip
- [ ] game-i-dont-want.zip
- [x] another-game.7z ((My Custom Name))

Search for Games

# Interactive search across all consoles
uv run search

Enter a search term (e.g., "mario", "zelda", "pokemon") and use arrow keys to navigate results. Press Enter to mark/unmark games.

Sync Downloads

# Download missing checked files
uv run sync

# Re-download all checked files
uv run sync --full

Advanced Usage

Auto-Mark Best-of Games

Quickly mark curated lists of highly-rated classic games:

# Mark best Nintendo DS games
python mark-best-games.py

# Mark best games for GB, GBA, NES, SNES, N64, PSX, PSP
python mark-all-best-games.py

These scripts use curated game lists from sources like Metacritic, Nintendo Life, and Backloggd. See best-of.md for the full Nintendo DS list.

Discover Myrient Structure

Generate an initial console mapping file:

uv run python discover_consoles.py

This one-time setup script:

1. Scrapes Myrient's folder structure
2. Auto-detects common console paths
3. Generates console-mapping-discovered.json
4. You review and rename to console-mapping.json

Configuration

Console Mapping

The console-mapping.json file maps Myrient paths to local console folders. The console_id values align with ArkOS's EASYROMS folder naming conventions.

See CONSOLE-FOLDERS.md for the complete list of valid target console folders.

Example mapping:

{
  "myrient_path": "/No-Intro/Nintendo - Game Boy",
  "console_id": "gb",
  "console_name": "Game Boy"
}

Downloads go to ./downloads/{console_id}/ which can then be synced to your ArkOS device's EASYROMS folder.

Documentation

• CONSOLE-FOLDERS.md - Complete list of valid ArkOS console folders
• SPEC.md - Detailed specification and implementation details
• TESTING.md - Test suite documentation (81 tests)
• CLAUDE.md - AI development context and architecture notes