Skip to content

sixtdreanight/ComiRadar

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

41 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Language: English | 简体中文 | 繁體中文 | 日本語

ComiRadar

CI Python License Status

Automatically discovers anime cons, doujin fairs, and ACG concerts across China for the next 90 days.

Scrapes 9 ticketing platforms (Bilibili, Damai, Showstart, Maoyan, Piaoxingqiu, Yongle, etc.) plus Weibo, Bilibili feeds, and Xiaohongshu. Each platform has its own parser, unified into a normalized event schema.

Data Sources

Bilibili (会员购) · Damai (大麦) · Showstart (秀动) · Maoyan (猫眼) · Piaoxingqiu (票星球) · Yongle (永乐) · Weibo (微博) · Bilibili Dynamics · Xiaohongshu (小红书)

Deduplication Strategy

Two-layer dedup: SHA256 fingerprint for exact matches + fuzzy merge on city/date/title for near-duplicates. Merged events combine fields from multiple sources.

Usage

pip install -r requirements.txt
python main.py scrape   # Scrape all platforms
python main.py export   # Export events.json
python main.py notify   # Push notifications (configure keys first)
python main.py run      # All three steps

Docker

# Build and run the full pipeline
docker compose run --rm comiradar

# Run individual steps
docker compose run --rm scrape   # scrape only
docker compose run --rm export   # export only
docker compose run --rm notify   # notifications only

# Data persists in ./data/

Configuration

Edit config.py to set notification keys:

NOTIFIERS = {
    "serverchan": {"key": "your-SendKey"},
    "bark": {"url": "https://api.day.app/your-key"},
}

Environment Variables

Variable Description
BILI_BUVID3 Bilibili cookie buvid3
BILI_SESSDATA Bilibili cookie SESSDATA
BILI_BILI_JCT Bilibili cookie bili_jct
DAMAI_M_H5_TK Damai cookie m_h5_tk
DAMAI_COOKIE2 Damai cookie cookie2
DEEPSEEK_API_KEY DeepSeek API key (for AI info extraction)
COMI_EXPORT_PATH Export path (optional, defaults to project directory)

Blog Integration

This repo is included as a git submodule in Blog-mizuki. GitHub Actions runs the scraper on schedule, and results auto-sync to the events page.

Tech Stack

  • Scraping: Python 3.11+ · Playwright · HTTPX
  • Storage: SQLite
  • Scheduling: GitHub Actions (cron)
  • Integration: Git submodule → Astro blog

Related

  • chinese-scraper-utils — Shared utilities extracted from this project
  • Blog-mizuki — Events data displayed at dreamnight.net.cn/anime-events (current)
  • myBlog — Previous blog (archived)

License

MIT


Language / 语言 / 言語

English | 简体中文 | 繁體中文 | 日本語

About

ComiRadar — Discovers anime cons & ACG events across 9 Chinese ticketing platforms. Auto-updates via GitHub Actions.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages