Skip to content

feat(route): add 4pda.to forum thread route - #23076

Open
untitaker wants to merge 2 commits into
DIYgod:masterfrom
untitaker:feat/4pda-forum-thread
Open

feat(route): add 4pda.to forum thread route#23076
untitaker wants to merge 2 commits into
DIYgod:masterfrom
untitaker:feat/4pda-forum-thread

Conversation

@untitaker

Copy link
Copy Markdown
Contributor

Involved Issue / 该 PR 相关 Issue

New route, no existing issue.

Example for the Proposed Route(s) / 路由地址示例

/4pda/forum/thread/669936

New RSS Route Checklist / 新 RSS 路由检查表

  • New Route / 新的路由
  • Anti-bot or rate limit / 反爬/频率限制
    • If yes, do your code reflect this sign? / 如果有, 是否有对应的措施? — Sets browser-like headers (User-Agent, Accept, Accept-Language) to avoid 403 responses. Route marked with antiCrawler: true.
  • Date and time / 日期和时间
    • Parsed / 可以解析 — Parses DD.MM.YY, HH:MM format
    • Correct time zone / 时区正确 — Moscow time (UTC+3)
  • New package added / 添加了新的包 — Uses existing iconv-lite
  • Puppeteer — Not required

Note / 说明

Adds a route to subscribe to posts in 4PDA forum threads, a popular Russian tech/mobile forum running Invision Power Board.

Features:

  • Fetches the last page of a thread to get the most recent posts
  • Handles windows-1251 encoding via iconv-lite (already a project dependency)
  • Includes radar rules for automatic route detection from 4pda.to URLs
  • Parses post author, date, floor number, and full post content
  • Supports ?limit=N query parameter

Example: /4pda/forum/thread/669936 subscribes to the TP-LINK M7350 thread.

Add a new route to subscribe to posts in 4PDA forum threads.

- Route: /4pda/forum/thread/:topicId
- Fetches the last page of a thread for the most recent posts
- Handles windows-1251 encoding via iconv-lite
- Includes radar rules for automatic route detection
- Parses post author, date (Moscow timezone UTC+3), and content
@github-actions github-actions Bot added route auto: not ready to review Users can't get the RSS feed output according to automated testing results labels Aug 22, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Successfully generated as following:

http://localhost:1200/4pda/forum/thread/669936 - Failed ❌
HTTPError: Response code 503 (Service Unavailable)

Error Message:<br/>FetchError: [GET] &quot;https://4pda.to/forum/index.php?showtopic=669936&amp;st=0&quot;: 403 Forbidden
Route: /4pda/forum/thread/:topicId
Full Route: /4pda/forum/thread/669936
Node Version: v24.19.0
Git Hash: 7b7cc5ae

@github-actions

github-actions Bot commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Auto Review

Previous findings (hardcoded User-Agent, custom limit handling) no longer apply after the rewrite to Playwright. Remaining items on the current diff:

[AGENTS 15 - Avoid loading HTML twice]

  • lib/routes/4pda/thread.ts: load(html) is called twice ($first, then $). When the thread has no pagination link, html is unchanged and the exact same markup is parsed a second time. Fix: parse once - only re-load() when a navigation actually happened (const $ = navigated ? load(await page.content()) : $first;) and reuse that object for both the pagination lookup and the post extraction.

[AGENTS 16 / 43 - Playwright resource handling]

  • lib/routes/4pda/thread.ts: await destroy() is only reached on the happy path. If page.content() or the second page.goto() throws (timeout / navigation error), the browser context is never released and leaks. Fix: follow the existing convention (see lib/routes/iwara/ranking.ts:61-87) and wrap the page work in try { ... } finally { await destroy(); }.

@untitaker
untitaker marked this pull request as draft August 22, 2026 16:21
- Use playwright (patchright) instead of ofetch to handle Cloudflare
  Turnstile challenge on 4pda.to
- Remove iconv-lite (playwright handles encoding natively)
- Remove custom limit handling (middleware handles ?limit= param)
- Remove hardcoded User-Agent (playwright uses configured UA)
- Fix author extraction to exclude icon text
- Mark route as requirePuppeteer: true
@untitaker
untitaker marked this pull request as ready for review August 22, 2026 16:33
@github-actions github-actions Bot removed the auto: not ready to review Users can't get the RSS feed output according to automated testing results label Aug 22, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Successfully generated as following:

http://localhost:1200/4pda/forum/thread/669936 - Failed ❌
HTTPError: Response code 503 (Service Unavailable)

Error Message:<br/>Error: this route is empty, please check the original site or &lt;a href=&quot;https://github.com/DIYgod/RSSHub/issues/new/choose&quot;&gt;create an issue&lt;/a&gt;
Route: /4pda/forum/thread/:topicId
Full Route: /4pda/forum/thread/669936
Node Version: v24.19.0
Git Hash: e4982ffb

@github-actions github-actions Bot added the auto: not ready to review Users can't get the RSS feed output according to automated testing results label Aug 22, 2026
@untitaker

Copy link
Copy Markdown
Contributor Author

I think turnstile is left as "exercise for the reader." we cannot solve it in rsshub.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto: not ready to review Users can't get the RSS feed output according to automated testing results route

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant