xiaohongshu search: API returns empty data, needs DOM fallback

[jack4world]

Problem

opencli xiaohongshu search --keyword "酒店" returns empty results.

Root Cause

The current search.ts uses Pinia store's loadMore() method with XHR interception to capture data from //edith.xiaohongshu.com/api/sns/web/v1/search/notes. The API now returns { success: true, data: { has_more: true } } without any items array, meaning xiaohongshu has changed their API response structure or added anti-scraping measures.

Debug Findings

• Pinia store search still exists with mutateSearchValue and loadMore methods ✅
• XHR interception correctly captures the request to search/notes ✅
• API response success: true but data only contains { has_more } — no items/notes ❌
• searchStore.feeds remains empty after loadMore() ❌
• However, navigating to https://www.xiaohongshu.com/search_result?keyword=酒店 does render results in the DOM via section.note-item elements ✅

Suggested Fix

Fall back to DOM-based extraction when the API interception returns empty:

1. Navigate to the search results page directly:

   https://www.xiaohongshu.com/search_result?keyword={keyword}&source=web_search_result_notes
   

2. Wait for section.note-item elements to render
3. Extract structured data from DOM:

   const notes = document.querySelectorAll('section.note-item');
   // Each note-item contains: .title, .name, .time, .count, a[href*="/explore/"]

DOM Structure (confirmed working)

section.note-item
  ├── .cover (thumbnail)
  └── .footer
       ├── .title → note title
       └── .card-bottom-wrapper
            ├── .author
            │    ├── .author-avatar
            │    └── .name-time-wrapper
            │         ├── .name → author name
            │         └── .time → publish date
            └── .like-wrapper
                 └── .count → like count

Note: filter out section.query-note-item (these are "related searches", not actual notes).

Environment

• @playwright/mcp@0.0.68
• macOS, Chrome with Playwright MCP Bridge extension
• User logged in to xiaohongshu

🤖 Generated with Claude Code

[jackwener]

感谢你极其详尽的 debug 报告和 DOM 结构分析！🙏

已在最新提交 b3850d9 中完全采纳你的建议进行了修复：

修改内容

src/clis/xiaohongshu/search.ts 完全重写：

1. 放弃 Pinia store + XHR 拦截方案（如你所述，API 现在返回空 items）
2. 改为直接导航到搜索结果页 search_result?keyword=xxx&source=web_search_result_notes
3. 基于 DOM 提取数据：从 section.note-item 中抓取 title/author/likes
4. 过滤掉 query-note-item（相关搜索噪音）
5. 自动滚动加载更多结果

使用方式不变

opencli xiaohongshu search --keyword "酒店" --limit 10

你已被设为此 commit 的 Co-authored-by 以感谢贡献。再次感谢！ 🎉