Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

通过豆瓣小组网页获取数据 #11

Merged
merged 2 commits into from
Jul 19, 2021
Merged

通过豆瓣小组网页获取数据 #11

merged 2 commits into from
Jul 19, 2021

Conversation

xwjdsh
Copy link
Contributor

@xwjdsh xwjdsh commented Jul 16, 2021

豆瓣 API 不能使用了，改成通过网页解析的方式来获取数据，有几个问题，

  • 现在在列表上获取不到内容和创建时间，如果对每一个帖子都获取详情，感觉很容易被限制，所以暂时没有实现，如果你觉得可行，我会继续更新。
  • 有些字段不知道在原来的接口上表示什么，不知道取值是否正确。

lesywix
lesywix reviewed Jul 17, 2021
View reviewed changes
Copy link
Owner

@lesywix lesywix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

其他没啥问题了

crawler_main.py Outdated Show resolved Hide resolved
crawler_main.py Outdated Show resolved Hide resolved
@xwjdsh
Copy link
Contributor Author

xwjdsh commented Jul 19, 2021

已更新。

@lesywix lesywix merged commit f6e9c79 into lesywix:master Jul 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lesywix lesywix lesywix left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@xwjdsh @lesywix