List view

希望POST请求JSON的时候HttpRequestBody里的关键词可以不用再parse出来

question

Status: Closed (completed).

#597 In code4craft/webmagic;

· by luchatex was closed on Jun 17, 2017

爬虫需要翻页，怎么处理？

Status: Closed (completed).

#600 In code4craft/webmagic;

· by tobecrazy was closed on Jun 18, 2017

请教大神，有几台linux服务器，怎么设置成代理服务器，供爬虫使用呢？ps：自己琢磨了很久，爬虫总是用代理下载不了

Status: Closed (completed).

#602 In code4craft/webmagic;

· by flamw was closed on Jun 18, 2017

参考指南: 配置代理是否有过更改

question

Status: Closed (completed).

#607 In code4craft/webmagic;

· by coding-dream was closed on Jun 18, 2017

可以实现这样的功能吗？爬列表->解析内容->解析第二页内容，第三页。。。

Status: Closed (completed).

#612 In code4craft/webmagic;

· by lidaoyang was closed on Jun 22, 2017

使用正则表达式抽取默认不是应该默认为第0组吗？为何要定义为默认第一组？

duplicate

Status: Closed (completed).

#616 In code4craft/webmagic;

· by qixian1991 was closed on Jun 24, 2017

是否可以在page 对象内添加byte[] 属性 以作 文件下载使用

duplicate

Status: Closed (completed).

#618 In code4craft/webmagic;

· by lin1005q was closed on Jul 29, 2017

java.lang.NoSuchMethodError: org.apache.commons.collections.CollectionUtils.isNotEmpty(Ljava/util/Collection;)Z

Status: Closed (completed).

#621 In code4craft/webmagic;

· by zgm4z was closed on Jul 1, 2017

status code不是200的请求response不能在process方法里拿到吗？

Status: Closed (completed).

#625 In code4craft/webmagic;

· by lifuchao was closed on Jul 22, 2017

爬取网站download时候报错ClientProtocolException: null

question

Status: Closed (completed).

#630 In code4craft/webmagic;

· by lidaoyang was closed on Jul 29, 2017

为什么源代码注释都是英文的

Status: Closed (completed).

#632 In code4craft/webmagic;

· by zsw12013 was closed on Jul 29, 2017

整合 spring boot schedule 做定时爬虫

Status: Closed (completed).

#634 In code4craft/webmagic;

· by Shaun-Yao was closed on Aug 21, 2017

实现PageProcessor接口类里面spring 注入不生效

Status: Closed (completed).

#637 In code4craft/webmagic;

· by Shaun-Yao was closed on Jul 30, 2017

SpiderListener可以加一个onStart方法吗

Status: Closed (completed).

#639 In code4craft/webmagic;

· by lisanlai was closed on Apr 11, 2018

us.codecraft.webmagic.downloader.HttpClientDownloader无法被继承

minor

Status: Closed (completed).

#647 In code4craft/webmagic;

· by sunyt was closed on Oct 26, 2020

weibo.com不能爬

Status: Closed (completed).

#648 In code4craft/webmagic;

· by yaoyuanyy was closed on Apr 11, 2020

Not in GZIP format

Status: Closed (completed).

#649 In code4craft/webmagic;

· by hyuga0410 was closed on Aug 19, 2018

backend browser support

question

Status: Closed (completed).

#482 In code4craft/webmagic;

· by dingjian2 was closed on Mar 11, 2017

可否配置不打印get page信息？页面太多导致页面太多，不容易发现自己的调试结果

question

Status: Closed (completed).

#485 In code4craft/webmagic;

· by buaanie was closed on Mar 14, 2017

xpath运算是封闭的，而jsonPath运算不是

Status: Closed (completed).

#496 In code4craft/webmagic;

· by zyfxgo was closed on Mar 23, 2017

爬取新浪首页新闻的时候会出现Cookie rejected

question

Status: Closed (completed).

#499 In code4craft/webmagic;

· by xuedygaga was closed on Mar 25, 2017

调用start()方法没什么反映是怎么回事

Status: Closed (completed).

#500 In code4craft/webmagic;

· by gggitpl was closed on Mar 22, 2017

addUrl添加的入口地址被Scheduler过滤后,下次无法启动

question

Status: Closed (completed).

#501 In code4craft/webmagic;

· by JsonSong89 was closed on Mar 25, 2017

ProxyPool类无法继承扩展

enhancement

Status: Closed (completed).

#503 In code4craft/webmagic;

· by wjmzmmd was closed on Jun 18, 2017

爬取结果后保存到数据库并返回给前台，这该怎么处理？

question

Status: Closed (completed).

#506 In code4craft/webmagic;

· by vosonwang was closed on Mar 25, 2017