woodpecker-requests是基于 requests 为woodpecker框架定制开发的httpclient库,目的是编写插件时能拥有像python requests一样的便利。特点为可以全局设置代理、全局设置UA等
Requests is a http request lib with fluent api for java, inspired by the python request module. Requests requires JDK 1.8+, the last version support Java7 is 4.18.* .
Requests is now in maven central repo.
https://mvnrepository.com/artifact/me.gv7.woodpecker/woodpecker-requests
<dependency>
<groupId>me.gv7.woodpecker</groupId>
<artifactId>woodpecker-requests</artifactId>
<version>0.1.0</version>
</dependency>public static boolean enable = false; // open proxy or close
public static String protocol = "http"; // http or socks
public static String host = "127.0.0.1";
public static int port = 8080;
public static String username; // socks proxy user,can set null
public static String password; // socks proxy pass,can set null
HttpConfigManager.setProxyConfig(enable
,protocol
,host
,port
,username
,password);
}Requests.method("GET","http://wwww.baidu.com/").proxy(Proxy.NO_PROXY).send();LinkedList<String> uaList = new LinkedList<>();
// set greater than 2 will random choise.
uaList.add("Mozilla/5.0 (Android; Mobile; rv:14.0) Gecko/14.0 Firefox/14.0");
uaList.add("Mozilla/5.0 (Android; Tablet; rv:14.0) Gecko/14.0 Firefox/14.0");
HttpConfigManager.setUserAgentConfig(uaList);int time = 5000; // !!!attention!!! millisecond
boolean enableMandatoryTimeout = false; // ignored user set
int mandatoryTimeout = 1;
HttpConfigManager.setTimeoutConfig(time
,enableMandatoryTimeout
,mandatoryTimeout);enableMandatoryTimeout will ignore user set,such us:
Requests.get("http://woodpecker.gv7.me/").timeout(10000).send() // timeout will be replaced by mandatoryTimeoutOne simple http request example that do http get request and read response as string:
String url = ...;
String resp = Requests.get(url).send().readToText();
// or
Response<String> resp = Requests.get(url).send().toTextResponse();Post and other method:
resp = Requests.post(url).send().readToText();
resp = Requests.head(url).send().readToText();
...The response object have several common http response fields can be used:
RawResponse resp = Requests.get(url).send();
int statusCode = resp.statusCode();
String contentLen = resp.getHeader("Content-Length");
Cookie cookie = resp.getCookie("_bd_name");
String body = resp.readToText();Make sure call readToText or other methods to consume resp, or call close method to close resp.
The readToText() method here trans http response body as String, more other methods provided:
// get response as string, use encoding get from response header
String resp = Requests.get(url).send().readToText();
// get response as bytes
byte[] resp1 = Requests.get(url).send().readToBytes();
// save response as file
boolean result = Requests.get(url).send().writeToFile("/path/to/save/file");Requests default use UTF-8 to encode parameters, post forms or request string body, you can set other charset by:
String resp = Requests.get(url).charset(StandardCharsets.ISO_8859_1).send().readToText();When read response to text-based result, use charset get from http response header, or UTF-8 if not found. You can force use specified charset by:
String resp = Requests.get(url).send().charset(StandardCharsets.ISO_8859_1).readToText();Pass parameters in urls using params method:
// set params by map
Map<String, Object> params = new HashMap<>();
params.put("k1", "v1");
params.put("k2", "v2");
String resp = Requests.get(url).params(params).send().readToText();
// set multi params
String resp = Requests.get(url)
.params(Parameter.of("k1", "v1"), Parameter.of("k2", "v2"))
.send().readToText();If you want to send post www-form-encoded parameters, use body() methods:
// set params by map
Map<String, Object> params = new HashMap<>();
params.put("k1", "v1");
params.put("k2", "v2");
String resp = Requests.post(url).body(params).send().readToText();
// set multi params
String resp = Requests.post(url)
.body(Parameter.of("k1", "v1"), Parameter.of("k2", "v2"))
.send().readToText();The forms parameter should only works with post method.
Http request headers can be set by headers method:
// set headers by map
Map<String, Object> headers = new HashMap<>();
headers.put("k1", "v1");
headers.put("k2", "v2");
String resp = Requests.get(url).headers(headers).send().readToText();
// set multi headers
String resp = Requests.get(url)
.headers(new Header("k1", "v1"), new Header("k2", "v2"))
.send().readToText();Cookies can be add by:
Map<String, Object> cookies = new HashMap<>();
cookies.put("k1", "v1");
cookies.put("k2", "v2");
// set cookies by map
String resp = Requests.get(url).cookies(cookies).send().readToText();
// set cookies
String resp = Requests.get(url)
.cookies(Parameter.of("k1", "v1"), Parameter.of("k2", "v2"))
.send().readToText();Http Post, Put, Patch method can send request body. Take Post for example:
// set post form data
String resp = Requests.post(url).body(Parameter.of("k1", "v1"), Parameter.of("k2", "v2"))
.send().readToText();
// set post form data by map
Map<String, Object> formData = new HashMap<>();
formData.put("k1", "v1");
formData.put("k2", "v2");
String resp = Requests.post(url).body(formData).send().readToText();
// send byte array data as body
byte[] data = ...;
resp = Requests.post(url).body(data).send().readToText();
// send string data as body
String str = ...;
resp = Requests.post(url).body(str).send().readToText();
// send data from inputStream
InputStreamSupplier supplier = ...;
resp = Requests.post(url).body(supplier).send().readToText();One more complicate situation is multiPart post request, this can be done via multiPart method, one simplified multi part request example which send files and param data:
// send form-encoded data
InputStreamSupplier supplier = ...;
byte[] bytes = ...;
String resp = Requests.post(url)
.multiPartBody(
Part.file("file1", new File(...)),
Part.file("file2", "second_file.dat", supplier),
Part.text("input", "on")
).send().readToText();Requests can handle json encoder(for request body)/decoder(for response body), if having Json Binding, Jackson, Gson, or Fastjson lib in classpath.
// send json body, content-type is set to application/json
RawResponse response = Requests.post("http://.../update_person")
.jsonBody(value)
.send();
// response body as json, to value
Person person = Requests.post("http://.../get_person")
.params(Parameter.of("id", 101))
.send().readToJson(Person.class);
// json body decoder to generic type
List<Person> persons = Requests.post("http://.../get_person_list")
.send().readToJson(new TypeInfer<List<Person>>() {});You may set your own json processor by:
JsonProcessor jsonProcessor = ...;
JsonLookup.getInstance().register(jsonProcessor);Set http basic auth param by auth method:
String resp = Requests.get(url).basicAuth("user", "passwd").send().readToText();Requests will handle 30x http redirect automatically, you can disable it by:
Requests.get(url).followRedirect(false).send();You can set connection connect timeout, and socket read/write timeout value, as blow:
// set connect timeout and socket timeout
Requests.get(url).socksTimeout(20_000).connectTimeout(30_000).send();Requests send Accept-Encoding: gzip, deflate, and auto handle response decompress in default. You can disable this by:
// do not send Accept-Encoding: gzip, deflate header
String resp = Requests.get(url).acceptCompress(false).send().readToText();
// do not decompress response body
String resp2 = Requests.get(url).send().decompress(false).readToText();Some https sites do not have trusted http certificate, Exception will be thrown when request. You can disable https certificate verify by:
Requests.get(url).verify(false).send();Set proxy by proxy method:
Requests.get(url).proxy(Proxies.httpProxy("127.0.0.1", 8081)).send(); // http proxy
Requests.get(url).proxy(Proxies.socksProxy("127.0.0.1", 1080)).send(); // socks proxy proxySession maintains cookies, basic auth and maybe other http context for you, useful when need login or other situations. Session have the same usage as Requests.
Session session = Requests.session();
String resp1 = session.get(url1).send().readToText();
String resp2 = session.get(url2).send().readToText();