+
- 
+ 
-
+
+ Huge thanks to 
for sponsoring Sindre Sorhus!
+
+ (they love Got too!)
+
+
+
+
+> Human-friendly and powerful HTTP request library for Node.js
+
+[](https://travis-ci.com/github/sindresorhus/got)
+[](https://coveralls.io/github/sindresorhus/got?branch=master)
+[](https://npmjs.com/got)
+[](https://packagephobia.now.sh/result?p=got)
+
+[Moving from Request?](documentation/migration-guides.md) [*(Note that Request is unmaintained)*](https://github.com/request/request/issues/3142)
+
+[See how Got compares to other HTTP libraries](#comparison)
+
+For browser usage, we recommend [Ky](https://github.com/sindresorhus/ky) by the same people.
+
+## Highlights
+
+- [Promise API](#api)
+- [Stream API](#streams)
+- [Pagination API](#pagination)
+- [HTTP2 support](#http2)
+- [Request cancelation](#aborting-the-request)
+- [RFC compliant caching](#cache-adapters)
+- [Follows redirects](#followredirect)
+- [Retries on failure](#retry)
+- [Progress events](#onuploadprogress-progress)
+- [Handles gzip/deflate/brotli](#decompress)
+- [Timeout handling](#timeout)
+- [Errors with metadata](#errors)
+- [JSON mode](#json-mode)
+- [WHATWG URL support](#url)
+- [HTTPS API](#advanced-https-api)
+- [Hooks](#hooks)
+- [Instances with custom defaults](#instances)
+- [Types](#types)
+- [Composable](documentation/advanced-creation.md#merging-instances)
+- [Plugins](documentation/lets-make-a-plugin.md)
+- [Used by 4K+ packages and 1.8M+ repos](https://github.com/sindresorhus/got/network/dependents)
+- [Actively maintained](https://github.com/sindresorhus/got/graphs/contributors)
+- [Trusted by many companies](#widely-used)
+
+## Install
+
+```
+$ npm install got
+```
+
+## Usage
+
+###### Promise
+
+```js
+const got = require('got');
+
+(async () => {
+ try {
+ const response = await got('https://sindresorhus.com');
+ console.log(response.body);
+ //=> ' ...'
+ } catch (error) {
+ console.log(error.response.body);
+ //=> 'Internal server error ...'
+ }
+})();
+```
+
+###### JSON
+
+```js
+const got = require('got');
+
+(async () => {
+ const {body} = await got.post('https://httpbin.org/anything', {
+ json: {
+ hello: 'world'
+ },
+ responseType: 'json'
+ });
+
+ console.log(body.data);
+ //=> {hello: 'world'}
+})();
+```
+
+See [JSON mode](#json-mode) for more details.
+
+###### Streams
+
+```js
+const stream = require('stream');
+const {promisify} = require('util');
+const fs = require('fs');
+const got = require('got');
+
+const pipeline = promisify(stream.pipeline);
+
+(async () => {
+ await pipeline(
+ got.stream('https://sindresorhus.com'),
+ fs.createWriteStream('index.html')
+ );
+
+ // For POST, PUT, PATCH, and DELETE methods, `got.stream` returns a `stream.Writable`.
+ await pipeline(
+ fs.createReadStream('index.html'),
+ got.stream.post('https://sindresorhus.com')
+ );
+})();
+```
+
+**Tip:** `from.pipe(to)` doesn't forward errors. Instead, use [`stream.pipeline(from, ..., to, callback)`](https://nodejs.org/api/stream.html#stream_stream_pipeline_streams_callback).
+
+**Note:** While `got.post('https://example.com')` resolves, `got.stream.post('https://example.com')` will hang indefinitely until a body is provided. If there's no body on purpose, remember to `.end()` the stream or set the [`body`](#body) option to an empty string.
+
+### API
+
+It's a `GET` request by default, but can be changed by using different methods or via [`options.method`](#method).
+
+**By default, Got will retry on failure. To disable this option, set [`options.retry`](#retry) to `0`.**
+
+#### got(url?, options?)
+
+Returns a Promise giving a [Response object](#response) or a [Got Stream](#streams-1) if `options.isStream` is set to true.
+
+##### url
+
+Type: `string | object`
+
+The URL to request, as a string, a [`https.request` options object](https://nodejs.org/api/https.html#https_https_request_options_callback), or a [WHATWG `URL`](https://nodejs.org/api/url.html#url_class_url).
+
+Properties from `options` will override properties in the parsed `url`.
+
+If no protocol is specified, it will throw a `TypeError`.
+
+**Note:** The query string is **not** parsed as search params. Example:
+
+```js
+got('https://example.com/?query=a b'); //=> https://example.com/?query=a%20b
+got('https://example.com/', {searchParams: {query: 'a b'}}); //=> https://example.com/?query=a+b
+
+// The query string is overridden by `searchParams`
+got('https://example.com/?query=a b', {searchParams: {query: 'a b'}}); //=> https://example.com/?query=a+b
+```
+
+##### options
+
+Type: `object`
+
+Any of the [`https.request`](https://nodejs.org/api/https.html#https_https_request_options_callback) options.
+
+**Note:** Legacy URL support is disabled. `options.path` is supported only for backwards compatibility. Use `options.pathname` and `options.searchParams` instead. `options.auth` has been replaced with `options.username` & `options.password`.
+
+###### method
+
+Type: `string`\
+Default: `GET`
+
+The HTTP method used to make the request.
+
+###### prefixUrl
+
+Type: `string | URL`
+
+When specified, `prefixUrl` will be prepended to `url`. The prefix can be any valid URL, either relative or absolute.\
+A trailing slash `/` is optional - one will be added automatically.
+
+**Note:** `prefixUrl` will be ignored if the `url` argument is a URL instance.
+
+**Note:** Leading slashes in `input` are disallowed when using this option to enforce consistency and avoid confusion. For example, when the prefix URL is `https://example.com/foo` and the input is `/bar`, there's ambiguity whether the resulting URL would become `https://example.com/foo/bar` or `https://example.com/bar`. The latter is used by browsers.
+
+**Tip:** Useful when used with [`got.extend()`](#custom-endpoints) to create niche-specific Got instances.
+
+**Tip:** You can change `prefixUrl` using hooks as long as the URL still includes the `prefixUrl`. If the URL doesn't include it anymore, it will throw.
+
+```js
+const got = require('got');
+
+(async () => {
+ await got('unicorn', {prefixUrl: 'https://cats.com'});
+ //=> 'https://cats.com/unicorn'
+
+ const instance = got.extend({
+ prefixUrl: 'https://google.com'
+ });
+
+ await instance('unicorn', {
+ hooks: {
+ beforeRequest: [
+ options => {
+ options.prefixUrl = 'https://cats.com';
+ }
+ ]
+ }
+ });
+ //=> 'https://cats.com/unicorn'
+})();
+```
+
+###### headers
+
+Type: `object`\
+Default: `{}`
+
+Request headers.
+
+Existing headers will be overwritten. Headers set to `undefined` will be omitted.
+
+###### isStream
+
+Type: `boolean`\
+Default: `false`
+
+Returns a `Stream` instead of a `Promise`. This is equivalent to calling `got.stream(url, options?)`.
+
+###### body
+
+Type: `string | Buffer | stream.Readable` or [`form-data` instance](https://github.com/form-data/form-data)
+
+**Note #1:** The `body` option cannot be used with the `json` or `form` option.
+
+**Note #2:** If you provide this option, `got.stream()` will be read-only.
+
+**Note #3:** If you provide a payload with the `GET` or `HEAD` method, it will throw a `TypeError` unless the method is `GET` and the `allowGetBody` option is set to `true`.
+
+**Note #4:** This option is not enumerable and will not be merged with the instance defaults.
+
+The `content-length` header will be automatically set if `body` is a `string` / `Buffer` / `fs.createReadStream` instance / [`form-data` instance](https://github.com/form-data/form-data), and `content-length` and `transfer-encoding` are not manually set in `options.headers`.
+
+###### json
+
+Type: `object | Array | number | string | boolean | null` *(JSON-serializable values)*
+
+**Note #1:** If you provide this option, `got.stream()` will be read-only.\
+**Note #2:** This option is not enumerable and will not be merged with the instance defaults.
+
+JSON body. If the `Content-Type` header is not set, it will be set to `application/json`.
+
+###### context
+
+Type: `object`
+
+User data. In contrast to other options, `context` is not enumerable.
+
+**Note:** The object is never merged, it's just passed through. Got will not modify the object in any way.
+
+It's very useful for storing auth tokens:
+
+```js
+const got = require('got');
+
+const instance = got.extend({
+ hooks: {
+ beforeRequest: [
+ options => {
+ if (!options.context || !options.context.token) {
+ throw new Error('Token required');
+ }
+
+ options.headers.token = options.context.token;
+ }
+ ]
+ }
+});
+
+(async () => {
+ const context = {
+ token: 'secret'
+ };
+
+ const response = await instance('https://httpbin.org/headers', {context});
+
+ // Let's see the headers
+ console.log(response.body);
+})();
+```
+
+###### responseType
+
+Type: `string`\
+Default: `'text'`
+
+**Note:** When using streams, this option is ignored.
+
+The parsing method. Can be `'text'`, `'json'` or `'buffer'`.
+
+The promise also has `.text()`, `.json()` and `.buffer()` methods which return another Got promise for the parsed body.\
+It's like setting the options to `{responseType: 'json', resolveBodyOnly: true}` but without affecting the main Got promise.
+
+Example:
+
+```js
+(async () => {
+ const responsePromise = got(url);
+ const bufferPromise = responsePromise.buffer();
+ const jsonPromise = responsePromise.json();
+
+ const [response, buffer, json] = await Promise.all([responsePromise, bufferPromise, jsonPromise]);
+ // `response` is an instance of Got Response
+ // `buffer` is an instance of Buffer
+ // `json` is an object
+})();
+```
+
+```js
+// This
+const body = await got(url).json();
+
+// is semantically the same as this
+const body = await got(url, {responseType: 'json', resolveBodyOnly: true});
+```
+
+**Note:** `buffer` will return the raw body buffer. Modifying it will also alter the result of `promise.text()` and `promise.json()`. Before overwritting the buffer, please copy it first via `Buffer.from(buffer)`. See https://github.com/nodejs/node/issues/27080
+
+###### parseJson
+
+Type: `(text: string) => unknown`\
+Default: `(text: string) => JSON.parse(text)`
+
+A function used to parse JSON responses.
+
+
+Example
+
+Using [`bourne`](https://github.com/hapijs/bourne) to prevent prototype pollution:
+
+```js
+const got = require('got');
+const Bourne = require('@hapi/bourne');
+
+(async () => {
+ const parsed = await got('https://example.com', {
+ parseJson: text => Bourne.parse(text)
+ }).json();
+
+ console.log(parsed);
+})();
+```
+
+
+###### stringifyJson
+
+Type: `(object: unknown) => string`\
+Default: `(object: unknown) => JSON.stringify(object)`
+
+A function used to stringify the body of JSON requests.
+
+
+Examples
+
+Ignore properties starting with `_`:
+
+```js
+const got = require('got');
+
+(async () => {
+ await got.post('https://example.com', {
+ stringifyJson: object => JSON.stringify(object, (key, value) => {
+ if (key.startsWith('_')) {
+ return;
+ }
+
+ return value;
+ }),
+ json: {
+ some: 'payload',
+ _ignoreMe: 1234
+ }
+ });
+})();
+```
+
+All numbers as strings:
+
+```js
+const got = require('got');
+
+(async () => {
+ await got.post('https://example.com', {
+ stringifyJson: object => JSON.stringify(object, (key, value) => {
+ if (typeof value === 'number') {
+ return value.toString();
+ }
+
+ return value;
+ }),
+ json: {
+ some: 'payload',
+ number: 1
+ }
+ });
+})();
+```
+
+
+###### resolveBodyOnly
+
+Type: `boolean`\
+Default: `false`
+
+When set to `true` the promise will return the [Response body](#body-1) instead of the [Response](#response) object.
+
+###### cookieJar
+
+Type: `object` | [`tough.CookieJar` instance](https://github.com/salesforce/tough-cookie#cookiejar)
+
+**Note:** If you provide this option, `options.headers.cookie` will be overridden.
+
+Cookie support. You don't have to care about parsing or how to store them. [Example](#cookies).
+
+###### cookieJar.setCookie
+
+Type: `Function`
+
+The function takes two arguments: `rawCookie` (`string`) and `url` (`string`).
+
+###### cookieJar.getCookieString
+
+Type: `Function`
+
+The function takes one argument: `url` (`string`).
+
+###### ignoreInvalidCookies
+
+Type: `boolean`\
+Default: `false`
+
+Ignore invalid cookies instead of throwing an error. Only useful when the `cookieJar` option has been set. Not recommended.
+
+###### encoding
+
+Type: `string`\
+Default: `'utf8'`
+
+[Encoding](https://nodejs.org/api/buffer.html#buffer_buffers_and_character_encodings) to be used on `setEncoding` of the response data.
+
+To get a [`Buffer`](https://nodejs.org/api/buffer.html), you need to set [`responseType`](#responseType) to `buffer` instead. Don't set this option to `null`.
+
+**Note:** This doesn't affect streams! Instead, you need to do `got.stream(...).setEncoding(encoding)`.
+
+###### form
+
+Type: `object`
+
+**Note #1:** If you provide this option, `got.stream()` will be read-only.\
+**Note #2:** This option is not enumerable and will not be merged with the instance defaults.
+
+The form body is converted to a query string using [`(new URLSearchParams(object)).toString()`](https://nodejs.org/api/url.html#url_constructor_new_urlsearchparams_obj).
+
+If the `Content-Type` header is not present, it will be set to `application/x-www-form-urlencoded`.
+
+###### searchParams
+
+Type: `string | object | URLSearchParams`
+
+Query string that will be added to the request URL. This will override the query string in `url`.
+
+If you need to pass in an array, you can do it using a `URLSearchParams` instance:
+
+```js
+const got = require('got');
+
+const searchParams = new URLSearchParams([['key', 'a'], ['key', 'b']]);
+
+got('https://example.com', {searchParams});
+
+console.log(searchParams.toString());
+//=> 'key=a&key=b'
+```
+
+There are some exceptions in regards to `URLSearchParams` behavior:
+
+**Note #1:** `null` values are not stringified, an empty string is used instead.
+
+**Note #2:** `undefined` values are not stringified, the entry is skipped instead.
+
+###### timeout
+
+Type: `number | object`
+
+Milliseconds to wait for the server to end the response before aborting the request with [`got.TimeoutError`](#gottimeouterror) error (a.k.a. `request` property). By default, there's no timeout.
+
+This also accepts an `object` with the following fields to constrain the duration of each phase of the request lifecycle:
+
+- `lookup` starts when a socket is assigned and ends when the hostname has been resolved. Does not apply when using a Unix domain socket.
+- `connect` starts when `lookup` completes (or when the socket is assigned if lookup does not apply to the request) and ends when the socket is connected.
+- `secureConnect` starts when `connect` completes and ends when the handshaking process completes (HTTPS only).
+- `socket` starts when the socket is connected. See [request.setTimeout](https://nodejs.org/api/http.html#http_request_settimeout_timeout_callback).
+- `response` starts when the request has been written to the socket and ends when the response headers are received.
+- `send` starts when the socket is connected and ends with the request has been written to the socket.
+- `request` starts when the request is initiated and ends when the response's end event fires.
+
+###### retry
+
+Type: `number | object`\
+Default:
+- limit: `2`
+- calculateDelay: `({attemptCount, retryOptions, error, computedValue}) => computedValue | Promise`
+- methods: `GET` `PUT` `HEAD` `DELETE` `OPTIONS` `TRACE`
+- statusCodes: [`408`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/408) [`413`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/413) [`429`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/429) [`500`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500) [`502`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/502) [`503`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/503) [`504`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/504) [`521`](https://support.cloudflare.com/hc/en-us/articles/115003011431#521error) [`522`](https://support.cloudflare.com/hc/en-us/articles/115003011431#522error) [`524`](https://support.cloudflare.com/hc/en-us/articles/115003011431#524error)
+- maxRetryAfter: `undefined`
+- errorCodes: `ETIMEDOUT` `ECONNRESET` `EADDRINUSE` `ECONNREFUSED` `EPIPE` `ENOTFOUND` `ENETUNREACH` `EAI_AGAIN`
+
+An object representing `limit`, `calculateDelay`, `methods`, `statusCodes`, `maxRetryAfter` and `errorCodes` fields for maximum retry count, retry handler, allowed methods, allowed status codes, maximum [`Retry-After`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After) time and allowed error codes.
+
+If `maxRetryAfter` is set to `undefined`, it will use `options.timeout`.\
+If [`Retry-After`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After) header is greater than `maxRetryAfter`, it will cancel the request.
+
+Delays between retries counts with function `1000 * Math.pow(2, retry - 1) + Math.random() * 100`, where `retry` is attempt number (starts from 1).
+
+The `calculateDelay` property is a `function` that receives an object with `attemptCount`, `retryOptions`, `error` and `computedValue` properties for current retry count, the retry options, error and default computed value. The function must return a delay in milliseconds (or a Promise resolving with it) (`0` return value cancels retry).
+
+**Note:** The `calculateDelay` function is responsible for the entire cache mechanism, including the `limit` property. To support it, you need to check whether `computedValue` is different than `0`.
+
+By default, it retries *only* on the specified methods, status codes, and on these network errors:
+- `ETIMEDOUT`: One of the [timeout](#timeout) limits were reached.
+- `ECONNRESET`: Connection was forcibly closed by a peer.
+- `EADDRINUSE`: Could not bind to any free port.
+- `ECONNREFUSED`: Connection was refused by the server.
+- `EPIPE`: The remote side of the stream being written has been closed.
+- `ENOTFOUND`: Couldn't resolve the hostname to an IP address.
+- `ENETUNREACH`: No internet connection.
+- `EAI_AGAIN`: DNS lookup timed out.
+
+
+
+You can retry Got streams too. The implementation looks like this:
+
+```js
+const got = require('got');
+const fs = require('fs');
+
+let writeStream;
+
+const fn = (retryCount = 0) => {
+ const stream = got.stream('https://example.com');
+ stream.retryCount = retryCount;
+
+ if (writeStream) {
+ writeStream.destroy();
+ }
+
+ writeStream = fs.createWriteStream('example.com');
+
+ stream.pipe(writeStream);
+
+ // If you don't attach the listener, it will NOT make a retry.
+ // It automatically checks the listener count so it knows whether to retry or not :)
+ stream.once('retry', fn);
+};
+
+fn();
+```
+
+###### followRedirect
+
+Type: `boolean`\
+Default: `true`
+
+Defines if redirect responses should be followed automatically.
+
+Note that if a `303` is sent by the server in response to any request type (`POST`, `DELETE`, etc.), Got will automatically request the resource pointed to in the location header via `GET`. This is in accordance with [the spec](https://tools.ietf.org/html/rfc7231#section-6.4.4).
+
+###### methodRewriting
+
+Type: `boolean`\
+Default: `true`
+
+By default, redirects will use [method rewriting](https://tools.ietf.org/html/rfc7231#section-6.4). For example, when sending a POST request and receiving a `302`, it will resend the body to the new location using the same HTTP method (`POST` in this case).
+
+###### allowGetBody
+
+Type: `boolean`\
+Default: `false`
+
+**Note:** The [RFC 7321](https://tools.ietf.org/html/rfc7231#section-4.3.1) doesn't specify any particular behavior for the GET method having a payload, therefore **it's considered an [anti-pattern](https://en.wikipedia.org/wiki/Anti-pattern)**.
+
+Set this to `true` to allow sending body for the `GET` method. However, the [HTTP/2 specification](https://tools.ietf.org/html/rfc7540#section-8.1.3) says that `An HTTP GET request includes request header fields and no payload body`, therefore when using the HTTP/2 protocol this option will have no effect. This option is only meant to interact with non-compliant servers when you have no other choice.
+
+###### maxRedirects
+
+Type: `number`\
+Default: `10`
+
+If exceeded, the request will be aborted and a `MaxRedirectsError` will be thrown.
+
+###### decompress
+
+Type: `boolean`\
+Default: `true`
+
+Decompress the response automatically. This will set the `accept-encoding` header to `gzip, deflate, br` on Node.js 11.7.0+ or `gzip, deflate` for older Node.js versions, unless you set it yourself.
+
+Brotli (`br`) support requires Node.js 11.7.0 or later.
+
+If this is disabled, a compressed response is returned as a `Buffer`. This may be useful if you want to handle decompression yourself or stream the raw compressed data.
+
+###### cache
+
+Type: `object | false`\
+Default: `false`
+
+[Cache adapter instance](#cache-adapters) for storing cached response data.
+
+###### cacheOptions
+
+Type: `object | undefined`\
+Default: `{}`
+
+[Cache options](https://github.com/kornelski/http-cache-semantics#constructor-options) used for the specified request.
+
+###### dnsCache
+
+Type: `CacheableLookup | false`\
+Default: `false`
+
+An instance of [`CacheableLookup`](https://github.com/szmarczak/cacheable-lookup) used for making DNS lookups. Useful when making lots of requests to different *public* hostnames.
+
+**Note:** This should stay disabled when making requests to internal hostnames such as `localhost`, `database.local` etc.\
+`CacheableLookup` uses `dns.resolver4(..)` and `dns.resolver6(...)` under the hood and fall backs to `dns.lookup(...)` when the first two fail, which may lead to additional delay.
+
+###### dnsLookupIpVersion
+
+Type: `'auto' | 'ipv4' | 'ipv6'`\
+Default: `'auto'`
+
+Indicates which DNS record family to use.\
+Values:
+ - `auto`: IPv4 (if present) or IPv6
+ - `ipv4`: Only IPv4
+ - `ipv6`: Only IPv6
+
+Note: If you are using the undocumented option `family`, `dnsLookupIpVersion` will override it.
+
+```js
+// `api6.ipify.org` will be resolved as IPv4 and the request will be over IPv4 (the website will respond with your public IPv4)
+await got('https://api6.ipify.org', {
+ dnsLookupIpVersion: 'ipv4'
+});
+
+// `api6.ipify.org` will be resolved as IPv6 and the request will be over IPv6 (the website will respond with your public IPv6)
+await got('https://api6.ipify.org', {
+ dnsLookupIpVersion: 'ipv6'
+});
+```
+
+###### lookup
+
+Type: `Function`\
+Default: [`dns.lookup`](https://nodejs.org/api/dns.html#dns_dns_lookup_hostname_options_callback)
+
+Custom DNS resolution logic.
+
+The function signature is the same as [`dns.lookup`](https://nodejs.org/api/dns.html#dns_dns_lookup_hostname_options_callback).
+
+###### request
+
+Type: `Function`\
+Default: `http.request | https.request` *(Depending on the protocol)*
+
+Custom request function. The main purpose of this is to [support HTTP2 using a wrapper](https://github.com/szmarczak/http2-wrapper).
+
+###### http2
+
+Type: `boolean`\
+Default: `false`
+
+If set to `true`, Got will additionally accept HTTP2 requests.\
+It will choose either HTTP/1.1 or HTTP/2 depending on the ALPN protocol.
+
+**Note:** Overriding `options.request` will disable HTTP2 support.
+
+**Note:** This option will default to `true` in the next upcoming major release.
+
+```js
+const got = require('got');
+
+(async () => {
+ const {headers} = await got('https://nghttp2.org/httpbin/anything', {http2: true});
+ console.log(headers.via);
+ //=> '2 nghttpx'
+})();
+```
+
+###### throwHttpErrors
+
+Type: `boolean`\
+Default: `true`
+
+Determines if a [`got.HTTPError`](#gothttperror) is thrown for unsuccessful responses.
+
+If this is disabled, requests that encounter an error status code will be resolved with the `response` instead of throwing. This may be useful if you are checking for resource availability and are expecting error responses.
+
+###### agent
+
+Type: `object`
+
+An object representing `http`, `https` and `http2` keys for [`http.Agent`](https://nodejs.org/api/http.html#http_class_http_agent), [`https.Agent`](https://nodejs.org/api/https.html#https_class_https_agent) and [`http2wrapper.Agent`](https://github.com/szmarczak/http2-wrapper#new-http2agentoptions) instance. This is necessary because a request to one protocol might redirect to another. In such a scenario, Got will switch over to the right protocol agent for you.
+
+If a key is not present, it will default to a global agent.
+
+```js
+const got = require('got');
+const HttpAgent = require('agentkeepalive');
+const {HttpsAgent} = HttpAgent;
+
+got('https://sindresorhus.com', {
+ agent: {
+ http: new HttpAgent(),
+ https: new HttpsAgent()
+ }
+});
+```
+
+###### hooks
+
+Type: `object`
+
+Hooks allow modifications during the request lifecycle. Hook functions may be async and are run serially.
+
+###### hooks.init
+
+Type: `Function[]`\
+Default: `[]`
+
+Called with plain [request options](#options), right before their normalization. This is especially useful in conjunction with [`got.extend()`](#instances) when the input needs custom handling.
+
+See the [Request migration guide](documentation/migration-guides.md#breaking-changes) for an example.
+
+**Note #1:** This hook must be synchronous!\
+**Note #2:** Errors in this hook will be converted into an instances of [`RequestError`](#gotrequesterror).\
+**Note #3:** The options object may not have a `url` property. To modify it, use a `beforeRequest` hook instead.
+
+###### hooks.beforeRequest
+
+Type: `Function[]`\
+Default: `[]`
+
+Called with [normalized](source/core/index.ts) [request options](#options). Got will make no further changes to the request before it is sent. This is especially useful in conjunction with [`got.extend()`](#instances) when you want to create an API client that, for example, uses HMAC-signing.
+
+**Note:** Changing `options.json` or `options.form` has no effect on the request, you should change `options.body` instead. If needed, update the `options.headers` accordingly. Example:
+
+```js
+const got = require('got');
+
+got.post({
+ json: {payload: 'old'},
+ hooks: {
+ beforeRequest: [
+ options => {
+ options.body = JSON.stringify({payload: 'new'});
+ options.headers['content-length'] = options.body.length.toString();
+ }
+ ]
+ }
+});
+```
+
+**Tip:** You can override the `request` function by returning a [`ClientRequest`-like](https://nodejs.org/api/http.html#http_class_http_clientrequest) instance or a [`IncomingMessage`-like](https://nodejs.org/api/http.html#http_class_http_incomingmessage) instance. This is very useful when creating a custom cache mechanism.
+
+###### hooks.beforeRedirect
+
+Type: `Function[]`\
+Default: `[]`
+
+Called with [normalized](source/core/index.ts) [request options](#options) and the redirect [response](#response). Got will make no further changes to the request. This is especially useful when you want to avoid dead sites. Example:
+
+```js
+const got = require('got');
+
+got('https://example.com', {
+ hooks: {
+ beforeRedirect: [
+ (options, response) => {
+ if (options.hostname === 'deadSite') {
+ options.hostname = 'fallbackSite';
+ }
+ }
+ ]
+ }
+});
+```
+
+###### hooks.beforeRetry
+
+Type: `Function[]`\
+Default: `[]`
+
+**Note:** When using streams, this hook is ignored.
+
+Called with [normalized](source/normalize-arguments.ts) [request options](#options), the error and the retry count. Got will make no further changes to the request. This is especially useful when some extra work is required before the next try. Example:
+
+```js
+const got = require('got');
+
+got.post('https://example.com', {
+ hooks: {
+ beforeRetry: [
+ (options, error, retryCount) => {
+ if (error.response.statusCode === 413) { // Payload too large
+ options.body = getNewBody();
+ }
+ }
+ ]
+ }
+});
+```
+
+**Note:** When retrying in a `afterResponse` hook, all remaining `beforeRetry` hooks will be called without the `error` and `retryCount` arguments.
+
+###### hooks.afterResponse
+
+Type: `Function[]`\
+Default: `[]`
+
+**Note:** When using streams, this hook is ignored.
+
+Called with [response object](#response) and a retry function. Calling the retry function will trigger `beforeRetry` hooks.
+
+Each function should return the response. This is especially useful when you want to refresh an access token. Example:
+
+```js
+const got = require('got');
+
+const instance = got.extend({
+ hooks: {
+ afterResponse: [
+ (response, retryWithMergedOptions) => {
+ if (response.statusCode === 401) { // Unauthorized
+ const updatedOptions = {
+ headers: {
+ token: getNewToken() // Refresh the access token
+ }
+ };
+
+ // Save for further requests
+ instance.defaults.options = got.mergeOptions(instance.defaults.options, updatedOptions);
+
+ // Make a new retry
+ return retryWithMergedOptions(updatedOptions);
+ }
+
+ // No changes otherwise
+ return response;
+ }
+ ],
+ beforeRetry: [
+ (options, error, retryCount) => {
+ // This will be called on `retryWithMergedOptions(...)`
+ }
+ ]
+ },
+ mutableDefaults: true
+});
+```
+
+###### hooks.beforeError
+
+Type: `Function[]`\
+Default: `[]`
+
+Called with an `Error` instance. The error is passed to the hook right before it's thrown. This is especially useful when you want to have more detailed errors.
+
+**Note:** Errors thrown while normalizing input options are thrown directly and not part of this hook.
+
+```js
+const got = require('got');
+
+got('https://api.github.com/some-endpoint', {
+ hooks: {
+ beforeError: [
+ error => {
+ const {response} = error;
+ if (response && response.body) {
+ error.name = 'GitHubError';
+ error.message = `${response.body.message} (${response.statusCode})`;
+ }
+
+ return error;
+ }
+ ]
+ }
+});
+```
+
+##### pagination
+
+Type: `object`
+
+**Note:** We're [looking for feedback](https://github.com/sindresorhus/got/issues/1052), any ideas on how to improve the API are welcome.
+
+###### pagination.transform
+
+Type: `Function`\
+Default: `response => JSON.parse(response.body)`
+
+A function that transform [`Response`](#response) into an array of items. This is where you should do the parsing.
+
+###### pagination.paginate
+
+Type: `Function`\
+Default: [`Link` header logic](source/index.ts)
+
+The function takes three arguments:
+- `response` - The current response object.
+- `allItems` - An array of the emitted items.
+- `currentItems` - Items from the current response.
+
+It should return an object representing Got options pointing to the next page. The options are merged automatically with the previous request, therefore the options returned `pagination.paginate(...)` must reflect changes only. If there are no more pages, `false` should be returned.
+
+For example, if you want to stop when the response contains less items than expected, you can use something like this:
+
+```js
+const got = require('got');
+
+(async () => {
+ const limit = 10;
+
+ const items = got.paginate('https://example.com/items', {
+ searchParams: {
+ limit,
+ offset: 0
+ },
+ pagination: {
+ paginate: (response, allItems, currentItems) => {
+ const previousSearchParams = response.request.options.searchParams;
+ const previousOffset = previousSearchParams.get('offset');
+
+ if (currentItems.length < limit) {
+ return false;
+ }
+
+ return {
+ searchParams: {
+ ...previousSearchParams,
+ offset: Number(previousOffset) + limit,
+ }
+ };
+ }
+ }
+ });
+
+ console.log('Items from all pages:', items);
+})();
+```
+
+###### pagination.filter
+
+Type: `Function`\
+Default: `(item, allItems, currentItems) => true`
+
+Checks whether the item should be emitted or not.
+
+###### pagination.shouldContinue
+
+Type: `Function`\
+Default: `(item, allItems, currentItems) => true`
+
+Checks whether the pagination should continue.
+
+For example, if you need to stop **before** emitting an entry with some flag, you should use `(item, allItems, currentItems) => !item.flag`. If you want to stop **after** emitting the entry, you should use `(item, allItems, currentItems) => allItems.some(entry => entry.flag)` instead.
+
+###### pagination.countLimit
+
+Type: `number`\
+Default: `Infinity`
+
+The maximum amount of items that should be emitted.
+
+###### pagination.backoff
+
+Type: `number`\
+Default: `0`
+
+Milliseconds to wait before the next request is triggered.
+
+###### pagination.requestLimit
+
+Type: `number`\
+Default: `10000`
+
+The maximum amount of request that should be triggered. [Retries on failure](#retry) are not counted towards this limit.
+
+For example, it can be helpful during development to avoid an infinite number of requests.
+
+###### pagination.stackAllItems
+
+Type: `boolean`\
+Default: `true`
+
+Defines how the parameter `allItems` in [pagination.paginate](#pagination.paginate), [pagination.filter](#pagination.filter) and [pagination.shouldContinue](#pagination.shouldContinue) is managed. When set to `false`, the parameter `allItems` is always an empty array.
+
+This option can be helpful to save on memory usage when working with a large dataset.
+
+##### localAddress
+
+Type: `string`
+
+The IP address used to send the request from.
+
+### Advanced HTTPS API
+
+Note: If the request is not HTTPS, these options will be ignored.
+
+##### https.certificateAuthority
+
+Type: `string | Buffer | Array`
+
+Override the default Certificate Authorities ([from Mozilla](https://ccadb-public.secure.force.com/mozilla/IncludedCACertificateReport))
+
+```js
+// Single Certificate Authority
+got('https://example.com', {
+ https: {
+ certificateAuthority: fs.readFileSync('./my_ca.pem')
+ }
+});
+```
+
+##### https.key
+
+Type: `string | Buffer | Array | object[]`
+
+Private keys in [PEM](https://en.wikipedia.org/wiki/Privacy-Enhanced_Mail) format.\
+[PEM](https://en.wikipedia.org/wiki/Privacy-Enhanced_Mail) allows the option of private keys being encrypted. Encrypted keys will be decrypted with `options.https.passphrase`.\
+Multiple keys with different passphrases can be provided as an array of `{pem: , passphrase: }`
+
+##### https.certificate
+
+Type: `string | Buffer | (string | Buffer)[]`
+
+[Certificate chains](https://en.wikipedia.org/wiki/X.509#Certificate_chains_and_cross-certification) in [PEM](https://en.wikipedia.org/wiki/Privacy-Enhanced_Mail) format.\
+One cert chain should be provided per private key (`options.https.key`).\
+When providing multiple cert chains, they do not have to be in the same order as their private keys in `options.https.key`.\
+If the intermediate certificates are not provided, the peer will not be able to validate the certificate, and the handshake will fail.
+
+##### https.passphrase
+
+Type: `string`
+
+The passphrase to decrypt the `options.https.key` (if different keys have different passphrases refer to `options.https.key` documentation).
+
+##### https.pfx
+
+Type: `string | Buffer | Array`
+
+[PFX or PKCS12](https://en.wikipedia.org/wiki/PKCS_12) encoded private key and certificate chain. Using `options.https.pfx` is an alternative to providing `options.https.key` and `options.https.certificate` individually. A PFX is usually encrypted, and if it is, `options.https.passphrase` will be used to decrypt it.
+
+Multiple PFX's can be be provided as an array of unencrypted buffers or an array of objects like:
+
+```ts
+{
+ buffer: string | Buffer,
+ passphrase?: string
+}
+```
+
+This object form can only occur in an array. If the provided buffers are encrypted, `object.passphrase` can be used to decrypt them. If `object.passphrase` is not provided, `options.https.passphrase` will be used for decryption.
+
+##### Examples for `https.key`, `https.certificate`, `https.passphrase`, and `https.pfx`
+
+```js
+// Single key with certificate
+got('https://example.com', {
+ https: {
+ key: fs.readFileSync('./client_key.pem'),
+ certificate: fs.readFileSync('./client_cert.pem')
+ }
+});
+
+// Multiple keys with certificates (out of order)
+got('https://example.com', {
+ https: {
+ key: [
+ fs.readFileSync('./client_key1.pem'),
+ fs.readFileSync('./client_key2.pem')
+ ],
+ certificate: [
+ fs.readFileSync('./client_cert2.pem'),
+ fs.readFileSync('./client_cert1.pem')
+ ]
+ }
+});
+
+// Single key with passphrase
+got('https://example.com', {
+ https: {
+ key: fs.readFileSync('./client_key.pem'),
+ certificate: fs.readFileSync('./client_cert.pem'),
+ passphrase: 'client_key_passphrase'
+ }
+});
+
+// Multiple keys with different passphrases
+got('https://example.com', {
+ https: {
+ key: [
+ {pem: fs.readFileSync('./client_key1.pem'), passphrase: 'passphrase1'},
+ {pem: fs.readFileSync('./client_key2.pem'), passphrase: 'passphrase2'},
+ ],
+ certificate: [
+ fs.readFileSync('./client_cert1.pem'),
+ fs.readFileSync('./client_cert2.pem')
+ ]
+ }
+});
+
+// Single encrypted PFX with passphrase
+got('https://example.com', {
+ https: {
+ pfx: fs.readFileSync('./fake.pfx'),
+ passphrase: 'passphrase'
+ }
+});
+
+// Multiple encrypted PFX's with different passphrases
+got('https://example.com', {
+ https: {
+ pfx: [
+ {
+ buffer: fs.readFileSync('./key1.pfx'),
+ passphrase: 'passphrase1'
+ },
+ {
+ buffer: fs.readFileSync('./key2.pfx'),
+ passphrase: 'passphrase2'
+ }
+ ]
+ }
+});
+
+// Multiple encrypted PFX's with single passphrase
+got('https://example.com', {
+ https: {
+ passphrase: 'passphrase',
+ pfx: [
+ {
+ buffer: fs.readFileSync('./key1.pfx')
+ },
+ {
+ buffer: fs.readFileSync('./key2.pfx')
+ }
+ ]
+ }
+});
+```
+
+##### https.rejectUnauthorized
+
+Type: `boolean`\
+Default: `true`
+
+If set to `false`, all invalid SSL certificates will be ignored and no error will be thrown.\
+If set to `true`, it will throw an error whenever an invalid SSL certificate is detected.
+
+We strongly recommend to have this set to `true` for security reasons.
+
+```js
+const got = require('got');
+
+(async () => {
+ // Correct:
+ await got('https://example.com', {
+ https: {
+ rejectUnauthorized: true
+ }
+ });
+
+ // You can disable it when developing an HTTPS app:
+ await got('https://localhost', {
+ https: {
+ rejectUnauthorized: false
+ }
+ });
+
+ // Never do this:
+ await got('https://example.com', {
+ https: {
+ rejectUnauthorized: false
+ }
+ });
+```
+
+##### https.checkServerIdentity
+
+Type: `Function`\
+Signature: `(hostname: string, certificate: DetailedPeerCertificate) => Error | undefined`\
+Default: `tls.checkServerIdentity` (from the `tls` module)
+
+This function enable a custom check of the certificate.\
+Note: In order to have the function called the certificate must not be `expired`, `self-signed` or with an `untrusted-root`.\
+The function parameters are:
+- `hostname`: The server hostname (used when connecting)
+- `certificate`: The server certificate
+
+The function must return `undefined` if the check succeeded or an `Error` if it failed.
+
+```js
+await got('https://example.com', {
+ https: {
+ checkServerIdentity: (hostname, certificate) => {
+ if (hostname === 'example.com') {
+ return; // Certificate OK
+ }
+
+ return new Error('Invalid Hostname'); // Certificate NOT OK
+ }
+ }
+});
+```
+
+#### Response
+
+The response object will typically be a [Node.js HTTP response stream](https://nodejs.org/api/http.html#http_class_http_incomingmessage), however, if returned from the cache it will be a [response-like object](https://github.com/lukechilds/responselike) which behaves in the same way.
+
+##### request
+
+Type: `object`
+
+**Note:** This is not a [http.ClientRequest](https://nodejs.org/api/http.html#http_class_http_clientrequest).
+
+- `options` - The Got options that were set on this request.
+
+##### body
+
+Type: `string | object | Buffer` *(Depending on `options.responseType`)*
+
+The result of the request.
+
+##### rawBody
+
+Type: `Buffer`
+
+The raw result of the request.
+
+##### url
+
+Type: `string`
+
+The request URL or the final URL after redirects.
+
+##### ip
+
+Type: `string`
+
+The remote IP address.
+
+**Note:** Not available when the response is cached. This is hopefully a temporary limitation, see [lukechilds/cacheable-request#86](https://github.com/lukechilds/cacheable-request/issues/86).
+
+##### requestUrl
+
+Type: `string`
+
+The original request URL.
+
+##### timings
+
+Type: `object`
+
+The object contains the following properties:
+
+- `start` - Time when the request started.
+- `socket` - Time when a socket was assigned to the request.
+- `lookup` - Time when the DNS lookup finished.
+- `connect` - Time when the socket successfully connected.
+- `secureConnect` - Time when the socket securely connected.
+- `upload` - Time when the request finished uploading.
+- `response` - Time when the request fired `response` event.
+- `end` - Time when the response fired `end` event.
+- `error` - Time when the request fired `error` event.
+- `abort` - Time when the request fired `abort` event.
+- `phases`
+ - `wait` - `timings.socket - timings.start`
+ - `dns` - `timings.lookup - timings.socket`
+ - `tcp` - `timings.connect - timings.lookup`
+ - `tls` - `timings.secureConnect - timings.connect`
+ - `request` - `timings.upload - (timings.secureConnect || timings.connect)`
+ - `firstByte` - `timings.response - timings.upload`
+ - `download` - `timings.end - timings.response`
+ - `total` - `(timings.end || timings.error || timings.abort) - timings.start`
+
+If something has not been measured yet, it will be `undefined`.
+
+**Note:** The time is a `number` representing the milliseconds elapsed since the UNIX epoch.
+
+##### isFromCache
+
+Type: `boolean`
+
+Whether the response was retrieved from the cache.
+
+##### redirectUrls
+
+Type: `string[]`
+
+The redirect URLs.
+
+##### retryCount
+
+Type: `number`
+
+The number of times the request was retried.
+
+#### Streams
+
+**Note:** Progress events, redirect events and request/response events can also be used with promises.
+
+**Note:** To access `response.isFromCache` you need to use `got.stream(url, options).isFromCache`. The value will be undefined until the `response` event.
+
+#### got.stream(url, options?)
+
+Sets `options.isStream` to `true`.
+
+Returns a [duplex stream](https://nodejs.org/api/stream.html#stream_class_stream_duplex) with additional events:
+
+##### .on('request', request)
-> Simplified HTTP requests
+`request` event to get the request object of the request.
+
+**Tip:** You can use `request` event to abort request:
-[](https://travis-ci.org/sindresorhus/got) [](https://coveralls.io/github/sindresorhus/got?branch=master) [](https://npmjs.com/got)
+```js
+got.stream('https://github.com')
+ .on('request', request => setTimeout(() => request.destroy(), 50));
+```
-A nicer interface to the built-in [`http`](http://nodejs.org/api/http.html) module.
+##### .on('response', response)
-It supports following redirects, promises, streams, retries, automagically handling gzip/deflate and some convenience options.
+The `response` event to get the response object of the final request.
-Created because [`request`](https://github.com/request/request) is bloated *(several megabytes!)*.
+##### .on('redirect', response, nextOptions)
+The `redirect` event to get the response object of a redirect. The second argument is options for the next request to the redirect location.
-## Install
+##### .on('uploadProgress', progress)
+##### .uploadProgress
+##### .on('downloadProgress', progress)
+##### .downloadProgress
-**WARNING: Node.js 4 or higher is required for got@6 and above.** For older Node.js versions use [got@5](https://github.com/sindresorhus/got/tree/v5.x).
+Progress events for uploading (sending a request) and downloading (receiving a response). The `progress` argument is an object like:
+```js
+{
+ percent: 0.1,
+ transferred: 1024,
+ total: 10240
+}
```
-$ npm install --save got
+
+If the `content-length` header is missing, `total` will be `undefined`.
+
+```js
+(async () => {
+ const response = await got('https://sindresorhus.com')
+ .on('downloadProgress', progress => {
+ // Report download progress
+ })
+ .on('uploadProgress', progress => {
+ // Report upload progress
+ });
+
+ console.log(response);
+})();
```
+##### .once('retry', retryCount, error)
-## Usage
+To enable retrying on a Got stream, it is required to have a `retry` handler attached.\
+When this event is emitted, you should reset the stream you were writing to and prepare the body again.
+
+See the [`retry`](#retry-stream) option for an example implementation.
+
+##### .ip
+
+Type: `string`
+
+The remote IP address.
+
+##### .aborted
+
+Type: `boolean`
+
+Indicates whether the request has been aborted or not.
+
+##### .timings
+
+The same as `response.timings`.
+
+##### .isFromCache
+
+The same as `response.isFromCache`.
+
+##### .socket
+
+The same as `response.socket`.
+
+##### .on('error', error)
+
+The emitted `error` is an instance of [`RequestError`](#gotrequesterror).
+
+#### Pagination
+
+#### got.paginate(url, options?)
+#### got.paginate.each(url, options?)
+
+Returns an async iterator:
```js
-const fs = require('fs');
-const got = require('got');
+(async () => {
+ const countLimit = 10;
-got('todomvc.com')
- .then(response => {
- console.log(response.body);
- //=> ' ...'
- })
- .catch(error => {
- console.log(error.response.body);
- //=> 'Internal server error ...'
+ const pagination = got.paginate('https://api.github.com/repos/sindresorhus/got/commits', {
+ pagination: {countLimit}
});
-// Streams
-got.stream('todomvc.com').pipe(fs.createWriteStream('index.html'));
+ console.log(`Printing latest ${countLimit} Got commits (newest to oldest):`);
-// For POST, PUT and PATCH methods got.stream returns a WritableStream
-fs.createReadStream('index.html').pipe(got.stream.post('todomvc.com'));
+ for await (const commitData of pagination) {
+ console.log(commitData.commit.message);
+ }
+})();
```
+See [`options.pagination`](#pagination) for more pagination options.
-### API
+#### got.paginate.all(url, options?)
-It's a `GET` request by default, but can be changed in `options`.
+Returns a Promise for an array of all results:
-#### got(url, [options])
+```js
+(async () => {
+ const countLimit = 10;
-Returns a Promise for a `response` object with a `body` property, a `url` property with the request URL or the final URL after redirects, and a `requestUrl` property with the original request URL.
+ const results = await got.paginate.all('https://api.github.com/repos/sindresorhus/got/commits', {
+ pagination: {countLimit}
+ });
-##### url
+ console.log(`Printing latest ${countLimit} Got commits (newest to oldest):`);
+ console.log(results);
+})();
+```
-Type: `string`, `object`
+See [`options.pagination`](#pagination) for more pagination options.
-The URL to request or a [`http.request` options](https://nodejs.org/api/http.html#http_http_request_options_callback) object.
+#### got.get(url, options?)
+#### got.post(url, options?)
+#### got.put(url, options?)
+#### got.patch(url, options?)
+#### got.head(url, options?)
+#### got.delete(url, options?)
-Properties from `options` will override properties in the parsed `url`.
+Sets [`options.method`](#method) to the method name and makes a request.
-##### options
+### Instances
-Type: `object`
+#### got.extend(...options)
-Any of the [`http.request`](http://nodejs.org/api/http.html#http_http_request_options_callback) options.
+Configure a new `got` instance with default `options`. The `options` are merged with the parent instance's `defaults.options` using [`got.mergeOptions`](#gotmergeoptionsparentoptions-newoptions). You can access the resolved options with the `.defaults` property on the instance.
-###### body
+```js
+const client = got.extend({
+ prefixUrl: 'https://example.com',
+ headers: {
+ 'x-unicorn': 'rainbow'
+ }
+});
-Type: `string`, `buffer`, `readableStream`, `object`
+client.get('demo');
-*This is mutually exclusive with stream mode.*
+/* HTTP Request =>
+ * GET /demo HTTP/1.1
+ * Host: example.com
+ * x-unicorn: rainbow
+ */
+```
-Body that will be sent with a `POST` request.
+```js
+(async () => {
+ const client = got.extend({
+ prefixUrl: 'httpbin.org',
+ headers: {
+ 'x-foo': 'bar'
+ }
+ });
+ const {headers} = await client.get('headers').json();
+ //=> headers['x-foo'] === 'bar'
+
+ const jsonClient = client.extend({
+ responseType: 'json',
+ resolveBodyOnly: true,
+ headers: {
+ 'x-baz': 'qux'
+ }
+ });
+ const {headers: headers2} = await jsonClient.get('headers');
+ //=> headers2['x-foo'] === 'bar'
+ //=> headers2['x-baz'] === 'qux'
+})();
+```
-If present in `options` and `options.method` is not set, `options.method` will be set to `POST`.
+Additionally, `got.extend()` accepts two properties from the `defaults` object: `mutableDefaults` and `handlers`. Example:
-If `content-length` or `transfer-encoding` is not set in `options.headers` and `body` is a string or buffer, `content-length` will be set to the body length.
+```js
+// You can now modify `mutableGot.defaults.options`.
+const mutableGot = got.extend({mutableDefaults: true});
-If `body` is a plain object, it will be stringified with [`querystring.stringify`](https://nodejs.org/api/querystring.html#querystring_querystring_stringify_obj_sep_eq_options) and sent as `application/x-www-form-urlencoded`.
+const mergedHandlers = got.extend({
+ handlers: [
+ (options, next) => {
+ delete options.headers.referer;
-###### encoding
+ return next(options);
+ }
+ ]
+});
+```
-Type: `string`, `null`
-Default: `'utf8'`
+**Note:** Handlers can be asynchronous. The recommended approach is:
-Encoding to be used on `setEncoding` of the response data. If `null`, the body is returned as a Buffer.
+```js
+const handler = (options, next) => {
+ if (options.isStream) {
+ // It's a Stream
+ return next(options);
+ }
-###### json
+ // It's a Promise
+ return (async () => {
+ try {
+ const response = await next(options);
+ response.yourOwnProperty = true;
+ return response;
+ } catch (error) {
+ // Every error will be replaced by this one.
+ // Before you receive any error here,
+ // it will be passed to the `beforeError` hooks first.
+ // Note: this one won't be passed to `beforeError` hook. It's final.
+ throw new Error('Your very own error.');
+ }
+ })();
+};
-Type: `boolean`
-Default: `false`
+const instance = got.extend({handlers: [handler]});
+```
-*This is mutually exclusive with stream mode.*
+#### got.extend(...options, ...instances, ...)
-Parse response body with `JSON.parse` and set `accept` header to `application/json`.
+Merges many instances into a single one:
+- options are merged using [`got.mergeOptions()`](#gotmergeoptionsparentoptions-newoptions) (including hooks),
+- handlers are stored in an array (you can access them through `instance.defaults.handlers`).
-###### query
+```js
+const a = {headers: {cat: 'meow'}};
+const b = got.extend({
+ options: {
+ headers: {
+ cow: 'moo'
+ }
+ }
+});
-Type: `string`, `object`
+// The same as `got.extend(a).extend(b)`.
+// Note `a` is options and `b` is an instance.
+got.extend(a, b);
+//=> {headers: {cat: 'meow', cow: 'moo'}}
+```
-Query string object that will be added to the request URL. This will override the query string in `url`.
+#### got.mergeOptions(parent, ...sources)
-###### timeout
+Extends parent options. Avoid using [object spread](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Spread_syntax#Spread_in_object_literals) as it doesn't work recursively:
-Type: `number`, `object`
+```js
+const a = {headers: {cat: 'meow', wolf: ['bark', 'wrrr']}};
+const b = {headers: {cow: 'moo', wolf: ['auuu']}};
-Milliseconds to wait for a server to send response headers before aborting request with `ETIMEDOUT` error.
+{...a, ...b} // => {headers: {cow: 'moo', wolf: ['auuu']}}
+got.mergeOptions(a, b) // => {headers: {cat: 'meow', cow: 'moo', wolf: ['auuu']}}
+```
-Option accepts `object` with separate `connect` and `socket` fields for connection and socket inactivity timeouts.
+**Note:** Only Got options are merged! Custom user options should be defined via [`options.context`](#context).
-###### retries
+Options are deeply merged to a new object. The value of each key is determined as follows:
-Type: `number`, `function`
-Default: `5`
+- If the new property is not defined, the old value is used.
+- If the new property is explicitly set to `undefined`:
+ - If the parent property is a plain `object`, the parent value is deeply cloned.
+ - Otherwise, `undefined` is used.
+- If the parent value is an instance of `URLSearchParams`:
+ - If the new value is a `string`, an `object` or an instance of `URLSearchParams`, a new `URLSearchParams` instance is created. The values are merged using [`urlSearchParams.append(key, value)`](https://developer.mozilla.org/en-US/docs/Web/API/URLSearchParams/append). The keys defined in the new value override the keys defined in the parent value. Please note that `null` values point to an empty string and `undefined` values will exclude the entry.
+ - Otherwise, the only available value is `undefined`.
+- If the new property is a plain `object`:
+ - If the parent property is a plain `object` too, both values are merged recursively into a new `object`.
+ - Otherwise, only the new value is deeply cloned.
+- If the new property is an `Array`, it overwrites the old one with a deep clone of the new property.
+- Properties that are not enumerable, such as `context`, `body`, `json`, and `form`, will not be merged.
+- Otherwise, the new value is assigned to the key.
-Number of request retries when network errors happens. Delays between retries counts with function `1000 * Math.pow(2, retry) + Math.random() * 100`, where `retry` is attempt number (starts from 0).
+```js
+const a = {json: {cat: 'meow'}};
+const b = {json: {cow: 'moo'}};
-Option accepts `function` with `retry` and `error` arguments. Function must return delay in milliseconds (`0` return value cancels retry).
+got.mergeOptions(a, b);
+//=> {json: {cow: 'moo'}}
+```
-**Note:** if `retries` is `number`, `ENOTFOUND` and `ENETUNREACH` error will not be retried (see full list in [`is-retry-allowed`](https://github.com/floatdrop/is-retry-allowed/blob/master/index.js#L12) module).
+#### got.defaults
-###### followRedirect
+Type: `object`
-Type: `boolean`
-Default: `true`
+The Got defaults used in that instance.
-Defines if redirect responses should be followed automatically.
+##### [options](#options)
+##### handlers
-#### Streams
+Type: `Function[]`\
+Default: `[]`
-#### got.stream(url, [options])
+An array of functions. You execute them directly by calling `got()`. They are some sort of "global hooks" - these functions are called first. The last handler (*it's hidden*) is either [`asPromise`](source/core/as-promise/index.ts) or [`asStream`](source/core/index.ts), depending on the `options.isStream` property.
-`stream` method will return Duplex stream with additional events:
+Each handler takes two arguments:
-##### .on('request', request)
+###### [options](#options)
-`request` event to get the request object of the request.
+###### next()
-**Tip**: You can use `request` event to abort request:
+Returns a `Promise` or a `Stream` depending on [`options.isStream`](#isstream).
```js
-got.stream('github.com')
- .on('request', req => setTimeout(() => req.abort(), 50));
+const settings = {
+ handlers: [
+ (options, next) => {
+ if (options.isStream) {
+ // It's a Stream, so we can perform stream-specific actions on it
+ return next(options)
+ .on('request', request => {
+ setTimeout(() => {
+ request.abort();
+ }, 50);
+ });
+ }
+
+ // It's a Promise
+ return next(options);
+ }
+ ],
+ options: got.mergeOptions(got.defaults.options, {
+ responseType: 'json'
+ })
+};
+
+const jsonGot = got.extend(settings);
```
-##### .on('response', response)
+##### mutableDefaults
+
+Type: `boolean`\
+Default: `false`
-`response` event to get the response object of the final request.
+A read-only boolean describing whether the defaults are mutable or not. If set to `true`, you can [update headers over time](#hooksafterresponse), for example, update an access token when it expires.
-##### .on('redirect', response, nextOptions)
+## Types
-`redirect` event to get the response object of a redirect. The second argument is options for the next request to the redirect location.
+Got exports some handy TypeScript types and interfaces. See the type definition for all the exported types.
-##### .on('error', error, body, response)
+### Got
-`error` event emitted in case of protocol error (like `ENOTFOUND` etc.) or status error (4xx or 5xx). The second argument is the body of the server response in case of status error. The third argument is response object.
+TypeScript will automatically infer types for Got instances, but in case you want to define something like dependencies, you can import the available types directly from Got.
-#### got.get(url, [options])
-#### got.post(url, [options])
-#### got.put(url, [options])
-#### got.patch(url, [options])
-#### got.head(url, [options])
-#### got.delete(url, [options])
+```ts
+import {GotRequestFunction} from 'got';
-Sets `options.method` to the method name and makes a request.
+interface Dependencies {
+ readonly post: GotRequestFunction
+}
+```
+### Hooks
-## Errors
+When writing hooks, you can refer to their types to keep your interfaces consistent.
+
+```ts
+import {BeforeRequestHook} from 'got';
+
+const addAccessToken = (accessToken: string): BeforeRequestHook => options => {
+ options.path = `${options.path}?access_token=${accessToken}`;
+}
+```
-Each error contains (if available) `statusCode`, `statusMessage`, `host`, `hostname`, `method` and `path` properties to make debugging easier.
+## Errors
-In Promise mode, the `response` is attached to the error.
+Each error contains an `options` property which are the options Got used to create a request - just to make debugging easier.\
+Additionaly, the errors may have `request` (Got Stream) and `response` (Got Response) properties depending on which phase of the request failed.
#### got.RequestError
-When a request fails. Contains a `code` property with error class code, like `ECONNREFUSED`.
+When a request fails. Contains a `code` property with error class code, like `ECONNREFUSED`. If there is no specific code supplied, `code` defaults to `ERR_GOT_REQUEST_ERROR`. All the errors below inherit this one.
+
+#### got.CacheError
+
+When a cache method fails, for example, if the database goes down or there's a filesystem error. Contains a `code` property with `ERR_CACHE_ACCESS` or a more specific failure code.
#### got.ReadError
-When reading from response stream fails.
+When reading from response stream fails. Contains a `code` property with `ERR_READING_RESPONSE_STREAM` or a more specific failure code.
#### got.ParseError
-When `json` option is enabled and `JSON.parse` fails.
+When server response code is 2xx, and parsing body fails. Includes a `response` property. Contains a `code` property with `ERR_BODY_PARSE_FAILURE` or a more specific failure code.
+
+#### got.UploadError
+
+When the request body is a stream and an error occurs while reading from that stream. Contains a `code` property with `ERR_UPLOAD` or a more specific failure code.
#### got.HTTPError
-When server response code is not 2xx. Contains `statusCode` and `statusMessage`.
+When the server response code is not 2xx nor 3xx if `options.followRedirect` is `true`, but always except for 304. Includes a `response` property. Contains a `code` property with `ERR_NON_2XX_3XX_RESPONSE` or a more specific failure code.
+
#### got.MaxRedirectsError
-When server redirects you more than 10 times.
+When the server redirects you more than ten times. Includes a `response` property. Contains a `code` property with `ERR_TOO_MANY_REDIRECTS`.
+#### got.UnsupportedProtocolError
-## Proxies
+When given an unsupported protocol. Contains a `code` property with `ERR_UNSUPPORTED_PROTOCOL`.
+
+#### got.TimeoutError
+
+When the request is aborted due to a [timeout](#timeout). Includes an `event` and `timings` property. Contains a `code` property with `ETIMEDOUT`.
+
+#### got.CancelError
+
+When the request is aborted with `.cancel()`. Contains a `code` property with `ERR_CANCELED`.
-You can use the [`tunnel`](https://github.com/koichik/node-tunnel) module with the `agent` option to work with proxies:
+## Aborting the request
+
+The promise returned by Got has a [`.cancel()`](https://github.com/sindresorhus/p-cancelable) method which when called, aborts the request.
+
+```js
+(async () => {
+ const request = got(url, options);
+
+ // …
+
+ // In another part of the code
+ if (something) {
+ request.cancel();
+ }
+
+ // …
+
+ try {
+ await request;
+ } catch (error) {
+ if (request.isCanceled) { // Or `error instanceof got.CancelError`
+ // Handle cancelation
+ }
+
+ // Handle other errors
+ }
+})();
+```
+
+When using hooks, simply throw an error to abort the request.
```js
const got = require('got');
-const tunnel = require('tunnel');
-got('todomvc.com', {
- agent: tunnel.httpOverHttp({
- proxy: {
- host: 'localhost'
+(async () => {
+ const request = got(url, {
+ hooks: {
+ beforeRequest: [
+ () => {
+ throw new Error('Oops. Request canceled.');
+ }
+ ]
}
- })
-});
+ });
+
+ try {
+ await request;
+ } catch (error) {
+ // …
+ }
+})();
```
+To abort the Got Stream request, just call `stream.destroy()`.
-## Cookies
+```js
+const got = require('got');
+
+const stream = got.stream(url);
+stream.destroy();
+```
-You can use the [`cookie`](https://github.com/jshttp/cookie) module to include cookies in a request:
+
+## Cache
+
+Got implements [RFC 7234](https://httpwg.org/specs/rfc7234.html) compliant HTTP caching which works out of the box in-memory and is easily pluggable with a wide range of storage adapters. Fresh cache entries are served directly from the cache, and stale cache entries are revalidated with `If-None-Match`/`If-Modified-Since` headers. You can read more about the underlying cache behavior in the [`cacheable-request` documentation](https://github.com/lukechilds/cacheable-request). For DNS cache, Got uses [`cacheable-lookup`](https://github.com/szmarczak/cacheable-lookup).
+
+You can use the JavaScript `Map` type as an in-memory cache:
```js
const got = require('got');
-const cookie = require('cookie');
-got('google.com', {
- headers: {
- cookie: cookie.serialize('foo', 'bar')
+const map = new Map();
+
+(async () => {
+ let response = await got('https://sindresorhus.com', {cache: map});
+ console.log(response.isFromCache);
+ //=> false
+
+ response = await got('https://sindresorhus.com', {cache: map});
+ console.log(response.isFromCache);
+ //=> true
+})();
+```
+
+Got uses [Keyv](https://github.com/lukechilds/keyv) internally to support a wide range of storage adapters. For something more scalable you could use an [official Keyv storage adapter](https://github.com/lukechilds/keyv#official-storage-adapters):
+
+```
+$ npm install @keyv/redis
+```
+
+```js
+const got = require('got');
+const KeyvRedis = require('@keyv/redis');
+
+const redis = new KeyvRedis('redis://user:pass@localhost:6379');
+
+got('https://sindresorhus.com', {cache: redis});
+```
+
+Got supports anything that follows the Map API, so it's easy to write your own storage adapter or use a third-party solution.
+
+For example, the following are all valid storage adapters:
+
+```js
+const storageAdapter = new Map();
+// Or
+const storageAdapter = require('./my-storage-adapter');
+// Or
+const QuickLRU = require('quick-lru');
+const storageAdapter = new QuickLRU({maxSize: 1000});
+
+got('https://sindresorhus.com', {cache: storageAdapter});
+```
+
+View the [Keyv docs](https://github.com/lukechilds/keyv) for more information on how to use storage adapters.
+
+## Proxies
+
+You can use the [`tunnel`](https://github.com/koichik/node-tunnel) package with the `agent` option to work with proxies:
+
+```js
+const got = require('got');
+const tunnel = require('tunnel');
+
+got('https://sindresorhus.com', {
+ agent: {
+ https: tunnel.httpsOverHttp({
+ proxy: {
+ host: 'localhost'
+ }
+ })
+ }
+});
+```
+
+Otherwise, you can use the [`hpagent`](https://github.com/delvedor/hpagent) package, which keeps the internal sockets alive to be reused.
+
+```js
+const got = require('got');
+const {HttpsProxyAgent} = require('hpagent');
+
+got('https://sindresorhus.com', {
+ agent: {
+ https: new HttpsProxyAgent({
+ keepAlive: true,
+ keepAliveMsecs: 1000,
+ maxSockets: 256,
+ maxFreeSockets: 256,
+ scheduling: 'lifo',
+ proxy: 'https://localhost:8080'
+ })
}
});
```
+Alternatively, use [`global-agent`](https://github.com/gajus/global-agent) to configure a global proxy for all HTTP/HTTPS traffic in your program.
+
+Read the [`http2-wrapper`](https://github.com/szmarczak/http2-wrapper/#proxy-support) docs to learn about proxying for HTTP/2.
+
+## Cookies
+
+You can use the [`tough-cookie`](https://github.com/salesforce/tough-cookie) package:
+
+```js
+const {promisify} = require('util');
+const got = require('got');
+const {CookieJar} = require('tough-cookie');
+
+(async () => {
+ const cookieJar = new CookieJar();
+ const setCookie = promisify(cookieJar.setCookie.bind(cookieJar));
+
+ await setCookie('foo=bar', 'https://example.com');
+ await got('https://example.com', {cookieJar});
+})();
+```
## Form data
-You can use the [`form-data`](https://github.com/form-data/form-data) module to create POST request with form data:
+You can use the [`form-data`](https://github.com/form-data/form-data) package to create POST request with form data:
```js
const fs = require('fs');
const got = require('got');
const FormData = require('form-data');
+
const form = new FormData();
form.append('my_file', fs.createReadStream('/foo/bar.jpg'));
-got.post('google.com', {
+got.post('https://example.com', {
body: form
});
```
-
## OAuth
-You can use the [`oauth-1.0a`](https://github.com/ddo/oauth-1.0a) module to create a signed OAuth request:
+You can use the [`oauth-1.0a`](https://github.com/ddo/oauth-1.0a) package to create a signed OAuth request:
```js
const got = require('got');
@@ -280,56 +1967,511 @@ const url = 'https://api.twitter.com/1.1/statuses/home_timeline.json';
got(url, {
headers: oauth.toHeader(oauth.authorize({url, method: 'GET'}, token)),
- json: true
+ responseType: 'json'
});
```
-
## Unix Domain Sockets
Requests can also be sent via [unix domain sockets](http://serverfault.com/questions/124517/whats-the-difference-between-unix-socket-and-tcp-ip-socket). Use the following URL scheme: `PROTOCOL://unix:SOCKET:PATH`.
- `PROTOCOL` - `http` or `https` *(optional)*
-- `SOCKET` - absolute path to a unix domain socket, e.g. `/var/run/docker.sock`
-- `PATH` - request path, e.g. `/v2/keys`
+- `SOCKET` - Absolute path to a unix domain socket, for example: `/var/run/docker.sock`
+- `PATH` - Request path, for example: `/v2/keys`
```js
+const got = require('got');
+
got('http://unix:/var/run/docker.sock:/containers/json');
-// or without protocol (http by default)
+// Or without protocol (HTTP by default)
got('unix:/var/run/docker.sock:/containers/json');
```
+## AWS
+
+Requests to AWS services need to have their headers signed. This can be accomplished by using the [`got4aws`](https://www.npmjs.com/package/got4aws) package. This is an example for querying an ["API Gateway"](https://docs.aws.amazon.com/apigateway/api-reference/signing-requests/) with a signed request.
+
+```js
+const got4aws = require('got4aws');;
+
+const awsClient = got4aws();
+
+const response = await awsClient('https://.execute-api..amazonaws.com//endpoint/path', {
+ // Request-specific options
+});
+```
+
+## Testing
+
+You can test your requests by using the [`nock`](https://github.com/node-nock/nock) package to mock an endpoint:
+
+```js
+const got = require('got');
+const nock = require('nock');
+
+nock('https://sindresorhus.com')
+ .get('/')
+ .reply(200, 'Hello world!');
+
+(async () => {
+ const response = await got('https://sindresorhus.com');
+ console.log(response.body);
+ //=> 'Hello world!'
+})();
+```
+
+Bear in mind, that by default `nock` mocks only one request. Got will [retry](#retry) on failed requests by default, causing a `No match for request ...` error. The solution is to either disable retrying (set `options.retry` to `0`) or call `.persist()` on the mocked request.
+
+```js
+const got = require('got');
+const nock = require('nock');
+
+const scope = nock('https://sindresorhus.com')
+ .get('/')
+ .reply(500, 'Internal server error')
+ .persist();
+
+(async () => {
+ try {
+ await got('https://sindresorhus.com')
+ } catch (error) {
+ console.log(error.response.body);
+ //=> 'Internal server error'
+
+ console.log(error.response.retryCount);
+ //=> 2
+ }
+
+ scope.persist(false);
+})();
+```
+
+For real integration testing we recommend using [`ava`](https://github.com/avajs/ava) with [`create-test-server`](https://github.com/lukechilds/create-test-server). We're using a macro so we don't have to `server.listen()` and `server.close()` every test. Take a look at one of our tests:
+
+```js
+test('retry function gets iteration count', withServer, async (t, server, got) => {
+ let knocks = 0;
+ server.get('/', (request, response) => {
+ if (knocks++ === 1) {
+ response.end('who`s there?');
+ }
+ });
+
+ await got({
+ retry: {
+ calculateDelay: ({attemptCount}) => {
+ t.true(is.number(attemptCount));
+ return attemptCount < 2 ? 1 : 0;
+ }
+ }
+ });
+});
+```
+
+## Tips
+
+### JSON mode
+
+To pass an object as the body, you need to use the `json` option. It will be stringified using `JSON.stringify`. Example:
+
+```js
+const got = require('got');
+
+(async () => {
+ const {body} = await got.post('https://httpbin.org/anything', {
+ json: {
+ hello: 'world'
+ },
+ responseType: 'json'
+ });
+
+ console.log(body.data);
+ //=> '{"hello":"world"}'
+})();
+```
+
+To receive a JSON body you can either set `responseType` option to `json` or use `promise.json()`. Example:
+
+```js
+const got = require('got');
+
+(async () => {
+ const body = await got.post('https://httpbin.org/anything', {
+ json: {
+ hello: 'world'
+ }
+ }).json();
+
+ console.log(body);
+ //=> {…}
+})();
+```
-## Tip
+### User Agent
-It's a good idea to set the `'user-agent'` header so the provider can more easily see how their resource is used. By default, it's the URL to this repo.
+It's a good idea to set the `'user-agent'` header so the provider can more easily see how their resource is used. By default, it's the URL to this repo. You can omit this header by setting it to `undefined`.
```js
const got = require('got');
const pkg = require('./package.json');
-got('todomvc.com', {
+got('https://sindresorhus.com', {
+ headers: {
+ 'user-agent': `my-package/${pkg.version} (https://github.com/username/my-package)`
+ }
+});
+
+got('https://sindresorhus.com', {
headers: {
- 'user-agent': `my-module/${pkg.version} (https://github.com/username/my-module)`
+ 'user-agent': undefined
}
});
```
+### 304 Responses
-## Related
+Bear in mind; if you send an `if-modified-since` header and receive a `304 Not Modified` response, the body will be empty. It's your responsibility to cache and retrieve the body contents.
+
+### Custom endpoints
-- [gh-got](https://github.com/sindresorhus/gh-got) - Convenience wrapper for interacting with the GitHub API
-- [travis-got](https://github.com/samverschueren/travis-got) - Convenience wrapper for interacting with the Travis API
+Use `got.extend()` to make it nicer to work with REST APIs. Especially if you use the `prefixUrl` option.
+```js
+const got = require('got');
+const pkg = require('./package.json');
-## Created by
+const custom = got.extend({
+ prefixUrl: 'example.com',
+ responseType: 'json',
+ headers: {
+ 'user-agent': `my-package/${pkg.version} (https://github.com/username/my-package)`
+ }
+});
-[](https://sindresorhus.com) | [](https://github.com/floatdrop)
----|---
-[Sindre Sorhus](https://sindresorhus.com) | [Vsevolod Strukchinsky](https://github.com/floatdrop)
+// Use `custom` exactly how you use `got`
+(async () => {
+ const list = await custom('v1/users/list');
+})();
+```
+## FAQ
+
+### Why yet another HTTP client?
+
+Got was created because the popular [`request`](https://github.com/request/request) package is bloated: [](https://packagephobia.now.sh/result?p=request)\
+Furthermore, Got is fully written in TypeScript and actively maintained.
+
+### Electron support has been removed
+
+The Electron `net` module is not consistent with the Node.js `http` module. See [#899](https://github.com/sindresorhus/got/issues/899) for more info.
+
+## Comparison
+
+| | `got` | [`request`][r0] | [`node-fetch`][n0] | [`ky`][k0] | [`axios`][a0] | [`superagent`][s0] |
+|-----------------------|:------------------:|:------------------:|:--------------------:|:------------------------:|:------------------:|:----------------------:|
+| HTTP/2 support | :sparkle: | :x: | :x: | :x: | :x: | :heavy_check_mark:\*\* |
+| Browser support | :x: | :x: | :heavy_check_mark:\* | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
+| Promise API | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
+| Stream API | :heavy_check_mark: | :heavy_check_mark: | Node.js only | :x: | :x: | :heavy_check_mark: |
+| Pagination API | :heavy_check_mark: | :x: | :x: | :x: | :x: | :x: |
+| Request cancelation | :heavy_check_mark: | :x: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
+| RFC compliant caching | :heavy_check_mark: | :x: | :x: | :x: | :x: | :x: |
+| Cookies (out-of-box) | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: | :x: | :x: |
+| Follows redirects | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
+| Retries on failure | :heavy_check_mark: | :x: | :x: | :heavy_check_mark: | :x: | :heavy_check_mark: |
+| Progress events | :heavy_check_mark: | :x: | :x: | :heavy_check_mark:\*\*\* | Browser only | :heavy_check_mark: |
+| Handles gzip/deflate | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
+| Advanced timeouts | :heavy_check_mark: | :x: | :x: | :x: | :x: | :x: |
+| Timings | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: | :x: | :x: |
+| Errors with metadata | :heavy_check_mark: | :x: | :x: | :heavy_check_mark: | :heavy_check_mark: | :x: |
+| JSON mode | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
+| Custom defaults | :heavy_check_mark: | :heavy_check_mark: | :x: | :heavy_check_mark: | :heavy_check_mark: | :x: |
+| Composable | :heavy_check_mark: | :x: | :x: | :x: | :x: | :heavy_check_mark: |
+| Hooks | :heavy_check_mark: | :x: | :x: | :heavy_check_mark: | :heavy_check_mark: | :x: |
+| Issues open | [![][gio]][g1] | [![][rio]][r1] | [![][nio]][n1] | [![][kio]][k1] | [![][aio]][a1] | [![][sio]][s1] |
+| Issues closed | [![][gic]][g2] | [![][ric]][r2] | [![][nic]][n2] | [![][kic]][k2] | [![][aic]][a2] | [![][sic]][s2] |
+| Downloads | [![][gd]][g3] | [![][rd]][r3] | [![][nd]][n3] | [![][kd]][k3] | [![][ad]][a3] | [![][sd]][s3] |
+| Coverage | [![][gc]][g4] | [![][rc]][r4] | [![][nc]][n4] | [![][kc]][k4] | [![][ac]][a4] | [![][sc]][s4] |
+| Build | [![][gb]][g5] | [![][rb]][r5] | [![][nb]][n5] | [![][kb]][k5] | [![][ab]][a5] | [![][sb]][s5] |
+| Bugs | [![][gbg]][g6] | [![][rbg]][r6] | [![][nbg]][n6] | [![][kbg]][k6] | [![][abg]][a6] | [![][sbg]][s6] |
+| Dependents | [![][gdp]][g7] | [![][rdp]][r7] | [![][ndp]][n7] | [![][kdp]][k7] | [![][adp]][a7] | [![][sdp]][s7] |
+| Install size | [![][gis]][g8] | [![][ris]][r8] | [![][nis]][n8] | [![][kis]][k8] | [![][ais]][a8] | [![][sis]][s8] |
+| GitHub stars | [![][gs]][g9] | [![][rs]][r9] | [![][ns]][n9] | [![][ks]][k9] | [![][as]][a9] | [![][ss]][s9] |
+| TypeScript support | [![][gts]][g10] | [![][rts]][r10] | [![][nts]][n10] | [![][kts]][k10] | [![][ats]][a10] | [![][sts]][s11] |
+| Last commit | [![][glc]][g11] | [![][rlc]][r11] | [![][nlc]][n11] | [![][klc]][k11] | [![][alc]][a11] | [![][slc]][s11] |
+
+\* It's almost API compatible with the browser `fetch` API.\
+\*\* Need to switch the protocol manually. Doesn't accept PUSH streams and doesn't reuse HTTP/2 sessions.\
+\*\*\* Currently, only `DownloadProgress` event is supported, `UploadProgress` event is not supported.\
+:sparkle: Almost-stable feature, but the API may change. Don't hesitate to try it out!\
+:grey_question: Feature in early stage of development. Very experimental.
+
+
+[k0]: https://github.com/sindresorhus/ky
+[r0]: https://github.com/request/request
+[n0]: https://github.com/node-fetch/node-fetch
+[a0]: https://github.com/axios/axios
+[s0]: https://github.com/visionmedia/superagent
+
+
+[gio]: https://badgen.net/github/open-issues/sindresorhus/got?label
+[kio]: https://badgen.net/github/open-issues/sindresorhus/ky?label
+[rio]: https://badgen.net/github/open-issues/request/request?label
+[nio]: https://badgen.net/github/open-issues/bitinn/node-fetch?label
+[aio]: https://badgen.net/github/open-issues/axios/axios?label
+[sio]: https://badgen.net/github/open-issues/visionmedia/superagent?label
+
+[g1]: https://github.com/sindresorhus/got/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc
+[k1]: https://github.com/sindresorhus/ky/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc
+[r1]: https://github.com/request/request/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc
+[n1]: https://github.com/bitinn/node-fetch/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc
+[a1]: https://github.com/axios/axios/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc
+[s1]: https://github.com/visionmedia/superagent/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc
+
+
+[gic]: https://badgen.net/github/closed-issues/sindresorhus/got?label
+[kic]: https://badgen.net/github/closed-issues/sindresorhus/ky?label
+[ric]: https://badgen.net/github/closed-issues/request/request?label
+[nic]: https://badgen.net/github/closed-issues/bitinn/node-fetch?label
+[aic]: https://badgen.net/github/closed-issues/axios/axios?label
+[sic]: https://badgen.net/github/closed-issues/visionmedia/superagent?label
+
+[g2]: https://github.com/sindresorhus/got/issues?q=is%3Aissue+is%3Aclosed+sort%3Aupdated-desc
+[k2]: https://github.com/sindresorhus/ky/issues?q=is%3Aissue+is%3Aclosed+sort%3Aupdated-desc
+[r2]: https://github.com/request/request/issues?q=is%3Aissue+is%3Aclosed+sort%3Aupdated-desc
+[n2]: https://github.com/bitinn/node-fetch/issues?q=is%3Aissue+is%3Aclosed+sort%3Aupdated-desc
+[a2]: https://github.com/axios/axios/issues?q=is%3Aissue+is%3Aclosed+sort%3Aupdated-desc
+[s2]: https://github.com/visionmedia/superagent/issues?q=is%3Aissue+is%3Aclosed+sort%3Aupdated-desc
+
+
+[gd]: https://badgen.net/npm/dm/got?label
+[kd]: https://badgen.net/npm/dm/ky?label
+[rd]: https://badgen.net/npm/dm/request?label
+[nd]: https://badgen.net/npm/dm/node-fetch?label
+[ad]: https://badgen.net/npm/dm/axios?label
+[sd]: https://badgen.net/npm/dm/superagent?label
+
+[g3]: https://www.npmjs.com/package/got
+[k3]: https://www.npmjs.com/package/ky
+[r3]: https://www.npmjs.com/package/request
+[n3]: https://www.npmjs.com/package/node-fetch
+[a3]: https://www.npmjs.com/package/axios
+[s3]: https://www.npmjs.com/package/superagent
+
+
+[gc]: https://badgen.net/coveralls/c/github/sindresorhus/got?label
+[kc]: https://badgen.net/codecov/c/github/sindresorhus/ky?label
+[rc]: https://badgen.net/coveralls/c/github/request/request?label
+[nc]: https://badgen.net/coveralls/c/github/bitinn/node-fetch?label
+[ac]: https://badgen.net/coveralls/c/github/mzabriskie/axios?label
+[sc]: https://badgen.net/codecov/c/github/visionmedia/superagent?label
+
+[g4]: https://coveralls.io/github/sindresorhus/got
+[k4]: https://codecov.io/gh/sindresorhus/ky
+[r4]: https://coveralls.io/github/request/request
+[n4]: https://coveralls.io/github/bitinn/node-fetch
+[a4]: https://coveralls.io/github/mzabriskie/axios
+[s4]: https://codecov.io/gh/visionmedia/superagent
+
+
+[gb]: https://badgen.net/travis/sindresorhus/got?label
+[kb]: https://badgen.net/travis/sindresorhus/ky?label
+[rb]: https://badgen.net/travis/request/request?label
+[nb]: https://badgen.net/travis/bitinn/node-fetch?label
+[ab]: https://badgen.net/travis/axios/axios?label
+[sb]: https://badgen.net/travis/visionmedia/superagent?label
+
+[g5]: https://travis-ci.com/github/sindresorhus/got
+[k5]: https://travis-ci.com/github/sindresorhus/ky
+[r5]: https://travis-ci.org/github/request/request
+[n5]: https://travis-ci.org/github/bitinn/node-fetch
+[a5]: https://travis-ci.org/github/axios/axios
+[s5]: https://travis-ci.org/github/visionmedia/superagent
+
+
+[gbg]: https://badgen.net/github/label-issues/sindresorhus/got/bug/open?label
+[kbg]: https://badgen.net/github/label-issues/sindresorhus/ky/bug/open?label
+[rbg]: https://badgen.net/github/label-issues/request/request/Needs%20investigation/open?label
+[nbg]: https://badgen.net/github/label-issues/bitinn/node-fetch/bug/open?label
+[abg]: https://badgen.net/github/label-issues/axios/axios/type:confirmed%20bug/open?label
+[sbg]: https://badgen.net/github/label-issues/visionmedia/superagent/Bug/open?label
+
+[g6]: https://github.com/sindresorhus/got/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3Abug
+[k6]: https://github.com/sindresorhus/ky/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3Abug
+[r6]: https://github.com/request/request/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3A"Needs+investigation"
+[n6]: https://github.com/bitinn/node-fetch/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3Abug
+[a6]: https://github.com/axios/axios/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3A%22type%3Aconfirmed+bug%22
+[s6]: https://github.com/visionmedia/superagent/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3ABug
+
+
+[gdp]: https://badgen.net/npm/dependents/got?label
+[kdp]: https://badgen.net/npm/dependents/ky?label
+[rdp]: https://badgen.net/npm/dependents/request?label
+[ndp]: https://badgen.net/npm/dependents/node-fetch?label
+[adp]: https://badgen.net/npm/dependents/axios?label
+[sdp]: https://badgen.net/npm/dependents/superagent?label
+
+[g7]: https://www.npmjs.com/package/got?activeTab=dependents
+[k7]: https://www.npmjs.com/package/ky?activeTab=dependents
+[r7]: https://www.npmjs.com/package/request?activeTab=dependents
+[n7]: https://www.npmjs.com/package/node-fetch?activeTab=dependents
+[a7]: https://www.npmjs.com/package/axios?activeTab=dependents
+[s7]: https://www.npmjs.com/package/visionmedia?activeTab=dependents
+
+
+[gis]: https://badgen.net/packagephobia/install/got?label
+[kis]: https://badgen.net/packagephobia/install/ky?label
+[ris]: https://badgen.net/packagephobia/install/request?label
+[nis]: https://badgen.net/packagephobia/install/node-fetch?label
+[ais]: https://badgen.net/packagephobia/install/axios?label
+[sis]: https://badgen.net/packagephobia/install/superagent?label
+
+[g8]: https://packagephobia.now.sh/result?p=got
+[k8]: https://packagephobia.now.sh/result?p=ky
+[r8]: https://packagephobia.now.sh/result?p=request
+[n8]: https://packagephobia.now.sh/result?p=node-fetch
+[a8]: https://packagephobia.now.sh/result?p=axios
+[s8]: https://packagephobia.now.sh/result?p=superagent
+
+
+[gs]: https://badgen.net/github/stars/sindresorhus/got?label
+[ks]: https://badgen.net/github/stars/sindresorhus/ky?label
+[rs]: https://badgen.net/github/stars/request/request?label
+[ns]: https://badgen.net/github/stars/bitinn/node-fetch?label
+[as]: https://badgen.net/github/stars/axios/axios?label
+[ss]: https://badgen.net/github/stars/visionmedia/superagent?label
+
+[g9]: https://github.com/sindresorhus/got
+[k9]: https://github.com/sindresorhus/ky
+[r9]: https://github.com/request/request
+[n9]: https://github.com/node-fetch/node-fetch
+[a9]: https://github.com/axios/axios
+[s9]: https://github.com/visionmedia/superagent
+
+
+[gts]: https://badgen.net/npm/types/got?label
+[kts]: https://badgen.net/npm/types/ky?label
+[rts]: https://badgen.net/npm/types/request?label
+[nts]: https://badgen.net/npm/types/node-fetch?label
+[ats]: https://badgen.net/npm/types/axios?label
+[sts]: https://badgen.net/npm/types/superagent?label
+
+[g10]: https://github.com/sindresorhus/got
+[k10]: https://github.com/sindresorhus/ky
+[r10]: https://github.com/request/request
+[n10]: https://github.com/node-fetch/node-fetch
+[a10]: https://github.com/axios/axios
+[s10]: https://github.com/visionmedia/superagent
+
+
+[glc]: https://badgen.net/github/last-commit/sindresorhus/got?label
+[klc]: https://badgen.net/github/last-commit/sindresorhus/ky?label
+[rlc]: https://badgen.net/github/last-commit/request/request?label
+[nlc]: https://badgen.net/github/last-commit/bitinn/node-fetch?label
+[alc]: https://badgen.net/github/last-commit/axios/axios?label
+[slc]: https://badgen.net/github/last-commit/visionmedia/superagent?label
+
+[g11]: https://github.com/sindresorhus/got/commits
+[k11]: https://github.com/sindresorhus/ky/commits
+[r11]: https://github.com/request/request/commits
+[n11]: https://github.com/node-fetch/node-fetch/commits
+[a11]: https://github.com/axios/axios/commits
+[s11]: https://github.com/visionmedia/superagent/commits
+
+[Click here][InstallSizeOfTheDependencies] to see the install size of the Got dependencies.
+
+[InstallSizeOfTheDependencies]: https://packagephobia.com/result?p=@sindresorhus/is@3.0.0,@szmarczak/http-timer@4.0.5,@types/cacheable-request@6.0.1,@types/responselike@1.0.0,cacheable-lookup@5.0.3,cacheable-request@7.0.1,decompress-response@6.0.0,http2-wrapper@1.0.0,lowercase-keys@2.0.0,p-cancelable@2.0.0,responselike@2.0.0
-## License
+## Related
-MIT © [Sindre Sorhus](https://sindresorhus.com)
+- [gh-got](https://github.com/sindresorhus/gh-got) - Got convenience wrapper to interact with the GitHub API
+- [gl-got](https://github.com/singapore/gl-got) - Got convenience wrapper to interact with the GitLab API
+- [travis-got](https://github.com/samverschueren/travis-got) - Got convenience wrapper to interact with the Travis API
+- [graphql-got](https://github.com/kevva/graphql-got) - Got convenience wrapper to interact with GraphQL
+- [GotQL](https://github.com/khaosdoctor/gotql) - Got convenience wrapper to interact with GraphQL using JSON-parsed queries instead of strings
+- [got-fetch](https://github.com/alexghr/got-fetch) - Got with a `fetch` interface
+
+## Maintainers
+
+[](https://sindresorhus.com) | [](https://github.com/szmarczak) | [](https://github.com/Giotino)
+---|---|---
+[Sindre Sorhus](https://sindresorhus.com) | [Szymon Marczak](https://github.com/szmarczak) | [Giovanni Minotti](https://github.com/Giotino)
+
+###### Former
+
+- [Vsevolod Strukchinsky](https://github.com/floatdrop)
+- [Alexander Tesfamichael](https://github.com/alextes)
+- [Brandon Smith](https://github.com/brandon93s)
+- [Luke Childs](https://github.com/lukechilds)
+
+
+## These amazing companies are using Got
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+> Segment is a happy user of Got! Got powers the main backend API that our app talks to. It's used by our in-house RPC client that we use to communicate with all microservices.
+>
+> — Vadim Demedes
+
+> Antora, a static site generator for creating documentation sites, uses Got to download the UI bundle. In Antora, the UI bundle (aka theme) is maintained as a separate project. That project exports the UI as a zip file we call the UI bundle. The main site generator downloads that UI from a URL using Got and streams it to vinyl-zip to extract the files. Those files go on to be used to create the HTML pages and supporting assets.
+>
+> — Dan Allen
+
+> GetVoIP is happily using Got in production. One of the unique capabilities of Got is the ability to handle Unix sockets which enables us to build a full control interfaces for our docker stack.
+>
+> — Daniel Kalen
+
+> We're using Got inside of Exoframe to handle all the communication between CLI and server. Exoframe is a self-hosted tool that allows simple one-command deployments using Docker.
+>
+> — Tim Ermilov
+
+> Karaoke Mugen uses Got to fetch content updates from its online server.
+>
+> — Axel Terizaki
+
+> Renovate uses Got, gh-got and gl-got to send millions of queries per day to GitHub, GitLab, npmjs, PyPi, Packagist, Docker Hub, Terraform, CircleCI, and more.
+>
+> — Rhys Arkins
+
+> Resistbot uses Got to communicate from the API frontend where all correspondence ingresses to the officials lookup database in back.
+>
+> — Chris Erickson
+
+> Natural Cycles is using Got to communicate with all kinds of 3rd-party REST APIs (over 9000!).
+>
+> — Kirill Groshkov
+
+> Microlink is a cloud browser as an API service that uses Got widely as the main HTTP client, serving ~22M requests a month, every time a network call needs to be performed.
+>
+> — Kiko Beats
+
+> We’re using Got at Radity. Thanks for such an amazing work!
+>
+> — Mirzayev Farid
+
+## For enterprise
+
+Available as part of the Tidelift Subscription.
+
+The maintainers of `got` and thousands of other packages are working with Tidelift to deliver commercial support and maintenance for the open source dependencies you use to build your applications. Save time, reduce risk, and improve code health, while paying the maintainers of the exact dependencies you use. [Learn more.](https://tidelift.com/subscription/pkg/npm-got?utm_source=npm-got&utm_medium=referral&utm_campaign=enterprise&utm_term=repo)
diff --git a/source/as-promise/create-rejection.ts b/source/as-promise/create-rejection.ts
new file mode 100644
index 000000000..358e1fe76
--- /dev/null
+++ b/source/as-promise/create-rejection.ts
@@ -0,0 +1,31 @@
+import {CancelableRequest, BeforeErrorHook, RequestError} from './types';
+
+export default function createRejection(error: Error, ...beforeErrorGroups: Array): CancelableRequest {
+ const promise = (async () => {
+ if (error instanceof RequestError) {
+ try {
+ for (const hooks of beforeErrorGroups) {
+ if (hooks) {
+ for (const hook of hooks) {
+ // eslint-disable-next-line no-await-in-loop
+ error = await hook(error as RequestError);
+ }
+ }
+ }
+ } catch (error_) {
+ error = error_;
+ }
+ }
+
+ throw error;
+ })() as CancelableRequest;
+
+ const returnPromise = (): CancelableRequest => promise;
+
+ promise.json = returnPromise;
+ promise.text = returnPromise;
+ promise.buffer = returnPromise;
+ promise.on = returnPromise;
+
+ return promise;
+}
diff --git a/source/as-promise/index.ts b/source/as-promise/index.ts
new file mode 100644
index 000000000..8f3698ede
--- /dev/null
+++ b/source/as-promise/index.ts
@@ -0,0 +1,208 @@
+import {EventEmitter} from 'events';
+import is from '@sindresorhus/is';
+import PCancelable = require('p-cancelable');
+import {
+ NormalizedOptions,
+ CancelableRequest,
+ Response,
+ RequestError,
+ HTTPError,
+ CancelError
+} from './types';
+import parseBody from './parse-body';
+import Request from '../core';
+import proxyEvents from '../core/utils/proxy-events';
+import getBuffer from '../core/utils/get-buffer';
+import {isResponseOk} from '../core/utils/is-response-ok';
+
+const proxiedRequestEvents = [
+ 'request',
+ 'response',
+ 'redirect',
+ 'uploadProgress',
+ 'downloadProgress'
+];
+
+export default function asPromise(normalizedOptions: NormalizedOptions): CancelableRequest {
+ let globalRequest: Request;
+ let globalResponse: Response;
+ const emitter = new EventEmitter();
+
+ const promise = new PCancelable((resolve, reject, onCancel) => {
+ const makeRequest = (retryCount: number): void => {
+ const request = new Request(undefined, normalizedOptions);
+ request.retryCount = retryCount;
+ request._noPipe = true;
+
+ onCancel(() => request.destroy());
+
+ onCancel.shouldReject = false;
+ onCancel(() => reject(new CancelError(request)));
+
+ globalRequest = request;
+
+ request.once('response', async (response: Response) => {
+ response.retryCount = retryCount;
+
+ if (response.request.aborted) {
+ // Canceled while downloading - will throw a `CancelError` or `TimeoutError` error
+ return;
+ }
+
+ // Download body
+ let rawBody;
+ try {
+ rawBody = await getBuffer(request);
+ response.rawBody = rawBody;
+ } catch {
+ // The same error is caught below.
+ // See request.once('error')
+ return;
+ }
+
+ if (request._isAboutToError) {
+ return;
+ }
+
+ // Parse body
+ const contentEncoding = (response.headers['content-encoding'] ?? '').toLowerCase();
+ const isCompressed = ['gzip', 'deflate', 'br'].includes(contentEncoding);
+
+ const {options} = request;
+
+ if (isCompressed && !options.decompress) {
+ response.body = rawBody;
+ } else {
+ try {
+ response.body = parseBody(response, options.responseType, options.parseJson, options.encoding);
+ } catch (error) {
+ // Fallback to `utf8`
+ response.body = rawBody.toString();
+
+ if (isResponseOk(response)) {
+ request._beforeError(error);
+ return;
+ }
+ }
+ }
+
+ try {
+ for (const [index, hook] of options.hooks.afterResponse.entries()) {
+ // @ts-expect-error TS doesn't notice that CancelableRequest is a Promise
+ // eslint-disable-next-line no-await-in-loop
+ response = await hook(response, async (updatedOptions): CancelableRequest => {
+ const typedOptions = Request.normalizeArguments(undefined, {
+ ...updatedOptions,
+ retry: {
+ calculateDelay: () => 0
+ },
+ throwHttpErrors: false,
+ resolveBodyOnly: false
+ }, options);
+
+ // Remove any further hooks for that request, because we'll call them anyway.
+ // The loop continues. We don't want duplicates (asPromise recursion).
+ typedOptions.hooks.afterResponse = typedOptions.hooks.afterResponse.slice(0, index);
+
+ for (const hook of typedOptions.hooks.beforeRetry) {
+ // eslint-disable-next-line no-await-in-loop
+ await hook(typedOptions);
+ }
+
+ const promise: CancelableRequest = asPromise(typedOptions);
+
+ onCancel(() => {
+ promise.catch(() => {});
+ promise.cancel();
+ });
+
+ return promise;
+ });
+ }
+ } catch (error) {
+ request._beforeError(new RequestError(error.message, error, request));
+ return;
+ }
+
+ globalResponse = response;
+
+ if (!isResponseOk(response)) {
+ request._beforeError(new HTTPError(response));
+ return;
+ }
+
+ resolve(request.options.resolveBodyOnly ? response.body as T : response as unknown as T);
+ });
+
+ const onError = (error: RequestError) => {
+ if (promise.isCanceled) {
+ return;
+ }
+
+ const {options} = request;
+
+ if (error instanceof HTTPError && !options.throwHttpErrors) {
+ const {response} = error;
+ resolve(request.options.resolveBodyOnly ? response.body as T : response as unknown as T);
+ return;
+ }
+
+ reject(error);
+ };
+
+ request.once('error', onError);
+
+ const previousBody = request.options.body;
+
+ request.once('retry', (newRetryCount: number, error: RequestError) => {
+ if (previousBody === error.request?.options.body && is.nodeStream(error.request?.options.body)) {
+ onError(error);
+ return;
+ }
+
+ makeRequest(newRetryCount);
+ });
+
+ proxyEvents(request, emitter, proxiedRequestEvents);
+ };
+
+ makeRequest(0);
+ }) as CancelableRequest;
+
+ promise.on = (event: string, fn: (...args: any[]) => void) => {
+ emitter.on(event, fn);
+ return promise;
+ };
+
+ const shortcut = (responseType: NormalizedOptions['responseType']): CancelableRequest => {
+ const newPromise = (async () => {
+ // Wait until downloading has ended
+ await promise;
+
+ const {options} = globalResponse.request;
+
+ return parseBody(globalResponse, responseType, options.parseJson, options.encoding);
+ })();
+
+ Object.defineProperties(newPromise, Object.getOwnPropertyDescriptors(promise));
+
+ return newPromise as CancelableRequest;
+ };
+
+ promise.json = () => {
+ const {headers} = globalRequest.options;
+
+ if (!globalRequest.writableFinished && headers.accept === undefined) {
+ headers.accept = 'application/json';
+ }
+
+ return shortcut('json');
+ };
+
+ promise.buffer = () => shortcut('buffer');
+ promise.text = () => shortcut('text');
+
+ return promise;
+}
+
+export * from './types';
diff --git a/source/as-promise/normalize-arguments.ts b/source/as-promise/normalize-arguments.ts
new file mode 100644
index 000000000..1175b07c7
--- /dev/null
+++ b/source/as-promise/normalize-arguments.ts
@@ -0,0 +1,98 @@
+import is, {assert} from '@sindresorhus/is';
+import {
+ Options,
+ NormalizedOptions,
+ Defaults,
+ Method
+} from './types';
+
+const normalizeArguments = (options: NormalizedOptions, defaults?: Defaults): NormalizedOptions => {
+ if (is.null_(options.encoding)) {
+ throw new TypeError('To get a Buffer, set `options.responseType` to `buffer` instead');
+ }
+
+ assert.any([is.string, is.undefined], options.encoding);
+ assert.any([is.boolean, is.undefined], options.resolveBodyOnly);
+ assert.any([is.boolean, is.undefined], options.methodRewriting);
+ assert.any([is.boolean, is.undefined], options.isStream);
+ assert.any([is.string, is.undefined], options.responseType);
+
+ // `options.responseType`
+ if (options.responseType === undefined) {
+ options.responseType = 'text';
+ }
+
+ // `options.retry`
+ const {retry} = options;
+
+ if (defaults) {
+ options.retry = {...defaults.retry};
+ } else {
+ options.retry = {
+ calculateDelay: retryObject => retryObject.computedValue,
+ limit: 0,
+ methods: [],
+ statusCodes: [],
+ errorCodes: [],
+ maxRetryAfter: undefined
+ };
+ }
+
+ if (is.object(retry)) {
+ options.retry = {
+ ...options.retry,
+ ...retry
+ };
+
+ options.retry.methods = [...new Set(options.retry.methods.map(method => method.toUpperCase() as Method))];
+ options.retry.statusCodes = [...new Set(options.retry.statusCodes)];
+ options.retry.errorCodes = [...new Set(options.retry.errorCodes)];
+ } else if (is.number(retry)) {
+ options.retry.limit = retry;
+ }
+
+ if (is.undefined(options.retry.maxRetryAfter)) {
+ options.retry.maxRetryAfter = Math.min(
+ // TypeScript is not smart enough to handle `.filter(x => is.number(x))`.
+ // eslint-disable-next-line unicorn/no-fn-reference-in-iterator
+ ...[options.timeout.request, options.timeout.connect].filter(is.number)
+ );
+ }
+
+ // `options.pagination`
+ if (is.object(options.pagination)) {
+ if (defaults) {
+ (options as Options).pagination = {
+ ...defaults.pagination,
+ ...options.pagination
+ };
+ }
+
+ const {pagination} = options;
+
+ if (!is.function_(pagination.transform)) {
+ throw new Error('`options.pagination.transform` must be implemented');
+ }
+
+ if (!is.function_(pagination.shouldContinue)) {
+ throw new Error('`options.pagination.shouldContinue` must be implemented');
+ }
+
+ if (!is.function_(pagination.filter)) {
+ throw new TypeError('`options.pagination.filter` must be implemented');
+ }
+
+ if (!is.function_(pagination.paginate)) {
+ throw new Error('`options.pagination.paginate` must be implemented');
+ }
+ }
+
+ // JSON mode
+ if (options.responseType === 'json' && options.headers.accept === undefined) {
+ options.headers.accept = 'application/json';
+ }
+
+ return options;
+};
+
+export default normalizeArguments;
diff --git a/source/as-promise/parse-body.ts b/source/as-promise/parse-body.ts
new file mode 100644
index 000000000..204a8518e
--- /dev/null
+++ b/source/as-promise/parse-body.ts
@@ -0,0 +1,33 @@
+import {
+ ResponseType,
+ ParseError,
+ Response,
+ ParseJsonFunction
+} from './types';
+
+const parseBody = (response: Response, responseType: ResponseType, parseJson: ParseJsonFunction, encoding?: BufferEncoding): unknown => {
+ const {rawBody} = response;
+
+ try {
+ if (responseType === 'text') {
+ return rawBody.toString(encoding);
+ }
+
+ if (responseType === 'json') {
+ return rawBody.length === 0 ? '' : parseJson(rawBody.toString());
+ }
+
+ if (responseType === 'buffer') {
+ return rawBody;
+ }
+
+ throw new ParseError({
+ message: `Unknown body type '${responseType as string}'`,
+ name: 'Error'
+ }, response);
+ } catch (error) {
+ throw new ParseError(error, response);
+ }
+};
+
+export default parseBody;
diff --git a/source/as-promise/types.ts b/source/as-promise/types.ts
new file mode 100644
index 000000000..3ec9a96e3
--- /dev/null
+++ b/source/as-promise/types.ts
@@ -0,0 +1,297 @@
+import PCancelable = require('p-cancelable');
+import Request, {
+ Options,
+ Response,
+ RequestError,
+ RequestEvents
+} from '../core';
+
+/**
+All parsing methods supported by Got.
+*/
+export type ResponseType = 'json' | 'buffer' | 'text';
+
+export interface PaginationOptions {
+ /**
+ All options accepted by `got.paginate()`.
+ */
+ pagination?: {
+ /**
+ A function that transform [`Response`](#response) into an array of items.
+ This is where you should do the parsing.
+
+ @default response => JSON.parse(response.body)
+ */
+ transform?: (response: Response) => Promise | T[];
+
+ /**
+ Checks whether the item should be emitted or not.
+
+ @default (item, allItems, currentItems) => true
+ */
+ filter?: (item: T, allItems: T[], currentItems: T[]) => boolean;
+
+ /**
+ The function takes three arguments:
+ - `response` - The current response object.
+ - `allItems` - An array of the emitted items.
+ - `currentItems` - Items from the current response.
+
+ It should return an object representing Got options pointing to the next page.
+ The options are merged automatically with the previous request, therefore the options returned `pagination.paginate(...)` must reflect changes only.
+ If there are no more pages, `false` should be returned.
+
+ @example
+ ```
+ const got = require('got');
+
+ (async () => {
+ const limit = 10;
+
+ const items = got.paginate('https://example.com/items', {
+ searchParams: {
+ limit,
+ offset: 0
+ },
+ pagination: {
+ paginate: (response, allItems, currentItems) => {
+ const previousSearchParams = response.request.options.searchParams;
+ const previousOffset = previousSearchParams.get('offset');
+
+ if (currentItems.length < limit) {
+ return false;
+ }
+
+ return {
+ searchParams: {
+ ...previousSearchParams,
+ offset: Number(previousOffset) + limit,
+ }
+ };
+ }
+ }
+ });
+
+ console.log('Items from all pages:', items);
+ })();
+ ```
+ */
+ paginate?: (response: Response, allItems: T[], currentItems: T[]) => Options | false;
+
+ /**
+ Checks whether the pagination should continue.
+
+ For example, if you need to stop **before** emitting an entry with some flag, you should use `(item, allItems, currentItems) => !item.flag`.
+ If you want to stop **after** emitting the entry, you should use `(item, allItems, currentItems) => allItems.some(entry => entry.flag)` instead.
+
+ @default (item, allItems, currentItems) => true
+ */
+ shouldContinue?: (item: T, allItems: T[], currentItems: T[]) => boolean;
+
+ /**
+ The maximum amount of items that should be emitted.
+
+ @default Infinity
+ */
+ countLimit?: number;
+
+ /**
+ Milliseconds to wait before the next request is triggered.
+
+ @default 0
+ */
+ backoff?: number;
+ /**
+ The maximum amount of request that should be triggered.
+ Retries on failure are not counted towards this limit.
+
+ For example, it can be helpful during development to avoid an infinite number of requests.
+
+ @default 10000
+ */
+ requestLimit?: number;
+
+ /**
+ Defines how the parameter `allItems` in pagination.paginate, pagination.filter and pagination.shouldContinue is managed.
+ When set to `false`, the parameter `allItems` is always an empty array.
+
+ This option can be helpful to save on memory usage when working with a large dataset.
+ */
+ stackAllItems?: boolean;
+ };
+}
+
+export type AfterResponseHook = (response: Response, retryWithMergedOptions: (options: Options) => CancelableRequest) => Response | CancelableRequest | Promise>;
+
+// These should be merged into Options in core/index.ts
+export namespace PromiseOnly {
+ export interface Hooks {
+ /**
+ Called with [response object](#response) and a retry function.
+ Calling the retry function will trigger `beforeRetry` hooks.
+
+ Each function should return the response.
+ This is especially useful when you want to refresh an access token.
+
+ __Note__: When using streams, this hook is ignored.
+
+ @example
+ ```
+ const got = require('got');
+
+ const instance = got.extend({
+ hooks: {
+ afterResponse: [
+ (response, retryWithMergedOptions) => {
+ if (response.statusCode === 401) { // Unauthorized
+ const updatedOptions = {
+ headers: {
+ token: getNewToken() // Refresh the access token
+ }
+ };
+
+ // Save for further requests
+ instance.defaults.options = got.mergeOptions(instance.defaults.options, updatedOptions);
+
+ // Make a new retry
+ return retryWithMergedOptions(updatedOptions);
+ }
+
+ // No changes otherwise
+ return response;
+ }
+ ],
+ beforeRetry: [
+ (options, error, retryCount) => {
+ // This will be called on `retryWithMergedOptions(...)`
+ }
+ ]
+ },
+ mutableDefaults: true
+ });
+ ```
+ */
+ afterResponse?: AfterResponseHook[];
+ }
+
+ export interface Options extends PaginationOptions {
+ /**
+ The parsing method.
+
+ The promise also has `.text()`, `.json()` and `.buffer()` methods which return another Got promise for the parsed body.
+
+ It's like setting the options to `{responseType: 'json', resolveBodyOnly: true}` but without affecting the main Got promise.
+
+ __Note__: When using streams, this option is ignored.
+
+ @example
+ ```
+ (async () => {
+ const responsePromise = got(url);
+ const bufferPromise = responsePromise.buffer();
+ const jsonPromise = responsePromise.json();
+
+ const [response, buffer, json] = Promise.all([responsePromise, bufferPromise, jsonPromise]);
+ // `response` is an instance of Got Response
+ // `buffer` is an instance of Buffer
+ // `json` is an object
+ })();
+ ```
+
+ @example
+ ```
+ // This
+ const body = await got(url).json();
+
+ // is semantically the same as this
+ const body = await got(url, {responseType: 'json', resolveBodyOnly: true});
+ ```
+ */
+ responseType?: ResponseType;
+
+ /**
+ When set to `true` the promise will return the Response body instead of the Response object.
+
+ @default false
+ */
+ resolveBodyOnly?: boolean;
+
+ /**
+ Returns a `Stream` instead of a `Promise`.
+ This is equivalent to calling `got.stream(url, options?)`.
+
+ @default false
+ */
+ isStream?: boolean;
+
+ /**
+ [Encoding](https://nodejs.org/api/buffer.html#buffer_buffers_and_character_encodings) to be used on `setEncoding` of the response data.
+
+ To get a [`Buffer`](https://nodejs.org/api/buffer.html), you need to set `responseType` to `buffer` instead.
+ Don't set this option to `null`.
+
+ __Note__: This doesn't affect streams! Instead, you need to do `got.stream(...).setEncoding(encoding)`.
+
+ @default 'utf-8'
+ */
+ encoding?: BufferEncoding;
+ }
+
+ export interface NormalizedOptions {
+ responseType: ResponseType;
+ resolveBodyOnly: boolean;
+ isStream: boolean;
+ encoding?: BufferEncoding;
+ pagination?: Required['pagination']>;
+ }
+
+ export interface Defaults {
+ responseType: ResponseType;
+ resolveBodyOnly: boolean;
+ isStream: boolean;
+ pagination?: Required['pagination']>;
+ }
+
+ export type HookEvent = 'afterResponse';
+}
+
+/**
+An error to be thrown when server response code is 2xx, and parsing body fails.
+Includes a `response` property.
+*/
+export class ParseError extends RequestError {
+ declare readonly response: Response;
+
+ constructor(error: Error, response: Response) {
+ const {options} = response.request;
+
+ super(`${error.message} in "${options.url.toString()}"`, error, response.request);
+ this.name = 'ParseError';
+ this.code = this.code === 'ERR_GOT_REQUEST_ERROR' ? 'ERR_BODY_PARSE_FAILURE' : this.code;
+ }
+}
+
+/**
+An error to be thrown when the request is aborted with `.cancel()`.
+*/
+export class CancelError extends RequestError {
+ declare readonly response: Response;
+
+ constructor(request: Request) {
+ super('Promise was canceled', {}, request);
+ this.name = 'CancelError';
+ this.code = 'ERR_CANCELED';
+ }
+
+ get isCanceled() {
+ return true;
+ }
+}
+
+export interface CancelableRequest extends PCancelable, RequestEvents> {
+ json: () => CancelableRequest;
+ buffer: () => CancelableRequest;
+ text: () => CancelableRequest;
+}
+
+export * from '../core';
diff --git a/source/core/calculate-retry-delay.ts b/source/core/calculate-retry-delay.ts
new file mode 100644
index 000000000..3b584bfcc
--- /dev/null
+++ b/source/core/calculate-retry-delay.ts
@@ -0,0 +1,37 @@
+import {RetryFunction} from '.';
+
+type Returns unknown, V> = (...args: Parameters) => V;
+
+export const retryAfterStatusCodes: ReadonlySet = new Set([413, 429, 503]);
+
+const calculateRetryDelay: Returns = ({attemptCount, retryOptions, error, retryAfter}) => {
+ if (attemptCount > retryOptions.limit) {
+ return 0;
+ }
+
+ const hasMethod = retryOptions.methods.includes(error.options.method);
+ const hasErrorCode = retryOptions.errorCodes.includes(error.code);
+ const hasStatusCode = error.response && retryOptions.statusCodes.includes(error.response.statusCode);
+ if (!hasMethod || (!hasErrorCode && !hasStatusCode)) {
+ return 0;
+ }
+
+ if (error.response) {
+ if (retryAfter) {
+ if (retryOptions.maxRetryAfter === undefined || retryAfter > retryOptions.maxRetryAfter) {
+ return 0;
+ }
+
+ return retryAfter;
+ }
+
+ if (error.response.statusCode === 413) {
+ return 0;
+ }
+ }
+
+ const noise = Math.random() * 100;
+ return ((2 ** (attemptCount - 1)) * 1000) + noise;
+};
+
+export default calculateRetryDelay;
diff --git a/source/core/index.ts b/source/core/index.ts
new file mode 100644
index 000000000..bec7192fc
--- /dev/null
+++ b/source/core/index.ts
@@ -0,0 +1,2861 @@
+import {promisify} from 'util';
+import {Duplex, Writable, Readable} from 'stream';
+import {ReadStream} from 'fs';
+import {URL, URLSearchParams} from 'url';
+import {Socket} from 'net';
+import {SecureContextOptions, DetailedPeerCertificate} from 'tls';
+import http = require('http');
+import {ClientRequest, RequestOptions, IncomingMessage, ServerResponse, request as httpRequest} from 'http';
+import https = require('https');
+import timer, {ClientRequestWithTimings, Timings, IncomingMessageWithTimings} from '@szmarczak/http-timer';
+import CacheableLookup from 'cacheable-lookup';
+import CacheableRequest = require('cacheable-request');
+import decompressResponse = require('decompress-response');
+// @ts-expect-error Missing types
+import http2wrapper = require('http2-wrapper');
+import lowercaseKeys = require('lowercase-keys');
+import ResponseLike = require('responselike');
+import is, {assert} from '@sindresorhus/is';
+import getBodySize from './utils/get-body-size';
+import isFormData from './utils/is-form-data';
+import proxyEvents from './utils/proxy-events';
+import timedOut, {Delays, TimeoutError as TimedOutTimeoutError} from './utils/timed-out';
+import urlToOptions from './utils/url-to-options';
+import optionsToUrl, {URLOptions} from './utils/options-to-url';
+import WeakableMap from './utils/weakable-map';
+import getBuffer from './utils/get-buffer';
+import {DnsLookupIpVersion, isDnsLookupIpVersion, dnsLookupIpVersionToFamily} from './utils/dns-ip-version';
+import {isResponseOk} from './utils/is-response-ok';
+import deprecationWarning from '../utils/deprecation-warning';
+import normalizePromiseArguments from '../as-promise/normalize-arguments';
+import {PromiseOnly} from '../as-promise/types';
+import calculateRetryDelay from './calculate-retry-delay';
+
+let globalDnsCache: CacheableLookup;
+
+type HttpRequestFunction = typeof httpRequest;
+type Error = NodeJS.ErrnoException;
+
+const kRequest = Symbol('request');
+const kResponse = Symbol('response');
+const kResponseSize = Symbol('responseSize');
+const kDownloadedSize = Symbol('downloadedSize');
+const kBodySize = Symbol('bodySize');
+const kUploadedSize = Symbol('uploadedSize');
+const kServerResponsesPiped = Symbol('serverResponsesPiped');
+const kUnproxyEvents = Symbol('unproxyEvents');
+const kIsFromCache = Symbol('isFromCache');
+const kCancelTimeouts = Symbol('cancelTimeouts');
+const kStartedReading = Symbol('startedReading');
+const kStopReading = Symbol('stopReading');
+const kTriggerRead = Symbol('triggerRead');
+const kBody = Symbol('body');
+const kJobs = Symbol('jobs');
+const kOriginalResponse = Symbol('originalResponse');
+const kRetryTimeout = Symbol('retryTimeout');
+export const kIsNormalizedAlready = Symbol('isNormalizedAlready');
+
+const supportsBrotli = is.string((process.versions as any).brotli);
+
+export interface Agents {
+ http?: http.Agent;
+ https?: https.Agent;
+ http2?: unknown;
+}
+
+export const withoutBody: ReadonlySet = new Set(['GET', 'HEAD']);
+
+export interface ToughCookieJar {
+ getCookieString: ((currentUrl: string, options: Record, cb: (err: Error | null, cookies: string) => void) => void)
+ & ((url: string, callback: (error: Error | null, cookieHeader: string) => void) => void);
+ setCookie: ((cookieOrString: unknown, currentUrl: string, options: Record, cb: (err: Error | null, cookie: unknown) => void) => void)
+ & ((rawCookie: string, url: string, callback: (error: Error | null, result: unknown) => void) => void);
+}
+
+export interface PromiseCookieJar {
+ getCookieString: (url: string) => Promise;
+ setCookie: (rawCookie: string, url: string) => Promise;
+}
+
+/**
+All available HTTP request methods provided by Got.
+*/
+export type Method =
+ | 'GET'
+ | 'POST'
+ | 'PUT'
+ | 'PATCH'
+ | 'HEAD'
+ | 'DELETE'
+ | 'OPTIONS'
+ | 'TRACE'
+ | 'get'
+ | 'post'
+ | 'put'
+ | 'patch'
+ | 'head'
+ | 'delete'
+ | 'options'
+ | 'trace';
+
+type Promisable = T | Promise;
+
+export type InitHook = (options: Options) => void;
+export type BeforeRequestHook = (options: NormalizedOptions) => Promisable;
+export type BeforeRedirectHook = (options: NormalizedOptions, response: Response) => Promisable;
+export type BeforeErrorHook = (error: RequestError) => Promisable;
+export type BeforeRetryHook = (options: NormalizedOptions, error?: RequestError, retryCount?: number) => void | Promise;
+
+interface PlainHooks {
+ /**
+ Called with plain request options, right before their normalization.
+ This is especially useful in conjunction with `got.extend()` when the input needs custom handling.
+
+ __Note #1__: This hook must be synchronous!
+
+ __Note #2__: Errors in this hook will be converted into an instances of `RequestError`.
+
+ __Note #3__: The options object may not have a `url` property.
+ To modify it, use a `beforeRequest` hook instead.
+
+ @default []
+ */
+ init?: InitHook[];
+
+ /**
+ Called with normalized request options.
+ Got will make no further changes to the request before it is sent.
+ This is especially useful in conjunction with `got.extend()` when you want to create an API client that, for example, uses HMAC-signing.
+
+ @default []
+ */
+ beforeRequest?: BeforeRequestHook[];
+
+ /**
+ Called with normalized request options and the redirect response.
+ Got will make no further changes to the request.
+ This is especially useful when you want to avoid dead sites.
+
+ @default []
+
+ @example
+ ```
+ const got = require('got');
+
+ got('https://example.com', {
+ hooks: {
+ beforeRedirect: [
+ (options, response) => {
+ if (options.hostname === 'deadSite') {
+ options.hostname = 'fallbackSite';
+ }
+ }
+ ]
+ }
+ });
+ ```
+ */
+ beforeRedirect?: BeforeRedirectHook[];
+
+ /**
+ Called with an `Error` instance.
+ The error is passed to the hook right before it's thrown.
+ This is especially useful when you want to have more detailed errors.
+
+ __Note__: Errors thrown while normalizing input options are thrown directly and not part of this hook.
+
+ @default []
+
+ @example
+ ```
+ const got = require('got');
+
+ got('https://api.github.com/some-endpoint', {
+ hooks: {
+ beforeError: [
+ error => {
+ const {response} = error;
+ if (response && response.body) {
+ error.name = 'GitHubError';
+ error.message = `${response.body.message} (${response.statusCode})`;
+ }
+
+ return error;
+ }
+ ]
+ }
+ });
+ ```
+ */
+ beforeError?: BeforeErrorHook[];
+
+ /**
+ Called with normalized request options, the error and the retry count.
+ Got will make no further changes to the request.
+ This is especially useful when some extra work is required before the next try.
+
+ __Note__: When using streams, this hook is ignored.
+ __Note__: When retrying in a `afterResponse` hook, all remaining `beforeRetry` hooks will be called without the `error` and `retryCount` arguments.
+
+ @default []
+
+ @example
+ ```
+ const got = require('got');
+
+ got.post('https://example.com', {
+ hooks: {
+ beforeRetry: [
+ (options, error, retryCount) => {
+ if (error.response.statusCode === 413) { // Payload too large
+ options.body = getNewBody();
+ }
+ }
+ ]
+ }
+ });
+ ```
+ */
+ beforeRetry?: BeforeRetryHook[];
+}
+
+/**
+All available hook of Got.
+*/
+export interface Hooks extends PromiseOnly.Hooks, PlainHooks {}
+
+type PlainHookEvent = 'init' | 'beforeRequest' | 'beforeRedirect' | 'beforeError' | 'beforeRetry';
+
+/**
+All hook events acceptable by Got.
+*/
+export type HookEvent = PromiseOnly.HookEvent | PlainHookEvent;
+
+export const knownHookEvents: HookEvent[] = [
+ 'init',
+ 'beforeRequest',
+ 'beforeRedirect',
+ 'beforeError',
+ 'beforeRetry',
+
+ // Promise-Only
+ 'afterResponse'
+];
+
+type AcceptableResponse = IncomingMessageWithTimings | ResponseLike;
+type AcceptableRequestResult = AcceptableResponse | ClientRequest | Promise | undefined;
+
+export type RequestFunction = (url: URL, options: RequestOptions, callback?: (response: AcceptableResponse) => void) => AcceptableRequestResult;
+
+export type Headers = Record;
+
+type CacheableRequestFunction = (
+ options: string | URL | RequestOptions,
+ cb?: (response: ServerResponse | ResponseLike) => void
+) => CacheableRequest.Emitter;
+
+type CheckServerIdentityFunction = (hostname: string, certificate: DetailedPeerCertificate) => Error | void;
+export type ParseJsonFunction = (text: string) => unknown;
+export type StringifyJsonFunction = (object: unknown) => string;
+
+interface RealRequestOptions extends https.RequestOptions {
+ checkServerIdentity: CheckServerIdentityFunction;
+}
+
+export interface RetryObject {
+ attemptCount: number;
+ retryOptions: RequiredRetryOptions;
+ error: TimeoutError | RequestError;
+ computedValue: number;
+ retryAfter?: number;
+}
+
+export type RetryFunction = (retryObject: RetryObject) => number | Promise;
+
+/**
+An object representing `limit`, `calculateDelay`, `methods`, `statusCodes`, `maxRetryAfter` and `errorCodes` fields for maximum retry count, retry handler, allowed methods, allowed status codes, maximum [`Retry-After`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After) time and allowed error codes.
+
+Delays between retries counts with function `1000 * Math.pow(2, retry) + Math.random() * 100`, where `retry` is attempt number (starts from 1).
+
+The `calculateDelay` property is a `function` that receives an object with `attemptCount`, `retryOptions`, `error` and `computedValue` properties for current retry count, the retry options, error and default computed value.
+The function must return a delay in milliseconds (or a Promise resolving with it) (`0` return value cancels retry).
+
+By default, it retries *only* on the specified methods, status codes, and on these network errors:
+- `ETIMEDOUT`: One of the [timeout](#timeout) limits were reached.
+- `ECONNRESET`: Connection was forcibly closed by a peer.
+- `EADDRINUSE`: Could not bind to any free port.
+- `ECONNREFUSED`: Connection was refused by the server.
+- `EPIPE`: The remote side of the stream being written has been closed.
+- `ENOTFOUND`: Couldn't resolve the hostname to an IP address.
+- `ENETUNREACH`: No internet connection.
+- `EAI_AGAIN`: DNS lookup timed out.
+
+__Note__: If `maxRetryAfter` is set to `undefined`, it will use `options.timeout`.
+__Note__: If [`Retry-After`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After) header is greater than `maxRetryAfter`, it will cancel the request.
+*/
+export interface RequiredRetryOptions {
+ limit: number;
+ methods: Method[];
+ statusCodes: number[];
+ errorCodes: string[];
+ calculateDelay: RetryFunction;
+ maxRetryAfter?: number;
+}
+
+export interface CacheOptions {
+ shared?: boolean;
+ cacheHeuristic?: number;
+ immutableMinTimeToLive?: number;
+ ignoreCargoCult?: boolean;
+}
+
+interface PlainOptions extends URLOptions {
+ /**
+ Custom request function.
+ The main purpose of this is to [support HTTP2 using a wrapper](https://github.com/szmarczak/http2-wrapper).
+
+ @default http.request | https.request
+ */
+ request?: RequestFunction;
+
+ /**
+ An object representing `http`, `https` and `http2` keys for [`http.Agent`](https://nodejs.org/api/http.html#http_class_http_agent), [`https.Agent`](https://nodejs.org/api/https.html#https_class_https_agent) and [`http2wrapper.Agent`](https://github.com/szmarczak/http2-wrapper#new-http2agentoptions) instance.
+ This is necessary because a request to one protocol might redirect to another.
+ In such a scenario, Got will switch over to the right protocol agent for you.
+
+ If a key is not present, it will default to a global agent.
+
+ @example
+ ```
+ const got = require('got');
+ const HttpAgent = require('agentkeepalive');
+ const {HttpsAgent} = HttpAgent;
+
+ got('https://sindresorhus.com', {
+ agent: {
+ http: new HttpAgent(),
+ https: new HttpsAgent()
+ }
+ });
+ ```
+ */
+ agent?: Agents | false;
+
+ /**
+ Decompress the response automatically.
+ This will set the `accept-encoding` header to `gzip, deflate, br` on Node.js 11.7.0+ or `gzip, deflate` for older Node.js versions, unless you set it yourself.
+
+ Brotli (`br`) support requires Node.js 11.7.0 or later.
+
+ If this is disabled, a compressed response is returned as a `Buffer`.
+ This may be useful if you want to handle decompression yourself or stream the raw compressed data.
+
+ @default true
+ */
+ decompress?: boolean;
+
+ /**
+ Milliseconds to wait for the server to end the response before aborting the request with `got.TimeoutError` error (a.k.a. `request` property).
+ By default, there's no timeout.
+
+ This also accepts an `object` with the following fields to constrain the duration of each phase of the request lifecycle:
+
+ - `lookup` starts when a socket is assigned and ends when the hostname has been resolved.
+ Does not apply when using a Unix domain socket.
+ - `connect` starts when `lookup` completes (or when the socket is assigned if lookup does not apply to the request) and ends when the socket is connected.
+ - `secureConnect` starts when `connect` completes and ends when the handshaking process completes (HTTPS only).
+ - `socket` starts when the socket is connected. See [request.setTimeout](https://nodejs.org/api/http.html#http_request_settimeout_timeout_callback).
+ - `response` starts when the request has been written to the socket and ends when the response headers are received.
+ - `send` starts when the socket is connected and ends with the request has been written to the socket.
+ - `request` starts when the request is initiated and ends when the response's end event fires.
+ */
+ timeout?: Delays | number;
+
+ /**
+ When specified, `prefixUrl` will be prepended to `url`.
+ The prefix can be any valid URL, either relative or absolute.
+ A trailing slash `/` is optional - one will be added automatically.
+
+ __Note__: `prefixUrl` will be ignored if the `url` argument is a URL instance.
+
+ __Note__: Leading slashes in `input` are disallowed when using this option to enforce consistency and avoid confusion.
+ For example, when the prefix URL is `https://example.com/foo` and the input is `/bar`, there's ambiguity whether the resulting URL would become `https://example.com/foo/bar` or `https://example.com/bar`.
+ The latter is used by browsers.
+
+ __Tip__: Useful when used with `got.extend()` to create niche-specific Got instances.
+
+ __Tip__: You can change `prefixUrl` using hooks as long as the URL still includes the `prefixUrl`.
+ If the URL doesn't include it anymore, it will throw.
+
+ @example
+ ```
+ const got = require('got');
+
+ (async () => {
+ await got('unicorn', {prefixUrl: 'https://cats.com'});
+ //=> 'https://cats.com/unicorn'
+
+ const instance = got.extend({
+ prefixUrl: 'https://google.com'
+ });
+
+ await instance('unicorn', {
+ hooks: {
+ beforeRequest: [
+ options => {
+ options.prefixUrl = 'https://cats.com';
+ }
+ ]
+ }
+ });
+ //=> 'https://cats.com/unicorn'
+ })();
+ ```
+ */
+ prefixUrl?: string | URL;
+
+ /**
+ __Note #1__: The `body` option cannot be used with the `json` or `form` option.
+
+ __Note #2__: If you provide this option, `got.stream()` will be read-only.
+
+ __Note #3__: If you provide a payload with the `GET` or `HEAD` method, it will throw a `TypeError` unless the method is `GET` and the `allowGetBody` option is set to `true`.
+
+ __Note #4__: This option is not enumerable and will not be merged with the instance defaults.
+
+ The `content-length` header will be automatically set if `body` is a `string` / `Buffer` / `fs.createReadStream` instance / [`form-data` instance](https://github.com/form-data/form-data), and `content-length` and `transfer-encoding` are not manually set in `options.headers`.
+ */
+ body?: string | Buffer | Readable;
+
+ /**
+ The form body is converted to a query string using [`(new URLSearchParams(object)).toString()`](https://nodejs.org/api/url.html#url_constructor_new_urlsearchparams_obj).
+
+ If the `Content-Type` header is not present, it will be set to `application/x-www-form-urlencoded`.
+
+ __Note #1__: If you provide this option, `got.stream()` will be read-only.
+
+ __Note #2__: This option is not enumerable and will not be merged with the instance defaults.
+ */
+ form?: Record;
+
+ /**
+ JSON body. If the `Content-Type` header is not set, it will be set to `application/json`.
+
+ __Note #1__: If you provide this option, `got.stream()` will be read-only.
+
+ __Note #2__: This option is not enumerable and will not be merged with the instance defaults.
+ */
+ json?: Record;
+
+ /**
+ The URL to request, as a string, a [`https.request` options object](https://nodejs.org/api/https.html#https_https_request_options_callback), or a [WHATWG `URL`](https://nodejs.org/api/url.html#url_class_url).
+
+ Properties from `options` will override properties in the parsed `url`.
+
+ If no protocol is specified, it will throw a `TypeError`.
+
+ __Note__: The query string is **not** parsed as search params.
+
+ @example
+ ```
+ got('https://example.com/?query=a b'); //=> https://example.com/?query=a%20b
+ got('https://example.com/', {searchParams: {query: 'a b'}}); //=> https://example.com/?query=a+b
+
+ // The query string is overridden by `searchParams`
+ got('https://example.com/?query=a b', {searchParams: {query: 'a b'}}); //=> https://example.com/?query=a+b
+ ```
+ */
+ url?: string | URL;
+
+ /**
+ Cookie support. You don't have to care about parsing or how to store them.
+
+ __Note__: If you provide this option, `options.headers.cookie` will be overridden.
+ */
+ cookieJar?: PromiseCookieJar | ToughCookieJar;
+
+ /**
+ Ignore invalid cookies instead of throwing an error.
+ Only useful when the `cookieJar` option has been set. Not recommended.
+
+ @default false
+ */
+ ignoreInvalidCookies?: boolean;
+
+ /**
+ Query string that will be added to the request URL.
+ This will override the query string in `url`.
+
+ If you need to pass in an array, you can do it using a `URLSearchParams` instance.
+
+ @example
+ ```
+ const got = require('got');
+
+ const searchParams = new URLSearchParams([['key', 'a'], ['key', 'b']]);
+
+ got('https://example.com', {searchParams});
+
+ console.log(searchParams.toString());
+ //=> 'key=a&key=b'
+ ```
+ */
+ searchParams?: string | Record | URLSearchParams;
+
+ /**
+ An instance of [`CacheableLookup`](https://github.com/szmarczak/cacheable-lookup) used for making DNS lookups.
+ Useful when making lots of requests to different *public* hostnames.
+
+ `CacheableLookup` uses `dns.resolver4(..)` and `dns.resolver6(...)` under the hood and fall backs to `dns.lookup(...)` when the first two fail, which may lead to additional delay.
+
+ __Note__: This should stay disabled when making requests to internal hostnames such as `localhost`, `database.local` etc.
+
+ @default false
+ */
+ dnsCache?: CacheableLookup | boolean;
+
+ /**
+ User data. In contrast to other options, `context` is not enumerable.
+
+ __Note__: The object is never merged, it's just passed through.
+ Got will not modify the object in any way.
+
+ @example
+ ```
+ const got = require('got');
+
+ const instance = got.extend({
+ hooks: {
+ beforeRequest: [
+ options => {
+ if (!options.context || !options.context.token) {
+ throw new Error('Token required');
+ }
+
+ options.headers.token = options.context.token;
+ }
+ ]
+ }
+ });
+
+ (async () => {
+ const context = {
+ token: 'secret'
+ };
+
+ const response = await instance('https://httpbin.org/headers', {context});
+
+ // Let's see the headers
+ console.log(response.body);
+ })();
+ ```
+ */
+ context?: Record;
+
+ /**
+ Hooks allow modifications during the request lifecycle.
+ Hook functions may be async and are run serially.
+ */
+ hooks?: Hooks;
+
+ /**
+ Defines if redirect responses should be followed automatically.
+
+ Note that if a `303` is sent by the server in response to any request type (`POST`, `DELETE`, etc.), Got will automatically request the resource pointed to in the location header via `GET`.
+ This is in accordance with [the spec](https://tools.ietf.org/html/rfc7231#section-6.4.4).
+
+ @default true
+ */
+ followRedirect?: boolean;
+
+ /**
+ If exceeded, the request will be aborted and a `MaxRedirectsError` will be thrown.
+
+ @default 10
+ */
+ maxRedirects?: number;
+
+ /**
+ A cache adapter instance for storing cached response data.
+
+ @default false
+ */
+ cache?: string | CacheableRequest.StorageAdapter | false;
+
+ /**
+ Determines if a `got.HTTPError` is thrown for unsuccessful responses.
+
+ If this is disabled, requests that encounter an error status code will be resolved with the `response` instead of throwing.
+ This may be useful if you are checking for resource availability and are expecting error responses.
+
+ @default true
+ */
+ throwHttpErrors?: boolean;
+
+ username?: string;
+
+ password?: string;
+
+ /**
+ If set to `true`, Got will additionally accept HTTP2 requests.
+
+ It will choose either HTTP/1.1 or HTTP/2 depending on the ALPN protocol.
+
+ __Note__: Overriding `options.request` will disable HTTP2 support.
+
+ __Note__: This option will default to `true` in the next upcoming major release.
+
+ @default false
+
+ @example
+ ```
+ const got = require('got');
+
+ (async () => {
+ const {headers} = await got('https://nghttp2.org/httpbin/anything', {http2: true});
+ console.log(headers.via);
+ //=> '2 nghttpx'
+ })();
+ ```
+ */
+ http2?: boolean;
+
+ /**
+ Set this to `true` to allow sending body for the `GET` method.
+ However, the [HTTP/2 specification](https://tools.ietf.org/html/rfc7540#section-8.1.3) says that `An HTTP GET request includes request header fields and no payload body`, therefore when using the HTTP/2 protocol this option will have no effect.
+ This option is only meant to interact with non-compliant servers when you have no other choice.
+
+ __Note__: The [RFC 7321](https://tools.ietf.org/html/rfc7231#section-4.3.1) doesn't specify any particular behavior for the GET method having a payload, therefore __it's considered an [anti-pattern](https://en.wikipedia.org/wiki/Anti-pattern)__.
+
+ @default false
+ */
+ allowGetBody?: boolean;
+
+ lookup?: CacheableLookup['lookup'];
+
+ /**
+ Request headers.
+
+ Existing headers will be overwritten. Headers set to `undefined` will be omitted.
+
+ @default {}
+ */
+ headers?: Headers;
+
+ /**
+ By default, redirects will use [method rewriting](https://tools.ietf.org/html/rfc7231#section-6.4).
+ For example, when sending a POST request and receiving a `302`, it will resend the body to the new location using the same HTTP method (`POST` in this case).
+
+ @default true
+ */
+ methodRewriting?: boolean;
+
+ /**
+ Indicates which DNS record family to use.
+
+ Values:
+ - `auto`: IPv4 (if present) or IPv6
+ - `ipv4`: Only IPv4
+ - `ipv6`: Only IPv6
+
+ __Note__: If you are using the undocumented option `family`, `dnsLookupIpVersion` will override it.
+
+ @default 'auto'
+ */
+ dnsLookupIpVersion?: DnsLookupIpVersion;
+
+ /**
+ A function used to parse JSON responses.
+
+ @example
+ ```
+ const got = require('got');
+ const Bourne = require('@hapi/bourne');
+
+ (async () => {
+ const parsed = await got('https://example.com', {
+ parseJson: text => Bourne.parse(text)
+ }).json();
+
+ console.log(parsed);
+ })();
+ ```
+ */
+ parseJson?: ParseJsonFunction;
+
+ /**
+ A function used to stringify the body of JSON requests.
+
+ @example
+ ```
+ const got = require('got');
+
+ (async () => {
+ await got.post('https://example.com', {
+ stringifyJson: object => JSON.stringify(object, (key, value) => {
+ if (key.startsWith('_')) {
+ return;
+ }
+
+ return value;
+ }),
+ json: {
+ some: 'payload',
+ _ignoreMe: 1234
+ }
+ });
+ })();
+ ```
+
+ @example
+ ```
+ const got = require('got');
+
+ (async () => {
+ await got.post('https://example.com', {
+ stringifyJson: object => JSON.stringify(object, (key, value) => {
+ if (typeof value === 'number') {
+ return value.toString();
+ }
+
+ return value;
+ }),
+ json: {
+ some: 'payload',
+ number: 1
+ }
+ });
+ })();
+ ```
+ */
+ stringifyJson?: StringifyJsonFunction;
+
+ /**
+ An object representing `limit`, `calculateDelay`, `methods`, `statusCodes`, `maxRetryAfter` and `errorCodes` fields for maximum retry count, retry handler, allowed methods, allowed status codes, maximum [`Retry-After`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After) time and allowed error codes.
+
+ Delays between retries counts with function `1000 * Math.pow(2, retry) + Math.random() * 100`, where `retry` is attempt number (starts from 1).
+
+ The `calculateDelay` property is a `function` that receives an object with `attemptCount`, `retryOptions`, `error` and `computedValue` properties for current retry count, the retry options, error and default computed value.
+ The function must return a delay in milliseconds (or a Promise resolving with it) (`0` return value cancels retry).
+
+ By default, it retries *only* on the specified methods, status codes, and on these network errors:
+
+ - `ETIMEDOUT`: One of the [timeout](#timeout) limits were reached.
+ - `ECONNRESET`: Connection was forcibly closed by a peer.
+ - `EADDRINUSE`: Could not bind to any free port.
+ - `ECONNREFUSED`: Connection was refused by the server.
+ - `EPIPE`: The remote side of the stream being written has been closed.
+ - `ENOTFOUND`: Couldn't resolve the hostname to an IP address.
+ - `ENETUNREACH`: No internet connection.
+ - `EAI_AGAIN`: DNS lookup timed out.
+
+ __Note__: If `maxRetryAfter` is set to `undefined`, it will use `options.timeout`.
+ __Note__: If [`Retry-After`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After) header is greater than `maxRetryAfter`, it will cancel the request.
+ */
+ retry?: Partial | number;
+
+ // From `http.RequestOptions`
+ /**
+ The IP address used to send the request from.
+ */
+ localAddress?: string;
+
+ socketPath?: string;
+
+ /**
+ The HTTP method used to make the request.
+
+ @default 'GET'
+ */
+ method?: Method;
+
+ createConnection?: (options: http.RequestOptions, oncreate: (error: Error, socket: Socket) => void) => Socket;
+
+ // From `http-cache-semantics`
+ cacheOptions?: CacheOptions;
+
+ // TODO: remove when Got 12 gets released
+ /**
+ If set to `false`, all invalid SSL certificates will be ignored and no error will be thrown.
+
+ If set to `true`, it will throw an error whenever an invalid SSL certificate is detected.
+
+ We strongly recommend to have this set to `true` for security reasons.
+
+ @default true
+
+ @example
+ ```
+ const got = require('got');
+
+ (async () => {
+ // Correct:
+ await got('https://example.com', {rejectUnauthorized: true});
+
+ // You can disable it when developing an HTTPS app:
+ await got('https://localhost', {rejectUnauthorized: false});
+
+ // Never do this:
+ await got('https://example.com', {rejectUnauthorized: false});
+ })();
+ ```
+ */
+ rejectUnauthorized?: boolean; // Here for backwards compatibility
+
+ /**
+ Options for the advanced HTTPS API.
+ */
+ https?: HTTPSOptions;
+}
+
+export interface Options extends PromiseOnly.Options, PlainOptions {}
+
+export interface HTTPSOptions {
+ // From `http.RequestOptions` and `tls.CommonConnectionOptions`
+ rejectUnauthorized?: https.RequestOptions['rejectUnauthorized'];
+
+ // From `tls.ConnectionOptions`
+ checkServerIdentity?: CheckServerIdentityFunction;
+
+ // From `tls.SecureContextOptions`
+ /**
+ Override the default Certificate Authorities ([from Mozilla](https://ccadb-public.secure.force.com/mozilla/IncludedCACertificateReport)).
+
+ @example
+ ```
+ // Single Certificate Authority
+ got('https://example.com', {
+ https: {
+ certificateAuthority: fs.readFileSync('./my_ca.pem')
+ }
+ });
+ ```
+ */
+ certificateAuthority?: SecureContextOptions['ca'];
+
+ /**
+ Private keys in [PEM](https://en.wikipedia.org/wiki/Privacy-Enhanced_Mail) format.
+
+ [PEM](https://en.wikipedia.org/wiki/Privacy-Enhanced_Mail) allows the option of private keys being encrypted.
+ Encrypted keys will be decrypted with `options.https.passphrase`.
+
+ Multiple keys with different passphrases can be provided as an array of `{pem: , passphrase: }`
+ */
+ key?: SecureContextOptions['key'];
+
+ /**
+ [Certificate chains](https://en.wikipedia.org/wiki/X.509#Certificate_chains_and_cross-certification) in [PEM](https://en.wikipedia.org/wiki/Privacy-Enhanced_Mail) format.
+
+ One cert chain should be provided per private key (`options.https.key`).
+
+ When providing multiple cert chains, they do not have to be in the same order as their private keys in `options.https.key`.
+
+ If the intermediate certificates are not provided, the peer will not be able to validate the certificate, and the handshake will fail.
+ */
+ certificate?: SecureContextOptions['cert'];
+
+ /**
+ The passphrase to decrypt the `options.https.key` (if different keys have different passphrases refer to `options.https.key` documentation).
+ */
+ passphrase?: SecureContextOptions['passphrase'];
+ pfx?: SecureContextOptions['pfx'];
+}
+
+interface NormalizedPlainOptions extends PlainOptions {
+ method: Method;
+ url: URL;
+ timeout: Delays;
+ prefixUrl: string;
+ ignoreInvalidCookies: boolean;
+ decompress: boolean;
+ searchParams?: URLSearchParams;
+ cookieJar?: PromiseCookieJar;
+ headers: Headers;
+ context: Record;
+ hooks: Required;
+ followRedirect: boolean;
+ maxRedirects: number;
+ cache?: string | CacheableRequest.StorageAdapter;
+ throwHttpErrors: boolean;
+ dnsCache?: CacheableLookup;
+ http2: boolean;
+ allowGetBody: boolean;
+ rejectUnauthorized: boolean;
+ lookup?: CacheableLookup['lookup'];
+ methodRewriting: boolean;
+ username: string;
+ password: string;
+ parseJson: ParseJsonFunction;
+ stringifyJson: StringifyJsonFunction;
+ retry: RequiredRetryOptions;
+ cacheOptions: CacheOptions;
+ [kRequest]: HttpRequestFunction;
+ [kIsNormalizedAlready]?: boolean;
+}
+
+export interface NormalizedOptions extends PromiseOnly.NormalizedOptions, NormalizedPlainOptions {}
+
+interface PlainDefaults {
+ timeout: Delays;
+ prefixUrl: string;
+ method: Method;
+ ignoreInvalidCookies: boolean;
+ decompress: boolean;
+ context: Record;
+ cookieJar?: PromiseCookieJar | ToughCookieJar;
+ dnsCache?: CacheableLookup;
+ headers: Headers;
+ hooks: Required;
+ followRedirect: boolean;
+ maxRedirects: number;
+ cache?: string | CacheableRequest.StorageAdapter;
+ throwHttpErrors: boolean;
+ http2: boolean;
+ allowGetBody: boolean;
+ https?: HTTPSOptions;
+ methodRewriting: boolean;
+ parseJson: ParseJsonFunction;
+ stringifyJson: StringifyJsonFunction;
+ retry: RequiredRetryOptions;
+
+ // Optional
+ agent?: Agents | false;
+ request?: RequestFunction;
+ searchParams?: URLSearchParams;
+ lookup?: CacheableLookup['lookup'];
+ localAddress?: string;
+ createConnection?: Options['createConnection'];
+
+ // From `http-cache-semantics`
+ cacheOptions: CacheOptions;
+}
+
+export interface Defaults extends PromiseOnly.Defaults, PlainDefaults {}
+
+export interface Progress {
+ percent: number;
+ transferred: number;
+ total?: number;
+}
+
+export interface PlainResponse extends IncomingMessageWithTimings {
+ /**
+ The original request URL.
+ */
+ requestUrl: string;
+
+ /**
+ The redirect URLs.
+ */
+ redirectUrls: string[];
+
+ /**
+ - `options` - The Got options that were set on this request.
+
+ __Note__: This is not a [http.ClientRequest](https://nodejs.org/api/http.html#http_class_http_clientrequest).
+ */
+ request: Request;
+
+ /**
+ The remote IP address.
+
+ This is hopefully a temporary limitation, see [lukechilds/cacheable-request#86](https://github.com/lukechilds/cacheable-request/issues/86).
+
+ __Note__: Not available when the response is cached.
+ */
+ ip?: string;
+
+ /**
+ Whether the response was retrieved from the cache.
+ */
+ isFromCache: boolean;
+
+ /**
+ The status code of the response.
+ */
+ statusCode: number;
+
+ /**
+ The request URL or the final URL after redirects.
+ */
+ url: string;
+
+ /**
+ The object contains the following properties:
+
+ - `start` - Time when the request started.
+ - `socket` - Time when a socket was assigned to the request.
+ - `lookup` - Time when the DNS lookup finished.
+ - `connect` - Time when the socket successfully connected.
+ - `secureConnect` - Time when the socket securely connected.
+ - `upload` - Time when the request finished uploading.
+ - `response` - Time when the request fired `response` event.
+ - `end` - Time when the response fired `end` event.
+ - `error` - Time when the request fired `error` event.
+ - `abort` - Time when the request fired `abort` event.
+ - `phases`
+ - `wait` - `timings.socket - timings.start`
+ - `dns` - `timings.lookup - timings.socket`
+ - `tcp` - `timings.connect - timings.lookup`
+ - `tls` - `timings.secureConnect - timings.connect`
+ - `request` - `timings.upload - (timings.secureConnect || timings.connect)`
+ - `firstByte` - `timings.response - timings.upload`
+ - `download` - `timings.end - timings.response`
+ - `total` - `(timings.end || timings.error || timings.abort) - timings.start`
+
+ If something has not been measured yet, it will be `undefined`.
+
+ __Note__: The time is a `number` representing the milliseconds elapsed since the UNIX epoch.
+ */
+ timings: Timings;
+
+ /**
+ The number of times the request was retried.
+ */
+ retryCount: number;
+
+ // Defined only if request errored
+ /**
+ The raw result of the request.
+ */
+ rawBody?: Buffer;
+
+ /**
+ The result of the request.
+ */
+ body?: unknown;
+}
+
+// For Promise support
+export interface Response extends PlainResponse {
+ /**
+ The result of the request.
+ */
+ body: T;
+
+ /**
+ The raw result of the request.
+ */
+ rawBody: Buffer;
+}
+
+export interface RequestEvents {
+ /**
+ `request` event to get the request object of the request.
+
+ __Tip__: You can use `request` event to abort requests.
+
+ @example
+ ```
+ got.stream('https://github.com')
+ .on('request', request => setTimeout(() => request.destroy(), 50));
+ ```
+ */
+ on: ((name: 'request', listener: (request: http.ClientRequest) => void) => T)
+
+ /**
+ The `response` event to get the response object of the final request.
+ */
+ & ((name: 'response', listener: (response: R) => void) => T)
+
+ /**
+ The `redirect` event to get the response object of a redirect. The second argument is options for the next request to the redirect location.
+ */
+ & ((name: 'redirect', listener: (response: R, nextOptions: N) => void) => T)
+
+ /**
+ Progress events for uploading (sending a request) and downloading (receiving a response).
+ The `progress` argument is an object like:
+
+ ```js
+ {
+ percent: 0.1,
+ transferred: 1024,
+ total: 10240
+ }
+ ```
+
+ If the `content-length` header is missing, `total` will be `undefined`.
+
+ @example
+ ```js
+ (async () => {
+ const response = await got('https://sindresorhus.com')
+ .on('downloadProgress', progress => {
+ // Report download progress
+ })
+ .on('uploadProgress', progress => {
+ // Report upload progress
+ });
+
+ console.log(response);
+ })();
+ ```
+ */
+ & ((name: 'uploadProgress' | 'downloadProgress', listener: (progress: Progress) => void) => T)
+ /**
+ To enable retrying on a Got stream, it is required to have a `retry` handler attached.
+
+ When this event is emitted, you should reset the stream you were writing to and prepare the body again.
+
+ See `got.options.retry` for more information.
+ */
+ & ((name: 'retry', listener: (retryCount: number, error: RequestError) => void) => T);
+}
+
+function validateSearchParameters(searchParameters: Record): asserts searchParameters is Record {
+ // eslint-disable-next-line guard-for-in
+ for (const key in searchParameters) {
+ const value = searchParameters[key];
+
+ if (!is.string(value) && !is.number(value) && !is.boolean(value) && !is.null_(value) && !is.undefined(value)) {
+ throw new TypeError(`The \`searchParams\` value '${String(value)}' must be a string, number, boolean or null`);
+ }
+ }
+}
+
+function isClientRequest(clientRequest: unknown): clientRequest is ClientRequest {
+ return is.object(clientRequest) && !('statusCode' in clientRequest);
+}
+
+const cacheableStore = new WeakableMap();
+
+const waitForOpenFile = async (file: ReadStream): Promise => new Promise((resolve, reject) => {
+ const onError = (error: Error): void => {
+ reject(error);
+ };
+
+ // Node.js 12 has incomplete types
+ if (!(file as any).pending) {
+ resolve();
+ }
+
+ file.once('error', onError);
+ file.once('ready', () => {
+ file.off('error', onError);
+ resolve();
+ });
+});
+
+const redirectCodes: ReadonlySet = new Set([300, 301, 302, 303, 304, 307, 308]);
+
+type NonEnumerableProperty = 'context' | 'body' | 'json' | 'form';
+const nonEnumerableProperties: NonEnumerableProperty[] = [
+ 'context',
+ 'body',
+ 'json',
+ 'form'
+];
+
+export const setNonEnumerableProperties = (sources: Array, to: Options): void => {
+ // Non enumerable properties shall not be merged
+ const properties: Partial<{[Key in NonEnumerableProperty]: any}> = {};
+
+ for (const source of sources) {
+ if (!source) {
+ continue;
+ }
+
+ for (const name of nonEnumerableProperties) {
+ if (!(name in source)) {
+ continue;
+ }
+
+ properties[name] = {
+ writable: true,
+ configurable: true,
+ enumerable: false,
+ // @ts-expect-error TS doesn't see the check above
+ value: source[name]
+ };
+ }
+ }
+
+ Object.defineProperties(to, properties);
+};
+
+/**
+An error to be thrown when a request fails.
+Contains a `code` property with error class code, like `ECONNREFUSED`.
+*/
+export class RequestError extends Error {
+ code: string;
+ stack!: string;
+ declare readonly options: NormalizedOptions;
+ readonly response?: Response;
+ readonly request?: Request;
+ readonly timings?: Timings;
+
+ constructor(message: string, error: Partial, self: Request | NormalizedOptions) {
+ super(message);
+ Error.captureStackTrace(this, this.constructor);
+
+ this.name = 'RequestError';
+ this.code = error.code ?? 'ERR_GOT_REQUEST_ERROR';
+
+ if (self instanceof Request) {
+ Object.defineProperty(this, 'request', {
+ enumerable: false,
+ value: self
+ });
+
+ Object.defineProperty(this, 'response', {
+ enumerable: false,
+ value: self[kResponse]
+ });
+
+ Object.defineProperty(this, 'options', {
+ // This fails because of TS 3.7.2 useDefineForClassFields
+ // Ref: https://github.com/microsoft/TypeScript/issues/34972
+ enumerable: false,
+ value: self.options
+ });
+ } else {
+ Object.defineProperty(this, 'options', {
+ // This fails because of TS 3.7.2 useDefineForClassFields
+ // Ref: https://github.com/microsoft/TypeScript/issues/34972
+ enumerable: false,
+ value: self
+ });
+ }
+
+ this.timings = this.request?.timings;
+
+ // Recover the original stacktrace
+ if (is.string(error.stack) && is.string(this.stack)) {
+ const indexOfMessage = this.stack.indexOf(this.message) + this.message.length;
+ const thisStackTrace = this.stack.slice(indexOfMessage).split('\n').reverse();
+ const errorStackTrace = error.stack.slice(error.stack.indexOf(error.message!) + error.message!.length).split('\n').reverse();
+
+ // Remove duplicated traces
+ while (errorStackTrace.length !== 0 && errorStackTrace[0] === thisStackTrace[0]) {
+ thisStackTrace.shift();
+ }
+
+ this.stack = `${this.stack.slice(0, indexOfMessage)}${thisStackTrace.reverse().join('\n')}${errorStackTrace.reverse().join('\n')}`;
+ }
+ }
+}
+
+/**
+An error to be thrown when the server redirects you more than ten times.
+Includes a `response` property.
+*/
+export class MaxRedirectsError extends RequestError {
+ declare readonly response: Response;
+ declare readonly request: Request;
+ declare readonly timings: Timings;
+
+ constructor(request: Request) {
+ super(`Redirected ${request.options.maxRedirects} times. Aborting.`, {}, request);
+ this.name = 'MaxRedirectsError';
+ this.code = 'ERR_TOO_MANY_REDIRECTS';
+ }
+}
+
+/**
+An error to be thrown when the server response code is not 2xx nor 3xx if `options.followRedirect` is `true`, but always except for 304.
+Includes a `response` property.
+*/
+export class HTTPError extends RequestError {
+ declare readonly response: Response;
+ declare readonly request: Request;
+ declare readonly timings: Timings;
+
+ constructor(response: Response) {
+ super(`Response code ${response.statusCode} (${response.statusMessage!})`, {}, response.request);
+ this.name = 'HTTPError';
+ this.code = 'ERR_NON_2XX_3XX_RESPONSE';
+ }
+}
+/**
+An error to be thrown when a cache method fails.
+For example, if the database goes down or there's a filesystem error.
+*/
+export class CacheError extends RequestError {
+ declare readonly request: Request;
+
+ constructor(error: Error, request: Request) {
+ super(error.message, error, request);
+ this.name = 'CacheError';
+ this.code = this.code === 'ERR_GOT_REQUEST_ERROR' ? 'ERR_CACHE_ACCESS' : this.code;
+ }
+}
+
+/**
+An error to be thrown when the request body is a stream and an error occurs while reading from that stream.
+*/
+export class UploadError extends RequestError {
+ declare readonly request: Request;
+
+ constructor(error: Error, request: Request) {
+ super(error.message, error, request);
+ this.name = 'UploadError';
+ this.code = this.code === 'ERR_GOT_REQUEST_ERROR' ? 'ERR_UPLOAD' : this.code;
+ }
+}
+
+/**
+An error to be thrown when the request is aborted due to a timeout.
+Includes an `event` and `timings` property.
+*/
+export class TimeoutError extends RequestError {
+ declare readonly request: Request;
+ readonly timings: Timings;
+ readonly event: string;
+
+ constructor(error: TimedOutTimeoutError, timings: Timings, request: Request) {
+ super(error.message, error, request);
+ this.name = 'TimeoutError';
+ this.event = error.event;
+ this.timings = timings;
+ }
+}
+
+/**
+An error to be thrown when reading from response stream fails.
+*/
+export class ReadError extends RequestError {
+ declare readonly request: Request;
+ declare readonly response: Response;
+ declare readonly timings: Timings;
+
+ constructor(error: Error, request: Request) {
+ super(error.message, error, request);
+ this.name = 'ReadError';
+ this.code = this.code === 'ERR_GOT_REQUEST_ERROR' ? 'ERR_READING_RESPONSE_STREAM' : this.code;
+ }
+}
+
+/**
+An error to be thrown when given an unsupported protocol.
+*/
+export class UnsupportedProtocolError extends RequestError {
+ constructor(options: NormalizedOptions) {
+ super(`Unsupported protocol "${options.url.protocol}"`, {}, options);
+ this.name = 'UnsupportedProtocolError';
+ this.code = 'ERR_UNSUPPORTED_PROTOCOL';
+ }
+}
+
+const proxiedRequestEvents = [
+ 'socket',
+ 'connect',
+ 'continue',
+ 'information',
+ 'upgrade',
+ 'timeout'
+];
+
+export default class Request extends Duplex implements RequestEvents {
+ ['constructor']: typeof Request;
+
+ declare [kUnproxyEvents]: () => void;
+ declare _cannotHaveBody: boolean;
+ [kDownloadedSize]: number;
+ [kUploadedSize]: number;
+ [kStopReading]: boolean;
+ [kTriggerRead]: boolean;
+ [kBody]: Options['body'];
+ [kJobs]: Array<() => void>;
+ [kRetryTimeout]?: NodeJS.Timeout;
+ [kBodySize]?: number;
+ [kServerResponsesPiped]: Set;
+ [kIsFromCache]?: boolean;
+ [kStartedReading]?: boolean;
+ [kCancelTimeouts]?: () => void;
+ [kResponseSize]?: number;
+ [kResponse]?: IncomingMessageWithTimings;
+ [kOriginalResponse]?: IncomingMessageWithTimings;
+ [kRequest]?: ClientRequest;
+ _noPipe?: boolean;
+ _progressCallbacks: Array<() => void>;
+
+ declare options: NormalizedOptions;
+ declare requestUrl: string;
+ requestInitialized: boolean;
+ redirects: string[];
+ retryCount: number;
+
+ constructor(url: string | URL | undefined, options: Options = {}, defaults?: Defaults) {
+ super({
+ // This must be false, to enable throwing after destroy
+ // It is used for retry logic in Promise API
+ autoDestroy: false,
+ // It needs to be zero because we're just proxying the data to another stream
+ highWaterMark: 0
+ });
+
+ this[kDownloadedSize] = 0;
+ this[kUploadedSize] = 0;
+ this.requestInitialized = false;
+ this[kServerResponsesPiped] = new Set();
+ this.redirects = [];
+ this[kStopReading] = false;
+ this[kTriggerRead] = false;
+ this[kJobs] = [];
+ this.retryCount = 0;
+
+ // TODO: Remove this when targeting Node.js >= 12
+ this._progressCallbacks = [];
+
+ const unlockWrite = (): void => this._unlockWrite();
+ const lockWrite = (): void => this._lockWrite();
+
+ this.on('pipe', (source: Writable) => {
+ source.prependListener('data', unlockWrite);
+ source.on('data', lockWrite);
+
+ source.prependListener('end', unlockWrite);
+ source.on('end', lockWrite);
+ });
+
+ this.on('unpipe', (source: Writable) => {
+ source.off('data', unlockWrite);
+ source.off('data', lockWrite);
+
+ source.off('end', unlockWrite);
+ source.off('end', lockWrite);
+ });
+
+ this.on('pipe', source => {
+ if (source instanceof IncomingMessage) {
+ this.options.headers = {
+ ...source.headers,
+ ...this.options.headers
+ };
+ }
+ });
+
+ const {json, body, form} = options;
+ if (json || body || form) {
+ this._lockWrite();
+ }
+
+ if (kIsNormalizedAlready in options) {
+ this.options = options as NormalizedOptions;
+ } else {
+ try {
+ // @ts-expect-error Common TypeScript bug saying that `this.constructor` is not accessible
+ this.options = this.constructor.normalizeArguments(url, options, defaults);
+ } catch (error) {
+ // TODO: Move this to `_destroy()`
+ if (is.nodeStream(options.body)) {
+ options.body.destroy();
+ }
+
+ this.destroy(error);
+ return;
+ }
+ }
+
+ (async () => {
+ try {
+ if (this.options.body instanceof ReadStream) {
+ await waitForOpenFile(this.options.body);
+ }
+
+ const {url: normalizedURL} = this.options;
+
+ if (!normalizedURL) {
+ throw new TypeError('Missing `url` property');
+ }
+
+ this.requestUrl = normalizedURL.toString();
+ decodeURI(this.requestUrl);
+
+ await this._finalizeBody();
+ await this._makeRequest();
+
+ if (this.destroyed) {
+ this[kRequest]?.destroy();
+ return;
+ }
+
+ // Queued writes etc.
+ for (const job of this[kJobs]) {
+ job();
+ }
+
+ // Prevent memory leak
+ this[kJobs].length = 0;
+
+ this.requestInitialized = true;
+ } catch (error) {
+ if (error instanceof RequestError) {
+ this._beforeError(error);
+ return;
+ }
+
+ // This is a workaround for https://github.com/nodejs/node/issues/33335
+ if (!this.destroyed) {
+ this.destroy(error);
+ }
+ }
+ })();
+ }
+
+ static normalizeArguments(url?: string | URL, options?: Options, defaults?: Defaults): NormalizedOptions {
+ const rawOptions = options;
+
+ if (is.object(url) && !is.urlInstance(url)) {
+ options = {...defaults, ...(url as Options), ...options};
+ } else {
+ if (url && options && options.url !== undefined) {
+ throw new TypeError('The `url` option is mutually exclusive with the `input` argument');
+ }
+
+ options = {...defaults, ...options};
+
+ if (url !== undefined) {
+ options.url = url;
+ }
+
+ if (is.urlInstance(options.url)) {
+ options.url = new URL(options.url.toString());
+ }
+ }
+
+ // TODO: Deprecate URL options in Got 12.
+
+ // Support extend-specific options
+ if (options.cache === false) {
+ options.cache = undefined;
+ }
+
+ if (options.dnsCache === false) {
+ options.dnsCache = undefined;
+ }
+
+ // Nice type assertions
+ assert.any([is.string, is.undefined], options.method);
+ assert.any([is.object, is.undefined], options.headers);
+ assert.any([is.string, is.urlInstance, is.undefined], options.prefixUrl);
+ assert.any([is.object, is.undefined], options.cookieJar);
+ assert.any([is.object, is.string, is.undefined], options.searchParams);
+ assert.any([is.object, is.string, is.undefined], options.cache);
+ assert.any([is.object, is.number, is.undefined], options.timeout);
+ assert.any([is.object, is.undefined], options.context);
+ assert.any([is.object, is.undefined], options.hooks);
+ assert.any([is.boolean, is.undefined], options.decompress);
+ assert.any([is.boolean, is.undefined], options.ignoreInvalidCookies);
+ assert.any([is.boolean, is.undefined], options.followRedirect);
+ assert.any([is.number, is.undefined], options.maxRedirects);
+ assert.any([is.boolean, is.undefined], options.throwHttpErrors);
+ assert.any([is.boolean, is.undefined], options.http2);
+ assert.any([is.boolean, is.undefined], options.allowGetBody);
+ assert.any([is.string, is.undefined], options.localAddress);
+ assert.any([isDnsLookupIpVersion, is.undefined], options.dnsLookupIpVersion);
+ assert.any([is.object, is.undefined], options.https);
+ assert.any([is.boolean, is.undefined], options.rejectUnauthorized);
+
+ if (options.https) {
+ assert.any([is.boolean, is.undefined], options.https.rejectUnauthorized);
+ assert.any([is.function_, is.undefined], options.https.checkServerIdentity);
+ assert.any([is.string, is.object, is.array, is.undefined], options.https.certificateAuthority);
+ assert.any([is.string, is.object, is.array, is.undefined], options.https.key);
+ assert.any([is.string, is.object, is.array, is.undefined], options.https.certificate);
+ assert.any([is.string, is.undefined], options.https.passphrase);
+ assert.any([is.string, is.buffer, is.array, is.undefined], options.https.pfx);
+ }
+
+ assert.any([is.object, is.undefined], options.cacheOptions);
+
+ // `options.method`
+ if (is.string(options.method)) {
+ options.method = options.method.toUpperCase() as Method;
+ } else {
+ options.method = 'GET';
+ }
+
+ // `options.headers`
+ if (options.headers === defaults?.headers) {
+ options.headers = {...options.headers};
+ } else {
+ options.headers = lowercaseKeys({...(defaults?.headers), ...options.headers});
+ }
+
+ // Disallow legacy `url.Url`
+ if ('slashes' in options) {
+ throw new TypeError('The legacy `url.Url` has been deprecated. Use `URL` instead.');
+ }
+
+ // `options.auth`
+ if ('auth' in options) {
+ throw new TypeError('Parameter `auth` is deprecated. Use `username` / `password` instead.');
+ }
+
+ // `options.searchParams`
+ if ('searchParams' in options) {
+ if (options.searchParams && options.searchParams !== defaults?.searchParams) {
+ let searchParameters: URLSearchParams;
+
+ if (is.string(options.searchParams) || (options.searchParams instanceof URLSearchParams)) {
+ searchParameters = new URLSearchParams(options.searchParams);
+ } else {
+ validateSearchParameters(options.searchParams);
+
+ searchParameters = new URLSearchParams();
+
+ // eslint-disable-next-line guard-for-in
+ for (const key in options.searchParams) {
+ const value = options.searchParams[key];
+
+ if (value === null) {
+ searchParameters.append(key, '');
+ } else if (value !== undefined) {
+ searchParameters.append(key, value as string);
+ }
+ }
+ }
+
+ // `normalizeArguments()` is also used to merge options
+ defaults?.searchParams?.forEach((value, key) => {
+ // Only use default if one isn't already defined
+ if (!searchParameters.has(key)) {
+ searchParameters.append(key, value);
+ }
+ });
+
+ options.searchParams = searchParameters;
+ }
+ }
+
+ // `options.username` & `options.password`
+ options.username = options.username ?? '';
+ options.password = options.password ?? '';
+
+ // `options.prefixUrl` & `options.url`
+ if (is.undefined(options.prefixUrl)) {
+ options.prefixUrl = defaults?.prefixUrl ?? '';
+ } else {
+ options.prefixUrl = options.prefixUrl.toString();
+
+ if (options.prefixUrl !== '' && !options.prefixUrl.endsWith('/')) {
+ options.prefixUrl += '/';
+ }
+ }
+
+ if (is.string(options.url)) {
+ if (options.url.startsWith('/')) {
+ throw new Error('`input` must not start with a slash when using `prefixUrl`');
+ }
+
+ options.url = optionsToUrl(options.prefixUrl + options.url, options as Options & {searchParams?: URLSearchParams});
+ } else if ((is.undefined(options.url) && options.prefixUrl !== '') || options.protocol) {
+ options.url = optionsToUrl(options.prefixUrl, options as Options & {searchParams?: URLSearchParams});
+ }
+
+ if (options.url) {
+ if ('port' in options) {
+ delete options.port;
+ }
+
+ // Make it possible to change `options.prefixUrl`
+ let {prefixUrl} = options;
+ Object.defineProperty(options, 'prefixUrl', {
+ set: (value: string) => {
+ const url = options!.url as URL;
+
+ if (!url.href.startsWith(value)) {
+ throw new Error(`Cannot change \`prefixUrl\` from ${prefixUrl} to ${value}: ${url.href}`);
+ }
+
+ options!.url = new URL(value + url.href.slice(prefixUrl.length));
+ prefixUrl = value;
+ },
+ get: () => prefixUrl
+ });
+
+ // Support UNIX sockets
+ let {protocol} = options.url;
+
+ if (protocol === 'unix:') {
+ protocol = 'http:';
+
+ options.url = new URL(`http://unix${options.url.pathname}${options.url.search}`);
+ }
+
+ // Set search params
+ if (options.searchParams) {
+ // eslint-disable-next-line @typescript-eslint/no-base-to-string
+ options.url.search = options.searchParams.toString();
+ }
+
+ // Protocol check
+ if (protocol !== 'http:' && protocol !== 'https:') {
+ throw new UnsupportedProtocolError(options as NormalizedOptions);
+ }
+
+ // Update `username`
+ if (options.username === '') {
+ options.username = options.url.username;
+ } else {
+ options.url.username = options.username;
+ }
+
+ // Update `password`
+ if (options.password === '') {
+ options.password = options.url.password;
+ } else {
+ options.url.password = options.password;
+ }
+ }
+
+ // `options.cookieJar`
+ const {cookieJar} = options;
+ if (cookieJar) {
+ let {setCookie, getCookieString} = cookieJar;
+
+ assert.function_(setCookie);
+ assert.function_(getCookieString);
+
+ /* istanbul ignore next: Horrible `tough-cookie` v3 check */
+ if (setCookie.length === 4 && getCookieString.length === 0) {
+ setCookie = promisify(setCookie.bind(options.cookieJar));
+ getCookieString = promisify(getCookieString.bind(options.cookieJar));
+
+ options.cookieJar = {
+ setCookie,
+ getCookieString: getCookieString as PromiseCookieJar['getCookieString']
+ };
+ }
+ }
+
+ // `options.cache`
+ const {cache} = options;
+ if (cache) {
+ if (!cacheableStore.has(cache)) {
+ cacheableStore.set(cache, new CacheableRequest(
+ ((requestOptions: RequestOptions, handler?: (response: IncomingMessageWithTimings) => void): ClientRequest => {
+ const result = (requestOptions as Pick)[kRequest](requestOptions, handler);
+
+ // TODO: remove this when `cacheable-request` supports async request functions.
+ if (is.promise(result)) {
+ // @ts-expect-error
+ // We only need to implement the error handler in order to support HTTP2 caching.
+ // The result will be a promise anyway.
+ result.once = (event: string, handler: (reason: unknown) => void) => {
+ if (event === 'error') {
+ result.catch(handler);
+ } else if (event === 'abort') {
+ // The empty catch is needed here in case when
+ // it rejects before it's `await`ed in `_makeRequest`.
+ (async () => {
+ try {
+ const request = (await result) as ClientRequest;
+ request.once('abort', handler);
+ } catch {}
+ })();
+ } else {
+ /* istanbul ignore next: safety check */
+ throw new Error(`Unknown HTTP2 promise event: ${event}`);
+ }
+
+ return result;
+ };
+ }
+
+ return result;
+ }) as HttpRequestFunction,
+ cache as CacheableRequest.StorageAdapter
+ ));
+ }
+ }
+
+ // `options.cacheOptions`
+ options.cacheOptions = {...options.cacheOptions};
+
+ // `options.dnsCache`
+ if (options.dnsCache === true) {
+ if (!globalDnsCache) {
+ globalDnsCache = new CacheableLookup();
+ }
+
+ options.dnsCache = globalDnsCache;
+ } else if (!is.undefined(options.dnsCache) && !options.dnsCache.lookup) {
+ throw new TypeError(`Parameter \`dnsCache\` must be a CacheableLookup instance or a boolean, got ${is(options.dnsCache)}`);
+ }
+
+ // `options.timeout`
+ if (is.number(options.timeout)) {
+ options.timeout = {request: options.timeout};
+ } else if (defaults && options.timeout !== defaults.timeout) {
+ options.timeout = {
+ ...defaults.timeout,
+ ...options.timeout
+ };
+ } else {
+ options.timeout = {...options.timeout};
+ }
+
+ // `options.context`
+ if (!options.context) {
+ options.context = {};
+ }
+
+ // `options.hooks`
+ const areHooksDefault = options.hooks === defaults?.hooks;
+ options.hooks = {...options.hooks};
+
+ for (const event of knownHookEvents) {
+ if (event in options.hooks) {
+ if (is.array(options.hooks[event])) {
+ // See https://github.com/microsoft/TypeScript/issues/31445#issuecomment-576929044
+ (options.hooks as any)[event] = [...options.hooks[event]!];
+ } else {
+ throw new TypeError(`Parameter \`${event}\` must be an Array, got ${is(options.hooks[event])}`);
+ }
+ } else {
+ options.hooks[event] = [];
+ }
+ }
+
+ if (defaults && !areHooksDefault) {
+ for (const event of knownHookEvents) {
+ const defaultHooks = defaults.hooks[event];
+
+ if (defaultHooks.length > 0) {
+ // See https://github.com/microsoft/TypeScript/issues/31445#issuecomment-576929044
+ (options.hooks as any)[event] = [
+ ...defaults.hooks[event],
+ ...options.hooks[event]!
+ ];
+ }
+ }
+ }
+
+ // DNS options
+ if ('family' in options) {
+ deprecationWarning('"options.family" was never documented, please use "options.dnsLookupIpVersion"');
+ }
+
+ // HTTPS options
+ if (defaults?.https) {
+ options.https = {...defaults.https, ...options.https};
+ }
+
+ if ('rejectUnauthorized' in options) {
+ deprecationWarning('"options.rejectUnauthorized" is now deprecated, please use "options.https.rejectUnauthorized"');
+ }
+
+ if ('checkServerIdentity' in options) {
+ deprecationWarning('"options.checkServerIdentity" was never documented, please use "options.https.checkServerIdentity"');
+ }
+
+ if ('ca' in options) {
+ deprecationWarning('"options.ca" was never documented, please use "options.https.certificateAuthority"');
+ }
+
+ if ('key' in options) {
+ deprecationWarning('"options.key" was never documented, please use "options.https.key"');
+ }
+
+ if ('cert' in options) {
+ deprecationWarning('"options.cert" was never documented, please use "options.https.certificate"');
+ }
+
+ if ('passphrase' in options) {
+ deprecationWarning('"options.passphrase" was never documented, please use "options.https.passphrase"');
+ }
+
+ if ('pfx' in options) {
+ deprecationWarning('"options.pfx" was never documented, please use "options.https.pfx"');
+ }
+
+ // Other options
+ if ('followRedirects' in options) {
+ throw new TypeError('The `followRedirects` option does not exist. Use `followRedirect` instead.');
+ }
+
+ if (options.agent) {
+ for (const key in options.agent) {
+ if (key !== 'http' && key !== 'https' && key !== 'http2') {
+ throw new TypeError(`Expected the \`options.agent\` properties to be \`http\`, \`https\` or \`http2\`, got \`${key}\``);
+ }
+ }
+ }
+
+ options.maxRedirects = options.maxRedirects ?? 0;
+
+ // Set non-enumerable properties
+ setNonEnumerableProperties([defaults, rawOptions], options);
+
+ return normalizePromiseArguments(options as NormalizedOptions, defaults);
+ }
+
+ _lockWrite(): void {
+ const onLockedWrite = (): never => {
+ throw new TypeError('The payload has been already provided');
+ };
+
+ this.write = onLockedWrite;
+ this.end = onLockedWrite;
+ }
+
+ _unlockWrite(): void {
+ this.write = super.write;
+ this.end = super.end;
+ }
+
+ async _finalizeBody(): Promise {
+ const {options} = this;
+ const {headers} = options;
+
+ const isForm = !is.undefined(options.form);
+ const isJSON = !is.undefined(options.json);
+ const isBody = !is.undefined(options.body);
+ const hasPayload = isForm || isJSON || isBody;
+ const cannotHaveBody = withoutBody.has(options.method) && !(options.method === 'GET' && options.allowGetBody);
+
+ this._cannotHaveBody = cannotHaveBody;
+
+ if (hasPayload) {
+ if (cannotHaveBody) {
+ throw new TypeError(`The \`${options.method}\` method cannot be used with a body`);
+ }
+
+ if ([isBody, isForm, isJSON].filter(isTrue => isTrue).length > 1) {
+ throw new TypeError('The `body`, `json` and `form` options are mutually exclusive');
+ }
+
+ if (
+ isBody &&
+ !(options.body instanceof Readable) &&
+ !is.string(options.body) &&
+ !is.buffer(options.body) &&
+ !isFormData(options.body)
+ ) {
+ throw new TypeError('The `body` option must be a stream.Readable, string or Buffer');
+ }
+
+ if (isForm && !is.object(options.form)) {
+ throw new TypeError('The `form` option must be an Object');
+ }
+
+ {
+ // Serialize body
+ const noContentType = !is.string(headers['content-type']);
+
+ if (isBody) {
+ // Special case for https://github.com/form-data/form-data
+ if (isFormData(options.body) && noContentType) {
+ headers['content-type'] = `multipart/form-data; boundary=${options.body.getBoundary()}`;
+ }
+
+ this[kBody] = options.body;
+ } else if (isForm) {
+ if (noContentType) {
+ headers['content-type'] = 'application/x-www-form-urlencoded';
+ }
+
+ this[kBody] = (new URLSearchParams(options.form as Record)).toString();
+ } else {
+ if (noContentType) {
+ headers['content-type'] = 'application/json';
+ }
+
+ this[kBody] = options.stringifyJson(options.json);
+ }
+
+ const uploadBodySize = await getBodySize(this[kBody], options.headers);
+
+ // See https://tools.ietf.org/html/rfc7230#section-3.3.2
+ // A user agent SHOULD send a Content-Length in a request message when
+ // no Transfer-Encoding is sent and the request method defines a meaning
+ // for an enclosed payload body. For example, a Content-Length header
+ // field is normally sent in a POST request even when the value is 0
+ // (indicating an empty payload body). A user agent SHOULD NOT send a
+ // Content-Length header field when the request message does not contain
+ // a payload body and the method semantics do not anticipate such a
+ // body.
+ if (is.undefined(headers['content-length']) && is.undefined(headers['transfer-encoding'])) {
+ if (!cannotHaveBody && !is.undefined(uploadBodySize)) {
+ headers['content-length'] = String(uploadBodySize);
+ }
+ }
+ }
+ } else if (cannotHaveBody) {
+ this._lockWrite();
+ } else {
+ this._unlockWrite();
+ }
+
+ this[kBodySize] = Number(headers['content-length']) || undefined;
+ }
+
+ async _onResponseBase(response: IncomingMessageWithTimings): Promise {
+ const {options} = this;
+ const {url} = options;
+
+ this[kOriginalResponse] = response;
+
+ if (options.decompress) {
+ response = decompressResponse(response);
+ }
+
+ const statusCode = response.statusCode!;
+ const typedResponse = response as Response;
+
+ typedResponse.statusMessage = typedResponse.statusMessage ? typedResponse.statusMessage : http.STATUS_CODES[statusCode];
+ typedResponse.url = options.url.toString();
+ typedResponse.requestUrl = this.requestUrl;
+ typedResponse.redirectUrls = this.redirects;
+ typedResponse.request = this;
+ typedResponse.isFromCache = (response as any).fromCache || false;
+ typedResponse.ip = this.ip;
+ typedResponse.retryCount = this.retryCount;
+
+ this[kIsFromCache] = typedResponse.isFromCache;
+
+ this[kResponseSize] = Number(response.headers['content-length']) || undefined;
+ this[kResponse] = response;
+
+ response.once('end', () => {
+ this[kResponseSize] = this[kDownloadedSize];
+ this.emit('downloadProgress', this.downloadProgress);
+ });
+
+ response.once('error', (error: Error) => {
+ // Force clean-up, because some packages don't do this.
+ // TODO: Fix decompress-response
+ response.destroy();
+
+ this._beforeError(new ReadError(error, this));
+ });
+
+ response.once('aborted', () => {
+ this._beforeError(new ReadError({
+ name: 'Error',
+ message: 'The server aborted pending request',
+ code: 'ECONNRESET'
+ }, this));
+ });
+
+ this.emit('downloadProgress', this.downloadProgress);
+
+ const rawCookies = response.headers['set-cookie'];
+ if (is.object(options.cookieJar) && rawCookies) {
+ let promises: Array> = rawCookies.map(async (rawCookie: string) => (options.cookieJar as PromiseCookieJar).setCookie(rawCookie, url.toString()));
+
+ if (options.ignoreInvalidCookies) {
+ promises = promises.map(async p => p.catch(() => {}));
+ }
+
+ try {
+ await Promise.all(promises);
+ } catch (error) {
+ this._beforeError(error);
+ return;
+ }
+ }
+
+ if (options.followRedirect && response.headers.location && redirectCodes.has(statusCode)) {
+ // We're being redirected, we don't care about the response.
+ // It'd be best to abort the request, but we can't because
+ // we would have to sacrifice the TCP connection. We don't want that.
+ response.resume();
+
+ if (this[kRequest]) {
+ this[kCancelTimeouts]!();
+
+ // eslint-disable-next-line @typescript-eslint/no-dynamic-delete
+ delete this[kRequest];
+ this[kUnproxyEvents]();
+ }
+
+ const shouldBeGet = statusCode === 303 && options.method !== 'GET' && options.method !== 'HEAD';
+ if (shouldBeGet || !options.methodRewriting) {
+ // Server responded with "see other", indicating that the resource exists at another location,
+ // and the client should request it from that location via GET or HEAD.
+ options.method = 'GET';
+
+ if ('body' in options) {
+ delete options.body;
+ }
+
+ if ('json' in options) {
+ delete options.json;
+ }
+
+ if ('form' in options) {
+ delete options.form;
+ }
+
+ this[kBody] = undefined;
+ delete options.headers['content-length'];
+ }
+
+ if (this.redirects.length >= options.maxRedirects) {
+ this._beforeError(new MaxRedirectsError(this));
+ return;
+ }
+
+ try {
+ // Do not remove. See https://github.com/sindresorhus/got/pull/214
+ const redirectBuffer = Buffer.from(response.headers.location, 'binary').toString();
+
+ // Handles invalid URLs. See https://github.com/sindresorhus/got/issues/604
+ const redirectUrl = new URL(redirectBuffer, url);
+ const redirectString = redirectUrl.toString();
+ decodeURI(redirectString);
+
+ // eslint-disable-next-line no-inner-declarations
+ function isUnixSocketURL(url: URL) {
+ return url.protocol === 'unix:' || url.hostname === 'unix';
+ }
+
+ if (!isUnixSocketURL(url) && isUnixSocketURL(redirectUrl)) {
+ this._beforeError(new RequestError('Cannot redirect to UNIX socket', {}, this));
+ return;
+ }
+
+ // Redirecting to a different site, clear sensitive data.
+ if (redirectUrl.hostname !== url.hostname || redirectUrl.port !== url.port) {
+ if ('host' in options.headers) {
+ delete options.headers.host;
+ }
+
+ if ('cookie' in options.headers) {
+ delete options.headers.cookie;
+ }
+
+ if ('authorization' in options.headers) {
+ delete options.headers.authorization;
+ }
+
+ if (options.username || options.password) {
+ options.username = '';
+ options.password = '';
+ }
+ } else {
+ redirectUrl.username = options.username;
+ redirectUrl.password = options.password;
+ }
+
+ this.redirects.push(redirectString);
+ options.url = redirectUrl;
+
+ for (const hook of options.hooks.beforeRedirect) {
+ // eslint-disable-next-line no-await-in-loop
+ await hook(options, typedResponse);
+ }
+
+ this.emit('redirect', typedResponse, options);
+
+ await this._makeRequest();
+ } catch (error) {
+ this._beforeError(error);
+ return;
+ }
+
+ return;
+ }
+
+ if (options.isStream && options.throwHttpErrors && !isResponseOk(typedResponse)) {
+ this._beforeError(new HTTPError(typedResponse));
+ return;
+ }
+
+ response.on('readable', () => {
+ if (this[kTriggerRead]) {
+ this._read();
+ }
+ });
+
+ this.on('resume', () => {
+ response.resume();
+ });
+
+ this.on('pause', () => {
+ response.pause();
+ });
+
+ response.once('end', () => {
+ this.push(null);
+ });
+
+ this.emit('response', response);
+
+ for (const destination of this[kServerResponsesPiped]) {
+ if (destination.headersSent) {
+ continue;
+ }
+
+ // eslint-disable-next-line guard-for-in
+ for (const key in response.headers) {
+ const isAllowed = options.decompress ? key !== 'content-encoding' : true;
+ const value = response.headers[key];
+
+ if (isAllowed) {
+ destination.setHeader(key, value!);
+ }
+ }
+
+ destination.statusCode = statusCode;
+ }
+ }
+
+ async _onResponse(response: IncomingMessageWithTimings): Promise {
+ try {
+ await this._onResponseBase(response);
+ } catch (error) {
+ /* istanbul ignore next: better safe than sorry */
+ this._beforeError(error);
+ }
+ }
+
+ _onRequest(request: ClientRequest): void {
+ const {options} = this;
+ const {timeout, url} = options;
+
+ timer(request);
+
+ this[kCancelTimeouts] = timedOut(request, timeout, url);
+
+ const responseEventName = options.cache ? 'cacheableResponse' : 'response';
+
+ request.once(responseEventName, (response: IncomingMessageWithTimings) => {
+ void this._onResponse(response);
+ });
+
+ request.once('error', (error: Error) => {
+ // Force clean-up, because some packages (e.g. nock) don't do this.
+ request.destroy();
+
+ // Node.js <= 12.18.2 mistakenly emits the response `end` first.
+ (request as ClientRequest & {res: IncomingMessage | undefined}).res?.removeAllListeners('end');
+
+ error = error instanceof TimedOutTimeoutError ? new TimeoutError(error, this.timings!, this) : new RequestError(error.message, error, this);
+
+ this._beforeError(error as RequestError);
+ });
+
+ this[kUnproxyEvents] = proxyEvents(request, this, proxiedRequestEvents);
+
+ this[kRequest] = request;
+
+ this.emit('uploadProgress', this.uploadProgress);
+
+ // Send body
+ const body = this[kBody];
+ const currentRequest = this.redirects.length === 0 ? this : request;
+
+ if (is.nodeStream(body)) {
+ body.pipe(currentRequest);
+ body.once('error', (error: NodeJS.ErrnoException) => {
+ this._beforeError(new UploadError(error, this));
+ });
+ } else {
+ this._unlockWrite();
+
+ if (!is.undefined(body)) {
+ this._writeRequest(body, undefined, () => {});
+ currentRequest.end();
+
+ this._lockWrite();
+ } else if (this._cannotHaveBody || this._noPipe) {
+ currentRequest.end();
+
+ this._lockWrite();
+ }
+ }
+
+ this.emit('request', request);
+ }
+
+ async _createCacheableRequest(url: URL, options: RequestOptions): Promise {
+ return new Promise((resolve, reject) => {
+ // TODO: Remove `utils/url-to-options.ts` when `cacheable-request` is fixed
+ Object.assign(options, urlToOptions(url));
+
+ // `http-cache-semantics` checks this
+ // TODO: Fix this ignore.
+ // @ts-expect-error
+ delete (options as unknown as NormalizedOptions).url;
+
+ let request: ClientRequest | Promise;
+
+ // This is ugly
+ const cacheRequest = cacheableStore.get((options as any).cache)!(options, async response => {
+ // TODO: Fix `cacheable-response`
+ (response as any)._readableState.autoDestroy = false;
+
+ if (request) {
+ (await request).emit('cacheableResponse', response);
+ }
+
+ resolve(response as unknown as ResponseLike);
+ });
+
+ // Restore options
+ (options as unknown as NormalizedOptions).url = url;
+
+ cacheRequest.once('error', reject);
+ cacheRequest.once('request', async (requestOrPromise: ClientRequest | Promise) => {
+ request = requestOrPromise;
+ resolve(request);
+ });
+ });
+ }
+
+ async _makeRequest(): Promise {
+ const {options} = this;
+
+ const {headers} = options;
+
+ for (const key in headers) {
+ if (is.undefined(headers[key])) {
+ // eslint-disable-next-line @typescript-eslint/no-dynamic-delete
+ delete headers[key];
+ } else if (is.null_(headers[key])) {
+ throw new TypeError(`Use \`undefined\` instead of \`null\` to delete the \`${key}\` header`);
+ }
+ }
+
+ if (options.decompress && is.undefined(headers['accept-encoding'])) {
+ headers['accept-encoding'] = supportsBrotli ? 'gzip, deflate, br' : 'gzip, deflate';
+ }
+
+ // Set cookies
+ if (options.cookieJar) {
+ const cookieString: string = await options.cookieJar.getCookieString(options.url.toString());
+
+ if (is.nonEmptyString(cookieString)) {
+ options.headers.cookie = cookieString;
+ }
+ }
+
+ for (const hook of options.hooks.beforeRequest) {
+ // eslint-disable-next-line no-await-in-loop
+ const result = await hook(options);
+
+ if (!is.undefined(result)) {
+ // @ts-expect-error Skip the type mismatch to support abstract responses
+ options.request = () => result;
+ break;
+ }
+ }
+
+ if (options.body && this[kBody] !== options.body) {
+ this[kBody] = options.body;
+ }
+
+ const {agent, request, timeout, url} = options;
+
+ if (options.dnsCache && !('lookup' in options)) {
+ options.lookup = options.dnsCache.lookup;
+ }
+
+ // UNIX sockets
+ if (url.hostname === 'unix') {
+ const matches = /(?.+?):(?.+)/.exec(`${url.pathname}${url.search}`);
+
+ if (matches?.groups) {
+ const {socketPath, path} = matches.groups;
+
+ Object.assign(options, {
+ socketPath,
+ path,
+ host: ''
+ });
+ }
+ }
+
+ const isHttps = url.protocol === 'https:';
+
+ // Fallback function
+ let fallbackFn: HttpRequestFunction;
+ if (options.http2) {
+ fallbackFn = http2wrapper.auto;
+ } else {
+ fallbackFn = isHttps ? https.request : http.request;
+ }
+
+ const realFn = options.request ?? fallbackFn;
+
+ // Cache support
+ const fn = options.cache ? this._createCacheableRequest : realFn;
+
+ // Pass an agent directly when HTTP2 is disabled
+ if (agent && !options.http2) {
+ (options as unknown as RequestOptions).agent = agent[isHttps ? 'https' : 'http'];
+ }
+
+ // Prepare plain HTTP request options
+ options[kRequest] = realFn as HttpRequestFunction;
+ delete options.request;
+ // TODO: Fix this ignore.
+ // @ts-expect-error
+ delete options.timeout;
+
+ const requestOptions = options as unknown as (RealRequestOptions & CacheOptions);
+ requestOptions.shared = options.cacheOptions?.shared;
+ requestOptions.cacheHeuristic = options.cacheOptions?.cacheHeuristic;
+ requestOptions.immutableMinTimeToLive = options.cacheOptions?.immutableMinTimeToLive;
+ requestOptions.ignoreCargoCult = options.cacheOptions?.ignoreCargoCult;
+
+ // If `dnsLookupIpVersion` is not present do not override `family`
+ if (options.dnsLookupIpVersion !== undefined) {
+ try {
+ requestOptions.family = dnsLookupIpVersionToFamily(options.dnsLookupIpVersion);
+ } catch {
+ throw new Error('Invalid `dnsLookupIpVersion` option value');
+ }
+ }
+
+ // HTTPS options remapping
+ if (options.https) {
+ if ('rejectUnauthorized' in options.https) {
+ requestOptions.rejectUnauthorized = options.https.rejectUnauthorized;
+ }
+
+ if (options.https.checkServerIdentity) {
+ requestOptions.checkServerIdentity = options.https.checkServerIdentity;
+ }
+
+ if (options.https.certificateAuthority) {
+ requestOptions.ca = options.https.certificateAuthority;
+ }
+
+ if (options.https.certificate) {
+ requestOptions.cert = options.https.certificate;
+ }
+
+ if (options.https.key) {
+ requestOptions.key = options.https.key;
+ }
+
+ if (options.https.passphrase) {
+ requestOptions.passphrase = options.https.passphrase;
+ }
+
+ if (options.https.pfx) {
+ requestOptions.pfx = options.https.pfx;
+ }
+ }
+
+ try {
+ let requestOrResponse = await fn(url, requestOptions);
+
+ if (is.undefined(requestOrResponse)) {
+ requestOrResponse = fallbackFn(url, requestOptions);
+ }
+
+ // Restore options
+ options.request = request;
+ options.timeout = timeout;
+ options.agent = agent;
+
+ // HTTPS options restore
+ if (options.https) {
+ if ('rejectUnauthorized' in options.https) {
+ delete requestOptions.rejectUnauthorized;
+ }
+
+ if (options.https.checkServerIdentity) {
+ // @ts-expect-error - This one will be removed when we remove the alias.
+ delete requestOptions.checkServerIdentity;
+ }
+
+ if (options.https.certificateAuthority) {
+ delete requestOptions.ca;
+ }
+
+ if (options.https.certificate) {
+ delete requestOptions.cert;
+ }
+
+ if (options.https.key) {
+ delete requestOptions.key;
+ }
+
+ if (options.https.passphrase) {
+ delete requestOptions.passphrase;
+ }
+
+ if (options.https.pfx) {
+ delete requestOptions.pfx;
+ }
+ }
+
+ if (isClientRequest(requestOrResponse)) {
+ this._onRequest(requestOrResponse);
+
+ // Emit the response after the stream has been ended
+ } else if (this.writable) {
+ this.once('finish', () => {
+ void this._onResponse(requestOrResponse as IncomingMessageWithTimings);
+ });
+
+ this._unlockWrite();
+ this.end();
+ this._lockWrite();
+ } else {
+ void this._onResponse(requestOrResponse as IncomingMessageWithTimings);
+ }
+ } catch (error) {
+ if (error instanceof CacheableRequest.CacheError) {
+ throw new CacheError(error, this);
+ }
+
+ throw new RequestError(error.message, error, this);
+ }
+ }
+
+ async _error(error: RequestError): Promise {
+ try {
+ for (const hook of this.options.hooks.beforeError) {
+ // eslint-disable-next-line no-await-in-loop
+ error = await hook(error);
+ }
+ } catch (error_) {
+ error = new RequestError(error_.message, error_, this);
+ }
+
+ this.destroy(error);
+ }
+
+ _beforeError(error: Error): void {
+ if (this[kStopReading]) {
+ return;
+ }
+
+ const {options} = this;
+ const retryCount = this.retryCount + 1;
+
+ this[kStopReading] = true;
+
+ if (!(error instanceof RequestError)) {
+ error = new RequestError(error.message, error, this);
+ }
+
+ const typedError = error as RequestError;
+ const {response} = typedError;
+
+ void (async () => {
+ if (response && !response.body) {
+ response.setEncoding((this as any)._readableState.encoding);
+
+ try {
+ response.rawBody = await getBuffer(response);
+ response.body = response.rawBody.toString();
+ } catch {}
+ }
+
+ if (this.listenerCount('retry') !== 0) {
+ let backoff: number;
+
+ try {
+ let retryAfter;
+ if (response && 'retry-after' in response.headers) {
+ retryAfter = Number(response.headers['retry-after']);
+ if (Number.isNaN(retryAfter)) {
+ retryAfter = Date.parse(response.headers['retry-after']!) - Date.now();
+
+ if (retryAfter <= 0) {
+ retryAfter = 1;
+ }
+ } else {
+ retryAfter *= 1000;
+ }
+ }
+
+ backoff = await options.retry.calculateDelay({
+ attemptCount: retryCount,
+ retryOptions: options.retry,
+ error: typedError,
+ retryAfter,
+ computedValue: calculateRetryDelay({
+ attemptCount: retryCount,
+ retryOptions: options.retry,
+ error: typedError,
+ retryAfter,
+ computedValue: 0
+ })
+ });
+ } catch (error_) {
+ void this._error(new RequestError(error_.message, error_, this));
+ return;
+ }
+
+ if (backoff) {
+ const retry = async (): Promise => {
+ try {
+ for (const hook of this.options.hooks.beforeRetry) {
+ // eslint-disable-next-line no-await-in-loop
+ await hook(this.options, typedError, retryCount);
+ }
+ } catch (error_) {
+ void this._error(new RequestError(error_.message, error, this));
+ return;
+ }
+
+ // Something forced us to abort the retry
+ if (this.destroyed) {
+ return;
+ }
+
+ this.destroy();
+ this.emit('retry', retryCount, error);
+ };
+
+ this[kRetryTimeout] = setTimeout(retry, backoff);
+ return;
+ }
+ }
+
+ void this._error(typedError);
+ })();
+ }
+
+ _read(): void {
+ this[kTriggerRead] = true;
+
+ const response = this[kResponse];
+ if (response && !this[kStopReading]) {
+ // We cannot put this in the `if` above
+ // because `.read()` also triggers the `end` event
+ if (response.readableLength) {
+ this[kTriggerRead] = false;
+ }
+
+ let data;
+ while ((data = response.read()) !== null) {
+ this[kDownloadedSize] += data.length;
+ this[kStartedReading] = true;
+
+ const progress = this.downloadProgress;
+
+ if (progress.percent < 1) {
+ this.emit('downloadProgress', progress);
+ }
+
+ this.push(data);
+ }
+ }
+ }
+
+ // Node.js 12 has incorrect types, so the encoding must be a string
+ _write(chunk: any, encoding: string | undefined, callback: (error?: Error | null) => void): void {
+ const write = (): void => {
+ this._writeRequest(chunk, encoding as BufferEncoding, callback);
+ };
+
+ if (this.requestInitialized) {
+ write();
+ } else {
+ this[kJobs].push(write);
+ }
+ }
+
+ _writeRequest(chunk: any, encoding: BufferEncoding | undefined, callback: (error?: Error | null) => void): void {
+ if (this[kRequest]!.destroyed) {
+ // Probably the `ClientRequest` instance will throw
+ return;
+ }
+
+ this._progressCallbacks.push((): void => {
+ this[kUploadedSize] += Buffer.byteLength(chunk, encoding);
+
+ const progress = this.uploadProgress;
+
+ if (progress.percent < 1) {
+ this.emit('uploadProgress', progress);
+ }
+ });
+
+ // TODO: What happens if it's from cache? Then this[kRequest] won't be defined.
+
+ this[kRequest]!.write(chunk, encoding!, (error?: Error | null) => {
+ if (!error && this._progressCallbacks.length > 0) {
+ this._progressCallbacks.shift()!();
+ }
+
+ callback(error);
+ });
+ }
+
+ _final(callback: (error?: Error | null) => void): void {
+ const endRequest = (): void => {
+ // FIX: Node.js 10 calls the write callback AFTER the end callback!
+ while (this._progressCallbacks.length !== 0) {
+ this._progressCallbacks.shift()!();
+ }
+
+ // We need to check if `this[kRequest]` is present,
+ // because it isn't when we use cache.
+ if (!(kRequest in this)) {
+ callback();
+ return;
+ }
+
+ if (this[kRequest]!.destroyed) {
+ callback();
+ return;
+ }
+
+ this[kRequest]!.end((error?: Error | null) => {
+ if (!error) {
+ this[kBodySize] = this[kUploadedSize];
+
+ this.emit('uploadProgress', this.uploadProgress);
+ this[kRequest]!.emit('upload-complete');
+ }
+
+ callback(error);
+ });
+ };
+
+ if (this.requestInitialized) {
+ endRequest();
+ } else {
+ this[kJobs].push(endRequest);
+ }
+ }
+
+ _destroy(error: Error | null, callback: (error: Error | null) => void): void {
+ this[kStopReading] = true;
+
+ // Prevent further retries
+ clearTimeout(this[kRetryTimeout] as NodeJS.Timeout);
+
+ if (kRequest in this) {
+ this[kCancelTimeouts]!();
+
+ // TODO: Remove the next `if` when these get fixed:
+ // - https://github.com/nodejs/node/issues/32851
+ if (!this[kResponse]?.complete) {
+ this[kRequest]!.destroy();
+ }
+ }
+
+ if (error !== null && !is.undefined(error) && !(error instanceof RequestError)) {
+ error = new RequestError(error.message, error, this);
+ }
+
+ callback(error);
+ }
+
+ get _isAboutToError() {
+ return this[kStopReading];
+ }
+
+ /**
+ The remote IP address.
+ */
+ get ip(): string | undefined {
+ return this.socket?.remoteAddress;
+ }
+
+ /**
+ Indicates whether the request has been aborted or not.
+ */
+ get aborted(): boolean {
+ return (this[kRequest]?.destroyed ?? this.destroyed) && !(this[kOriginalResponse]?.complete);
+ }
+
+ get socket(): Socket | undefined {
+ return this[kRequest]?.socket ?? undefined;
+ }
+
+ /**
+ Progress event for downloading (receiving a response).
+ */
+ get downloadProgress(): Progress {
+ let percent;
+ if (this[kResponseSize]) {
+ percent = this[kDownloadedSize] / this[kResponseSize]!;
+ } else if (this[kResponseSize] === this[kDownloadedSize]) {
+ percent = 1;
+ } else {
+ percent = 0;
+ }
+
+ return {
+ percent,
+ transferred: this[kDownloadedSize],
+ total: this[kResponseSize]
+ };
+ }
+
+ /**
+ Progress event for uploading (sending a request).
+ */
+ get uploadProgress(): Progress {
+ let percent;
+ if (this[kBodySize]) {
+ percent = this[kUploadedSize] / this[kBodySize]!;
+ } else if (this[kBodySize] === this[kUploadedSize]) {
+ percent = 1;
+ } else {
+ percent = 0;
+ }
+
+ return {
+ percent,
+ transferred: this[kUploadedSize],
+ total: this[kBodySize]
+ };
+ }
+
+ /**
+ The object contains the following properties:
+
+ - `start` - Time when the request started.
+ - `socket` - Time when a socket was assigned to the request.
+ - `lookup` - Time when the DNS lookup finished.
+ - `connect` - Time when the socket successfully connected.
+ - `secureConnect` - Time when the socket securely connected.
+ - `upload` - Time when the request finished uploading.
+ - `response` - Time when the request fired `response` event.
+ - `end` - Time when the response fired `end` event.
+ - `error` - Time when the request fired `error` event.
+ - `abort` - Time when the request fired `abort` event.
+ - `phases`
+ - `wait` - `timings.socket - timings.start`
+ - `dns` - `timings.lookup - timings.socket`
+ - `tcp` - `timings.connect - timings.lookup`
+ - `tls` - `timings.secureConnect - timings.connect`
+ - `request` - `timings.upload - (timings.secureConnect || timings.connect)`
+ - `firstByte` - `timings.response - timings.upload`
+ - `download` - `timings.end - timings.response`
+ - `total` - `(timings.end || timings.error || timings.abort) - timings.start`
+
+ If something has not been measured yet, it will be `undefined`.
+
+ __Note__: The time is a `number` representing the milliseconds elapsed since the UNIX epoch.
+ */
+ get timings(): Timings | undefined {
+ return (this[kRequest] as ClientRequestWithTimings)?.timings;
+ }
+
+ /**
+ Whether the response was retrieved from the cache.
+ */
+ get isFromCache(): boolean | undefined {
+ return this[kIsFromCache];
+ }
+
+ pipe(destination: T, options?: {end?: boolean}): T {
+ if (this[kStartedReading]) {
+ throw new Error('Failed to pipe. The response has been emitted already.');
+ }
+
+ if (destination instanceof ServerResponse) {
+ this[kServerResponsesPiped].add(destination);
+ }
+
+ return super.pipe(destination, options);
+ }
+
+ unpipe(destination: T): this {
+ if (destination instanceof ServerResponse) {
+ this[kServerResponsesPiped].delete(destination);
+ }
+
+ super.unpipe(destination);
+
+ return this;
+ }
+}
diff --git a/source/core/utils/dns-ip-version.ts b/source/core/utils/dns-ip-version.ts
new file mode 100644
index 000000000..7ea331aaf
--- /dev/null
+++ b/source/core/utils/dns-ip-version.ts
@@ -0,0 +1,20 @@
+export type DnsLookupIpVersion = 'auto' | 'ipv4' | 'ipv6';
+type DnsIpFamily = 0 | 4 | 6;
+
+const conversionTable = {
+ auto: 0,
+ ipv4: 4,
+ ipv6: 6
+};
+
+export const isDnsLookupIpVersion = (value: any): boolean => {
+ return value in conversionTable;
+};
+
+export const dnsLookupIpVersionToFamily = (dnsLookupIpVersion: DnsLookupIpVersion): DnsIpFamily => {
+ if (isDnsLookupIpVersion(dnsLookupIpVersion)) {
+ return conversionTable[dnsLookupIpVersion] as DnsIpFamily;
+ }
+
+ throw new Error('Invalid DNS lookup IP version');
+};
diff --git a/source/core/utils/get-body-size.ts b/source/core/utils/get-body-size.ts
new file mode 100644
index 000000000..d7cf8f0c6
--- /dev/null
+++ b/source/core/utils/get-body-size.ts
@@ -0,0 +1,41 @@
+import {ReadStream, stat} from 'fs';
+import {promisify} from 'util';
+import {ClientRequestArgs} from 'http';
+import is from '@sindresorhus/is';
+import isFormData from './is-form-data';
+
+const statAsync = promisify(stat);
+
+export default async (body: unknown, headers: ClientRequestArgs['headers']): Promise => {
+ if (headers && 'content-length' in headers) {
+ return Number(headers['content-length']);
+ }
+
+ if (!body) {
+ return 0;
+ }
+
+ if (is.string(body)) {
+ return Buffer.byteLength(body);
+ }
+
+ if (is.buffer(body)) {
+ return body.length;
+ }
+
+ if (isFormData(body)) {
+ return promisify(body.getLength.bind(body))();
+ }
+
+ if (body instanceof ReadStream) {
+ const {size} = await statAsync(body.path);
+
+ if (size === 0) {
+ return undefined;
+ }
+
+ return size;
+ }
+
+ return undefined;
+};
diff --git a/source/core/utils/get-buffer.ts b/source/core/utils/get-buffer.ts
new file mode 100644
index 000000000..d62a2202c
--- /dev/null
+++ b/source/core/utils/get-buffer.ts
@@ -0,0 +1,21 @@
+import {Readable} from 'stream';
+
+// TODO: Update https://github.com/sindresorhus/get-stream
+
+const getBuffer = async (stream: Readable) => {
+ const chunks = [];
+ let length = 0;
+
+ for await (const chunk of stream) {
+ chunks.push(chunk);
+ length += Buffer.byteLength(chunk);
+ }
+
+ if (Buffer.isBuffer(chunks[0])) {
+ return Buffer.concat(chunks, length);
+ }
+
+ return Buffer.from(chunks.join(''));
+};
+
+export default getBuffer;
diff --git a/source/core/utils/is-form-data.ts b/source/core/utils/is-form-data.ts
new file mode 100644
index 000000000..34281a90a
--- /dev/null
+++ b/source/core/utils/is-form-data.ts
@@ -0,0 +1,9 @@
+import is from '@sindresorhus/is';
+import {Readable} from 'stream';
+
+interface FormData extends Readable {
+ getBoundary: () => string;
+ getLength: (callback: (error: Error | null, length: number) => void) => void;
+}
+
+export default (body: unknown): body is FormData => is.nodeStream(body) && is.function_((body as FormData).getBoundary);
diff --git a/source/core/utils/is-response-ok.ts b/source/core/utils/is-response-ok.ts
new file mode 100644
index 000000000..fe4186869
--- /dev/null
+++ b/source/core/utils/is-response-ok.ts
@@ -0,0 +1,8 @@
+import {Response} from '..';
+
+export const isResponseOk = (response: Response): boolean => {
+ const {statusCode} = response;
+ const limitStatusCode = response.request.options.followRedirect ? 299 : 399;
+
+ return (statusCode >= 200 && statusCode <= limitStatusCode) || statusCode === 304;
+};
diff --git a/source/core/utils/options-to-url.ts b/source/core/utils/options-to-url.ts
new file mode 100644
index 000000000..0f00e9805
--- /dev/null
+++ b/source/core/utils/options-to-url.ts
@@ -0,0 +1,73 @@
+/* istanbul ignore file: deprecated */
+import {URL} from 'url';
+
+export interface URLOptions {
+ href?: string;
+ protocol?: string;
+ host?: string;
+ hostname?: string;
+ port?: string | number;
+ pathname?: string;
+ search?: string;
+ searchParams?: unknown;
+ path?: string;
+}
+
+const keys: Array> = [
+ 'protocol',
+ 'host',
+ 'hostname',
+ 'port',
+ 'pathname',
+ 'search'
+];
+
+export default (origin: string, options: URLOptions): URL => {
+ if (options.path) {
+ if (options.pathname) {
+ throw new TypeError('Parameters `path` and `pathname` are mutually exclusive.');
+ }
+
+ if (options.search) {
+ throw new TypeError('Parameters `path` and `search` are mutually exclusive.');
+ }
+
+ if (options.searchParams) {
+ throw new TypeError('Parameters `path` and `searchParams` are mutually exclusive.');
+ }
+ }
+
+ if (options.search && options.searchParams) {
+ throw new TypeError('Parameters `search` and `searchParams` are mutually exclusive.');
+ }
+
+ if (!origin) {
+ if (!options.protocol) {
+ throw new TypeError('No URL protocol specified');
+ }
+
+ origin = `${options.protocol}//${options.hostname ?? options.host ?? ''}`;
+ }
+
+ const url = new URL(origin);
+
+ if (options.path) {
+ const searchIndex = options.path.indexOf('?');
+ if (searchIndex === -1) {
+ options.pathname = options.path;
+ } else {
+ options.pathname = options.path.slice(0, searchIndex);
+ options.search = options.path.slice(searchIndex + 1);
+ }
+
+ delete options.path;
+ }
+
+ for (const key of keys) {
+ if (options[key]) {
+ url[key] = options[key]!.toString();
+ }
+ }
+
+ return url;
+};
diff --git a/source/core/utils/proxy-events.ts b/source/core/utils/proxy-events.ts
new file mode 100644
index 000000000..b231e5360
--- /dev/null
+++ b/source/core/utils/proxy-events.ts
@@ -0,0 +1,22 @@
+import {EventEmitter} from 'events';
+
+type Fn = (...args: unknown[]) => void;
+type Fns = Record;
+
+export default function (from: EventEmitter, to: EventEmitter, events: string[]): () => void {
+ const fns: Fns = {};
+
+ for (const event of events) {
+ fns[event] = (...args: unknown[]) => {
+ to.emit(event, ...args);
+ };
+
+ from.on(event, fns[event]);
+ }
+
+ return () => {
+ for (const event of events) {
+ from.off(event, fns[event]);
+ }
+ };
+}
diff --git a/source/core/utils/timed-out.ts b/source/core/utils/timed-out.ts
new file mode 100644
index 000000000..cea5b8e5b
--- /dev/null
+++ b/source/core/utils/timed-out.ts
@@ -0,0 +1,178 @@
+import net = require('net');
+import {ClientRequest, IncomingMessage} from 'http';
+import unhandler from './unhandle';
+
+const reentry: unique symbol = Symbol('reentry');
+const noop = (): void => {};
+
+interface TimedOutOptions {
+ host?: string;
+ hostname?: string;
+ protocol?: string;
+}
+
+export interface Delays {
+ lookup?: number;
+ connect?: number;
+ secureConnect?: number;
+ socket?: number;
+ response?: number;
+ send?: number;
+ request?: number;
+}
+
+export type ErrorCode =
+ | 'ETIMEDOUT'
+ | 'ECONNRESET'
+ | 'EADDRINUSE'
+ | 'ECONNREFUSED'
+ | 'EPIPE'
+ | 'ENOTFOUND'
+ | 'ENETUNREACH'
+ | 'EAI_AGAIN';
+
+export class TimeoutError extends Error {
+ code: ErrorCode;
+
+ constructor(threshold: number, public event: string) {
+ super(`Timeout awaiting '${event}' for ${threshold}ms`);
+
+ this.name = 'TimeoutError';
+ this.code = 'ETIMEDOUT';
+ }
+}
+
+export default (request: ClientRequest, delays: Delays, options: TimedOutOptions): () => void => {
+ if (reentry in request) {
+ return noop;
+ }
+
+ request[reentry] = true;
+ const cancelers: Array = [];
+ const {once, unhandleAll} = unhandler();
+
+ const addTimeout = (delay: number, callback: (delay: number, event: string) => void, event: string): (typeof noop) => {
+ const timeout = setTimeout(callback, delay, delay, event) as unknown as NodeJS.Timeout;
+
+ timeout.unref?.();
+
+ const cancel = (): void => {
+ clearTimeout(timeout);
+ };
+
+ cancelers.push(cancel);
+
+ return cancel;
+ };
+
+ const {host, hostname} = options;
+
+ const timeoutHandler = (delay: number, event: string): void => {
+ request.destroy(new TimeoutError(delay, event));
+ };
+
+ const cancelTimeouts = (): void => {
+ for (const cancel of cancelers) {
+ cancel();
+ }
+
+ unhandleAll();
+ };
+
+ request.once('error', error => {
+ cancelTimeouts();
+
+ // Save original behavior
+ /* istanbul ignore next */
+ if (request.listenerCount('error') === 0) {
+ throw error;
+ }
+ });
+
+ request.once('close', cancelTimeouts);
+
+ once(request, 'response', (response: IncomingMessage): void => {
+ once(response, 'end', cancelTimeouts);
+ });
+
+ if (typeof delays.request !== 'undefined') {
+ addTimeout(delays.request, timeoutHandler, 'request');
+ }
+
+ if (typeof delays.socket !== 'undefined') {
+ const socketTimeoutHandler = (): void => {
+ timeoutHandler(delays.socket!, 'socket');
+ };
+
+ request.setTimeout(delays.socket, socketTimeoutHandler);
+
+ // `request.setTimeout(0)` causes a memory leak.
+ // We can just remove the listener and forget about the timer - it's unreffed.
+ // See https://github.com/sindresorhus/got/issues/690
+ cancelers.push(() => {
+ request.removeListener('timeout', socketTimeoutHandler);
+ });
+ }
+
+ once(request, 'socket', (socket: net.Socket): void => {
+ const {socketPath} = request as ClientRequest & {socketPath?: string};
+
+ /* istanbul ignore next: hard to test */
+ if (socket.connecting) {
+ const hasPath = Boolean(socketPath ?? net.isIP(hostname ?? host ?? '') !== 0);
+
+ if (typeof delays.lookup !== 'undefined' && !hasPath && typeof (socket.address() as net.AddressInfo).address === 'undefined') {
+ const cancelTimeout = addTimeout(delays.lookup, timeoutHandler, 'lookup');
+ once(socket, 'lookup', cancelTimeout);
+ }
+
+ if (typeof delays.connect !== 'undefined') {
+ const timeConnect = (): (() => void) => addTimeout(delays.connect!, timeoutHandler, 'connect');
+
+ if (hasPath) {
+ once(socket, 'connect', timeConnect());
+ } else {
+ once(socket, 'lookup', (error: Error): void => {
+ if (error === null) {
+ once(socket, 'connect', timeConnect());
+ }
+ });
+ }
+ }
+
+ if (typeof delays.secureConnect !== 'undefined' && options.protocol === 'https:') {
+ once(socket, 'connect', (): void => {
+ const cancelTimeout = addTimeout(delays.secureConnect!, timeoutHandler, 'secureConnect');
+ once(socket, 'secureConnect', cancelTimeout);
+ });
+ }
+ }
+
+ if (typeof delays.send !== 'undefined') {
+ const timeRequest = (): (() => void) => addTimeout(delays.send!, timeoutHandler, 'send');
+ /* istanbul ignore next: hard to test */
+ if (socket.connecting) {
+ once(socket, 'connect', (): void => {
+ once(request, 'upload-complete', timeRequest());
+ });
+ } else {
+ once(request, 'upload-complete', timeRequest());
+ }
+ }
+ });
+
+ if (typeof delays.response !== 'undefined') {
+ once(request, 'upload-complete', (): void => {
+ const cancelTimeout = addTimeout(delays.response!, timeoutHandler, 'response');
+ once(request, 'response', cancelTimeout);
+ });
+ }
+
+ return cancelTimeouts;
+};
+
+declare module 'http' {
+ interface ClientRequest {
+ [reentry]: boolean;
+ }
+}
diff --git a/source/core/utils/unhandle.ts b/source/core/utils/unhandle.ts
new file mode 100644
index 000000000..7f0a838ba
--- /dev/null
+++ b/source/core/utils/unhandle.ts
@@ -0,0 +1,40 @@
+import {EventEmitter} from 'events';
+
+type Origin = EventEmitter;
+type Event = string | symbol;
+type Fn = (...args: any[]) => void;
+
+interface Handler {
+ origin: Origin;
+ event: Event;
+ fn: Fn;
+}
+
+interface Unhandler {
+ once: (origin: Origin, event: Event, fn: Fn) => void;
+ unhandleAll: () => void;
+}
+
+// When attaching listeners, it's very easy to forget about them.
+// Especially if you do error handling and set timeouts.
+// So instead of checking if it's proper to throw an error on every timeout ever,
+// use this simple tool which will remove all listeners you have attached.
+export default (): Unhandler => {
+ const handlers: Handler[] = [];
+
+ return {
+ once(origin: Origin, event: Event, fn: Fn) {
+ origin.once(event, fn);
+ handlers.push({origin, event, fn});
+ },
+
+ unhandleAll() {
+ for (const handler of handlers) {
+ const {origin, event, fn} = handler;
+ origin.removeListener(event, fn);
+ }
+
+ handlers.length = 0;
+ }
+ };
+};
diff --git a/source/core/utils/url-to-options.ts b/source/core/utils/url-to-options.ts
new file mode 100644
index 000000000..06342f58b
--- /dev/null
+++ b/source/core/utils/url-to-options.ts
@@ -0,0 +1,43 @@
+import {URL, UrlWithStringQuery} from 'url';
+import is from '@sindresorhus/is';
+
+// TODO: Deprecate legacy URL at some point
+
+export interface LegacyUrlOptions {
+ protocol: string;
+ hostname: string;
+ host: string;
+ hash: string | null;
+ search: string | null;
+ pathname: string;
+ href: string;
+ path: string;
+ port?: number;
+ auth?: string;
+}
+
+export default (url: URL | UrlWithStringQuery): LegacyUrlOptions => {
+ // Cast to URL
+ url = url as URL;
+
+ const options: LegacyUrlOptions = {
+ protocol: url.protocol,
+ hostname: is.string(url.hostname) && url.hostname.startsWith('[') ? url.hostname.slice(1, -1) : url.hostname,
+ host: url.host,
+ hash: url.hash,
+ search: url.search,
+ pathname: url.pathname,
+ href: url.href,
+ path: `${url.pathname || ''}${url.search || ''}`
+ };
+
+ if (is.string(url.port) && url.port.length > 0) {
+ options.port = Number(url.port);
+ }
+
+ if (url.username || url.password) {
+ options.auth = `${url.username || ''}:${url.password || ''}`;
+ }
+
+ return options;
+};
diff --git a/source/core/utils/weakable-map.ts b/source/core/utils/weakable-map.ts
new file mode 100644
index 000000000..60861d487
--- /dev/null
+++ b/source/core/utils/weakable-map.ts
@@ -0,0 +1,33 @@
+export default class WeakableMap {
+ weakMap: WeakMap, V>;
+ map: Map;
+
+ constructor() {
+ this.weakMap = new WeakMap();
+ this.map = new Map();
+ }
+
+ set(key: K, value: V): void {
+ if (typeof key === 'object') {
+ this.weakMap.set(key as unknown as Record, value);
+ } else {
+ this.map.set(key, value);
+ }
+ }
+
+ get(key: K): V | undefined {
+ if (typeof key === 'object') {
+ return this.weakMap.get(key as unknown as Record);
+ }
+
+ return this.map.get(key);
+ }
+
+ has(key: K): boolean {
+ if (typeof key === 'object') {
+ return this.weakMap.has(key as unknown as Record);
+ }
+
+ return this.map.has(key);
+ }
+}
diff --git a/source/create.ts b/source/create.ts
new file mode 100644
index 000000000..92e21cf95
--- /dev/null
+++ b/source/create.ts
@@ -0,0 +1,322 @@
+import {URL} from 'url';
+import is from '@sindresorhus/is';
+import asPromise, {
+ // Response
+ Response,
+
+ // Options
+ Options,
+ NormalizedOptions,
+
+ // Hooks
+ InitHook,
+
+ // Errors
+ ParseError,
+ RequestError,
+ CacheError,
+ ReadError,
+ HTTPError,
+ MaxRedirectsError,
+ TimeoutError,
+ UnsupportedProtocolError,
+ UploadError,
+ CancelError
+} from './as-promise';
+import {
+ GotReturn,
+ ExtendOptions,
+ Got,
+ HTTPAlias,
+ HandlerFunction,
+ InstanceDefaults,
+ GotPaginate,
+ GotStream,
+ GotRequestFunction,
+ OptionsWithPagination,
+ StreamOptions
+} from './types';
+import createRejection from './as-promise/create-rejection';
+import Request, {kIsNormalizedAlready, setNonEnumerableProperties, Defaults} from './core';
+import deepFreeze from './utils/deep-freeze';
+
+const errors = {
+ RequestError,
+ CacheError,
+ ReadError,
+ HTTPError,
+ MaxRedirectsError,
+ TimeoutError,
+ ParseError,
+ CancelError,
+ UnsupportedProtocolError,
+ UploadError
+};
+
+// The `delay` package weighs 10KB (!)
+const delay = async (ms: number) => new Promise(resolve => {
+ setTimeout(resolve, ms);
+});
+
+const {normalizeArguments} = Request;
+
+const mergeOptions = (...sources: Options[]): NormalizedOptions => {
+ let mergedOptions: NormalizedOptions | undefined;
+
+ for (const source of sources) {
+ mergedOptions = normalizeArguments(undefined, source, mergedOptions);
+ }
+
+ return mergedOptions!;
+};
+
+const getPromiseOrStream = (options: NormalizedOptions): GotReturn => options.isStream ? new Request(undefined, options) : asPromise(options);
+
+const isGotInstance = (value: Got | ExtendOptions): value is Got => (
+ 'defaults' in value && 'options' in value.defaults
+);
+
+const aliases: readonly HTTPAlias[] = [
+ 'get',
+ 'post',
+ 'put',
+ 'patch',
+ 'head',
+ 'delete'
+];
+
+export const defaultHandler: HandlerFunction = (options, next) => next(options);
+
+const callInitHooks = (hooks: InitHook[] | undefined, options?: Options): void => {
+ if (hooks) {
+ for (const hook of hooks) {
+ hook(options!);
+ }
+ }
+};
+
+const create = (defaults: InstanceDefaults): Got => {
+ // Proxy properties from next handlers
+ defaults._rawHandlers = defaults.handlers;
+ defaults.handlers = defaults.handlers.map(fn => ((options, next) => {
+ // This will be assigned by assigning result
+ let root!: ReturnType;
+
+ const result = fn(options, newOptions => {
+ root = next(newOptions);
+ return root;
+ });
+
+ if (result !== root && !options.isStream && root) {
+ const typedResult = result as Promise;
+
+ const {then: promiseThen, catch: promiseCatch, finally: promiseFianlly} = typedResult;
+ Object.setPrototypeOf(typedResult, Object.getPrototypeOf(root));
+ Object.defineProperties(typedResult, Object.getOwnPropertyDescriptors(root));
+
+ // These should point to the new promise
+ // eslint-disable-next-line promise/prefer-await-to-then
+ typedResult.then = promiseThen;
+ typedResult.catch = promiseCatch;
+ typedResult.finally = promiseFianlly;
+ }
+
+ return result;
+ }));
+
+ // Got interface
+ const got: Got = ((url: string | URL, options: Options = {}, _defaults?: Defaults): GotReturn => {
+ let iteration = 0;
+ const iterateHandlers = (newOptions: NormalizedOptions): GotReturn => {
+ return defaults.handlers[iteration++](
+ newOptions,
+ iteration === defaults.handlers.length ? getPromiseOrStream : iterateHandlers
+ ) as GotReturn;
+ };
+
+ // TODO: Remove this in Got 12.
+ if (is.plainObject(url)) {
+ const mergedOptions = {
+ ...url as Options,
+ ...options
+ };
+
+ setNonEnumerableProperties([url as Options, options], mergedOptions);
+
+ options = mergedOptions;
+ url = undefined as any;
+ }
+
+ try {
+ // Call `init` hooks
+ let initHookError: Error | undefined;
+ try {
+ callInitHooks(defaults.options.hooks.init, options);
+ callInitHooks(options.hooks?.init, options);
+ } catch (error) {
+ initHookError = error;
+ }
+
+ // Normalize options & call handlers
+ const normalizedOptions = normalizeArguments(url, options, _defaults ?? defaults.options);
+ normalizedOptions[kIsNormalizedAlready] = true;
+
+ if (initHookError) {
+ throw new RequestError(initHookError.message, initHookError, normalizedOptions);
+ }
+
+ return iterateHandlers(normalizedOptions);
+ } catch (error) {
+ if (options.isStream) {
+ throw error;
+ } else {
+ return createRejection(error, defaults.options.hooks.beforeError, options.hooks?.beforeError);
+ }
+ }
+ }) as Got;
+
+ got.extend = (...instancesOrOptions) => {
+ const optionsArray: Options[] = [defaults.options];
+ let handlers: HandlerFunction[] = [...defaults._rawHandlers!];
+ let isMutableDefaults: boolean | undefined;
+
+ for (const value of instancesOrOptions) {
+ if (isGotInstance(value)) {
+ optionsArray.push(value.defaults.options);
+ handlers.push(...value.defaults._rawHandlers!);
+ isMutableDefaults = value.defaults.mutableDefaults;
+ } else {
+ optionsArray.push(value);
+
+ if ('handlers' in value) {
+ handlers.push(...value.handlers!);
+ }
+
+ isMutableDefaults = value.mutableDefaults;
+ }
+ }
+
+ handlers = handlers.filter(handler => handler !== defaultHandler);
+
+ if (handlers.length === 0) {
+ handlers.push(defaultHandler);
+ }
+
+ return create({
+ options: mergeOptions(...optionsArray),
+ handlers,
+ mutableDefaults: Boolean(isMutableDefaults)
+ });
+ };
+
+ // Pagination
+ const paginateEach = (async function * (url: string | URL, options?: OptionsWithPagination): AsyncIterableIterator {
+ // TODO: Remove this `@ts-expect-error` when upgrading to TypeScript 4.
+ // Error: Argument of type 'Merge> | undefined' is not assignable to parameter of type 'Options | undefined'.
+ // @ts-expect-error
+ let normalizedOptions = normalizeArguments(url, options, defaults.options);
+ normalizedOptions.resolveBodyOnly = false;
+
+ const pagination = normalizedOptions.pagination!;
+
+ if (!is.object(pagination)) {
+ throw new TypeError('`options.pagination` must be implemented');
+ }
+
+ const all: T[] = [];
+ let {countLimit} = pagination;
+
+ let numberOfRequests = 0;
+ while (numberOfRequests < pagination.requestLimit) {
+ if (numberOfRequests !== 0) {
+ // eslint-disable-next-line no-await-in-loop
+ await delay(pagination.backoff);
+ }
+
+ // @ts-expect-error FIXME!
+ // TODO: Throw when result is not an instance of Response
+ // eslint-disable-next-line no-await-in-loop
+ const result = (await got(undefined, undefined, normalizedOptions)) as Response;
+
+ // eslint-disable-next-line no-await-in-loop
+ const parsed = await pagination.transform(result);
+ const current: T[] = [];
+
+ for (const item of parsed) {
+ if (pagination.filter(item, all, current)) {
+ if (!pagination.shouldContinue(item, all, current)) {
+ return;
+ }
+
+ yield item as T;
+
+ if (pagination.stackAllItems) {
+ all.push(item as T);
+ }
+
+ current.push(item as T);
+
+ if (--countLimit <= 0) {
+ return;
+ }
+ }
+ }
+
+ const optionsToMerge = pagination.paginate(result, all, current);
+
+ if (optionsToMerge === false) {
+ return;
+ }
+
+ if (optionsToMerge === result.request.options) {
+ normalizedOptions = result.request.options;
+ } else if (optionsToMerge !== undefined) {
+ normalizedOptions = normalizeArguments(undefined, optionsToMerge, normalizedOptions);
+ }
+
+ numberOfRequests++;
+ }
+ });
+
+ got.paginate = paginateEach as GotPaginate;
+
+ got.paginate.all = (async (url: string | URL, options?: OptionsWithPagination) => {
+ const results: T[] = [];
+
+ for await (const item of paginateEach(url, options)) {
+ results.push(item);
+ }
+
+ return results;
+ }) as GotPaginate['all'];
+
+ // For those who like very descriptive names
+ got.paginate.each = paginateEach as GotPaginate['each'];
+
+ // Stream API
+ got.stream = ((url: string | URL, options?: StreamOptions) => got(url, {...options, isStream: true})) as GotStream;
+
+ // Shortcuts
+ for (const method of aliases) {
+ got[method] = ((url: string | URL, options?: Options): GotReturn => got(url, {...options, method})) as GotRequestFunction;
+
+ got.stream[method] = ((url: string | URL, options?: StreamOptions) => {
+ return got(url, {...options, method, isStream: true});
+ }) as GotStream;
+ }
+
+ Object.assign(got, errors);
+ Object.defineProperty(got, 'defaults', {
+ value: defaults.mutableDefaults ? defaults : deepFreeze(defaults),
+ writable: defaults.mutableDefaults,
+ configurable: defaults.mutableDefaults,
+ enumerable: true
+ });
+
+ got.mergeOptions = mergeOptions;
+
+ return got;
+};
+
+export default create;
+export * from './types';
diff --git a/source/index.ts b/source/index.ts
new file mode 100644
index 000000000..712c803cf
--- /dev/null
+++ b/source/index.ts
@@ -0,0 +1,133 @@
+import {URL} from 'url';
+import {Response, Options} from './as-promise';
+import create, {defaultHandler, InstanceDefaults} from './create';
+
+const defaults: InstanceDefaults = {
+ options: {
+ method: 'GET',
+ retry: {
+ limit: 2,
+ methods: [
+ 'GET',
+ 'PUT',
+ 'HEAD',
+ 'DELETE',
+ 'OPTIONS',
+ 'TRACE'
+ ],
+ statusCodes: [
+ 408,
+ 413,
+ 429,
+ 500,
+ 502,
+ 503,
+ 504,
+ 521,
+ 522,
+ 524
+ ],
+ errorCodes: [
+ 'ETIMEDOUT',
+ 'ECONNRESET',
+ 'EADDRINUSE',
+ 'ECONNREFUSED',
+ 'EPIPE',
+ 'ENOTFOUND',
+ 'ENETUNREACH',
+ 'EAI_AGAIN'
+ ],
+ maxRetryAfter: undefined,
+ calculateDelay: ({computedValue}) => computedValue
+ },
+ timeout: {},
+ headers: {
+ 'user-agent': 'got (https://github.com/sindresorhus/got)'
+ },
+ hooks: {
+ init: [],
+ beforeRequest: [],
+ beforeRedirect: [],
+ beforeRetry: [],
+ beforeError: [],
+ afterResponse: []
+ },
+ cache: undefined,
+ dnsCache: undefined,
+ decompress: true,
+ throwHttpErrors: true,
+ followRedirect: true,
+ isStream: false,
+ responseType: 'text',
+ resolveBodyOnly: false,
+ maxRedirects: 10,
+ prefixUrl: '',
+ methodRewriting: true,
+ ignoreInvalidCookies: false,
+ context: {},
+ // TODO: Set this to `true` when Got 12 gets released
+ http2: false,
+ allowGetBody: false,
+ https: undefined,
+ pagination: {
+ transform: (response: Response) => {
+ if (response.request.options.responseType === 'json') {
+ return response.body;
+ }
+
+ return JSON.parse(response.body as string);
+ },
+ paginate: response => {
+ if (!Reflect.has(response.headers, 'link')) {
+ return false;
+ }
+
+ const items = (response.headers.link as string).split(',');
+
+ let next: string | undefined;
+ for (const item of items) {
+ const parsed = item.split(';');
+
+ if (parsed[1].includes('next')) {
+ next = parsed[0].trimStart().trim();
+ next = next.slice(1, -1);
+ break;
+ }
+ }
+
+ if (next) {
+ const options: Options = {
+ url: new URL(next)
+ };
+
+ return options;
+ }
+
+ return false;
+ },
+ filter: () => true,
+ shouldContinue: () => true,
+ countLimit: Infinity,
+ backoff: 0,
+ requestLimit: 10000,
+ stackAllItems: true
+ },
+ parseJson: (text: string) => JSON.parse(text),
+ stringifyJson: (object: unknown) => JSON.stringify(object),
+ cacheOptions: {}
+ },
+ handlers: [defaultHandler],
+ mutableDefaults: false
+};
+
+const got = create(defaults);
+
+export default got;
+
+// For CommonJS default export support
+module.exports = got;
+module.exports.default = got;
+module.exports.__esModule = true; // Workaround for TS issue: https://github.com/sindresorhus/got/pull/1267
+
+export * from './create';
+export * from './as-promise';
diff --git a/source/types.ts b/source/types.ts
new file mode 100644
index 000000000..f2ea8516b
--- /dev/null
+++ b/source/types.ts
@@ -0,0 +1,391 @@
+import {URL} from 'url';
+import {CancelError} from 'p-cancelable';
+import {
+ // Request & Response
+ CancelableRequest,
+ Response,
+
+ // Options
+ Options,
+ NormalizedOptions,
+ Defaults as DefaultOptions,
+ PaginationOptions,
+
+ // Errors
+ ParseError,
+ RequestError,
+ CacheError,
+ ReadError,
+ HTTPError,
+ MaxRedirectsError,
+ TimeoutError,
+ UnsupportedProtocolError,
+ UploadError
+} from './as-promise';
+import Request from './core';
+
+// `type-fest` utilities
+type Except = Pick>;
+type Merge = Except> & SecondType;
+
+/**
+Defaults for each Got instance.
+*/
+export interface InstanceDefaults {
+ /**
+ An object containing the default options of Got.
+ */
+ options: DefaultOptions;
+
+ /**
+ An array of functions. You execute them directly by calling `got()`.
+ They are some sort of "global hooks" - these functions are called first.
+ The last handler (*it's hidden*) is either `asPromise` or `asStream`, depending on the `options.isStream` property.
+
+ @default []
+ */
+ handlers: HandlerFunction[];
+
+ /**
+ A read-only boolean describing whether the defaults are mutable or not.
+ If set to `true`, you can update headers over time, for example, update an access token when it expires.
+
+ @default false
+ */
+ mutableDefaults: boolean;
+
+ _rawHandlers?: HandlerFunction[];
+}
+
+/**
+A Request object returned by calling Got, or any of the Got HTTP alias request functions.
+*/
+export type GotReturn = Request | CancelableRequest;
+
+/**
+A function to handle options and returns a Request object.
+It acts sort of like a "global hook", and will be called before any actual request is made.
+*/
+export type HandlerFunction = (options: NormalizedOptions, next: (options: NormalizedOptions) => T) => T | Promise;
+
+/**
+The options available for `got.extend()`.
+*/
+export interface ExtendOptions extends Options {
+ /**
+ An array of functions. You execute them directly by calling `got()`.
+ They are some sort of "global hooks" - these functions are called first.
+ The last handler (*it's hidden*) is either `asPromise` or `asStream`, depending on the `options.isStream` property.
+
+ @default []
+ */
+ handlers?: HandlerFunction[];
+
+ /**
+ A read-only boolean describing whether the defaults are mutable or not.
+ If set to `true`, you can update headers over time, for example, update an access token when it expires.
+
+ @default false
+ */
+ mutableDefaults?: boolean;
+}
+
+export type OptionsOfTextResponseBody = Merge;
+export type OptionsOfJSONResponseBody = Merge;
+export type OptionsOfBufferResponseBody = Merge;
+export type OptionsOfUnknownResponseBody = Merge;
+export type StrictOptions = Except;
+export type StreamOptions = Merge;
+type ResponseBodyOnly = {resolveBodyOnly: true};
+
+export type OptionsWithPagination = Merge>;
+
+/**
+An instance of `got.paginate`.
+*/
+export interface GotPaginate {
+ /**
+ Same as `GotPaginate.each`.
+ */
+ (url: string | URL, options?: OptionsWithPagination): AsyncIterableIterator;
+
+ /**
+ Same as `GotPaginate.each`.
+ */
+ (options?: OptionsWithPagination): AsyncIterableIterator;
+
+ /**
+ Returns an async iterator.
+
+ See pagination.options for more pagination options.
+
+ @example
+ ```
+ (async () => {
+ const countLimit = 10;
+
+ const pagination = got.paginate('https://api.github.com/repos/sindresorhus/got/commits', {
+ pagination: {countLimit}
+ });
+
+ console.log(`Printing latest ${countLimit} Got commits (newest to oldest):`);
+
+ for await (const commitData of pagination) {
+ console.log(commitData.commit.message);
+ }
+ })();
+ ```
+ */
+ each: ((url: string | URL, options?: OptionsWithPagination) => AsyncIterableIterator)
+ & ((options?: OptionsWithPagination) => AsyncIterableIterator);
+
+ /**
+ Returns a Promise for an array of all results.
+
+ See pagination.options for more pagination options.
+
+ @example
+ ```
+ (async () => {
+ const countLimit = 10;
+
+ const results = await got.paginate.all('https://api.github.com/repos/sindresorhus/got/commits', {
+ pagination: {countLimit}
+ });
+
+ console.log(`Printing latest ${countLimit} Got commits (newest to oldest):`);
+ console.log(results);
+ })();
+ ```
+ */
+ all: ((url: string | URL, options?: OptionsWithPagination) => Promise)
+ & ((options?: OptionsWithPagination) => Promise);
+}
+
+export interface GotRequestFunction {
+ // `asPromise` usage
+ (url: string | URL, options?: OptionsOfTextResponseBody): CancelableRequest>;
+ (url: string | URL, options?: OptionsOfJSONResponseBody): CancelableRequest>;
+ (url: string | URL, options?: OptionsOfBufferResponseBody): CancelableRequest>;
+ (url: string | URL, options?: OptionsOfUnknownResponseBody): CancelableRequest;
+
+ (options: OptionsOfTextResponseBody): CancelableRequest>;
+ (options: OptionsOfJSONResponseBody): CancelableRequest>;
+ (options: OptionsOfBufferResponseBody): CancelableRequest>;
+ (options: OptionsOfUnknownResponseBody): CancelableRequest;
+
+ // `resolveBodyOnly` usage
+ (url: string | URL, options?: (Merge)): CancelableRequest;
+ (url: string | URL, options?: (Merge)): CancelableRequest;
+ (url: string | URL, options?: (Merge)): CancelableRequest;
+
+ (options: (Merge)): CancelableRequest;
+ (options: (Merge)): CancelableRequest;
+ (options: (Merge)): CancelableRequest;
+
+ // `asStream` usage
+ (url: string | URL, options?: Merge): Request;
+
+ (options: Merge): Request;
+
+ // Fallback
+ (url: string | URL, options?: Options): CancelableRequest | Request;
+
+ (options: Options): CancelableRequest | Request;
+}
+
+/**
+All available HTTP request methods provided by Got.
+*/
+export type HTTPAlias =
+ | 'get'
+ | 'post'
+ | 'put'
+ | 'patch'
+ | 'head'
+ | 'delete';
+
+interface GotStreamFunction {
+ (url: string | URL, options?: Merge): Request;
+ (options?: Merge): Request;
+}
+
+/**
+An instance of `got.stream()`.
+*/
+export type GotStream = GotStreamFunction & Record;
+
+/**
+An instance of `got`.
+*/
+export interface Got extends Record, GotRequestFunction {
+ /**
+ Sets `options.isStream` to `true`.
+
+ Returns a [duplex stream](https://nodejs.org/api/stream.html#stream_class_stream_duplex) with additional events:
+ - request
+ - response
+ - redirect
+ - uploadProgress
+ - downloadProgress
+ - error
+ */
+ stream: GotStream;
+
+ /**
+ Returns an async iterator.
+
+ See pagination.options for more pagination options.
+
+ @example
+ ```
+ (async () => {
+ const countLimit = 10;
+
+ const pagination = got.paginate('https://api.github.com/repos/sindresorhus/got/commits', {
+ pagination: {countLimit}
+ });
+
+ console.log(`Printing latest ${countLimit} Got commits (newest to oldest):`);
+
+ for await (const commitData of pagination) {
+ console.log(commitData.commit.message);
+ }
+ })();
+ ```
+ */
+ paginate: GotPaginate;
+
+ /**
+ The Got defaults used in that instance.
+ */
+ defaults: InstanceDefaults;
+
+ /**
+ An error to be thrown when a cache method fails. For example, if the database goes down or there's a filesystem error.
+ Contains a `code` property with `ERR_CACHE_ACCESS` or a more specific failure code.
+ */
+ CacheError: typeof CacheError;
+
+ /**
+ An error to be thrown when a request fails. Contains a `code` property with error class code, like `ECONNREFUSED`.
+ If there is no specific code supplied, `code` defaults to `ERR_GOT_REQUEST_ERROR`.
+ */
+ RequestError: typeof RequestError;
+
+ /**
+ An error to be thrown when reading from response stream fails. Contains a `code` property with
+ `ERR_READING_RESPONSE_STREAM` or a more specific failure code.
+ */
+ ReadError: typeof ReadError;
+
+ /**
+ An error to be thrown when server response code is 2xx, and parsing body fails. Includes a
+ `response` property. Contains a `code` property with `ERR_BODY_PARSE_FAILURE` or a more specific failure code.
+ */
+ ParseError: typeof ParseError;
+
+ /**
+ An error to be thrown when the server response code is not 2xx nor 3xx if `options.followRedirect` is `true`, but always except for 304.
+ Includes a `response` property. Contains a `code` property with `ERR_NON_2XX_3XX_RESPONSE` or a more specific failure code.
+ */
+ HTTPError: typeof HTTPError;
+
+ /**
+ An error to be thrown when the server redirects you more than ten times.
+ Includes a `response` property. Contains a `code` property with `ERR_TOO_MANY_REDIRECTS`.
+ */
+ MaxRedirectsError: typeof MaxRedirectsError;
+
+ /**
+ An error to be thrown when given an unsupported protocol. Contains a `code` property with `ERR_UNSUPPORTED_PROTOCOL`.
+ */
+ UnsupportedProtocolError: typeof UnsupportedProtocolError;
+
+ /**
+ An error to be thrown when the request is aborted due to a timeout.
+ Includes an `event` and `timings` property. Contains a `code` property with `ETIMEDOUT`.
+ */
+ TimeoutError: typeof TimeoutError;
+
+ /**
+ An error to be thrown when the request body is a stream and an error occurs while reading from that stream.
+ Contains a `code` property with `ERR_UPLOAD` or a more specific failure code.
+ */
+ UploadError: typeof UploadError;
+
+ /**
+ An error to be thrown when the request is aborted with `.cancel()`. Contains a `code` property with `ERR_CANCELED`.
+ */
+ CancelError: typeof CancelError;
+
+ /**
+ Configure a new `got` instance with default `options`.
+ The `options` are merged with the parent instance's `defaults.options` using `got.mergeOptions`.
+ You can access the resolved options with the `.defaults` property on the instance.
+
+ Additionally, `got.extend()` accepts two properties from the `defaults` object: `mutableDefaults` and `handlers`.
+
+ It is also possible to merges many instances into a single one:
+ - options are merged using `got.mergeOptions()` (including hooks),
+ - handlers are stored in an array (you can access them through `instance.defaults.handlers`).
+
+ @example
+ ```js
+ const client = got.extend({
+ prefixUrl: 'https://example.com',
+ headers: {
+ 'x-unicorn': 'rainbow'
+ }
+ });
+
+ client.get('demo');
+
+ // HTTP Request =>
+ // GET /demo HTTP/1.1
+ // Host: example.com
+ // x-unicorn: rainbow
+ ```
+ */
+ extend: (...instancesOrOptions: Array) => Got;
+
+ /**
+ Merges multiple `got` instances into the parent.
+ */
+ mergeInstances: (parent: Got, ...instances: Got[]) => Got;
+
+ /**
+ Extends parent options.
+ Avoid using [object spread](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Spread_syntax#Spread_in_object_literals) as it doesn't work recursively.
+
+ Options are deeply merged to a new object. The value of each key is determined as follows:
+
+ - If the new property is not defined, the old value is used.
+ - If the new property is explicitly set to `undefined`:
+ - If the parent property is a plain `object`, the parent value is deeply cloned.
+ - Otherwise, `undefined` is used.
+ - If the parent value is an instance of `URLSearchParams`:
+ - If the new value is a `string`, an `object` or an instance of `URLSearchParams`, a new `URLSearchParams` instance is created.
+ The values are merged using [`urlSearchParams.append(key, value)`](https://developer.mozilla.org/en-US/docs/Web/API/URLSearchParams/append).
+ The keys defined in the new value override the keys defined in the parent value.
+ - Otherwise, the only available value is `undefined`.
+ - If the new property is a plain `object`:
+ - If the parent property is a plain `object` too, both values are merged recursively into a new `object`.
+ - Otherwise, only the new value is deeply cloned.
+ - If the new property is an `Array`, it overwrites the old one with a deep clone of the new property.
+ - Properties that are not enumerable, such as `context`, `body`, `json`, and `form`, will not be merged.
+ - Otherwise, the new value is assigned to the key.
+
+ **Note:** Only Got options are merged! Custom user options should be defined via [`options.context`](#context).
+
+ @example
+ ```
+ const a = {headers: {cat: 'meow', wolf: ['bark', 'wrrr']}};
+ const b = {headers: {cow: 'moo', wolf: ['auuu']}};
+
+ {...a, ...b} // => {headers: {cow: 'moo', wolf: ['auuu']}}
+ got.mergeOptions(a, b) // => {headers: {cat: 'meow', cow: 'moo', wolf: ['auuu']}}
+ ```
+ */
+ mergeOptions: (...sources: Options[]) => NormalizedOptions;
+}
diff --git a/source/utils/deep-freeze.ts b/source/utils/deep-freeze.ts
new file mode 100644
index 000000000..2b1f45b0d
--- /dev/null
+++ b/source/utils/deep-freeze.ts
@@ -0,0 +1,11 @@
+import is from '@sindresorhus/is';
+
+export default function deepFreeze>(object: T): Readonly {
+ for (const value of Object.values(object)) {
+ if (is.plainObject(value) || is.array(value)) {
+ deepFreeze(value);
+ }
+ }
+
+ return Object.freeze(object);
+}
diff --git a/source/utils/deprecation-warning.ts b/source/utils/deprecation-warning.ts
new file mode 100644
index 000000000..928f0a2ff
--- /dev/null
+++ b/source/utils/deprecation-warning.ts
@@ -0,0 +1,14 @@
+const alreadyWarned: Set = new Set();
+
+export default (message: string) => {
+ if (alreadyWarned.has(message)) {
+ return;
+ }
+
+ alreadyWarned.add(message);
+
+ // @ts-expect-error Missing types.
+ process.emitWarning(`Got: ${message}`, {
+ type: 'DeprecationWarning'
+ });
+};
diff --git a/test/agent.ts b/test/agent.ts
new file mode 100644
index 000000000..0d2c24edf
--- /dev/null
+++ b/test/agent.ts
@@ -0,0 +1,207 @@
+import {Agent as HttpAgent} from 'http';
+import {Agent as HttpsAgent} from 'https';
+import {Socket} from 'net';
+import test, {Constructor} from 'ava';
+import sinon = require('sinon');
+import withServer, {withHttpsServer} from './helpers/with-server';
+
+const createAgentSpy = (AgentClass: Constructor): {agent: T; spy: sinon.SinonSpy} => {
+ const agent: T = new AgentClass({keepAlive: true});
+ // @ts-expect-error This IS correct
+ const spy = sinon.spy(agent, 'addRequest');
+ return {agent, spy};
+};
+
+test('non-object agent option works with http', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end('ok');
+ });
+
+ const {agent, spy} = createAgentSpy(HttpAgent);
+
+ t.truthy((await got({
+ https: {
+ rejectUnauthorized: false
+ },
+ agent: {
+ http: agent
+ }
+ })).body);
+ t.true(spy.calledOnce);
+
+ // Make sure to close all open sockets
+ agent.destroy();
+});
+
+test('non-object agent option works with https', withHttpsServer(), async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end('ok');
+ });
+
+ const {agent, spy} = createAgentSpy(HttpsAgent);
+
+ t.truthy((await got({
+ https: {
+ rejectUnauthorized: false
+ },
+ agent: {
+ https: agent
+ }
+ })).body);
+ t.true(spy.calledOnce);
+
+ // Make sure to close all open sockets
+ agent.destroy();
+});
+
+test('redirects from http to https work with an agent object', withServer, async (t, serverHttp) => {
+ await withHttpsServer()(t, async (t, serverHttps, got) => {
+ serverHttp.get('/', (_request, response) => {
+ response.end('http');
+ });
+
+ serverHttps.get('/', (_request, response) => {
+ response.end('https');
+ });
+
+ serverHttp.get('/httpToHttps', (_request, response) => {
+ response.writeHead(302, {
+ location: serverHttps.url
+ });
+ response.end();
+ });
+
+ const {agent: httpAgent, spy: httpSpy} = createAgentSpy(HttpAgent);
+ const {agent: httpsAgent, spy: httpsSpy} = createAgentSpy(HttpsAgent);
+
+ t.truthy((await got('httpToHttps', {
+ prefixUrl: serverHttp.url,
+ agent: {
+ http: httpAgent,
+ https: httpsAgent
+ }
+ })).body);
+ t.true(httpSpy.calledOnce);
+ t.true(httpsSpy.calledOnce);
+
+ // Make sure to close all open sockets
+ httpAgent.destroy();
+ httpsAgent.destroy();
+ });
+});
+
+test('redirects from https to http work with an agent object', withHttpsServer(), async (t, serverHttps, got) => {
+ await withServer(t, async (t, serverHttp) => {
+ serverHttp.get('/', (_request, response) => {
+ response.end('http');
+ });
+
+ serverHttps.get('/', (_request, response) => {
+ response.end('https');
+ });
+
+ serverHttps.get('/httpsToHttp', (_request, response) => {
+ response.writeHead(302, {
+ location: serverHttp.url
+ });
+ response.end();
+ });
+
+ const {agent: httpAgent, spy: httpSpy} = createAgentSpy(HttpAgent);
+ const {agent: httpsAgent, spy: httpsSpy} = createAgentSpy(HttpsAgent);
+
+ t.truthy((await got('httpsToHttp', {
+ prefixUrl: serverHttps.url,
+ agent: {
+ http: httpAgent,
+ https: httpsAgent
+ }
+ })).body);
+ t.true(httpSpy.calledOnce);
+ t.true(httpsSpy.calledOnce);
+
+ // Make sure to close all open sockets
+ httpAgent.destroy();
+ httpsAgent.destroy();
+ });
+});
+
+test('socket connect listener cleaned up after request', withHttpsServer(), async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end('ok');
+ });
+
+ const {agent} = createAgentSpy(HttpsAgent);
+
+ // Make sure there are no memory leaks when reusing keep-alive sockets
+ for (let i = 0; i < 20; i++) {
+ // eslint-disable-next-line no-await-in-loop
+ await got({
+ agent: {
+ https: agent
+ }
+ });
+ }
+
+ // Node.js 12 has incomplete types
+ for (const value of Object.values((agent as any).freeSockets) as [Socket[]]) {
+ for (const sock of value) {
+ t.is(sock.listenerCount('connect'), 0);
+ }
+ }
+
+ // Make sure to close all open sockets
+ agent.destroy();
+});
+
+{
+ const testFn = Number(process.versions.node.split('.')[0]) < 12 ? test.failing : test;
+
+ testFn('no socket hung up regression', withServer, async (t, server, got) => {
+ const agent = new HttpAgent({keepAlive: true});
+ const token = 'helloworld';
+
+ server.get('/', (request, response) => {
+ if (request.headers.token !== token) {
+ response.statusCode = 401;
+ response.end();
+ return;
+ }
+
+ response.end('ok');
+ });
+
+ const {body} = await got({
+ prefixUrl: 'http://127.0.0.1:3000',
+ agent: {
+ http: agent
+ },
+ hooks: {
+ afterResponse: [
+ async (response, retryWithMergedOptions) => {
+ // Force clean-up
+ response.socket?.destroy();
+
+ // Unauthorized
+ if (response.statusCode === 401) {
+ return retryWithMergedOptions({
+ headers: {
+ token
+ }
+ });
+ }
+
+ // No changes otherwise
+ return response;
+ }
+ ]
+ },
+ // Disable automatic retries, manual retries are allowed
+ retry: 0
+ });
+
+ t.is(body, 'ok');
+
+ agent.destroy();
+ });
+}
diff --git a/test/arguments.js b/test/arguments.js
deleted file mode 100644
index a772a5764..000000000
--- a/test/arguments.js
+++ /dev/null
@@ -1,70 +0,0 @@
-import test from 'ava';
-import got from '../';
-import {createServer} from './helpers/server';
-
-let s;
-
-test.before('setup', async () => {
- s = await createServer();
-
- s.on('/', (req, res) => {
- res.statusCode = 404;
- res.end();
- });
-
- s.on('/test', (req, res) => {
- res.end(req.url);
- });
-
- s.on('/?test=wow', (req, res) => {
- res.end(req.url);
- });
-
- await s.listen(s.port);
-});
-
-test('url is required', async t => {
- try {
- await got();
- t.fail('Exception was not thrown');
- } catch (err) {
- t.regex(err.message, /Parameter `url` must be a string or object, not undefined/);
- }
-});
-
-test('options are optional', async t => {
- t.is((await got(`${s.url}/test`)).body, '/test');
-});
-
-test('accepts url.parse object as first argument', async t => {
- t.is((await got({
- hostname: s.host,
- port: s.port,
- path: '/test'
- })).body, '/test');
-});
-
-test('requestUrl with url.parse object as first argument', async t => {
- t.is((await got({
- hostname: s.host,
- port: s.port,
- path: '/test'
- })).requestUrl, `${s.url}/test`);
-});
-
-test('overrides querystring from opts', async t => {
- t.is((await got(`${s.url}/?test=doge`, {query: {test: 'wow'}})).body, '/?test=wow');
-});
-
-test('should throw with auth in url', async t => {
- try {
- await got('https://test:45d3ps453@account.myservice.com/api/token');
- t.fail('Exception was not thrown');
- } catch (err) {
- t.regex(err.message, /Basic authentication must be done with auth option/);
- }
-});
-
-test.after('cleanup', async () => {
- await s.close();
-});
diff --git a/test/arguments.ts b/test/arguments.ts
new file mode 100644
index 000000000..37b3e1264
--- /dev/null
+++ b/test/arguments.ts
@@ -0,0 +1,550 @@
+/* eslint-disable node/no-deprecated-api */
+import {parse, URL, URLSearchParams} from 'url';
+import test from 'ava';
+import {Handler} from 'express';
+import pEvent = require('p-event');
+import got, {StrictOptions} from '../source';
+import withServer, {withBodyParsingServer} from './helpers/with-server';
+
+const echoUrl: Handler = (request, response) => {
+ response.end(request.url);
+};
+
+test('`url` is required', async t => {
+ await t.throwsAsync(
+ // @ts-expect-error No argument on purpose.
+ got(),
+ {
+ message: 'Missing `url` property'
+ }
+ );
+
+ await t.throwsAsync(
+ got(''),
+ {
+ message: 'No URL protocol specified'
+ }
+ );
+
+ await t.throwsAsync(
+ got({
+ url: ''
+ }),
+ {
+ message: 'No URL protocol specified'
+ }
+ );
+});
+
+test('`url` should be utf-8 encoded', async t => {
+ await t.throwsAsync(
+ got('https://example.com/%D2%E0%EB%EB%E8%ED'),
+ {
+ message: 'URI malformed'
+ }
+ );
+});
+
+test('throws if no arguments provided', async t => {
+ // @ts-expect-error Error tests
+ await t.throwsAsync(got(), {
+ message: 'Missing `url` property'
+ });
+});
+
+test('throws an error if the protocol is not specified', async t => {
+ await t.throwsAsync(got('example.com'), {
+ instanceOf: TypeError,
+ message: 'Invalid URL: example.com'
+ });
+
+ await t.throwsAsync(got({}), {
+ message: 'Missing `url` property'
+ });
+});
+
+test('properly encodes query string', withServer, async (t, server, got) => {
+ server.get('/', echoUrl);
+
+ const path = '?test=http://example.com?foo=bar';
+ const {body} = await got(path);
+ t.is(body, '/?test=http://example.com?foo=bar');
+});
+
+test('options are optional', withServer, async (t, server, got) => {
+ server.get('/test', echoUrl);
+
+ t.is((await got('test')).body, '/test');
+});
+
+test('methods are normalized', withServer, async (t, server, got) => {
+ server.post('/test', echoUrl);
+
+ const instance = got.extend({
+ handlers: [
+ (options, next) => {
+ if (options.method === options.method.toUpperCase()) {
+ t.pass();
+ } else {
+ t.fail();
+ }
+
+ return next(options);
+ }
+ ]
+ });
+
+ await instance('test', {method: 'post'});
+});
+
+test.failing('throws an error when legacy URL is passed', withServer, async (t, server) => {
+ server.get('/test', echoUrl);
+
+ await t.throwsAsync(
+ // @ts-expect-error Error tests
+ got(parse(`${server.url}/test`)),
+ {message: 'The legacy `url.Url` has been deprecated. Use `URL` instead.'}
+ );
+
+ await t.throwsAsync(
+ got({
+ protocol: 'http:',
+ hostname: 'localhost',
+ port: server.port
+ }),
+ {message: 'The legacy `url.Url` has been deprecated. Use `URL` instead.'}
+ );
+});
+
+test('accepts legacy URL options', withServer, async (t, server) => {
+ server.get('/test', echoUrl);
+
+ const {body: secondBody} = await got({
+ protocol: 'http:',
+ hostname: 'localhost',
+ port: server.port,
+ pathname: '/test'
+ });
+
+ t.is(secondBody, '/test');
+});
+
+test('overrides `searchParams` from options', withServer, async (t, server, got) => {
+ server.get('/', echoUrl);
+
+ const {body} = await got(
+ '?drop=this',
+ {
+ searchParams: {
+ test: 'wow'
+ }
+ }
+ );
+
+ t.is(body, '/?test=wow');
+});
+
+test('does not duplicate `searchParams`', withServer, async (t, server, got) => {
+ server.get('/', echoUrl);
+
+ const instance = got.extend({
+ searchParams: new URLSearchParams({foo: '123'})
+ });
+
+ const body = await instance('?bar=456').text();
+
+ t.is(body, '/?foo=123');
+});
+
+test('escapes `searchParams` parameter values', withServer, async (t, server, got) => {
+ server.get('/', echoUrl);
+
+ const {body} = await got({
+ searchParams: {
+ test: 'it’s ok'
+ }
+ });
+
+ t.is(body, '/?test=it%E2%80%99s+ok');
+});
+
+test('the `searchParams` option can be a URLSearchParams', withServer, async (t, server, got) => {
+ server.get('/', echoUrl);
+
+ const searchParameters = new URLSearchParams({test: 'wow'});
+ const {body} = await got({searchParams: searchParameters});
+ t.is(body, '/?test=wow');
+});
+
+test('ignores empty searchParams object', withServer, async (t, server, got) => {
+ server.get('/test', echoUrl);
+
+ t.is((await got('test', {searchParams: {}})).requestUrl, `${server.url}/test`);
+});
+
+test('throws when passing body with a non payload method', async t => {
+ await t.throwsAsync(got('https://example.com', {body: 'asdf'}), {
+ message: 'The `GET` method cannot be used with a body'
+ });
+});
+
+test('`allowGetBody` option', withServer, async (t, server, got) => {
+ server.get('/test', echoUrl);
+
+ const url = new URL(`${server.url}/test`);
+ await t.notThrowsAsync(got(url, {body: 'asdf', allowGetBody: true}));
+});
+
+test('WHATWG URL support', withServer, async (t, server, got) => {
+ server.get('/test', echoUrl);
+
+ const url = new URL(`${server.url}/test`);
+ await t.notThrowsAsync(got(url));
+});
+
+test('returns streams when using `isStream` option', withServer, async (t, server, got) => {
+ server.get('/stream', (_request, response) => {
+ response.end('ok');
+ });
+
+ const data = await pEvent(got('stream', {isStream: true}), 'data');
+ t.is(data.toString(), 'ok');
+});
+
+test('accepts `url` as an option', withServer, async (t, server, got) => {
+ server.get('/test', echoUrl);
+
+ await t.notThrowsAsync(got({url: 'test'}));
+});
+
+test('can omit `url` option if using `prefixUrl`', withServer, async (t, server, got) => {
+ server.get('/', echoUrl);
+
+ await t.notThrowsAsync(got({}));
+});
+
+test('throws TypeError when known `options.hooks` value is not an array', async t => {
+ await t.throwsAsync(
+ // @ts-expect-error Error tests
+ got('https://example.com', {hooks: {beforeRequest: {}}}),
+ {
+ message: 'Parameter `beforeRequest` must be an Array, got Object'
+ }
+ );
+});
+
+test('throws TypeError when known `options.hooks` array item is not a function', async t => {
+ await t.throwsAsync(
+ // @ts-expect-error Error tests
+ got('https://example.com', {hooks: {beforeRequest: [{}]}}),
+ {
+ message: 'hook is not a function'
+ }
+ );
+});
+
+test('allows extra keys in `options.hooks`', withServer, async (t, server, got) => {
+ server.get('/test', echoUrl);
+
+ // @ts-expect-error We do not allow extra keys in hooks but this won't throw
+ await t.notThrowsAsync(got('test', {hooks: {extra: []}}));
+});
+
+test('`prefixUrl` option works', withServer, async (t, server, got) => {
+ server.get('/test/foobar', echoUrl);
+
+ const instanceA = got.extend({prefixUrl: `${server.url}/test`});
+ const {body} = await instanceA('foobar');
+ t.is(body, '/test/foobar');
+});
+
+test('accepts WHATWG URL as the `prefixUrl` option', withServer, async (t, server, got) => {
+ server.get('/test/foobar', echoUrl);
+
+ const instanceA = got.extend({prefixUrl: new URL(`${server.url}/test`)});
+ const {body} = await instanceA('foobar');
+ t.is(body, '/test/foobar');
+});
+
+test('backslash in the end of `prefixUrl` option is optional', withServer, async (t, server) => {
+ server.get('/test/foobar', echoUrl);
+
+ const instanceA = got.extend({prefixUrl: `${server.url}/test/`});
+ const {body} = await instanceA('foobar');
+ t.is(body, '/test/foobar');
+});
+
+test('`prefixUrl` can be changed if the URL contains the old one', withServer, async (t, server) => {
+ server.get('/', echoUrl);
+
+ const instanceA = got.extend({
+ prefixUrl: `${server.url}/meh`,
+ handlers: [
+ (options, next) => {
+ options.prefixUrl = server.url;
+ return next(options);
+ }
+ ]
+ });
+
+ const {body} = await instanceA('');
+ t.is(body, '/');
+});
+
+test('throws if cannot change `prefixUrl`', async t => {
+ const instanceA = got.extend({
+ prefixUrl: 'https://example.com',
+ handlers: [
+ (options, next) => {
+ options.url = new URL('https://google.pl');
+ options.prefixUrl = 'https://example.com';
+ return next(options);
+ }
+ ]
+ });
+
+ await t.throwsAsync(instanceA(''), {message: 'Cannot change `prefixUrl` from https://example.com/ to https://example.com: https://google.pl/'});
+});
+
+test('throws if the `searchParams` value is invalid', async t => {
+ await t.throwsAsync(got('https://example.com', {
+ searchParams: {
+ // @ts-expect-error Error tests
+ foo: []
+ }
+ }), {
+ instanceOf: TypeError,
+ message: 'The `searchParams` value \'\' must be a string, number, boolean or null'
+ });
+});
+
+test('`context` option is not enumerable', withServer, async (t, server, got) => {
+ server.get('/', echoUrl);
+
+ const context = {
+ foo: 'bar'
+ };
+
+ await got({
+ context,
+ hooks: {
+ beforeRequest: [
+ options => {
+ t.is(options.context, context);
+ t.false({}.propertyIsEnumerable.call(options, 'context'));
+ }
+ ]
+ }
+ });
+});
+
+test('`context` option is accessible when using hooks', withServer, async (t, server, got) => {
+ server.get('/', echoUrl);
+
+ const context = {
+ foo: 'bar'
+ };
+
+ await got({
+ context,
+ hooks: {
+ beforeRequest: [
+ options => {
+ t.is(options.context, context);
+ t.false({}.propertyIsEnumerable.call(options, 'context'));
+ }
+ ]
+ }
+ });
+});
+
+test('`context` option is accessible when extending instances', t => {
+ const context = {
+ foo: 'bar'
+ };
+
+ const instance = got.extend({context});
+
+ t.is(instance.defaults.options.context, context);
+ t.false({}.propertyIsEnumerable.call(instance.defaults.options, 'context'));
+});
+
+test('throws if `options.encoding` is `null`', async t => {
+ await t.throwsAsync(got('https://example.com', {
+ // @ts-expect-error For testing purposes
+ encoding: null
+ }), {message: 'To get a Buffer, set `options.responseType` to `buffer` instead'});
+});
+
+test('`url` option and input argument are mutually exclusive', async t => {
+ await t.throwsAsync(got('https://example.com', {
+ url: 'https://example.com'
+ }), {message: 'The `url` option is mutually exclusive with the `input` argument'});
+});
+
+test('throws a helpful error when passing `followRedirects`', async t => {
+ await t.throwsAsync(got('https://example.com', {
+ // @ts-expect-error For testing purposes
+ followRedirects: true
+ }), {message: 'The `followRedirects` option does not exist. Use `followRedirect` instead.'});
+});
+
+test('merges `searchParams` instances', t => {
+ const instance = got.extend({
+ searchParams: new URLSearchParams('a=1')
+ }, {
+ searchParams: new URLSearchParams('b=2')
+ });
+
+ t.is(instance.defaults.options.searchParams!.get('a'), '1');
+ t.is(instance.defaults.options.searchParams!.get('b'), '2');
+});
+
+test('throws a helpful error when passing `auth`', async t => {
+ await t.throwsAsync(got('https://example.com', {
+ // @ts-expect-error For testing purposes
+ auth: 'username:password'
+ }), {
+ message: 'Parameter `auth` is deprecated. Use `username` / `password` instead.'
+ });
+});
+
+test('throws on leading slashes', async t => {
+ await t.throwsAsync(got('/asdf', {prefixUrl: 'https://example.com'}), {
+ message: '`input` must not start with a slash when using `prefixUrl`'
+ });
+});
+
+test('throws on invalid `dnsCache` option', async t => {
+ await t.throwsAsync(got('https://example.com', {
+ // @ts-expect-error Error tests
+ dnsCache: 123
+ }), {message: 'Parameter `dnsCache` must be a CacheableLookup instance or a boolean, got number'});
+});
+
+test('throws on invalid `agent` option', async t => {
+ await t.throwsAsync(got('https://example.com', {
+ agent: {
+ // @ts-expect-error Error tests
+ asdf: 123
+ }
+ }), {message: 'Expected the `options.agent` properties to be `http`, `https` or `http2`, got `asdf`'});
+});
+
+test('fallbacks to native http if `request(...)` returns undefined', withServer, async (t, server, got) => {
+ server.get('/', echoUrl);
+
+ const {body} = await got('', {request: () => undefined});
+
+ t.is(body, '/');
+});
+
+test('strict options', withServer, async (t, server, got) => {
+ server.get('/', echoUrl);
+
+ const options: StrictOptions = {};
+
+ const {body} = await got(options);
+
+ t.is(body, '/');
+});
+
+test('does not throw on frozen options', withServer, async (t, server, got) => {
+ server.get('/', echoUrl);
+
+ const options: StrictOptions = {};
+
+ Object.freeze(options);
+
+ const {body} = await got(options);
+
+ t.is(body, '/');
+});
+
+test('encodes query string included in input', t => {
+ const {url} = got.mergeOptions({
+ url: new URL('https://example.com/?a=b c')
+ });
+
+ t.is(url.search, '?a=b%20c');
+});
+
+test('normalizes search params included in options', t => {
+ const {url} = got.mergeOptions({
+ url: new URL('https://example.com'),
+ searchParams: 'a=b c'
+ });
+
+ t.is(url.search, '?a=b+c');
+});
+
+test('reuse options while using init hook', withServer, async (t, server, got) => {
+ t.plan(2);
+
+ server.get('/', echoUrl);
+
+ const options = {
+ hooks: {
+ init: [
+ () => {
+ t.pass();
+ }
+ ]
+ }
+ };
+
+ await got('', options);
+ await got('', options);
+});
+
+test('allowGetBody sends json payload', withBodyParsingServer, async (t, server, got) => {
+ server.get('/', (request, response) => {
+ if (request.body.hello !== 'world') {
+ response.statusCode = 400;
+ }
+
+ response.end();
+ });
+
+ const {statusCode} = await got({
+ allowGetBody: true,
+ json: {hello: 'world'},
+ retry: 0,
+ throwHttpErrors: false
+ });
+ t.is(statusCode, 200);
+});
+
+test('no URL pollution', withServer, async (t, server) => {
+ server.get('/ok', echoUrl);
+
+ const url = new URL(server.url);
+
+ const {body} = await got(url, {
+ hooks: {
+ beforeRequest: [
+ options => {
+ options.url.pathname = '/ok';
+ }
+ ]
+ }
+ });
+
+ t.is(url.pathname, '/');
+ t.is(body, '/ok');
+});
+
+test('prefixUrl is properly replaced when extending', withServer, async (t, server) => {
+ server.get('/', (request, response) => {
+ response.end(request.url);
+ });
+
+ server.get('/other/path/', (request, response) => {
+ response.end(request.url);
+ });
+
+ const parent = got.extend({prefixUrl: server.url});
+ const child = parent.extend({prefixUrl: `${server.url}/other/path/`});
+
+ t.is(await child.get('').text(), '/other/path/');
+});
diff --git a/test/cache.ts b/test/cache.ts
new file mode 100644
index 000000000..89693a814
--- /dev/null
+++ b/test/cache.ts
@@ -0,0 +1,366 @@
+import {promisify} from 'util';
+import {gzip} from 'zlib';
+import test from 'ava';
+import pEvent = require('p-event');
+import getStream = require('get-stream');
+import {Handler} from 'express';
+import got, {Response} from '../source';
+import withServer from './helpers/with-server';
+import CacheableLookup from 'cacheable-lookup';
+import delay = require('delay');
+
+const cacheEndpoint: Handler = (_request, response) => {
+ response.setHeader('Cache-Control', 'public, max-age=60');
+ response.end(Date.now().toString());
+};
+
+test('non-cacheable responses are not cached', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.setHeader('Cache-Control', 'public, no-cache, no-store');
+ response.end(Date.now().toString());
+ });
+
+ const cache = new Map();
+
+ const firstResponseInt = Number((await got({cache})).body);
+ const secondResponseInt = Number((await got({cache})).body);
+
+ t.is(cache.size, 0);
+ t.true(firstResponseInt < secondResponseInt);
+});
+
+test('cacheable responses are cached', withServer, async (t, server, got) => {
+ server.get('/', cacheEndpoint);
+
+ const cache = new Map();
+
+ const firstResponse = await got({cache});
+ const secondResponse = await got({cache});
+
+ t.is(cache.size, 1);
+ t.is(firstResponse.body, secondResponse.body);
+});
+
+test('cached response is re-encoded to current encoding option', withServer, async (t, server, got) => {
+ server.get('/', cacheEndpoint);
+
+ const cache = new Map();
+ const firstEncoding = 'base64';
+ const secondEncoding = 'hex';
+
+ const firstResponse = await got({cache, encoding: firstEncoding});
+ const secondResponse = await got({cache, encoding: secondEncoding});
+
+ const expectedSecondResponseBody = Buffer.from(firstResponse.body, firstEncoding).toString(secondEncoding);
+
+ t.is(cache.size, 1);
+ t.is(secondResponse.body, expectedSecondResponseBody);
+});
+
+test('redirects are cached and re-used internally', withServer, async (t, server, got) => {
+ let status301Index = 0;
+ server.get('/301', (_request, response) => {
+ if (status301Index === 0) {
+ response.setHeader('Cache-Control', 'public, max-age=60');
+ response.setHeader('Location', '/');
+ response.statusCode = 301;
+ }
+
+ response.end();
+ status301Index++;
+ });
+
+ let status302Index = 0;
+ server.get('/302', (_request, response) => {
+ if (status302Index === 0) {
+ response.setHeader('Cache-Control', 'public, max-age=60');
+ response.setHeader('Location', '/');
+ response.statusCode = 302;
+ }
+
+ response.end();
+ status302Index++;
+ });
+
+ server.get('/', cacheEndpoint);
+
+ const cache = new Map();
+ const A1 = await got('301', {cache});
+ const B1 = await got('302', {cache});
+
+ const A2 = await got('301', {cache});
+ const B2 = await got('302', {cache});
+
+ t.is(cache.size, 3);
+ t.is(A1.body, B1.body);
+ t.is(A1.body, A2.body);
+ t.is(B1.body, B2.body);
+});
+
+test('cached response has got options', withServer, async (t, server, got) => {
+ server.get('/', cacheEndpoint);
+
+ const cache = new Map();
+ const options = {
+ username: 'foo',
+ cache
+ };
+
+ await got(options);
+ const secondResponse = await got(options);
+
+ t.is(secondResponse.request.options.username, options.username);
+});
+
+test('cache error throws `got.CacheError`', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end('ok');
+ });
+
+ const cache = {};
+
+ // @ts-expect-error Error tests
+ await t.throwsAsync(got({cache}), {
+ instanceOf: got.CacheError,
+ code: 'ERR_CACHE_ACCESS'
+ });
+});
+
+test('doesn\'t cache response when received HTTP error', withServer, async (t, server, got) => {
+ let isFirstErrorCalled = false;
+ server.get('/', (_request, response) => {
+ if (!isFirstErrorCalled) {
+ response.end('ok');
+ return;
+ }
+
+ isFirstErrorCalled = true;
+ response.statusCode = 502;
+ response.end('received 502');
+ });
+
+ const cache = new Map();
+
+ const {statusCode, body} = await got({url: '', cache, throwHttpErrors: false});
+ t.is(statusCode, 200);
+ t.is(body, 'ok');
+});
+
+test('DNS cache works', async t => {
+ const instance = got.extend({
+ dnsCache: true
+ });
+
+ await t.notThrowsAsync(instance('https://example.com'));
+
+ // @ts-expect-error
+ t.is(instance.defaults.options.dnsCache!._cache.size, 1);
+});
+
+test('DNS cache works - CacheableLookup instance', async t => {
+ const cache = new CacheableLookup();
+ await t.notThrowsAsync(got('https://example.com', {dnsCache: cache}));
+
+ t.is((cache as any)._cache.size, 1);
+});
+
+test('`isFromCache` stream property is undefined before the `response` event', withServer, async (t, server, got) => {
+ server.get('/', cacheEndpoint);
+
+ const cache = new Map();
+ const stream = got.stream({cache});
+ t.is(stream.isFromCache, undefined);
+
+ await getStream(stream);
+});
+
+test('`isFromCache` stream property is false after the `response` event', withServer, async (t, server, got) => {
+ server.get('/', cacheEndpoint);
+
+ const cache = new Map();
+ const stream = got.stream({cache});
+
+ const response: Response = await pEvent(stream, 'response');
+ t.is(response.isFromCache, false);
+ t.is(stream.isFromCache, false);
+
+ await getStream(stream);
+});
+
+test('`isFromCache` stream property is true if the response was cached', withServer, async (t, server, got) => {
+ server.get('/', cacheEndpoint);
+
+ const cache = new Map();
+
+ await getStream(got.stream({cache}));
+ const stream = got.stream({cache});
+
+ const response: Response = await pEvent(stream, 'response');
+ t.is(response.isFromCache, true);
+ t.is(stream.isFromCache, true);
+
+ await getStream(stream);
+});
+
+test('can disable cache by extending the instance', withServer, async (t, server, got) => {
+ server.get('/', cacheEndpoint);
+
+ const cache = new Map();
+
+ const instance = got.extend({cache});
+
+ await getStream(instance.stream(''));
+ const stream = instance.extend({cache: false}).stream('');
+
+ const response: Response = await pEvent(stream, 'response');
+ t.is(response.isFromCache, false);
+ t.is(stream.isFromCache, false);
+
+ await getStream(stream);
+});
+
+test('does not break POST requests', withServer, async (t, server, got) => {
+ server.post('/', async (request, response) => {
+ request.resume();
+ response.end(JSON.stringify(request.headers));
+ });
+
+ const headers = await got.post('', {
+ body: '',
+ cache: new Map()
+ }).json<{'content-length': string}>();
+
+ t.is(headers['content-length'], '0');
+});
+
+test('decompresses cached responses', withServer, async (t, server, got) => {
+ const etag = 'foobar';
+
+ const payload = JSON.stringify({foo: 'bar'});
+ const compressed = await promisify(gzip)(payload);
+
+ server.get('/', (request, response) => {
+ if (request.headers['if-none-match'] === etag) {
+ response.statusCode = 304;
+ response.end();
+ } else {
+ response.setHeader('content-encoding', 'gzip');
+ response.setHeader('cache-control', 'public, max-age=60');
+ response.setHeader('etag', etag);
+ response.end(compressed);
+ }
+ });
+
+ const cache = new Map();
+
+ for (let i = 0; i < 2; i++) {
+ // eslint-disable-next-line no-await-in-loop
+ await t.notThrowsAsync(got({
+ cache,
+ responseType: 'json',
+ decompress: true,
+ retry: 2
+ }));
+ }
+
+ t.is(cache.size, 1);
+});
+
+test('can replace the instance\'s HTTP cache', withServer, async (t, server, got) => {
+ server.get('/', cacheEndpoint);
+
+ const cache = new Map();
+ const secondCache = new Map();
+
+ const instance = got.extend({
+ mutableDefaults: true,
+ cache
+ });
+
+ await t.notThrowsAsync(instance(''));
+ await t.notThrowsAsync(instance(''));
+
+ instance.defaults.options.cache = secondCache;
+
+ await t.notThrowsAsync(instance(''));
+ await t.notThrowsAsync(instance(''));
+
+ t.is(cache.size, 1);
+ t.is(secondCache.size, 1);
+});
+
+test('does not hang on huge response', withServer, async (t, server, got) => {
+ const bufferSize = 3 * 16 * 1024;
+ const times = 10;
+
+ const buffer = Buffer.alloc(bufferSize);
+
+ server.get('/', async (_request, response) => {
+ for (let i = 0; i < 10; i++) {
+ response.write(buffer);
+
+ // eslint-disable-next-line no-await-in-loop
+ await delay(100);
+ }
+
+ response.end();
+ });
+
+ const body = await got('', {
+ cache: new Map()
+ }).buffer();
+
+ t.is(body.length, bufferSize * times);
+});
+
+test('cached response ETag', withServer, async (t, server, got) => {
+ const etag = 'foobar';
+ const body = 'responseBody';
+
+ server.get('/', (request, response) => {
+ if (request.headers['if-none-match'] === etag) {
+ response.writeHead(304);
+ response.end();
+ } else {
+ response.writeHead(200, {ETag: etag});
+ response.end(body);
+ }
+ });
+
+ const cache = new Map();
+
+ const originalResponse = await got({cache});
+
+ t.false(originalResponse.isFromCache);
+ t.is(originalResponse.body, body);
+
+ await delay(100); // Added small delay in order to wait the cache to be populated
+
+ t.is(cache.size, 1);
+
+ const cachedResponse = await got({cache});
+
+ t.true(cachedResponse.isFromCache);
+ t.is(cachedResponse.body, body);
+});
+
+test('works with http2', async t => {
+ const cache = new Map();
+
+ const client = got.extend({
+ http2: true,
+ cache
+ });
+
+ await t.notThrowsAsync(client('https://httpbin.org/anything'));
+});
+
+test('http-cache-semantics typings', t => {
+ const instance = got.extend({
+ cacheOptions: {
+ shared: false
+ }
+ });
+
+ t.is(instance.defaults.options.cacheOptions.shared, false);
+});
diff --git a/test/cancel.ts b/test/cancel.ts
new file mode 100644
index 000000000..fa6e16cb4
--- /dev/null
+++ b/test/cancel.ts
@@ -0,0 +1,290 @@
+import {EventEmitter} from 'events';
+import {Readable as ReadableStream} from 'stream';
+import stream = require('stream');
+import test from 'ava';
+import delay = require('delay');
+import pEvent = require('p-event');
+import getStream = require('get-stream');
+import {Handler} from 'express';
+import got, {CancelError} from '../source';
+import slowDataStream from './helpers/slow-data-stream';
+import {GlobalClock} from './helpers/types';
+import {ExtendedHttpTestServer} from './helpers/create-http-test-server';
+import withServer, {withServerAndFakeTimers} from './helpers/with-server';
+
+const prepareServer = (server: ExtendedHttpTestServer, clock: GlobalClock): {emitter: EventEmitter; promise: Promise} => {
+ const emitter = new EventEmitter();
+
+ const promise = new Promise((resolve, reject) => {
+ server.all('/abort', async (request, response) => {
+ emitter.emit('connection');
+
+ request.once('aborted', resolve);
+ response.once('finish', reject.bind(null, new Error('Request finished instead of aborting.')));
+
+ try {
+ await pEvent(request, 'end');
+ } catch {
+ // Node.js 15.0.0 throws AND emits `aborted`
+ }
+
+ response.end();
+ });
+
+ server.get('/redirect', (_request, response) => {
+ response.writeHead(302, {
+ location: `${server.url}/abort`
+ });
+ response.end();
+
+ emitter.emit('sentRedirect');
+
+ clock.tick(3000);
+ resolve();
+ });
+ });
+
+ return {emitter, promise};
+};
+
+const downloadHandler = (clock: GlobalClock): Handler => (_request, response) => {
+ response.writeHead(200, {
+ 'transfer-encoding': 'chunked'
+ });
+
+ response.flushHeaders();
+
+ stream.pipeline(
+ slowDataStream(clock),
+ response,
+ () => {
+ response.end();
+ }
+ );
+};
+
+test.serial('does not retry after cancelation', withServerAndFakeTimers, async (t, server, got, clock) => {
+ const {emitter, promise} = prepareServer(server, clock);
+
+ const gotPromise = got('redirect', {
+ retry: {
+ calculateDelay: () => {
+ t.fail('Makes a new try after cancelation');
+ return 0;
+ }
+ }
+ });
+
+ emitter.once('sentRedirect', () => {
+ gotPromise.cancel();
+ });
+
+ await t.throwsAsync(gotPromise, {
+ instanceOf: CancelError,
+ code: 'ERR_CANCELED'
+ });
+ await t.notThrowsAsync(promise, 'Request finished instead of aborting.');
+});
+
+test.serial('cleans up request timeouts', withServer, async (t, server, got) => {
+ server.get('/', () => {});
+
+ const gotPromise = got({
+ timeout: 10,
+ retry: {
+ calculateDelay: ({computedValue}) => {
+ process.nextTick(() => gotPromise.cancel());
+
+ if (computedValue) {
+ return 20;
+ }
+
+ return 0;
+ },
+ limit: 1
+ }
+ });
+
+ await t.throwsAsync(gotPromise, {
+ instanceOf: CancelError,
+ code: 'ERR_CANCELED'
+ });
+
+ // Wait for unhandled errors
+ await delay(40);
+});
+
+test.serial('cancels in-progress request', withServerAndFakeTimers, async (t, server, got, clock) => {
+ const {emitter, promise} = prepareServer(server, clock);
+
+ const body = new ReadableStream({
+ read() {}
+ });
+ body.push('1');
+
+ const gotPromise = got.post('abort', {body});
+
+ // Wait for the connection to be established before canceling
+ emitter.once('connection', () => {
+ gotPromise.cancel();
+ body.push(null);
+ });
+
+ await t.throwsAsync(gotPromise, {
+ instanceOf: CancelError,
+ code: 'ERR_CANCELED'
+ });
+ await t.notThrowsAsync(promise, 'Request finished instead of aborting.');
+});
+
+test.serial('cancels in-progress request with timeout', withServerAndFakeTimers, async (t, server, got, clock) => {
+ const {emitter, promise} = prepareServer(server, clock);
+
+ const body = new ReadableStream({
+ read() {}
+ });
+ body.push('1');
+
+ const gotPromise = got.post('abort', {body, timeout: 10000});
+
+ // Wait for the connection to be established before canceling
+ emitter.once('connection', () => {
+ gotPromise.cancel();
+ body.push(null);
+ });
+
+ await t.throwsAsync(gotPromise, {
+ instanceOf: CancelError,
+ code: 'ERR_CANCELED'
+ });
+ await t.notThrowsAsync(promise, 'Request finished instead of aborting.');
+});
+
+test.serial('cancel immediately', withServerAndFakeTimers, async (t, server, got, clock) => {
+ const promise = new Promise((resolve, reject) => {
+ // We won't get an abort or even a connection
+ // We assume no request within 1000ms equals a (client side) aborted request
+ server.get('/abort', (_request, response) => {
+ response.once('finish', reject.bind(global, new Error('Request finished instead of aborting.')));
+ response.end();
+ });
+
+ clock.tick(1000);
+ resolve();
+ });
+
+ const gotPromise = got('abort');
+ gotPromise.cancel();
+
+ await t.throwsAsync(gotPromise);
+ await t.notThrowsAsync(promise, 'Request finished instead of aborting.');
+});
+
+test('recover from cancelation using cancelable promise attribute', async t => {
+ // Canceled before connection started
+ const p = got('http://example.com');
+ const recover = p.catch((error: Error) => {
+ if (p.isCanceled) {
+ return;
+ }
+
+ throw error;
+ });
+
+ p.cancel();
+
+ await t.notThrowsAsync(recover);
+});
+
+test('recover from cancellation using error instance', async t => {
+ // Canceled before connection started
+ const p = got('http://example.com');
+ const recover = p.catch((error: Error) => {
+ if (error instanceof got.CancelError) {
+ return;
+ }
+
+ throw error;
+ });
+
+ p.cancel();
+
+ await t.notThrowsAsync(recover);
+});
+
+test.serial('throws on incomplete (canceled) response - promise', withServerAndFakeTimers, async (t, server, got, clock) => {
+ server.get('/', downloadHandler(clock));
+
+ await t.throwsAsync(
+ got({
+ timeout: {request: 500},
+ retry: 0
+ }),
+ {instanceOf: got.TimeoutError}
+ );
+});
+
+test.serial('throws on incomplete (canceled) response - promise #2', withServerAndFakeTimers, async (t, server, got, clock) => {
+ server.get('/', downloadHandler(clock));
+
+ const promise = got('').on('response', () => {
+ clock.tick(500);
+ promise.cancel();
+ });
+
+ await t.throwsAsync(promise, {
+ instanceOf: got.CancelError,
+ code: 'ERR_CANCELED'
+ });
+});
+
+test.serial('throws on incomplete (canceled) response - stream', withServerAndFakeTimers, async (t, server, got, clock) => {
+ server.get('/', downloadHandler(clock));
+
+ const errorString = 'Foobar';
+
+ const stream = got.stream('').on('response', () => {
+ clock.tick(500);
+ stream.destroy(new Error(errorString));
+ });
+
+ await t.throwsAsync(getStream(stream), {message: errorString});
+});
+
+// Note: it will throw, but the response is loaded already.
+test('throws when canceling cached request', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.setHeader('Cache-Control', 'public, max-age=60');
+ response.end(Date.now().toString());
+ });
+
+ const cache = new Map();
+ await got({cache});
+
+ const promise = got({cache}).on('response', () => {
+ promise.cancel();
+ });
+
+ await t.throwsAsync(promise, {
+ instanceOf: got.CancelError,
+ code: 'ERR_CANCELED'
+ });
+});
+
+test('throws when canceling cached request #2', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.setHeader('Cache-Control', 'public, max-age=60');
+ response.end(Date.now().toString());
+ });
+
+ const cache = new Map();
+ await got({cache});
+
+ const promise = got({cache});
+ promise.cancel();
+
+ await t.throwsAsync(promise, {
+ instanceOf: got.CancelError,
+ code: 'ERR_CANCELED'
+ });
+});
diff --git a/test/cookies.ts b/test/cookies.ts
new file mode 100644
index 000000000..a2c334754
--- /dev/null
+++ b/test/cookies.ts
@@ -0,0 +1,213 @@
+import net = require('net');
+import test from 'ava';
+import toughCookie = require('tough-cookie');
+import delay = require('delay');
+import got from '../source';
+import withServer from './helpers/with-server';
+
+test('reads a cookie', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.setHeader('set-cookie', 'hello=world');
+ response.end();
+ });
+
+ const cookieJar = new toughCookie.CookieJar();
+
+ await got({cookieJar});
+
+ const cookie = cookieJar.getCookiesSync(server.url)[0];
+ t.is(cookie.key, 'hello');
+ t.is(cookie.value, 'world');
+});
+
+test('reads multiple cookies', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.setHeader('set-cookie', ['hello=world', 'foo=bar']);
+ response.end();
+ });
+
+ const cookieJar = new toughCookie.CookieJar();
+
+ await got({cookieJar});
+
+ const cookies = cookieJar.getCookiesSync(server.url);
+ const cookieA = cookies[0];
+ t.is(cookieA.key, 'hello');
+ t.is(cookieA.value, 'world');
+
+ const cookieB = cookies[1];
+ t.is(cookieB.key, 'foo');
+ t.is(cookieB.value, 'bar');
+});
+
+test('cookies doesn\'t break on redirects', withServer, async (t, server, got) => {
+ server.get('/redirect', (_request, response) => {
+ response.setHeader('set-cookie', ['hello=world', 'foo=bar']);
+ response.setHeader('location', '/');
+ response.statusCode = 302;
+ response.end();
+ });
+
+ server.get('/', (request, response) => {
+ response.end(request.headers.cookie ?? '');
+ });
+
+ const cookieJar = new toughCookie.CookieJar();
+
+ const {body} = await got('redirect', {cookieJar});
+ t.is(body, 'hello=world; foo=bar');
+});
+
+test('throws on invalid cookies', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.setHeader('set-cookie', 'hello=world; domain=localhost');
+ response.end();
+ });
+
+ const cookieJar = new toughCookie.CookieJar();
+
+ await t.throwsAsync(got({cookieJar}), {message: 'Cookie has domain set to a public suffix'});
+});
+
+test('does not throw on invalid cookies when options.ignoreInvalidCookies is set', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.setHeader('set-cookie', 'hello=world; domain=localhost');
+ response.end();
+ });
+
+ const cookieJar = new toughCookie.CookieJar();
+
+ await got({
+ cookieJar,
+ ignoreInvalidCookies: true
+ });
+
+ const cookies = cookieJar.getCookiesSync(server.url);
+ t.is(cookies.length, 0);
+});
+
+test('catches store errors', async t => {
+ const error = 'Some error';
+ const cookieJar = new toughCookie.CookieJar({
+ findCookies: (_, __, ___, callback) => {
+ callback(new Error(error), []);
+ },
+ findCookie: () => {},
+ getAllCookies: () => {},
+ putCookie: () => {},
+ removeCookies: () => {},
+ removeCookie: () => {},
+ updateCookie: () => {},
+ synchronous: false
+ });
+
+ await t.throwsAsync(got('https://example.com', {cookieJar}), {message: error});
+});
+
+test('overrides options.headers.cookie', withServer, async (t, server, got) => {
+ server.get('/redirect', (_request, response) => {
+ response.setHeader('set-cookie', ['hello=world', 'foo=bar']);
+ response.setHeader('location', '/');
+ response.statusCode = 302;
+ response.end();
+ });
+
+ server.get('/', (request, response) => {
+ response.end(request.headers.cookie ?? '');
+ });
+
+ const cookieJar = new toughCookie.CookieJar();
+ const {body} = await got('redirect', {
+ cookieJar,
+ headers: {
+ cookie: 'a=b'
+ }
+ });
+ t.is(body, 'hello=world; foo=bar');
+});
+
+test('no unhandled errors', async t => {
+ const server = net.createServer(connection => {
+ connection.end('blah');
+ }).listen(0);
+
+ const message = 'snap!';
+
+ const options = {
+ cookieJar: {
+ setCookie: async (_rawCookie: string, _url: string) => {},
+ getCookieString: async (_url: string) => {
+ throw new Error(message);
+ }
+ }
+ };
+
+ await t.throwsAsync(got(`http://127.0.0.1:${(server.address() as net.AddressInfo).port}`, options), {message});
+ await delay(500);
+ t.pass();
+
+ server.close();
+});
+
+test('accepts custom `cookieJar` object', withServer, async (t, server, got) => {
+ server.get('/', (request, response) => {
+ response.setHeader('set-cookie', ['hello=world']);
+ response.end(request.headers.cookie);
+ });
+
+ const cookies: Record = {};
+ const cookieJar = {
+ async getCookieString(url: string) {
+ t.is(typeof url, 'string');
+
+ return cookies[url] || '';
+ },
+
+ async setCookie(rawCookie: string, url: string) {
+ cookies[url] = rawCookie;
+ }
+ };
+
+ const first = await got('', {cookieJar});
+ const second = await got('', {cookieJar});
+
+ t.is(first.body, '');
+ t.is(second.body, 'hello=world');
+});
+
+test('throws on invalid `options.cookieJar.setCookie`', async t => {
+ await t.throwsAsync(got('https://example.com', {
+ cookieJar: {
+ // @ts-expect-error Error tests
+ setCookie: 123
+ }
+ }), {message: 'Expected value which is `Function`, received value of type `number`.'});
+});
+
+test('throws on invalid `options.cookieJar.getCookieString`', async t => {
+ await t.throwsAsync(got('https://example.com', {
+ cookieJar: {
+ setCookie: async () => {},
+ // @ts-expect-error Error tests
+ getCookieString: 123
+ }
+ }), {message: 'Expected value which is `Function`, received value of type `number`.'});
+});
+
+test('cookies are cleared when redirecting to a different hostname (no cookieJar)', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.writeHead(302, {
+ location: 'https://httpbin.org/anything'
+ });
+ response.end();
+ });
+
+ const {headers} = await got('', {
+ headers: {
+ cookie: 'foo=bar',
+ 'user-agent': 'custom'
+ }
+ }).json();
+ t.is(headers.Cookie, undefined);
+ t.is(headers['User-Agent'], 'custom');
+});
diff --git a/test/create.ts b/test/create.ts
new file mode 100644
index 000000000..4b767ac95
--- /dev/null
+++ b/test/create.ts
@@ -0,0 +1,344 @@
+import {Agent as HttpAgent, IncomingMessage, request as httpRequest, RequestOptions} from 'http';
+import {URL} from 'url';
+import test from 'ava';
+import is from '@sindresorhus/is';
+import {Handler} from 'express';
+import got, {
+ BeforeRequestHook,
+ Headers,
+ Hooks,
+ RequestFunction
+} from '../source';
+import withServer from './helpers/with-server';
+
+const echoHeaders: Handler = (request, response) => {
+ request.resume();
+ response.end(JSON.stringify(request.headers));
+};
+
+test('preserves global defaults', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ const globalHeaders = await got('').json();
+ const instanceHeaders = await got.extend()('').json();
+ t.deepEqual(instanceHeaders, globalHeaders);
+});
+
+test('supports instance defaults', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ const instance = got.extend({
+ headers: {
+ 'user-agent': 'custom-ua-string'
+ }
+ });
+ const headers = await instance('').json();
+ t.is(headers['user-agent'], 'custom-ua-string');
+});
+
+test('supports invocation overrides', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ const instance = got.extend({
+ headers: {
+ 'user-agent': 'custom-ua-string'
+ }
+ });
+ const headers = await instance({
+ headers: {
+ 'user-agent': 'different-ua-string'
+ }
+ }).json();
+ t.is(headers['user-agent'], 'different-ua-string');
+});
+
+test('carries previous instance defaults', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ const instanceA = got.extend({
+ headers: {
+ 'x-foo': 'foo'
+ }
+ });
+ const instanceB = instanceA.extend({
+ headers: {
+ 'x-bar': 'bar'
+ }
+ });
+ const headers = await instanceB('').json();
+ t.is(headers['x-foo'], 'foo');
+ t.is(headers['x-bar'], 'bar');
+});
+
+test('custom headers (extend)', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ const options = {headers: {unicorn: 'rainbow'}};
+
+ const instance = got.extend(options);
+ const headers = await instance('').json();
+ t.is(headers.unicorn, 'rainbow');
+});
+
+test('extend overwrites arrays with a deep clone', t => {
+ const beforeRequest = [0];
+ const a = got.extend({hooks: {beforeRequest} as unknown as Hooks});
+ beforeRequest[0] = 1;
+ t.deepEqual(a.defaults.options.hooks.beforeRequest, [0] as unknown as BeforeRequestHook[]);
+ t.not(a.defaults.options.hooks.beforeRequest, beforeRequest as unknown as BeforeRequestHook[]);
+});
+
+test('extend keeps the old value if the new one is undefined', t => {
+ const a = got.extend({headers: undefined});
+ t.deepEqual(
+ a.defaults.options.headers,
+ got.defaults.options.headers
+ );
+});
+
+test('hooks are merged on got.extend()', t => {
+ const hooksA = [() => {}];
+ const hooksB = [() => {}];
+
+ const instanceA = got.extend({hooks: {beforeRequest: hooksA}});
+
+ const extended = instanceA.extend({hooks: {beforeRequest: hooksB}});
+ t.deepEqual(extended.defaults.options.hooks.beforeRequest, hooksA.concat(hooksB));
+});
+
+test('custom endpoint with custom headers (extend)', withServer, async (t, server) => {
+ server.all('/', echoHeaders);
+
+ const instance = got.extend({headers: {unicorn: 'rainbow'}, prefixUrl: server.url});
+ const headers = await instance('').json();
+ t.is(headers.unicorn, 'rainbow');
+ t.not(headers['user-agent'], undefined);
+});
+
+test('no tampering with defaults', t => {
+ t.throws(() => {
+ got.defaults.options.prefixUrl = 'http://google.com';
+ });
+
+ t.is(got.defaults.options.prefixUrl, '');
+});
+
+test('can set defaults to `got.mergeOptions(...)`', t => {
+ const instance = got.extend({
+ mutableDefaults: true,
+ followRedirect: false
+ });
+
+ t.notThrows(() => {
+ instance.defaults.options = got.mergeOptions(instance.defaults.options, {
+ followRedirect: true
+ });
+ });
+
+ t.true(instance.defaults.options.followRedirect);
+
+ t.notThrows(() => {
+ instance.defaults.options = got.mergeOptions({});
+ });
+
+ t.is(instance.defaults.options.followRedirect, undefined);
+});
+
+test('can set mutable defaults using got.extend', t => {
+ const instance = got.extend({
+ mutableDefaults: true,
+ followRedirect: false
+ });
+
+ t.notThrows(() => {
+ instance.defaults.options.followRedirect = true;
+ });
+
+ t.true(instance.defaults.options.followRedirect);
+});
+
+test('only plain objects are freezed', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ const instance = got.extend({
+ agent: {
+ http: new HttpAgent({keepAlive: true})
+ },
+ mutableDefaults: true
+ });
+
+ t.notThrows(() => {
+ (instance.defaults.options.agent as any).http.keepAlive = true;
+ });
+});
+
+test('defaults are cloned on instance creation', t => {
+ const options = {foo: 'bar', hooks: {beforeRequest: [() => {}]}};
+ const instance = got.extend(options);
+
+ t.notThrows(() => {
+ options.foo = 'foo';
+ delete options.hooks.beforeRequest[0];
+ });
+
+ // @ts-expect-error This IS correct
+ t.not(options.foo, instance.defaults.options.foo);
+ t.not(options.hooks.beforeRequest, instance.defaults.options.hooks.beforeRequest);
+});
+
+test('ability to pass a custom request method', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ let isCalled = false;
+
+ const request: RequestFunction = (...args: [
+ string | URL | RequestOptions,
+ (RequestOptions | ((response: IncomingMessage) => void))?,
+ ((response: IncomingMessage) => void)?
+ ]) => {
+ isCalled = true;
+ // @ts-expect-error Overload error
+ return httpRequest(...args);
+ };
+
+ const instance = got.extend({request});
+ await instance('');
+
+ t.true(isCalled);
+});
+
+test('does not include the `request` option in normalized `http` options', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ let isCalled = false;
+
+ const request: RequestFunction = (...args: [
+ string | URL | RequestOptions,
+ (RequestOptions | ((response: IncomingMessage) => void))?,
+ ((response: IncomingMessage) => void)?
+ ]) => {
+ isCalled = true;
+
+ t.false(Reflect.has(args[0] as RequestOptions, 'request'));
+
+ // @ts-expect-error Overload error
+ return httpRequest(...args);
+ };
+
+ const instance = got.extend({request});
+ await instance('');
+
+ t.true(isCalled);
+});
+
+test('should pass an options object into an initialization hook after .extend', withServer, async (t, server, got) => {
+ t.plan(1);
+
+ server.get('/', echoHeaders);
+
+ const instance = got.extend({
+ hooks: {
+ init: [
+ options => {
+ t.deepEqual(options, {});
+ }
+ ]
+ }
+ });
+
+ await instance('');
+});
+
+test('hooks aren\'t overriden when merging options', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ let isCalled = false;
+ const instance = got.extend({
+ hooks: {
+ beforeRequest: [
+ () => {
+ isCalled = true;
+ }
+ ]
+ }
+ });
+
+ await instance({});
+
+ t.true(isCalled);
+});
+
+test('extend with custom handlers', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ const instance = got.extend({
+ handlers: [
+ (options, next) => {
+ options.headers.unicorn = 'rainbow';
+ return next(options);
+ }
+ ]
+ });
+ const headers = await instance('').json();
+ t.is(headers.unicorn, 'rainbow');
+});
+
+test('extend with instances', t => {
+ const a = got.extend({prefixUrl: new URL('https://example.com/')});
+ const b = got.extend(a);
+ t.is(b.defaults.options.prefixUrl.toString(), 'https://example.com/');
+});
+
+test('extend with a chain', t => {
+ const a = got.extend({prefixUrl: 'https://example.com/'});
+ const b = got.extend(a, {headers: {foo: 'bar'}});
+ t.is(b.defaults.options.prefixUrl.toString(), 'https://example.com/');
+ t.is(b.defaults.options.headers.foo, 'bar');
+});
+
+test('async handlers', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ const instance = got.extend({
+ handlers: [
+ async (options, next) => {
+ const result = await next(options);
+ // @ts-expect-error Manual tests
+ result.modified = true;
+
+ return result;
+ }
+ ]
+ });
+
+ const promise = instance('');
+ t.true(is.function_(promise.cancel));
+ // @ts-expect-error Manual tests
+ t.true((await promise).modified);
+});
+
+test('async handlers can throw', async t => {
+ const message = 'meh';
+
+ const instance = got.extend({
+ handlers: [
+ async () => {
+ throw new Error(message);
+ }
+ ]
+ });
+
+ await t.throwsAsync(instance('https://example.com'), {message});
+});
+
+test('setting dnsCache to true points to global cache', t => {
+ const a = got.extend({
+ dnsCache: true
+ });
+
+ const b = got.extend({
+ dnsCache: true
+ });
+
+ t.is(a.defaults.options.dnsCache, b.defaults.options.dnsCache);
+});
diff --git a/test/error.js b/test/error.js
deleted file mode 100644
index c8ad3850d..000000000
--- a/test/error.js
+++ /dev/null
@@ -1,56 +0,0 @@
-import test from 'ava';
-import got from '../';
-import {createServer} from './helpers/server';
-
-let s;
-
-test.before('setup', async () => {
- s = await createServer();
-
- s.on('/', (req, res) => {
- res.statusCode = 404;
- res.end('not');
- });
-
- await s.listen(s.port);
-});
-
-test('properties', async t => {
- try {
- await got(s.url);
- t.fail('Exception was not thrown');
- } catch (err) {
- t.truthy(err);
- t.truthy(err.response);
- t.false({}.propertyIsEnumerable.call(err, 'response'));
- t.false({}.hasOwnProperty.call(err, 'code'));
- t.is(err.message, 'Response code 404 (Not Found)');
- t.is(err.host, `${s.host}:${s.port}`);
- t.is(err.method, 'GET');
- }
-});
-
-test('dns message', async t => {
- try {
- await got('.com', {retries: 0});
- t.fail('Exception was not thrown');
- } catch (err) {
- t.truthy(err);
- t.regex(err.message, /getaddrinfo ENOTFOUND/);
- t.is(err.host, '.com');
- t.is(err.method, 'GET');
- }
-});
-
-test('options.body error message', async t => {
- try {
- await got(s.url, {body: () => {}});
- t.fail('Exception was not thrown');
- } catch (err) {
- t.regex(err.message, /options.body must be a ReadableStream, string, Buffer or plain Object/);
- }
-});
-
-test.after('cleanup', async () => {
- await s.close();
-});
diff --git a/test/error.ts b/test/error.ts
new file mode 100644
index 000000000..122c9d773
--- /dev/null
+++ b/test/error.ts
@@ -0,0 +1,326 @@
+import {promisify} from 'util';
+import net = require('net');
+import http = require('http');
+import stream = require('stream');
+import test from 'ava';
+import getStream = require('get-stream');
+import is from '@sindresorhus/is';
+import got, {RequestError, HTTPError, TimeoutError} from '../source';
+import withServer from './helpers/with-server';
+
+const pStreamPipeline = promisify(stream.pipeline);
+
+test('properties', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.statusCode = 404;
+ response.end('not');
+ });
+
+ const url = new URL(server.url);
+
+ const error = await t.throwsAsync(got(''));
+ t.truthy(error);
+ t.truthy(error.response);
+ t.truthy(error.options);
+ t.false({}.propertyIsEnumerable.call(error, 'options'));
+ t.false({}.propertyIsEnumerable.call(error, 'response'));
+ // This fails because of TS 3.7.2 useDefineForClassFields
+ // Class fields will always be initialized, even though they are undefined
+ // A test to check for undefined is in place below
+ // t.false({}.hasOwnProperty.call(error, 'code'));
+ t.is(error.code, 'ERR_NON_2XX_3XX_RESPONSE');
+ t.is(error.message, 'Response code 404 (Not Found)');
+ t.deepEqual(error.options.url, url);
+ t.is(error.response.headers.connection, 'close');
+ t.is(error.response.body, 'not');
+});
+
+test('catches dns errors', async t => {
+ const error = await t.throwsAsync(got('http://doesntexist', {retry: 0}));
+ t.truthy(error);
+ t.regex(error.message, /ENOTFOUND|EAI_AGAIN/);
+ t.is(error.options.url.host, 'doesntexist');
+ t.is(error.options.method, 'GET');
+ t.is(error.code, 'ENOTFOUND');
+});
+
+test('`options.body` form error message', async t => {
+ // @ts-expect-error Error tests
+ await t.throwsAsync(got.post('https://example.com', {body: Buffer.from('test'), form: ''}), {
+ message: 'The `body`, `json` and `form` options are mutually exclusive'
+ });
+});
+
+test('no plain object restriction on json body', withServer, async (t, server, got) => {
+ server.post('/body', async (request, response) => {
+ await pStreamPipeline(request, response);
+ });
+
+ class CustomObject {
+ a = 123;
+ }
+
+ const body = await got.post('body', {json: new CustomObject()}).json();
+
+ t.deepEqual(body, {a: 123});
+});
+
+test('default status message', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.statusCode = 400;
+ response.end('body');
+ });
+
+ const error = await t.throwsAsync(got(''));
+ t.is(error.response.statusCode, 400);
+ t.is(error.response.statusMessage, 'Bad Request');
+});
+
+test('custom status message', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.statusCode = 400;
+ response.statusMessage = 'Something Exploded';
+ response.end('body');
+ });
+
+ const error = await t.throwsAsync(got(''));
+ t.is(error.response.statusCode, 400);
+ t.is(error.response.statusMessage, 'Something Exploded');
+});
+
+test('custom body', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.statusCode = 404;
+ response.end('not');
+ });
+
+ const error = await t.throwsAsync(got(''));
+ t.is(error.response.statusCode, 404);
+ t.is(error.response.body, 'not');
+});
+
+test('contains Got options', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.statusCode = 404;
+ response.end();
+ });
+
+ const options: {agent: false} = {
+ agent: false
+ };
+
+ const error = await t.throwsAsync(got(options));
+ t.is(error.options.agent, options.agent);
+});
+
+test('empty status message is overriden by the default one', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.writeHead(400, '');
+ response.end('body');
+ });
+
+ const error = await t.throwsAsync(got(''));
+ t.is(error.response.statusCode, 400);
+ t.is(error.response.statusMessage, http.STATUS_CODES[400]);
+});
+
+test('`http.request` error', async t => {
+ await t.throwsAsync(got('https://example.com', {
+ request: () => {
+ throw new TypeError('The header content contains invalid characters');
+ }
+ }), {
+ instanceOf: got.RequestError,
+ code: 'ERR_GOT_REQUEST_ERROR',
+ message: 'The header content contains invalid characters'
+ });
+});
+
+test('`http.request` pipe error', async t => {
+ const message = 'snap!';
+
+ await t.throwsAsync(got('https://example.com', {
+ // @ts-expect-error Error tests
+ request: () => {
+ const proxy = new stream.PassThrough();
+
+ const anyProxy = proxy as any;
+ anyProxy.socket = {
+ remoteAddress: '',
+ prependOnceListener: () => {}
+ };
+
+ anyProxy.headers = {};
+
+ anyProxy.abort = () => {};
+
+ proxy.resume();
+ proxy.read = () => {
+ proxy.destroy(new Error(message));
+
+ return null;
+ };
+
+ return proxy;
+ },
+ throwHttpErrors: false
+ }), {
+ instanceOf: got.RequestError,
+ message
+ });
+});
+
+test('`http.request` error through CacheableRequest', async t => {
+ await t.throwsAsync(got('https://example.com', {
+ request: () => {
+ throw new TypeError('The header content contains invalid characters');
+ },
+ cache: new Map()
+ }), {
+ instanceOf: got.RequestError,
+ message: 'The header content contains invalid characters'
+ });
+});
+
+test('normalization errors using convenience methods', async t => {
+ const url = 'undefined/https://example.com';
+ await t.throwsAsync(got(url).json().text().buffer(), {message: `Invalid URL: ${url}`});
+});
+
+test('errors can have request property', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.statusCode = 404;
+ response.end();
+ });
+
+ const error = await t.throwsAsync(got(''));
+
+ t.truthy(error.response);
+ t.truthy(error.request.downloadProgress);
+});
+
+test('promise does not hang on timeout on HTTP error', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.statusCode = 404;
+ response.write('asdf');
+ });
+
+ await t.throwsAsync(got({
+ timeout: 100
+ }), {
+ instanceOf: TimeoutError,
+ code: 'ETIMEDOUT'
+ });
+});
+
+test('no uncaught parse errors', async t => {
+ const server = net.createServer();
+
+ const listen = promisify(server.listen.bind(server));
+ const close = promisify(server.close.bind(server));
+
+ // @ts-expect-error TS is sooo dumb. It doesn't need an argument at all.
+ await listen();
+
+ server.on('connection', socket => {
+ socket.resume();
+ socket.end([
+ 'HTTP/1.1 404 Not Found',
+ 'transfer-encoding: chunked',
+ '',
+ '0',
+ '',
+ ''
+ ].join('\r\n'));
+ });
+
+ await t.throwsAsync(got.head(`http://localhost:${(server.address() as net.AddressInfo).port}`), {
+ message: /^Parse Error/
+ });
+
+ await close();
+});
+
+// Fails randomly on Node 10:
+// Blocked by https://github.com/istanbuljs/nyc/issues/619
+// eslint-disable-next-line ava/no-skip-test
+test.skip('the old stacktrace is recovered', async t => {
+ const error = await t.throwsAsync(got('https://example.com', {
+ request: () => {
+ throw new Error('foobar');
+ }
+ }));
+
+ t.true(error.stack!.includes('at Object.request'));
+
+ // The first `at get` points to where the error was wrapped,
+ // the second `at get` points to the real cause.
+ t.not(error.stack!.indexOf('at get'), error.stack!.lastIndexOf('at get'));
+});
+
+test.serial('custom stack trace', withServer, async (t, _server, got) => {
+ const ErrorCaptureStackTrace = Error.captureStackTrace;
+
+ const enable = () => {
+ Error.captureStackTrace = (target: {stack: any}) => {
+ target.stack = [
+ 'line 1',
+ 'line 2'
+ ];
+ };
+ };
+
+ const disable = () => {
+ Error.captureStackTrace = ErrorCaptureStackTrace;
+ };
+
+ // Node.js default behavior
+ {
+ const stream = got.stream('');
+ stream.destroy(new Error('oh no'));
+
+ const caught = await t.throwsAsync(getStream(stream));
+ t.is(is(caught.stack), 'string');
+ }
+
+ // Passing a custom error
+ {
+ enable();
+ const error = new Error('oh no');
+ disable();
+
+ const stream = got.stream('');
+ stream.destroy(error);
+
+ const caught = await t.throwsAsync(getStream(stream));
+ t.is(is(caught.stack), 'string');
+ }
+
+ // Custom global behavior
+ {
+ enable();
+ const error = new Error('oh no');
+
+ const stream = got.stream('');
+ stream.destroy(error);
+
+ const caught = await t.throwsAsync(getStream(stream));
+ t.is(is(caught.stack), 'Array');
+
+ disable();
+ }
+
+ // Passing a default error that needs some processing
+ {
+ const error = new Error('oh no');
+ enable();
+
+ const stream = got.stream('');
+ stream.destroy(error);
+
+ const caught = await t.throwsAsync(getStream(stream));
+ t.is(is(caught.stack), 'Array');
+
+ disable();
+ }
+});
diff --git a/test/fixtures/ok b/test/fixtures/ok
new file mode 100644
index 000000000..9766475a4
--- /dev/null
+++ b/test/fixtures/ok
@@ -0,0 +1 @@
+ok
diff --git a/test/fixtures/stream-content-length b/test/fixtures/stream-content-length
new file mode 100644
index 000000000..068915a85
--- /dev/null
+++ b/test/fixtures/stream-content-length
@@ -0,0 +1 @@
+Unicorns
diff --git a/test/gzip.js b/test/gzip.js
deleted file mode 100644
index 183153cf3..000000000
--- a/test/gzip.js
+++ /dev/null
@@ -1,83 +0,0 @@
-import zlib from 'zlib';
-import test from 'ava';
-import getStream from 'get-stream';
-import got from '../';
-import {createServer} from './helpers/server';
-
-const testContent = 'Compressible response content.\n';
-
-let s;
-
-test.before('setup', async () => {
- s = await createServer();
-
- s.on('/', (req, res) => {
- res.statusCode = 200;
- res.setHeader('Content-Type', 'text/plain');
- res.setHeader('Content-Encoding', 'gzip');
-
- if (req.method === 'HEAD') {
- res.end();
- return;
- }
-
- zlib.gzip(testContent, (_, data) => res.end(data));
- });
-
- s.on('/corrupted', (req, res) => {
- res.statusCode = 200;
- res.setHeader('Content-Type', 'text/plain');
- res.setHeader('Content-Encoding', 'gzip');
- res.end('Not gzipped content');
- });
-
- s.on('/missing-data', (req, res) => {
- res.statusCode = 200;
- res.setHeader('Content-Type', 'text/plain');
- res.setHeader('Content-Encoding', 'gzip');
- zlib.gzip(testContent, (_, data) => res.end(data.slice(0, -1)));
- });
-
- await s.listen(s.port);
-});
-
-test('decompress content', async t => {
- t.is((await got(s.url)).body, testContent);
-});
-
-test('decompress content - stream', async t => {
- t.is(await getStream(got.stream(s.url)), testContent);
-});
-
-test('handles gzip error', async t => {
- try {
- await got(`${s.url}/corrupted`);
- t.fail('Exception was not thrown');
- } catch (err) {
- t.is(err.message, 'incorrect header check');
- t.is(err.path, '/corrupted');
- t.is(err.name, 'ReadError');
- }
-});
-
-test('preserve headers property', async t => {
- t.truthy((await got(s.url)).headers);
-});
-
-test('do not break HEAD responses', async t => {
- t.is((await got.head(s.url)).body, '');
-});
-
-test('ignore missing data', async t => {
- t.is((await got(`${s.url}/missing-data`)).body, testContent);
-});
-
-test('has url and requestUrl properties', async t => {
- const res = await got(s.url);
- t.truthy(res.url);
- t.truthy(res.requestUrl);
-});
-
-test.after('cleanup', async () => {
- await s.close();
-});
diff --git a/test/gzip.ts b/test/gzip.ts
new file mode 100644
index 000000000..06d680a1f
--- /dev/null
+++ b/test/gzip.ts
@@ -0,0 +1,139 @@
+import {promisify} from 'util';
+import zlib = require('zlib');
+import test from 'ava';
+import getStream = require('get-stream');
+import withServer from './helpers/with-server';
+import {HTTPError, ReadError} from '../source';
+
+const testContent = 'Compressible response content.\n';
+const testContentUncompressed = 'Uncompressed response content.\n';
+
+let gzipData: Buffer;
+test.before('setup', async () => {
+ gzipData = await promisify(zlib.gzip)(testContent);
+});
+
+test('decompress content', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.setHeader('Content-Encoding', 'gzip');
+ response.end(gzipData);
+ });
+
+ t.is((await got('')).body, testContent);
+});
+
+test('decompress content on error', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.setHeader('Content-Encoding', 'gzip');
+ response.status(404);
+ response.end(gzipData);
+ });
+
+ const error = await t.throwsAsync(got(''));
+
+ t.is(error.response.body, testContent);
+});
+
+test('decompress content - stream', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.setHeader('Content-Encoding', 'gzip');
+ response.end(gzipData);
+ });
+
+ t.is((await getStream(got.stream(''))), testContent);
+});
+
+test('handles gzip error', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.setHeader('Content-Encoding', 'gzip');
+ response.end('Not gzipped content');
+ });
+
+ await t.throwsAsync(got(''), {
+ name: 'ReadError',
+ message: 'incorrect header check'
+ });
+});
+
+test('no unhandled `Premature close` error', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.setHeader('Content-Encoding', 'gzip');
+ response.write('Not gzipped content');
+ });
+
+ await t.throwsAsync(got(''), {
+ name: 'ReadError',
+ message: 'incorrect header check'
+ });
+});
+
+test('handles gzip error - stream', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.setHeader('Content-Encoding', 'gzip');
+ response.end('Not gzipped content');
+ });
+
+ await t.throwsAsync(getStream(got.stream('')), {
+ name: 'ReadError',
+ message: 'incorrect header check'
+ });
+});
+
+test('decompress option opts out of decompressing', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.setHeader('Content-Encoding', 'gzip');
+ response.end(gzipData);
+ });
+
+ const {body} = await got({decompress: false, responseType: 'buffer'});
+ t.is(Buffer.compare(body, gzipData), 0);
+});
+
+test('decompress option doesn\'t alter encoding of uncompressed responses', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end(testContentUncompressed);
+ });
+
+ const {body} = await got({decompress: false});
+ t.is(body, testContentUncompressed);
+});
+
+test('preserves `headers` property', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.setHeader('Content-Encoding', 'gzip');
+ response.end(gzipData);
+ });
+
+ t.truthy((await got('')).headers);
+});
+
+test('does not break HEAD responses', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end();
+ });
+
+ t.is((await got.head('')).body, '');
+});
+
+test('does not ignore missing data', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.setHeader('Content-Encoding', 'gzip');
+ response.end(gzipData.slice(0, -1));
+ });
+
+ await t.throwsAsync(got(''), {
+ instanceOf: ReadError,
+ message: 'unexpected end of file'
+ });
+});
+
+test('response has `url` and `requestUrl` properties', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.setHeader('Content-Encoding', 'gzip');
+ response.end(gzipData);
+ });
+
+ const response = await got('');
+ t.truthy(response.url);
+ t.truthy(response.requestUrl);
+});
diff --git a/test/headers.js b/test/headers.js
deleted file mode 100644
index 3eb8b381a..000000000
--- a/test/headers.js
+++ /dev/null
@@ -1,94 +0,0 @@
-import test from 'ava';
-import FormData from 'form-data';
-import got from '../';
-import pkg from '../package';
-import {createServer} from './helpers/server';
-
-let s;
-
-test.before('setup', async () => {
- s = await createServer();
-
- s.on('/', (req, res) => {
- req.resume();
- res.end(JSON.stringify(req.headers));
- });
-
- await s.listen(s.port);
-});
-
-test('user-agent', async t => {
- const headers = (await got(s.url, {json: true})).body;
- t.is(headers['user-agent'], `${pkg.name}/${pkg.version} (https://github.com/sindresorhus/got)`);
-});
-
-test('accept-encoding', async t => {
- const headers = (await got(s.url, {json: true})).body;
- t.is(headers['accept-encoding'], 'gzip,deflate');
-});
-
-test('accept header with json option', async t => {
- let headers = (await got(s.url, {json: true})).body;
- t.is(headers.accept, 'application/json');
-
- headers = (await got(s.url, {
- headers: {
- accept: ''
- },
- json: true
- })).body;
- t.is(headers.accept, '');
-});
-
-test('host', async t => {
- const headers = (await got(s.url, {json: true})).body;
- t.is(headers.host, `localhost:${s.port}`);
-});
-
-test('transform names to lowercase', async t => {
- const headers = (await got(s.url, {
- headers: {
- 'USER-AGENT': 'test'
- },
- json: true
- })).body;
- t.is(headers['user-agent'], 'test');
-});
-
-test('zero content-length', async t => {
- const headers = (await got(s.url, {
- headers: {
- 'content-length': 0
- },
- body: 'sup',
- json: true
- })).body;
- t.is(headers['content-length'], '0');
-});
-
-test('form-data manual content-type', async t => {
- const form = new FormData();
- form.append('a', 'b');
- const headers = (await got(s.url, {
- headers: {
- 'content-type': 'custom'
- },
- body: form,
- json: true
- })).body;
- t.is(headers['content-type'], 'custom');
-});
-
-test('form-data automatic content-type', async t => {
- const form = new FormData();
- form.append('a', 'b');
- const headers = (await got(s.url, {
- body: form,
- json: true
- })).body;
- t.is(headers['content-type'], `multipart/form-data; boundary=${form.getBoundary()}`);
-});
-
-test.after('cleanup', async () => {
- await s.close();
-});
diff --git a/test/headers.ts b/test/headers.ts
new file mode 100644
index 000000000..20107a6fa
--- /dev/null
+++ b/test/headers.ts
@@ -0,0 +1,261 @@
+import fs = require('fs');
+import {promisify} from 'util';
+import path = require('path');
+import test from 'ava';
+import {Handler} from 'express';
+import FormData = require('form-data');
+import got, {Headers} from '../source';
+import withServer from './helpers/with-server';
+
+const supportsBrotli = typeof (process.versions as any).brotli === 'string';
+
+const echoHeaders: Handler = (request, response) => {
+ request.resume();
+ response.end(JSON.stringify(request.headers));
+};
+
+test('`user-agent`', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ const headers = await got('').json();
+ t.is(headers['user-agent'], 'got (https://github.com/sindresorhus/got)');
+});
+
+test('`accept-encoding`', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ const headers = await got('').json();
+ t.is(headers['accept-encoding'], supportsBrotli ? 'gzip, deflate, br' : 'gzip, deflate');
+});
+
+test('does not override provided `accept-encoding`', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ const headers = await got({
+ headers: {
+ 'accept-encoding': 'gzip'
+ }
+ }).json();
+ t.is(headers['accept-encoding'], 'gzip');
+});
+
+test('does not remove user headers from `url` object argument', withServer, async (t, server) => {
+ server.get('/', echoHeaders);
+
+ const headers = (await got({
+ url: `http://${server.hostname}:${server.port}`,
+ responseType: 'json',
+ headers: {
+ 'X-Request-Id': 'value'
+ }
+ })).body;
+
+ t.is(headers.accept, 'application/json');
+ t.is(headers['user-agent'], 'got (https://github.com/sindresorhus/got)');
+ t.is(headers['accept-encoding'], supportsBrotli ? 'gzip, deflate, br' : 'gzip, deflate');
+ t.is(headers['x-request-id'], 'value');
+});
+
+test('does not set `accept-encoding` header when `options.decompress` is false', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ const headers = await got({
+ decompress: false
+ }).json();
+ // @ts-expect-error Error tests
+ t.false(Reflect.has(headers, 'accept-encoding'));
+});
+
+test('`accept` header with `json` option', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ let headers = await got('').json();
+ t.is(headers.accept, 'application/json');
+
+ headers = await got({
+ headers: {
+ accept: ''
+ }
+ }).json();
+ t.is(headers.accept, '');
+});
+
+test('`host` header', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ const headers = await got('').json();
+ t.is(headers.host, `localhost:${server.port}`);
+});
+
+test('transforms names to lowercase', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ const headers = (await got({
+ headers: {
+ 'ACCEPT-ENCODING': 'identity'
+ },
+ responseType: 'json'
+ })).body;
+ t.is(headers['accept-encoding'], 'identity');
+});
+
+test('setting `content-length` to 0', withServer, async (t, server, got) => {
+ server.post('/', echoHeaders);
+
+ const {body} = await got.post({
+ headers: {
+ 'content-length': '0'
+ },
+ body: 'sup'
+ });
+ const headers = JSON.parse(body);
+ t.is(headers['content-length'], '0');
+});
+
+test('sets `content-length` to `0` when requesting PUT with empty body', withServer, async (t, server, got) => {
+ server.put('/', echoHeaders);
+
+ const {body} = await got({
+ method: 'PUT'
+ });
+ const headers = JSON.parse(body);
+ t.is(headers['content-length'], '0');
+});
+
+test('form manual `content-type` header', withServer, async (t, server, got) => {
+ server.post('/', echoHeaders);
+
+ const {body} = await got.post({
+ headers: {
+ 'content-type': 'custom'
+ },
+ form: {
+ a: 1
+ }
+ });
+ const headers = JSON.parse(body);
+ t.is(headers['content-type'], 'custom');
+});
+
+test('form-data manual `content-type` header', withServer, async (t, server, got) => {
+ server.post('/', echoHeaders);
+
+ const form = new FormData();
+ form.append('a', 'b');
+ const {body} = await got.post({
+ headers: {
+ 'content-type': 'custom'
+ },
+ body: form
+ });
+ const headers = JSON.parse(body);
+ t.is(headers['content-type'], 'custom');
+});
+
+test('form-data automatic `content-type` header', withServer, async (t, server, got) => {
+ server.post('/', echoHeaders);
+
+ const form = new FormData();
+ form.append('a', 'b');
+ const {body} = await got.post({
+ body: form
+ });
+ const headers = JSON.parse(body);
+ t.is(headers['content-type'], `multipart/form-data; boundary=${form.getBoundary()}`);
+});
+
+test('form-data sets `content-length` header', withServer, async (t, server, got) => {
+ server.post('/', echoHeaders);
+
+ const form = new FormData();
+ form.append('a', 'b');
+ const {body} = await got.post({body: form});
+ const headers = JSON.parse(body);
+ t.is(headers['content-length'], '157');
+});
+
+test('stream as `options.body` sets `content-length` header', withServer, async (t, server, got) => {
+ server.post('/', echoHeaders);
+
+ const fixture = path.resolve('test/fixtures/stream-content-length');
+ const {size} = await promisify(fs.stat)(fixture);
+ const {body} = await got.post({
+ body: fs.createReadStream(fixture)
+ });
+ const headers = JSON.parse(body);
+ t.is(Number(headers['content-length']), size);
+});
+
+test('buffer as `options.body` sets `content-length` header', withServer, async (t, server, got) => {
+ server.post('/', echoHeaders);
+
+ const buffer = Buffer.from('unicorn');
+ const {body} = await got.post({
+ body: buffer
+ });
+ const headers = JSON.parse(body);
+ t.is(Number(headers['content-length']), buffer.length);
+});
+
+test('throws on null value headers', async t => {
+ await t.throwsAsync(got({
+ url: 'https://example.com',
+ headers: {
+ // @ts-expect-error Testing purposes
+ 'user-agent': null
+ }
+ }), {
+ message: 'Use `undefined` instead of `null` to delete the `user-agent` header'
+ });
+});
+
+test('removes undefined value headers', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ const {body} = await got({
+ headers: {
+ 'user-agent': undefined
+ }
+ });
+ const headers = JSON.parse(body);
+ t.is(headers['user-agent'], undefined);
+});
+
+test('non-existent headers set to undefined are omitted', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ const {body} = await got({
+ headers: {
+ blah: undefined
+ }
+ });
+ const headers = JSON.parse(body);
+ t.false(Reflect.has(headers, 'blah'));
+});
+
+test('preserve port in host header if non-standard port', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ const body = await got('').json();
+ t.is(body.host, `localhost:${server.port}`);
+});
+
+test('strip port in host header if explicit standard port (:80) & protocol (HTTP)', async t => {
+ const body = await got('http://httpbin.org:80/headers').json<{headers: Headers}>();
+ t.is(body.headers.Host, 'httpbin.org');
+});
+
+test('strip port in host header if explicit standard port (:443) & protocol (HTTPS)', async t => {
+ const body = await got('https://httpbin.org:443/headers').json<{headers: Headers}>();
+ t.is(body.headers.Host, 'httpbin.org');
+});
+
+test('strip port in host header if implicit standard port & protocol (HTTP)', async t => {
+ const body = await got('http://httpbin.org/headers').json<{headers: Headers}>();
+ t.is(body.headers.Host, 'httpbin.org');
+});
+
+test('strip port in host header if implicit standard port & protocol (HTTPS)', async t => {
+ const body = await got('https://httpbin.org/headers').json<{headers: Headers}>();
+ t.is(body.headers.Host, 'httpbin.org');
+});
diff --git a/test/helpers.js b/test/helpers.js
deleted file mode 100644
index 778a6dbbd..000000000
--- a/test/helpers.js
+++ /dev/null
@@ -1,42 +0,0 @@
-import test from 'ava';
-import got from '../';
-import {createServer} from './helpers/server';
-
-let s;
-
-test.before('setup', async () => {
- s = await createServer();
-
- s.on('/', (req, res) => {
- res.end('ok');
- });
-
- s.on('/404', (req, res) => {
- res.statusCode = 404;
- res.end('not found');
- });
-
- await s.listen(s.port);
-});
-
-test('promise mode', async t => {
- t.is((await got.get(s.url)).body, 'ok');
-
- try {
- await got.get(`${s.url}/404`);
- t.fail('Exception was not thrown');
- } catch (err) {
- t.is(err.response.body, 'not found');
- }
-
- try {
- await got.get('.com', {retries: 0});
- t.fail('Exception was not thrown');
- } catch (err) {
- t.truthy(err);
- }
-});
-
-test.after('cleanup', async () => {
- await s.close();
-});
diff --git a/test/helpers.ts b/test/helpers.ts
new file mode 100644
index 000000000..15bb4c266
--- /dev/null
+++ b/test/helpers.ts
@@ -0,0 +1,21 @@
+import test from 'ava';
+import got, {HTTPError} from '../source';
+import withServer from './helpers/with-server';
+
+test('works', withServer, async (t, server) => {
+ server.get('/', (_request, response) => {
+ response.end('ok');
+ });
+
+ server.get('/404', (_request, response) => {
+ response.statusCode = 404;
+ response.end('not found');
+ });
+
+ t.is((await got.get(server.url)).body, 'ok');
+
+ const error = await t.throwsAsync(got.get(`${server.url}/404`), {instanceOf: HTTPError});
+ t.is(error.response.body, 'not found');
+
+ await t.throwsAsync(got.get('.com', {retry: 0}), {message: 'Invalid URL: .com'});
+});
diff --git a/test/helpers/create-http-test-server.ts b/test/helpers/create-http-test-server.ts
new file mode 100644
index 000000000..bf3446a52
--- /dev/null
+++ b/test/helpers/create-http-test-server.ts
@@ -0,0 +1,43 @@
+import http = require('http');
+import net = require('net');
+import express = require('express');
+import pify = require('pify');
+import bodyParser = require('body-parser');
+
+export type HttpServerOptions = {
+ bodyParser?: express.NextFunction | false;
+};
+
+export interface ExtendedHttpTestServer extends express.Express {
+ http: http.Server;
+ url: string;
+ port: number;
+ hostname: string;
+ close: () => Promise;
+}
+
+const createHttpTestServer = async (options: HttpServerOptions = {}): Promise => {
+ const server = express() as ExtendedHttpTestServer;
+ server.http = http.createServer(server);
+
+ server.set('etag', false);
+
+ if (options.bodyParser !== false) {
+ server.use(bodyParser.json({limit: '1mb', type: 'application/json', ...options.bodyParser}));
+ server.use(bodyParser.text({limit: '1mb', type: 'text/plain', ...options.bodyParser}));
+ server.use(bodyParser.urlencoded({limit: '1mb', type: 'application/x-www-form-urlencoded', extended: true, ...options.bodyParser}));
+ server.use(bodyParser.raw({limit: '1mb', type: 'application/octet-stream', ...options.bodyParser}));
+ }
+
+ await pify(server.http.listen.bind(server.http))();
+
+ server.port = (server.http.address() as net.AddressInfo).port;
+ server.url = `http://localhost:${(server.port)}`;
+ server.hostname = 'localhost';
+
+ server.close = async () => pify(server.http.close.bind(server.http))();
+
+ return server;
+};
+
+export default createHttpTestServer;
diff --git a/test/helpers/create-https-test-server.ts b/test/helpers/create-https-test-server.ts
new file mode 100644
index 000000000..4dc8a12d9
--- /dev/null
+++ b/test/helpers/create-https-test-server.ts
@@ -0,0 +1,71 @@
+import https = require('https');
+import net = require('net');
+import express = require('express');
+import pify = require('pify');
+import pem = require('pem');
+
+export type HttpsServerOptions = {
+ commonName?: string;
+ days?: number;
+};
+
+export interface ExtendedHttpsTestServer extends express.Express {
+ https: https.Server;
+ caKey: Buffer;
+ caCert: Buffer;
+ url: string;
+ port: number;
+ close: () => Promise;
+}
+
+const createHttpsTestServer = async (options: HttpsServerOptions = {}): Promise => {
+ const createCSR = pify(pem.createCSR);
+ const createCertificate = pify(pem.createCertificate);
+
+ const caCSRResult = await createCSR({commonName: 'authority'});
+ const caResult = await createCertificate({
+ csr: caCSRResult.csr,
+ clientKey: caCSRResult.clientKey,
+ selfSigned: true
+ });
+ const caKey = caResult.clientKey;
+ const caCert = caResult.certificate;
+
+ const serverCSRResult = await createCSR({commonName: options.commonName ?? 'localhost'});
+ const serverResult = await createCertificate({
+ csr: serverCSRResult.csr,
+ clientKey: serverCSRResult.clientKey,
+ serviceKey: caKey,
+ serviceCertificate: caCert,
+ days: options.days ?? 365
+ });
+ const serverKey = serverResult.clientKey;
+ const serverCert = serverResult.certificate;
+
+ const server = express() as ExtendedHttpsTestServer;
+ server.https = https.createServer(
+ {
+ key: serverKey,
+ cert: serverCert,
+ ca: caCert,
+ requestCert: true,
+ rejectUnauthorized: false // This should be checked by the test
+ },
+ server
+ );
+
+ server.set('etag', false);
+
+ await pify(server.https.listen.bind(server.https))();
+
+ server.caKey = caKey;
+ server.caCert = caCert;
+ server.port = (server.https.address() as net.AddressInfo).port;
+ server.url = `https://localhost:${(server.port)}`;
+
+ server.close = async () => pify(server.https.close.bind(server.https))();
+
+ return server;
+};
+
+export default createHttpsTestServer;
diff --git a/test/helpers/server.js b/test/helpers/server.js
deleted file mode 100644
index 0a9c1c7fb..000000000
--- a/test/helpers/server.js
+++ /dev/null
@@ -1,39 +0,0 @@
-'use strict';
-const http = require('http');
-const https = require('https');
-const pify = require('pify');
-const getPort = require('get-port');
-
-const host = exports.host = 'localhost';
-
-exports.createServer = function () {
- return getPort().then(port => {
- const s = http.createServer((req, resp) => s.emit(req.url, req, resp));
-
- s.host = host;
- s.port = port;
- s.url = `http://${host}:${port}`;
- s.protocol = 'http';
-
- s.listen = pify(s.listen, Promise);
- s.close = pify(s.close, Promise);
-
- return s;
- });
-};
-
-exports.createSSLServer = function (opts) {
- return getPort().then(port => {
- const s = https.createServer(opts, (req, resp) => s.emit(req.url, req, resp));
-
- s.host = host;
- s.port = port;
- s.url = `https://${host}:${port}`;
- s.protocol = 'https';
-
- s.listen = pify(s.listen, Promise);
- s.close = pify(s.close, Promise);
-
- return s;
- });
-};
diff --git a/test/helpers/slow-data-stream.ts b/test/helpers/slow-data-stream.ts
new file mode 100644
index 000000000..3f8c592e8
--- /dev/null
+++ b/test/helpers/slow-data-stream.ts
@@ -0,0 +1,17 @@
+import {Readable} from 'stream';
+import {Clock} from '@sinonjs/fake-timers';
+
+export default (clock: Clock): Readable => {
+ let i = 0;
+
+ return new Readable({
+ read() {
+ if (i++ < 10) {
+ this.push('data\n'.repeat(100));
+ clock.tick(100);
+ } else {
+ this.push(null);
+ }
+ }
+ });
+};
diff --git a/test/helpers/types.ts b/test/helpers/types.ts
new file mode 100644
index 000000000..7c38421e5
--- /dev/null
+++ b/test/helpers/types.ts
@@ -0,0 +1,15 @@
+import {Server} from 'http';
+import {TestServer} from 'create-test-server';
+import * as FakeTimers from '@sinonjs/fake-timers';
+
+export interface ExtendedHttpServer extends Server {
+ socketPath: string;
+}
+
+export interface ExtendedTestServer extends TestServer {
+ hostname: string;
+ sslHostname: string;
+}
+
+export type InstalledClock = ReturnType;
+export type GlobalClock = InstalledClock | FakeTimers.NodeClock;
diff --git a/test/helpers/with-server.ts b/test/helpers/with-server.ts
new file mode 100644
index 000000000..5fb3cfc75
--- /dev/null
+++ b/test/helpers/with-server.ts
@@ -0,0 +1,122 @@
+import {promisify} from 'util';
+import * as test from 'ava';
+import is from '@sindresorhus/is';
+import http = require('http');
+import tempy = require('tempy');
+import createHttpsTestServer, {ExtendedHttpsTestServer, HttpsServerOptions} from './create-https-test-server';
+import createHttpTestServer, {ExtendedHttpTestServer, HttpServerOptions} from './create-http-test-server';
+import FakeTimers = require('@sinonjs/fake-timers');
+import got, {InstanceDefaults, Got} from '../../source';
+import {ExtendedHttpServer, GlobalClock, InstalledClock} from './types';
+
+export type RunTestWithServer = (t: test.ExecutionContext, server: ExtendedHttpTestServer, got: Got, clock: GlobalClock) => Promise | void;
+export type RunTestWithHttpsServer = (t: test.ExecutionContext, server: ExtendedHttpsTestServer, got: Got, fakeTimer?: GlobalClock) => Promise | void;
+export type RunTestWithSocket = (t: test.ExecutionContext, server: ExtendedHttpServer) => Promise | void;
+
+const generateHook = ({install, options: testServerOptions}: {install?: boolean; options?: HttpServerOptions}): test.Macro<[RunTestWithServer]> => async (t, run) => {
+ const clock = install ? FakeTimers.install() : FakeTimers.createClock() as GlobalClock;
+
+ // Re-enable body parsing to investigate https://github.com/sindresorhus/got/issues/1186
+ const server = await createHttpTestServer(is.plainObject(testServerOptions) ? testServerOptions : {
+ bodyParser: {
+ type: () => false
+ } as any
+ });
+
+ const options: InstanceDefaults = {
+ // @ts-expect-error Augmenting for test detection
+ avaTest: t.title,
+ handlers: [
+ (options, next) => {
+ const result = next(options);
+
+ clock.tick(0);
+
+ // @ts-expect-error FIXME: Incompatible union type signatures
+ result.on('response', () => {
+ clock.tick(0);
+ });
+
+ return result;
+ }
+ ]
+ };
+
+ const preparedGot = got.extend({prefixUrl: server.url, ...options});
+
+ try {
+ await run(t, server, preparedGot, clock);
+ } finally {
+ await server.close();
+ }
+
+ if (install) {
+ (clock as InstalledClock).uninstall();
+ }
+};
+
+export const withBodyParsingServer = generateHook({install: false, options: {}});
+export default generateHook({install: false});
+
+export const withServerAndFakeTimers = generateHook({install: true});
+
+const generateHttpsHook = (options?: HttpsServerOptions, installFakeTimer = false): test.Macro<[RunTestWithHttpsServer]> => async (t, run) => {
+ const fakeTimer = installFakeTimer ? FakeTimers.install() as GlobalClock : undefined;
+
+ const server = await createHttpsTestServer(options);
+
+ const preparedGot = got.extend({
+ // @ts-expect-error Augmenting for test detection
+ avaTest: t.title,
+ handlers: [
+ (options, next) => {
+ const result = next(options);
+
+ fakeTimer?.tick(0);
+
+ // @ts-expect-error FIXME: Incompatible union type signatures
+ result.on('response', () => {
+ fakeTimer?.tick(0);
+ });
+
+ return result;
+ }
+ ],
+ prefixUrl: server.url,
+ https: {
+ certificateAuthority: (server as any).caCert,
+ rejectUnauthorized: true
+ }
+ });
+
+ try {
+ await run(t, server, preparedGot, fakeTimer);
+ } finally {
+ await server.close();
+ }
+
+ if (installFakeTimer) {
+ (fakeTimer as InstalledClock).uninstall();
+ }
+};
+
+export const withHttpsServer = generateHttpsHook;
+
+// TODO: remove this when `create-test-server` supports custom listen
+export const withSocketServer: test.Macro<[RunTestWithSocket]> = async (t, run) => {
+ const socketPath = tempy.file({extension: 'socket'});
+
+ const server = http.createServer((request, response) => {
+ server.emit(request.url!, request, response);
+ }) as ExtendedHttpServer;
+
+ server.socketPath = socketPath;
+
+ await promisify(server.listen.bind(server))(socketPath);
+
+ try {
+ await run(t, server);
+ } finally {
+ await promisify(server.close.bind(server))();
+ }
+};
diff --git a/test/hooks.ts b/test/hooks.ts
new file mode 100644
index 000000000..adb8d19bf
--- /dev/null
+++ b/test/hooks.ts
@@ -0,0 +1,1256 @@
+import {URL} from 'url';
+import {Agent as HttpAgent} from 'http';
+import test, {Constructor} from 'ava';
+import nock = require('nock');
+import getStream = require('get-stream');
+import FormData = require('form-data');
+import sinon = require('sinon');
+import delay = require('delay');
+import {Handler} from 'express';
+import Responselike = require('responselike');
+import got, {RequestError, HTTPError, Response} from '../source';
+import withServer from './helpers/with-server';
+
+const errorString = 'oops';
+const error = new Error(errorString);
+
+const echoHeaders: Handler = (request, response) => {
+ response.end(JSON.stringify(request.headers));
+};
+
+const echoBody: Handler = async (request, response) => {
+ response.end(await getStream(request));
+};
+
+const echoUrl: Handler = (request, response) => {
+ response.end(request.url);
+};
+
+const retryEndpoint: Handler = (request, response) => {
+ if (request.headers.foo) {
+ response.statusCode = 302;
+ response.setHeader('location', '/');
+ response.end();
+ }
+
+ response.statusCode = 500;
+ response.end();
+};
+
+const redirectEndpoint: Handler = (_request, response) => {
+ response.statusCode = 302;
+ response.setHeader('location', '/');
+ response.end();
+};
+
+const createAgentSpy = (AgentClass: Constructor): {agent: T; spy: sinon.SinonSpy} => {
+ const agent: T = new AgentClass({keepAlive: true});
+ // @ts-expect-error This IS correct
+ const spy = sinon.spy(agent, 'addRequest');
+ return {agent, spy};
+};
+
+test('async hooks', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ const {body} = await got>({
+ responseType: 'json',
+ hooks: {
+ beforeRequest: [
+ async options => {
+ await delay(100);
+ options.headers.foo = 'bar';
+ }
+ ]
+ }
+ });
+ t.is(body.foo, 'bar');
+});
+
+test('catches init thrown errors', async t => {
+ await t.throwsAsync(got('https://example.com', {
+ hooks: {
+ init: [() => {
+ throw error;
+ }]
+ }
+ }), {
+ instanceOf: RequestError,
+ message: errorString
+ });
+});
+
+test('passes init thrown errors to beforeError hooks (promise-only)', async t => {
+ t.plan(2);
+
+ await t.throwsAsync(got('https://example.com', {
+ hooks: {
+ init: [() => {
+ throw error;
+ }],
+ beforeError: [error => {
+ t.is(error.message, errorString);
+
+ return error;
+ }]
+ }
+ }), {
+ instanceOf: RequestError,
+ message: errorString
+ });
+});
+
+test('passes init thrown errors to beforeError hooks (promise-only) - beforeError rejection', async t => {
+ const message = 'foo, bar!';
+
+ await t.throwsAsync(got('https://example.com', {
+ hooks: {
+ init: [() => {
+ throw error;
+ }],
+ beforeError: [() => {
+ throw new Error(message);
+ }]
+ }
+ }), {message});
+});
+
+test('catches beforeRequest thrown errors', async t => {
+ await t.throwsAsync(got('https://example.com', {
+ hooks: {
+ beforeRequest: [() => {
+ throw error;
+ }]
+ }
+ }), {
+ instanceOf: RequestError,
+ message: errorString
+ });
+});
+
+test('catches beforeRedirect thrown errors', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+ server.get('/redirect', redirectEndpoint);
+
+ await t.throwsAsync(got('redirect', {
+ hooks: {
+ beforeRedirect: [() => {
+ throw error;
+ }]
+ }
+ }), {
+ instanceOf: RequestError,
+ message: errorString
+ });
+});
+
+test('catches beforeRetry thrown errors', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+ server.get('/retry', retryEndpoint);
+
+ await t.throwsAsync(got('retry', {
+ hooks: {
+ beforeRetry: [() => {
+ throw error;
+ }]
+ }
+ }), {
+ instanceOf: RequestError,
+ message: errorString
+ });
+});
+
+test('catches afterResponse thrown errors', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ await t.throwsAsync(got({
+ hooks: {
+ afterResponse: [() => {
+ throw error;
+ }]
+ }
+ }), {
+ instanceOf: RequestError,
+ message: errorString
+ });
+});
+
+test('accepts an async function as init hook', async t => {
+ await got('https://example.com', {
+ hooks: {
+ init: [
+ async () => {
+ t.pass();
+ }
+ ]
+ }
+ });
+});
+
+test('catches beforeRequest promise rejections', async t => {
+ await t.throwsAsync(got('https://example.com', {
+ hooks: {
+ beforeRequest: [
+ async () => {
+ throw error;
+ }
+ ]
+ }
+ }), {
+ instanceOf: RequestError,
+ message: errorString
+ });
+});
+
+test('catches beforeRedirect promise rejections', withServer, async (t, server, got) => {
+ server.get('/', redirectEndpoint);
+
+ await t.throwsAsync(got({
+ hooks: {
+ beforeRedirect: [
+ async () => {
+ throw error;
+ }
+ ]
+ }
+ }), {
+ instanceOf: RequestError,
+ message: errorString
+ });
+});
+
+test('catches beforeRetry promise rejections', withServer, async (t, server, got) => {
+ server.get('/retry', retryEndpoint);
+
+ await t.throwsAsync(got('retry', {
+ hooks: {
+ beforeRetry: [
+ async () => {
+ throw error;
+ }
+ ]
+ }
+ }), {
+ instanceOf: RequestError,
+ message: errorString
+ });
+});
+
+test('catches afterResponse promise rejections', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ await t.throwsAsync(got({
+ hooks: {
+ afterResponse: [
+ async () => {
+ throw error;
+ }
+ ]
+ }
+ }), {message: errorString});
+});
+
+test('catches beforeError errors', async t => {
+ await t.throwsAsync(got('https://example.com', {
+ request: () => {
+ throw new Error('No way');
+ },
+ hooks: {
+ beforeError: [
+ async () => {
+ throw error;
+ }
+ ]
+ }
+ }), {message: errorString});
+});
+
+test('init is called with options', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ const context = {};
+
+ await got({
+ hooks: {
+ init: [
+ options => {
+ t.is(options.context, context);
+ }
+ ]
+ },
+ context
+ });
+});
+
+test('init from defaults is called with options', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ const context = {};
+
+ const instance = got.extend({
+ hooks: {
+ init: [
+ options => {
+ t.is(options.context, context);
+ }
+ ]
+ }
+ });
+
+ await instance({context});
+});
+
+test('init allows modifications', withServer, async (t, server, got) => {
+ server.get('/', (request, response) => {
+ response.end(request.headers.foo);
+ });
+
+ const {body} = await got('', {
+ headers: {},
+ hooks: {
+ init: [
+ options => {
+ options.headers!.foo = 'bar';
+ }
+ ]
+ }
+ });
+ t.is(body, 'bar');
+});
+
+test('beforeRequest is called with options', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ await got({
+ responseType: 'json',
+ hooks: {
+ beforeRequest: [
+ options => {
+ t.is(options.url.pathname, '/');
+ t.is(options.url.hostname, 'localhost');
+ }
+ ]
+ }
+ });
+});
+
+test('beforeRequest allows modifications', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ const {body} = await got>({
+ responseType: 'json',
+ hooks: {
+ beforeRequest: [
+ options => {
+ options.headers.foo = 'bar';
+ }
+ ]
+ }
+ });
+ t.is(body.foo, 'bar');
+});
+
+test('returning HTTP response from a beforeRequest hook', withServer, async (t, server, got) => {
+ server.get('/', echoUrl);
+
+ const {statusCode, headers, body} = await got({
+ hooks: {
+ beforeRequest: [
+ () => {
+ return new Responselike(
+ 200,
+ {foo: 'bar'},
+ Buffer.from('Hi!'),
+ ''
+ );
+ }
+ ]
+ }
+ });
+
+ t.is(statusCode, 200);
+ t.is(headers.foo, 'bar');
+ t.is(body, 'Hi!');
+});
+
+test('beforeRedirect is called with options and response', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+ server.get('/redirect', redirectEndpoint);
+
+ await got('redirect', {
+ responseType: 'json',
+ hooks: {
+ beforeRedirect: [
+ (options, response) => {
+ t.is(options.url.pathname, '/');
+ t.is(options.url.hostname, 'localhost');
+
+ t.is(response.statusCode, 302);
+ t.is(new URL(response.url).pathname, '/redirect');
+ t.is(response.redirectUrls.length, 1);
+ }
+ ]
+ }
+ });
+});
+
+test('beforeRedirect allows modifications', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+ server.get('/redirect', redirectEndpoint);
+
+ const {body} = await got>('redirect', {
+ responseType: 'json',
+ hooks: {
+ beforeRedirect: [
+ options => {
+ options.headers.foo = 'bar';
+ }
+ ]
+ }
+ });
+ t.is(body.foo, 'bar');
+});
+
+test('beforeRetry is called with options', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+ server.get('/retry', retryEndpoint);
+
+ const context = {};
+
+ await got('retry', {
+ responseType: 'json',
+ retry: 1,
+ throwHttpErrors: false,
+ context,
+ hooks: {
+ beforeRetry: [
+ (options, error, retryCount) => {
+ t.is(options.url.hostname, 'localhost');
+ t.is(options.context, context);
+ t.truthy(error);
+ t.true(retryCount! >= 1);
+ }
+ ]
+ }
+ });
+});
+
+test('beforeRetry allows modifications', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+ server.get('/retry', retryEndpoint);
+
+ const {body} = await got>('retry', {
+ responseType: 'json',
+ hooks: {
+ beforeRetry: [
+ options => {
+ options.headers.foo = 'bar';
+ }
+ ]
+ }
+ });
+ t.is(body.foo, 'bar');
+});
+
+test('beforeRetry allows stream body if different from original', withServer, async (t, server, got) => {
+ server.post('/retry', async (request, response) => {
+ if (request.headers.foo) {
+ response.send('test');
+ } else {
+ response.statusCode = 500;
+ }
+
+ response.end();
+ });
+
+ const generateBody = () => {
+ const form = new FormData();
+ form.append('A', 'B');
+ return form;
+ };
+
+ const {body} = await got.post('retry', {
+ body: generateBody(),
+ retry: {
+ methods: ['POST']
+ },
+ hooks: {
+ beforeRetry: [
+ options => {
+ const form = generateBody();
+ options.body = form;
+ options.headers['content-type'] = `multipart/form-data; boundary=${form.getBoundary()}`;
+ options.headers.foo = 'bar';
+ }
+ ]
+ }
+ });
+
+ t.is(body, 'test');
+});
+
+test('afterResponse is called with response', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ await got({
+ responseType: 'json',
+ hooks: {
+ afterResponse: [
+ response => {
+ t.is(typeof response.body, 'object');
+
+ return response;
+ }
+ ]
+ }
+ });
+});
+
+test('afterResponse allows modifications', withServer, async (t, server, got) => {
+ server.get('/', echoHeaders);
+
+ const {body} = await got>({
+ responseType: 'json',
+ hooks: {
+ afterResponse: [
+ response => {
+ response.body = {hello: 'world'};
+ return response;
+ }
+ ]
+ }
+ });
+ t.is(body.hello, 'world');
+});
+
+test('afterResponse allows to retry', withServer, async (t, server, got) => {
+ server.get('/', (request, response) => {
+ if (request.headers.token !== 'unicorn') {
+ response.statusCode = 401;
+ }
+
+ response.end();
+ });
+
+ const {statusCode} = await got({
+ hooks: {
+ afterResponse: [
+ (response, retryWithMergedOptions) => {
+ if (response.statusCode === 401) {
+ return retryWithMergedOptions({
+ headers: {
+ token: 'unicorn'
+ }
+ });
+ }
+
+ return response;
+ }
+ ]
+ }
+ });
+ t.is(statusCode, 200);
+});
+
+test('cancelling the request after retrying in a afterResponse hook', withServer, async (t, server, got) => {
+ let requests = 0;
+ server.get('/', (_request, response) => {
+ requests++;
+ response.end();
+ });
+
+ const gotPromise = got({
+ hooks: {
+ afterResponse: [
+ (_response, retryWithMergedOptions) => {
+ const promise = retryWithMergedOptions({
+ headers: {
+ token: 'unicorn'
+ }
+ });
+
+ gotPromise.cancel();
+
+ return promise;
+ }
+ ]
+ },
+ retry: {
+ calculateDelay: () => 1
+ }
+ });
+
+ await t.throwsAsync(gotPromise);
+ await delay(100);
+ t.is(requests, 1);
+});
+
+test('afterResponse allows to retry - `beforeRetry` hook', withServer, async (t, server, got) => {
+ server.get('/', (request, response) => {
+ if (request.headers.token !== 'unicorn') {
+ response.statusCode = 401;
+ }
+
+ response.end();
+ });
+
+ let isCalled = false;
+
+ const {statusCode} = await got({
+ hooks: {
+ afterResponse: [
+ (response, retryWithMergedOptions) => {
+ if (response.statusCode === 401) {
+ return retryWithMergedOptions({
+ headers: {
+ token: 'unicorn'
+ }
+ });
+ }
+
+ return response;
+ }
+ ],
+ beforeRetry: [
+ options => {
+ t.truthy(options);
+ isCalled = true;
+ }
+ ]
+ }
+ });
+ t.is(statusCode, 200);
+ t.true(isCalled);
+});
+
+test('no infinity loop when retrying on afterResponse', withServer, async (t, server, got) => {
+ server.get('/', (request, response) => {
+ if (request.headers.token !== 'unicorn') {
+ response.statusCode = 401;
+ }
+
+ response.end();
+ });
+
+ await t.throwsAsync(got({
+ retry: 0,
+ hooks: {
+ afterResponse: [
+ (_response, retryWithMergedOptions) => {
+ return retryWithMergedOptions({
+ headers: {
+ token: 'invalid'
+ }
+ });
+ }
+ ]
+ }
+ }), {instanceOf: got.HTTPError, message: 'Response code 401 (Unauthorized)'});
+});
+
+test('throws on afterResponse retry failure', withServer, async (t, server, got) => {
+ let didVisit401then500: boolean;
+ server.get('/', (_request, response) => {
+ if (didVisit401then500) {
+ response.statusCode = 500;
+ } else {
+ didVisit401then500 = true;
+ response.statusCode = 401;
+ }
+
+ response.end();
+ });
+
+ await t.throwsAsync(got({
+ retry: 1,
+ hooks: {
+ afterResponse: [
+ (response, retryWithMergedOptions) => {
+ if (response.statusCode === 401) {
+ return retryWithMergedOptions({
+ headers: {
+ token: 'unicorn'
+ }
+ });
+ }
+
+ return response;
+ }
+ ]
+ }
+ }), {instanceOf: got.HTTPError, message: 'Response code 500 (Internal Server Error)'});
+});
+
+test('doesn\'t throw on afterResponse retry HTTP failure if throwHttpErrors is false', withServer, async (t, server, got) => {
+ let didVisit401then500: boolean;
+ server.get('/', (_request, response) => {
+ if (didVisit401then500) {
+ response.statusCode = 500;
+ } else {
+ didVisit401then500 = true;
+ response.statusCode = 401;
+ }
+
+ response.end();
+ });
+
+ const {statusCode} = await got({
+ throwHttpErrors: false,
+ retry: 1,
+ hooks: {
+ afterResponse: [
+ (response, retryWithMergedOptions) => {
+ if (response.statusCode === 401) {
+ return retryWithMergedOptions({
+ headers: {
+ token: 'unicorn'
+ }
+ });
+ }
+
+ return response;
+ }
+ ]
+ }
+ });
+ t.is(statusCode, 500);
+});
+
+test('throwing in a beforeError hook - promise', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end('ok');
+ });
+
+ await t.throwsAsync(got({
+ hooks: {
+ afterResponse: [
+ () => {
+ throw error;
+ }
+ ],
+ beforeError: [
+ (): never => {
+ throw new Error('foobar');
+ },
+ () => {
+ throw new Error('This shouldn\'t be called at all');
+ }
+ ]
+ }
+ }), {message: 'foobar'});
+});
+
+test('throwing in a beforeError hook - stream', withServer, async (t, _server, got) => {
+ await t.throwsAsync(getStream(got.stream({
+ hooks: {
+ beforeError: [
+ () => {
+ throw new Error('foobar');
+ },
+ () => {
+ throw new Error('This shouldn\'t be called at all');
+ }
+ ]
+ }
+ })), {message: 'foobar'});
+});
+
+test('beforeError is called with an error - promise', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end('ok');
+ });
+
+ await t.throwsAsync(got({
+ hooks: {
+ afterResponse: [
+ () => {
+ throw error;
+ }
+ ],
+ beforeError: [error2 => {
+ t.true(error2 instanceof Error);
+ return error2;
+ }]
+ }
+ }), {message: errorString});
+});
+
+test('beforeError is called with an error - stream', withServer, async (t, _server, got) => {
+ await t.throwsAsync(getStream(got.stream({
+ hooks: {
+ beforeError: [error2 => {
+ t.true(error2 instanceof Error);
+ return error2;
+ }]
+ }
+ })), {message: 'Response code 404 (Not Found)'});
+});
+
+test('beforeError allows modifications', async t => {
+ const errorString2 = 'foobar';
+
+ await t.throwsAsync(got('https://example.com', {
+ request: () => {
+ throw error;
+ },
+ hooks: {
+ beforeError: [
+ error => {
+ const newError = new Error(errorString2);
+
+ return new RequestError(newError.message, newError, error.options);
+ }
+ ]
+ }
+ }), {message: errorString2});
+});
+
+test('does not break on `afterResponse` hook with JSON mode', withServer, async (t, server, got) => {
+ server.get('/foobar', echoHeaders);
+
+ await t.notThrowsAsync(got('', {
+ hooks: {
+ afterResponse: [
+ (response, retryWithMergedOptions) => {
+ if (response.statusCode === 404) {
+ const url = new URL('/foobar', response.url);
+
+ return retryWithMergedOptions({url});
+ }
+
+ return response;
+ }
+ ]
+ },
+ responseType: 'json'
+ }));
+});
+
+test('catches HTTPErrors', withServer, async (t, _server, got) => {
+ t.plan(2);
+
+ await t.throwsAsync(got({
+ hooks: {
+ beforeError: [
+ error => {
+ t.true(error instanceof got.HTTPError);
+ return error;
+ }
+ ]
+ }
+ }));
+});
+
+test('timeout can be modified using a hook', withServer, async (t, server, got) => {
+ server.get('/', () => {});
+
+ await t.throwsAsync(got({
+ timeout: 1000,
+ hooks: {
+ beforeRequest: [
+ options => {
+ options.timeout.request = 500;
+ }
+ ]
+ },
+ retry: 0
+ }), {message: 'Timeout awaiting \'request\' for 500ms'});
+});
+
+test('beforeRequest hook is called before each request', withServer, async (t, server, got) => {
+ server.post('/', echoUrl);
+ server.post('/redirect', redirectEndpoint);
+
+ const buffer = Buffer.from('Hello, Got!');
+ let counts = 0;
+
+ await got.post('redirect', {
+ body: buffer,
+ hooks: {
+ beforeRequest: [
+ options => {
+ counts++;
+ t.is(options.headers['content-length'], String(buffer.length));
+ }
+ ]
+ }
+ });
+
+ t.is(counts, 2);
+});
+
+test('beforeError emits valid promise `HTTPError`s', async t => {
+ t.plan(3);
+
+ nock('https://ValidHTTPErrors.com').get('/').reply(() => [422, 'no']);
+
+ const instance = got.extend({
+ hooks: {
+ beforeError: [
+ error => {
+ t.true(error instanceof HTTPError);
+ t.truthy(error.response!.body);
+
+ return error;
+ }
+ ]
+ },
+ retry: 0
+ });
+
+ await t.throwsAsync(instance('https://ValidHTTPErrors.com'));
+});
+
+test('hooks are not duplicated', withServer, async (t, _server, got) => {
+ let calls = 0;
+
+ await t.throwsAsync(got({
+ hooks: {
+ beforeError: [
+ error => {
+ calls++;
+
+ return error;
+ }
+ ]
+ },
+ retry: 0
+ }), {message: 'Response code 404 (Not Found)'});
+
+ t.is(calls, 1);
+});
+
+test('async afterResponse allows to retry with allowGetBody and json payload', withServer, async (t, server, got) => {
+ server.get('/', (request, response) => {
+ if (request.headers.token !== 'unicorn') {
+ response.statusCode = 401;
+ }
+
+ response.end();
+ });
+
+ const {statusCode} = await got({
+ allowGetBody: true,
+ json: {hello: 'world'},
+ hooks: {
+ afterResponse: [
+ (response, retryWithMergedOptions) => {
+ if (response.statusCode === 401) {
+ return retryWithMergedOptions({headers: {token: 'unicorn'}});
+ }
+
+ return response;
+ }
+ ]
+ },
+ retry: 0,
+ throwHttpErrors: false
+ });
+ t.is(statusCode, 200);
+});
+
+test('beforeRequest hook respect `agent` option', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end('ok');
+ });
+
+ const {agent, spy} = createAgentSpy(HttpAgent);
+
+ t.truthy((await got({
+ hooks: {
+ beforeRequest: [
+ options => {
+ options.agent = {
+ http: agent
+ };
+ }
+ ]
+ }
+ })).body);
+ t.true(spy.calledOnce);
+
+ // Make sure to close all open sockets
+ agent.destroy();
+});
+
+test('beforeRequest hook respect `url` option', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end('ko');
+ });
+
+ server.get('/changed', (_request, response) => {
+ response.end('ok');
+ });
+
+ t.is((await got(server.hostname, {
+ hooks: {
+ beforeRequest: [
+ options => {
+ options.url = new URL(server.url + '/changed');
+ }
+ ]
+ }
+ })).body, 'ok');
+});
+
+test('no duplicate hook calls in single-page paginated requests', withServer, async (t, server, got) => {
+ server.get('/get', (_request, response) => {
+ response.end('i <3 koalas');
+ });
+
+ let beforeHookCount = 0;
+ let beforeHookCountAdditional = 0;
+ let afterHookCount = 0;
+ let afterHookCountAdditional = 0;
+
+ const hooks = {
+ beforeRequest: [
+ () => {
+ beforeHookCount++;
+ }
+ ],
+ afterResponse: [
+ (response: any) => {
+ afterHookCount++;
+ return response;
+ }
+ ]
+ };
+
+ // Test only one request
+ const instance = got.extend({
+ hooks,
+ pagination: {
+ paginate: () => false,
+ countLimit: 2009,
+ transform: response => [response]
+ }
+ });
+
+ await instance.paginate.all('get');
+ t.is(beforeHookCount, 1);
+ t.is(afterHookCount, 1);
+
+ await instance.paginate.all('get', {
+ hooks: {
+ beforeRequest: [
+ () => {
+ beforeHookCountAdditional++;
+ }
+ ],
+ afterResponse: [
+ (response: any) => {
+ afterHookCountAdditional++;
+ return response;
+ }
+ ]
+ }
+ });
+ t.is(beforeHookCount, 2);
+ t.is(afterHookCount, 2);
+ t.is(beforeHookCountAdditional, 1);
+ t.is(afterHookCountAdditional, 1);
+
+ await got.paginate.all('get', {
+ hooks,
+ pagination: {
+ paginate: () => false,
+ transform: response => [response]
+ }
+ });
+
+ t.is(beforeHookCount, 3);
+ t.is(afterHookCount, 3);
+});
+
+test('no duplicate hook calls in sequential paginated requests', withServer, async (t, server, got) => {
+ server.get('/get', (_request, response) => {
+ response.end('i <3 unicorns');
+ });
+
+ let requestNumber = 0;
+ let beforeHookCount = 0;
+ let afterHookCount = 0;
+
+ const hooks = {
+ beforeRequest: [
+ () => {
+ beforeHookCount++;
+ }
+ ],
+ afterResponse: [
+ (response: any) => {
+ afterHookCount++;
+ return response;
+ }
+ ]
+ };
+
+ // Test only two requests, one after another
+ const paginate = () => requestNumber++ === 0 ? {} : false;
+
+ const instance = got.extend({
+ hooks,
+ pagination: {
+ paginate,
+ countLimit: 2009,
+ transform: response => [response]
+ }
+ });
+
+ await instance.paginate.all('get');
+
+ t.is(beforeHookCount, 2);
+ t.is(afterHookCount, 2);
+ requestNumber = 0;
+
+ await got.paginate.all('get', {
+ hooks,
+ pagination: {
+ paginate,
+ transform: response => [response]
+ }
+ });
+
+ t.is(beforeHookCount, 4);
+ t.is(afterHookCount, 4);
+});
+
+test('intentional duplicate hooks in pagination with extended instance', withServer, async (t, server, got) => {
+ server.get('/get', (_request, response) => {
+ response.end('<3');
+ });
+
+ let beforeCount = 0; // Number of times the hooks from `extend` are called
+ let afterCount = 0;
+ let beforeCountAdditional = 0; // Number of times the added hooks are called
+ let afterCountAdditional = 0;
+
+ const beforeHook = () => {
+ beforeCount++;
+ };
+
+ const afterHook = (response: any) => {
+ afterCount++;
+ return response;
+ };
+
+ const instance = got.extend({
+ hooks: {
+ beforeRequest: [
+ beforeHook,
+ beforeHook
+ ],
+ afterResponse: [
+ afterHook,
+ afterHook
+ ]
+ },
+ pagination: {
+ paginate: () => false,
+ countLimit: 2009,
+ transform: response => [response]
+ }
+ });
+
+ // Add duplicate hooks when calling paginate
+ const beforeHookAdditional = () => {
+ beforeCountAdditional++;
+ };
+
+ const afterHookAdditional = (response: any) => {
+ afterCountAdditional++;
+ return response;
+ };
+
+ await instance.paginate.all('get', {
+ hooks: {
+ beforeRequest: [
+ beforeHook,
+ beforeHookAdditional,
+ beforeHookAdditional
+ ],
+ afterResponse: [
+ afterHook,
+ afterHookAdditional,
+ afterHookAdditional
+ ]
+ }
+ });
+
+ t.is(beforeCount, 3);
+ t.is(afterCount, 3);
+ t.is(beforeCountAdditional, 2);
+ t.is(afterCountAdditional, 2);
+});
+
+test('no duplicate hook calls when returning original request options', withServer, async (t, server, got) => {
+ server.get('/get', (_request, response) => {
+ response.end('i <3 unicorns');
+ });
+
+ let requestNumber = 0;
+ let beforeHookCount = 0;
+ let afterHookCount = 0;
+
+ const hooks = {
+ beforeRequest: [
+ () => {
+ beforeHookCount++;
+ }
+ ],
+ afterResponse: [
+ (response: any) => {
+ afterHookCount++;
+ return response;
+ }
+ ]
+ };
+
+ // Test only two requests, one after another
+ const paginate = (response: Response) => requestNumber++ === 0 ? response.request.options : false;
+
+ const instance = got.extend({
+ hooks,
+ pagination: {
+ paginate,
+ countLimit: 2009,
+ transform: response => [response]
+ }
+ });
+
+ await instance.paginate.all('get');
+
+ t.is(beforeHookCount, 2);
+ t.is(afterHookCount, 2);
+ requestNumber = 0;
+
+ await got.paginate.all('get', {
+ hooks,
+ pagination: {
+ paginate,
+ transform: response => [response]
+ }
+ });
+
+ t.is(beforeHookCount, 4);
+ t.is(afterHookCount, 4);
+});
+
+test('`beforeRequest` change body', withServer, async (t, server, got) => {
+ server.post('/', echoBody);
+
+ const response = await got.post({
+ json: {payload: 'old'},
+ hooks: {
+ beforeRequest: [
+ options => {
+ options.body = JSON.stringify({payload: 'new'});
+ options.headers['content-length'] = options.body.length.toString();
+ }
+ ]
+ }
+ });
+
+ t.is(JSON.parse(response.body).payload, 'new');
+});
diff --git a/test/http.js b/test/http.js
deleted file mode 100644
index 193ba80df..000000000
--- a/test/http.js
+++ /dev/null
@@ -1,104 +0,0 @@
-import test from 'ava';
-import got from '../';
-import {createServer} from './helpers/server';
-
-let s;
-
-test.before('setup', async () => {
- s = await createServer();
-
- s.on('/', (req, res) => {
- res.end('ok');
- });
-
- s.on('/empty', (req, res) => {
- res.end();
- });
-
- s.on('/404', (req, res) => {
- setTimeout(() => {
- res.statusCode = 404;
- res.end('not');
- }, 10);
- });
-
- s.on('/?recent=true', (req, res) => {
- res.end('recent');
- });
-
- await s.listen(s.port);
-});
-
-test('simple request', async t => {
- t.is((await got(s.url)).body, 'ok');
-});
-
-test('protocol-less URLs', async t => {
- t.is((await got(s.url.replace(/^http:\/\//, ''))).body, 'ok');
-});
-
-test('empty response', async t => {
- t.is((await got(`${s.url}/empty`)).body, '');
-});
-
-test('requestUrl response', async t => {
- t.is((await got(s.url)).requestUrl, `${s.url}/`);
- t.is((await got(`${s.url}/empty`)).requestUrl, `${s.url}/empty`);
-});
-
-test('error with code', async t => {
- try {
- await got(`${s.url}/404`);
- t.fail('Exception was not thrown');
- } catch (err) {
- t.is(err.statusCode, 404);
- t.is(err.response.body, 'not');
- }
-});
-
-test('buffer on encoding === null', async t => {
- const data = (await got(s.url, {encoding: null})).body;
- t.truthy(Buffer.isBuffer(data));
-});
-
-test('timeout option', async t => {
- try {
- await got(`${s.url}/404`, {
- timeout: 1,
- retries: 0
- });
- t.fail('Exception was not thrown');
- } catch (err) {
- t.is(err.code, 'ETIMEDOUT');
- }
-});
-
-test('timeout option as object', async t => {
- try {
- await got(`${s.url}/404`, {
- timeout: {connect: 1},
- retries: 0
- });
- t.fail('Exception was not thrown');
- } catch (err) {
- t.is(err.code, 'ETIMEDOUT');
- }
-});
-
-test('query option', async t => {
- t.is((await got(s.url, {query: {recent: true}})).body, 'recent');
- t.is((await got(s.url, {query: 'recent=true'})).body, 'recent');
-});
-
-test('requestUrl response when sending url as param', async t => {
- t.is((await got(s.url, {hostname: s.host, port: s.port})).requestUrl, `${s.url}/`);
- t.is((await got({hostname: s.host, port: s.port})).requestUrl, `${s.url}/`);
-});
-
-test('response contains url', async t => {
- t.is((await got(s.url)).url, `${s.url}/`);
-});
-
-test.after('cleanup', async () => {
- await s.close();
-});
diff --git a/test/http.ts b/test/http.ts
new file mode 100644
index 000000000..4c98defa8
--- /dev/null
+++ b/test/http.ts
@@ -0,0 +1,375 @@
+import {STATUS_CODES, Agent} from 'http';
+import test from 'ava';
+import {Handler} from 'express';
+import {isIPv4, isIPv6} from 'net';
+import nock = require('nock');
+import getStream = require('get-stream');
+import pEvent from 'p-event';
+import got, {HTTPError, UnsupportedProtocolError, CancelableRequest, ReadError} from '../source';
+import withServer from './helpers/with-server';
+import os = require('os');
+
+const IPv6supported = Object.values(os.networkInterfaces()).some(iface => iface?.some(addr => !addr.internal && addr.family === 'IPv6'));
+
+const testIPv6 = (IPv6supported && process.env.TRAVIS_DIST !== 'bionic' && process.env.TRAVIS_DIST !== 'focal') ? test : test.skip;
+
+const echoIp: Handler = (request, response) => {
+ const address = request.connection.remoteAddress;
+ if (address === undefined) {
+ response.end();
+ return;
+ }
+
+ // IPv4 address mapped to IPv6
+ response.end(address === '::ffff:127.0.0.1' ? '127.0.0.1' : address);
+};
+
+const echoBody: Handler = async (request, response) => {
+ response.end(await getStream(request));
+};
+
+test('simple request', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end('ok');
+ });
+
+ t.is((await got('')).body, 'ok');
+});
+
+test('empty response', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end();
+ });
+
+ t.is((await got('')).body, '');
+});
+
+test('response has `requestUrl` property', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end('ok');
+ });
+
+ server.get('/empty', (_request, response) => {
+ response.end();
+ });
+
+ t.is((await got('')).requestUrl, `${server.url}/`);
+ t.is((await got('empty')).requestUrl, `${server.url}/empty`);
+});
+
+test('http errors have `response` property', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.statusCode = 404;
+ response.end('not');
+ });
+
+ const error = await t.throwsAsync(got(''), {instanceOf: HTTPError});
+ t.is(error.response.statusCode, 404);
+ t.is(error.response.body, 'not');
+});
+
+test('status code 304 doesn\'t throw', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.statusCode = 304;
+ response.end();
+ });
+
+ const promise = got('');
+ await t.notThrowsAsync(promise);
+ const {statusCode, body} = await promise;
+ t.is(statusCode, 304);
+ t.is(body, '');
+});
+
+test('doesn\'t throw if `options.throwHttpErrors` is false', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.statusCode = 404;
+ response.end('not');
+ });
+
+ t.is((await got({throwHttpErrors: false})).body, 'not');
+});
+
+test('invalid protocol throws', async t => {
+ await t.throwsAsync(got('c:/nope.com').json(), {
+ instanceOf: UnsupportedProtocolError,
+ code: 'ERR_UNSUPPORTED_PROTOCOL',
+ message: 'Unsupported protocol "c:"'
+ });
+});
+
+test('custom `options.encoding`', withServer, async (t, server, got) => {
+ const string = 'ok';
+
+ server.get('/', (_request, response) => {
+ response.end(string);
+ });
+
+ const data = (await got({encoding: 'base64'})).body;
+ t.is(data, Buffer.from(string).toString('base64'));
+});
+
+test('`options.encoding` doesn\'t affect streams', withServer, async (t, server, got) => {
+ const string = 'ok';
+
+ server.get('/', (_request, response) => {
+ response.end(string);
+ });
+
+ const data = await getStream(got.stream({encoding: 'base64'}));
+ t.is(data, string);
+});
+
+test('`got.stream(...).setEncoding(...)` works', withServer, async (t, server, got) => {
+ const string = 'ok';
+
+ server.get('/', (_request, response) => {
+ response.end(string);
+ });
+
+ const data = await getStream(got.stream('').setEncoding('base64'));
+ t.is(data, Buffer.from(string).toString('base64'));
+});
+
+test('`searchParams` option', withServer, async (t, server, got) => {
+ server.get('/', (request, response) => {
+ t.is(request.query.recent, 'true');
+ response.end('recent');
+ });
+
+ t.is((await got({searchParams: {recent: true}})).body, 'recent');
+ t.is((await got({searchParams: 'recent=true'})).body, 'recent');
+});
+
+test('response contains url', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end('ok');
+ });
+
+ t.is((await got('')).url, `${server.url}/`);
+});
+
+test('response contains got options', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end('ok');
+ });
+
+ {
+ const options = {
+ username: 'foo',
+ password: 'bar'
+ };
+
+ const {options: normalizedOptions} = (await got(options)).request;
+
+ t.is(normalizedOptions.username, options.username);
+ t.is(normalizedOptions.password, options.password);
+ }
+
+ {
+ const options = {
+ username: 'foo'
+ };
+
+ const {options: normalizedOptions} = (await got(options)).request;
+
+ t.is(normalizedOptions.username, options.username);
+ t.is(normalizedOptions.password, '');
+ }
+
+ {
+ const options = {
+ password: 'bar'
+ };
+
+ const {options: normalizedOptions} = (await got(options)).request;
+
+ t.is(normalizedOptions.username, '');
+ t.is(normalizedOptions.password, options.password);
+ }
+});
+
+test('socket destroyed by the server throws ECONNRESET', withServer, async (t, server, got) => {
+ server.get('/', request => {
+ request.socket.destroy();
+ });
+
+ await t.throwsAsync(got('', {retry: 0}), {
+ code: 'ECONNRESET'
+ });
+});
+
+test('the response contains timings property', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end('ok');
+ });
+
+ const {timings} = await got('');
+
+ t.truthy(timings);
+ t.true(timings.phases.total! >= 0);
+});
+
+test('throws an error if the server aborted the request', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.writeHead(200, {
+ 'content-type': 'text/plain'
+ });
+ response.write('chunk 1');
+
+ setImmediate(() => {
+ response.write('chunk 2');
+
+ setImmediate(() => {
+ response.destroy();
+ });
+ });
+ });
+
+ const error = await t.throwsAsync(got(''), {
+ code: 'ECONNRESET',
+ message: 'The server aborted pending request'
+ });
+
+ t.truthy(error.response.retryCount);
+});
+
+test('statusMessage fallback', async t => {
+ nock('http://statusMessageFallback').get('/').reply(503);
+
+ const {statusMessage} = await got('http://statusMessageFallback', {
+ throwHttpErrors: false,
+ retry: 0
+ });
+
+ t.is(statusMessage, STATUS_CODES[503]);
+});
+
+test('does not destroy completed requests', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.setHeader('content-encoding', 'gzip');
+ response.end('');
+ });
+
+ const options = {
+ agent: {
+ http: new Agent({keepAlive: true})
+ },
+ retry: 0
+ };
+
+ const stream = got.stream(options);
+ stream.resume();
+
+ const endPromise = pEvent(stream, 'end');
+
+ const socket = await pEvent(stream, 'socket');
+
+ const closeListener = () => {
+ t.fail('Socket has been destroyed');
+ };
+
+ socket.once('close', closeListener);
+
+ await new Promise(resolve => {
+ setTimeout(resolve, 10);
+ });
+
+ socket.off('close', closeListener);
+
+ await endPromise;
+
+ options.agent.http.destroy();
+
+ t.pass();
+});
+
+testIPv6('IPv6 request', withServer, async (t, server) => {
+ server.get('/ok', echoIp);
+
+ const response = await got(`http://[::1]:${server.port}/ok`);
+
+ t.is(response.body, '::1');
+});
+
+test('DNS auto', withServer, async (t, server, got) => {
+ server.get('/ok', echoIp);
+
+ const response = await got('ok', {
+ dnsLookupIpVersion: 'auto'
+ });
+
+ t.true(isIPv4(response.body));
+});
+
+test('DNS IPv4', withServer, async (t, server, got) => {
+ server.get('/ok', echoIp);
+
+ const response = await got('ok', {
+ dnsLookupIpVersion: 'ipv4'
+ });
+
+ t.true(isIPv4(response.body));
+});
+
+// Travis CI Ubuntu Focal VM does not resolve IPv6 hostnames
+testIPv6('DNS IPv6', withServer, async (t, server, got) => {
+ server.get('/ok', echoIp);
+
+ const response = await got('ok', {
+ dnsLookupIpVersion: 'ipv6'
+ });
+
+ t.true(isIPv6(response.body));
+});
+
+test('invalid `dnsLookupIpVersion`', withServer, async (t, server, got) => {
+ server.get('/ok', echoIp);
+
+ await t.throwsAsync(got('ok', {
+ dnsLookupIpVersion: 'test'
+ } as any));
+});
+
+test.serial('deprecated `family` option', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end('ok');
+ });
+
+ await new Promise(resolve => {
+ let request: CancelableRequest;
+ (async () => {
+ const warning = await pEvent(process, 'warning');
+ t.is(warning.name, 'DeprecationWarning');
+ request!.cancel();
+ resolve();
+ })();
+
+ (async () => {
+ request = got({
+ family: '4'
+ } as any);
+
+ try {
+ await request;
+ t.fail();
+ } catch {
+ t.true(request!.isCanceled);
+ }
+
+ resolve();
+ })();
+ });
+});
+
+test('JSON request custom stringifier', withServer, async (t, server, got) => {
+ server.post('/', echoBody);
+
+ const payload = {a: 'b'};
+ const customStringify = (object: any) => JSON.stringify({...object, c: 'd'});
+
+ t.deepEqual((await got.post({
+ stringifyJson: customStringify,
+ json: payload
+ })).body, customStringify(payload));
+});
diff --git a/test/https.js b/test/https.js
deleted file mode 100644
index 90e5308cb..000000000
--- a/test/https.js
+++ /dev/null
@@ -1,59 +0,0 @@
-import test from 'ava';
-import pem from 'pem';
-import pify from 'pify';
-import got from '../';
-import {createSSLServer} from './helpers/server';
-
-let s;
-let caRootCert;
-
-const pemP = pify(pem, Promise);
-
-test.before('setup', async () => {
- const caKeys = await pemP.createCertificate({
- days: 1,
- selfSigned: true
- });
-
- const caRootKey = caKeys.serviceKey;
- caRootCert = caKeys.certificate;
-
- const keys = await pemP.createCertificate({
- serviceCertificate: caRootCert,
- serviceKey: caRootKey,
- serial: Date.now(),
- days: 500,
- country: '',
- state: '',
- locality: '',
- organization: '',
- organizationUnit: '',
- commonName: 'sindresorhus.com'
- });
-
- const key = keys.clientKey;
- const cert = keys.certificate;
-
- s = await createSSLServer({key, cert}); // eslint-disable-line object-property-newline
-
- s.on('/', (req, res) => res.end('ok'));
-
- await s.listen(s.port);
-});
-
-test('make request to https server without ca', async t => {
- t.truthy((await got(s.url, {rejectUnauthorized: false})).body);
-});
-
-test('make request to https server with ca', async t => {
- const {body} = await got(s.url, {
- strictSSL: true,
- ca: caRootCert,
- headers: {host: 'sindresorhus.com'}
- });
- t.is(body, 'ok');
-});
-
-test.after('cleanup', async () => {
- await s.close();
-});
diff --git a/test/https.ts b/test/https.ts
new file mode 100644
index 000000000..60b33712c
--- /dev/null
+++ b/test/https.ts
@@ -0,0 +1,468 @@
+import test from 'ava';
+import got, {CancelableRequest} from '../source';
+import {withHttpsServer} from './helpers/with-server';
+import {DetailedPeerCertificate} from 'tls';
+import pEvent from 'p-event';
+import pify = require('pify');
+import pem = require('pem');
+
+const createPrivateKey = pify(pem.createPrivateKey);
+const createCSR = pify(pem.createCSR);
+const createCertificate = pify(pem.createCertificate);
+const createPkcs12 = pify(pem.createPkcs12);
+
+test('https request without ca', withHttpsServer(), async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end('ok');
+ });
+
+ t.truthy((await got({
+ https: {
+ certificateAuthority: [],
+ rejectUnauthorized: false
+ }
+ })).body);
+});
+
+test('https request with ca', withHttpsServer(), async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end('ok');
+ });
+
+ const {body} = await got({});
+
+ t.is(body, 'ok');
+});
+
+test('https request with ca and afterResponse hook', withHttpsServer(), async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end('ok');
+ });
+
+ const warningListener = (warning: any) => {
+ if (
+ warning.name === 'DeprecationWarning' &&
+ warning.message === 'Got: "options.ca" was never documented, please use ' +
+ '"options.https.certificateAuthority"'
+ ) {
+ process.off('warning', warningListener);
+ t.fail('unexpected deprecation warning');
+ }
+ };
+
+ process.once('warning', warningListener);
+
+ let shouldRetry = true;
+ const {body} = await got({
+ hooks: {
+ afterResponse: [
+ (response, retry) => {
+ if (shouldRetry) {
+ shouldRetry = false;
+
+ return retry({});
+ }
+
+ return response;
+ }
+ ]
+ }
+ });
+
+ t.is(body, 'ok');
+});
+
+test('https request with `checkServerIdentity` OK', withHttpsServer(), async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end('ok');
+ });
+
+ const {body} = await got({
+ https: {
+ checkServerIdentity: (hostname: string, certificate: DetailedPeerCertificate) => {
+ t.is(hostname, 'localhost');
+ t.is(certificate.subject.CN, 'localhost');
+ t.is(certificate.issuer.CN, 'authority');
+ }
+ }
+ });
+
+ t.is(body, 'ok');
+});
+
+test('https request with `checkServerIdentity` NOT OK', withHttpsServer(), async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end('ok');
+ });
+
+ const promise = got({
+ https: {
+ checkServerIdentity: (hostname: string, certificate: DetailedPeerCertificate) => {
+ t.is(hostname, 'localhost');
+ t.is(certificate.subject.CN, 'localhost');
+ t.is(certificate.issuer.CN, 'authority');
+
+ return new Error('CUSTOM_ERROR');
+ }
+ }
+ });
+
+ await t.throwsAsync(
+ promise,
+ {
+ message: 'CUSTOM_ERROR'
+ }
+ );
+});
+
+// The built-in `openssl` on macOS does not support negative days.
+{
+ const testFn = process.platform === 'darwin' ? test.skip : test;
+ testFn('https request with expired certificate', withHttpsServer({days: -1}), async (t, _server, got) => {
+ await t.throwsAsync(
+ got({}),
+ {
+ code: 'CERT_HAS_EXPIRED'
+ }
+ );
+ });
+}
+
+test('https request with wrong host', withHttpsServer({commonName: 'not-localhost.com'}), async (t, _server, got) => {
+ await t.throwsAsync(
+ got({}),
+ {
+ code: 'ERR_TLS_CERT_ALTNAME_INVALID'
+ }
+ );
+});
+
+test('http2', async t => {
+ const promise = got('https://httpbin.org/anything', {
+ http2: true
+ });
+
+ const {headers, body} = await promise;
+ await promise.json();
+
+ // @ts-expect-error Pseudo headers may not be strings
+ t.is(headers[':status'], 200);
+ t.is(typeof body, 'string');
+});
+
+test.serial('deprecated `rejectUnauthorized` option', withHttpsServer(), async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end('ok');
+ });
+
+ await new Promise(resolve => {
+ let request: CancelableRequest;
+ (async () => {
+ const warning = await pEvent(process, 'warning');
+ t.is(warning.name, 'DeprecationWarning');
+ request!.cancel();
+ resolve();
+ })();
+
+ (async () => {
+ request = got({
+ rejectUnauthorized: false
+ });
+
+ try {
+ await request;
+ t.fail();
+ } catch {
+ t.true(request!.isCanceled);
+ }
+
+ resolve();
+ })();
+ });
+});
+
+test.serial('non-deprecated `rejectUnauthorized` option', withHttpsServer(), async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end('ok');
+ });
+
+ (async () => {
+ const warning = await pEvent(process, 'warning');
+ t.not(warning.name, 'DeprecationWarning');
+ })();
+
+ await got({
+ https: {
+ rejectUnauthorized: false
+ }
+ });
+
+ t.pass();
+});
+
+test.serial('no double deprecated warning', withHttpsServer(), async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end('ok');
+ });
+
+ (async () => {
+ const warning = await pEvent(process, 'warning');
+ t.is(warning.name, 'DeprecationWarning');
+ })();
+
+ await got({
+ rejectUnauthorized: false
+ });
+
+ (async () => {
+ const warning = await pEvent(process, 'warning');
+ t.not(warning.name, 'DeprecationWarning');
+ })();
+
+ await got({
+ rejectUnauthorized: false
+ });
+
+ t.pass();
+});
+
+test('client certificate', withHttpsServer(), async (t, server, got) => {
+ server.get('/', (request, response) => {
+ const peerCertificate = (request.socket as any).getPeerCertificate(true);
+ peerCertificate.issuerCertificate.issuerCertificate = undefined; // Circular structure
+
+ response.json({
+ authorized: (request.socket as any).authorized,
+ peerCertificate
+ });
+ });
+
+ const clientCSRResult = await createCSR({commonName: 'client'});
+ const clientResult = await createCertificate({
+ csr: clientCSRResult.csr,
+ clientKey: clientCSRResult.clientKey,
+ serviceKey: (server as any).caKey,
+ serviceCertificate: (server as any).caCert
+ });
+ // eslint-disable-next-line prefer-destructuring
+ const clientKey = clientResult.clientKey;
+ const clientCert = clientResult.certificate;
+
+ const response: any = await got({
+ https: {
+ key: clientKey,
+ certificate: clientCert
+ }
+ }).json();
+
+ t.true(response.authorized);
+ t.is(response.peerCertificate.subject.CN, 'client');
+ t.is(response.peerCertificate.issuer.CN, 'authority');
+});
+
+test('invalid client certificate (self-signed)', withHttpsServer(), async (t, server, got) => {
+ server.get('/', (request, response) => {
+ const peerCertificate = (request.socket as any).getPeerCertificate(true);
+ peerCertificate.issuerCertificate = undefined; // Circular structure
+
+ response.json({
+ authorized: (request.socket as any).authorized,
+ peerCertificate
+ });
+ });
+
+ const clientCSRResult = await createCSR({commonName: 'other-client'});
+ const clientResult = await createCertificate({
+ csr: clientCSRResult.csr,
+ clientKey: clientCSRResult.clientKey,
+ selfSigned: true
+ });
+ // eslint-disable-next-line prefer-destructuring
+ const clientKey = clientResult.clientKey;
+ const clientCert = clientResult.certificate;
+
+ const response: any = await got({
+ https: {
+ key: clientKey,
+ certificate: clientCert
+ }
+ }).json();
+
+ t.is(response.authorized, false);
+});
+
+test('invalid client certificate (other CA)', withHttpsServer(), async (t, server, got) => {
+ server.get('/', (request, response) => {
+ const peerCertificate = (request.socket as any).getPeerCertificate(true);
+
+ response.json({
+ authorized: (request.socket as any).authorized,
+ peerCertificate
+ });
+ });
+
+ const caCSRResult = await createCSR({commonName: 'other-authority'});
+ const caResult = await createCertificate({
+ csr: caCSRResult.csr,
+ clientKey: caCSRResult.clientKey,
+ selfSigned: true
+ });
+ const caKey = caResult.clientKey;
+ const caCert = caResult.certificate;
+
+ const clientCSRResult = await createCSR({commonName: 'other-client'});
+ const clientResult = await createCertificate({
+ csr: clientCSRResult.csr,
+ clientKey: clientCSRResult.clientKey,
+ serviceKey: caKey,
+ serviceCertificate: caCert
+ });
+ // eslint-disable-next-line prefer-destructuring
+ const clientKey = clientResult.clientKey;
+ const clientCert = clientResult.certificate;
+
+ const response: any = await got({
+ https: {
+ key: clientKey,
+ certificate: clientCert
+ }
+ }).json();
+
+ t.false(response.authorized);
+ t.is(response.peerCertificate.subject.CN, 'other-client');
+ t.is(response.peerCertificate.issuer.CN, 'other-authority');
+});
+
+test('key passphrase', withHttpsServer(), async (t, server, got) => {
+ server.get('/', (request, response) => {
+ const peerCertificate = (request.socket as any).getPeerCertificate(true);
+ peerCertificate.issuerCertificate.issuerCertificate = undefined; // Circular structure
+
+ response.json({
+ authorized: (request.socket as any).authorized,
+ peerCertificate
+ });
+ });
+
+ const {key: clientKey} = await createPrivateKey(2048, {
+ cipher: 'aes256',
+ password: 'randomPassword'
+ });
+ const clientCSRResult = await createCSR({
+ // eslint-disable-next-line object-shorthand
+ clientKey: clientKey,
+ clientKeyPassword: 'randomPassword',
+ commonName: 'client'
+ });
+ const clientResult = await createCertificate({
+ csr: clientCSRResult.csr,
+ clientKey: clientCSRResult.clientKey,
+ clientKeyPassword: 'randomPassword',
+ serviceKey: (server as any).caKey,
+ serviceCertificate: (server as any).caCert
+ });
+ const clientCert = clientResult.certificate;
+
+ const response: any = await got({
+ https: {
+ key: clientKey,
+ passphrase: 'randomPassword',
+ certificate: clientCert
+ }
+ }).json();
+
+ t.true(response.authorized);
+ t.is(response.peerCertificate.subject.CN, 'client');
+ t.is(response.peerCertificate.issuer.CN, 'authority');
+});
+
+test('invalid key passphrase', withHttpsServer(), async (t, server, got) => {
+ server.get('/', (request, response) => {
+ const peerCertificate = (request.socket as any).getPeerCertificate(true);
+ peerCertificate.issuerCertificate.issuerCertificate = undefined; // Circular structure
+
+ response.json({
+ authorized: (request.socket as any).authorized,
+ peerCertificate
+ });
+ });
+
+ const {key: clientKey} = await createPrivateKey(2048, {
+ cipher: 'aes256',
+ password: 'randomPassword'
+ });
+ const clientCSRResult = await createCSR({
+ // eslint-disable-next-line object-shorthand
+ clientKey: clientKey,
+ clientKeyPassword: 'randomPassword',
+ commonName: 'client'
+ });
+ const clientResult = await createCertificate({
+ csr: clientCSRResult.csr,
+ clientKey: clientCSRResult.clientKey,
+ clientKeyPassword: 'randomPassword',
+ serviceKey: (server as any).caKey,
+ serviceCertificate: (server as any).caCert
+ });
+ const clientCert = clientResult.certificate;
+
+ const NODE_10 = process.versions.node.split('.')[0] === '10';
+
+ const request = got({
+ https: {
+ key: clientKey,
+ passphrase: 'wrongPassword',
+ certificate: clientCert
+ }
+ });
+
+ // Node.JS 10 does not have an error code, it only has a mesage
+ if (NODE_10) {
+ try {
+ await request;
+ t.fail();
+ } catch (error) {
+ t.true((error.message as string).includes('bad decrypt'), error.message);
+ }
+ } else {
+ await t.throwsAsync(request, {
+ code: 'ERR_OSSL_EVP_BAD_DECRYPT'
+ });
+ }
+});
+
+test('client certificate PFX', withHttpsServer(), async (t, server, got) => {
+ server.get('/', (request, response) => {
+ const peerCertificate = (request.socket as any).getPeerCertificate(true);
+ peerCertificate.issuerCertificate = undefined; // Circular structure
+
+ response.json({
+ authorized: (request.socket as any).authorized,
+ peerCertificate
+ });
+ });
+
+ const clientCSRResult = await createCSR({commonName: 'client'});
+ const clientResult = await createCertificate({
+ csr: clientCSRResult.csr,
+ clientKey: clientCSRResult.clientKey,
+ serviceKey: (server as any).caKey,
+ serviceCertificate: (server as any).caCert
+ });
+ // eslint-disable-next-line prefer-destructuring
+ const clientKey = clientResult.clientKey;
+ const clientCert = clientResult.certificate;
+
+ const {pkcs12} = await createPkcs12(clientKey, clientCert, 'randomPassword');
+
+ const response: any = await got({
+ https: {
+ pfx: pkcs12,
+ passphrase: 'randomPassword'
+ }
+ }).json();
+
+ t.true(response.authorized);
+ t.is(response.peerCertificate.subject.CN, 'client');
+ t.is(response.peerCertificate.issuer.CN, 'authority');
+});
diff --git a/test/json.js b/test/json.js
deleted file mode 100644
index 6e748368e..000000000
--- a/test/json.js
+++ /dev/null
@@ -1,87 +0,0 @@
-import test from 'ava';
-import got from '../';
-import {createServer} from './helpers/server';
-
-let s;
-
-test.before('setup', async () => {
- s = await createServer();
-
- s.on('/', (req, res) => {
- res.end('{"data":"dog"}');
- });
-
- s.on('/invalid', (req, res) => {
- res.end('/');
- });
-
- s.on('/no-body', (req, res) => {
- res.statusCode = 200;
- res.end();
- });
-
- s.on('/non200', (req, res) => {
- res.statusCode = 500;
- res.end('{"data":"dog"}');
- });
-
- s.on('/non200-invalid', (req, res) => {
- res.statusCode = 500;
- res.end('Internal error');
- });
-
- await s.listen(s.port);
-});
-
-test('parses response', async t => {
- t.deepEqual((await got(s.url, {json: true})).body, {data: 'dog'});
-});
-
-test('not parses responses without a body', async t => {
- const {body} = await got(`${s.url}/no-body`, {json: true});
- t.is(body, '');
-});
-
-test('wraps parsing errors', async t => {
- try {
- await got(`${s.url}/invalid`, {json: true});
- t.fail('Exception was not thrown');
- } catch (err) {
- t.regex(err.message, /Unexpected token/);
- t.true(err.message.indexOf(err.hostname) !== -1, err.message);
- t.is(err.path, '/invalid');
- }
-});
-
-test('parses non-200 responses', async t => {
- try {
- await got(`${s.url}/non200`, {json: true});
- t.fail('Exception was not thrown');
- } catch (err) {
- t.deepEqual(err.response.body, {data: 'dog'});
- }
-});
-
-test('catches errors on invalid non-200 responses', async t => {
- try {
- await got(`${s.url}/non200-invalid`, {json: true});
- t.fail('Exception was not thrown');
- } catch (err) {
- t.regex(err.message, /Unexpected token/);
- t.is(err.response.body, 'Internal error');
- t.is(err.path, '/non200-invalid');
- }
-});
-
-test('should have statusCode in err', async t => {
- try {
- await got(`${s.url}/non200-invalid`, {json: true});
- t.fail('Exception was not thrown');
- } catch (err) {
- t.is(err.statusCode, 500);
- }
-});
-
-test.after('cleanup', async () => {
- await s.close();
-});
diff --git a/test/merge-instances.ts b/test/merge-instances.ts
new file mode 100644
index 000000000..f32141e39
--- /dev/null
+++ b/test/merge-instances.ts
@@ -0,0 +1,177 @@
+import test from 'ava';
+import {Handler} from 'express';
+import got, {BeforeRequestHook, Got, Headers, NormalizedOptions} from '../source';
+import withServer from './helpers/with-server';
+
+const echoHeaders: Handler = (request, response) => {
+ response.end(JSON.stringify(request.headers));
+};
+
+test('merging instances', withServer, async (t, server) => {
+ server.get('/', echoHeaders);
+
+ const instanceA = got.extend({headers: {unicorn: 'rainbow'}});
+ const instanceB = got.extend({prefixUrl: server.url});
+ const merged = instanceA.extend(instanceB);
+
+ const headers = await merged('').json();
+ t.is(headers.unicorn, 'rainbow');
+ t.not(headers['user-agent'], undefined);
+});
+
+test('merges default handlers & custom handlers', withServer, async (t, server) => {
+ server.get('/', echoHeaders);
+
+ const instanceA = got.extend({headers: {unicorn: 'rainbow'}});
+ const instanceB = got.extend({
+ handlers: [
+ (options, next) => {
+ options.headers.cat = 'meow';
+ return next(options);
+ }
+ ]
+ });
+ const merged = instanceA.extend(instanceB);
+
+ const headers = await merged(server.url).json();
+ t.is(headers.unicorn, 'rainbow');
+ t.is(headers.cat, 'meow');
+});
+
+test('merging one group & one instance', withServer, async (t, server) => {
+ server.get('/', echoHeaders);
+
+ const instanceA = got.extend({headers: {dog: 'woof'}});
+ const instanceB = got.extend({headers: {cat: 'meow'}});
+ const instanceC = got.extend({headers: {bird: 'tweet'}});
+ const instanceD = got.extend({headers: {mouse: 'squeek'}});
+
+ const merged = instanceA.extend(instanceB, instanceC);
+ const doubleMerged = merged.extend(instanceD);
+
+ const headers = await doubleMerged(server.url).json();
+ t.is(headers.dog, 'woof');
+ t.is(headers.cat, 'meow');
+ t.is(headers.bird, 'tweet');
+ t.is(headers.mouse, 'squeek');
+});
+
+test('merging two groups of merged instances', withServer, async (t, server) => {
+ server.get('/', echoHeaders);
+
+ const instanceA = got.extend({headers: {dog: 'woof'}});
+ const instanceB = got.extend({headers: {cat: 'meow'}});
+ const instanceC = got.extend({headers: {bird: 'tweet'}});
+ const instanceD = got.extend({headers: {mouse: 'squeek'}});
+
+ const groupA = instanceA.extend(instanceB);
+ const groupB = instanceC.extend(instanceD);
+
+ const merged = groupA.extend(groupB);
+
+ const headers = await merged(server.url).json();
+ t.is(headers.dog, 'woof');
+ t.is(headers.cat, 'meow');
+ t.is(headers.bird, 'tweet');
+ t.is(headers.mouse, 'squeek');
+});
+
+test('hooks are merged', t => {
+ const getBeforeRequestHooks = (instance: Got): BeforeRequestHook[] => instance.defaults.options.hooks.beforeRequest;
+
+ const instanceA = got.extend({hooks: {
+ beforeRequest: [
+ options => {
+ options.headers.dog = 'woof';
+ }
+ ]
+ }});
+ const instanceB = got.extend({hooks: {
+ beforeRequest: [
+ options => {
+ options.headers.cat = 'meow';
+ }
+ ]
+ }});
+
+ const merged = instanceA.extend(instanceB);
+ t.deepEqual(getBeforeRequestHooks(merged), getBeforeRequestHooks(instanceA).concat(getBeforeRequestHooks(instanceB)));
+});
+
+test('default handlers are not duplicated', t => {
+ const instance = got.extend(got);
+ t.is(instance.defaults.handlers.length, 1);
+});
+
+test('URL is not polluted', withServer, async (t, server, got) => {
+ server.get('/', (_request, response) => {
+ response.end('ok');
+ });
+
+ await got({
+ username: 'foo'
+ });
+
+ const {options: normalizedOptions} = (await got({})).request;
+
+ t.is(normalizedOptions.username, '');
+});
+
+test('merging instances with HTTPS options', t => {
+ const instanceA = got.extend({https: {
+ rejectUnauthorized: true,
+ certificate: 'FIRST'
+ }});
+ const instanceB = got.extend({https: {
+ certificate: 'SECOND'
+ }});
+
+ const merged = instanceA.extend(instanceB);
+
+ t.true(merged.defaults.options.https?.rejectUnauthorized);
+ t.is(merged.defaults.options.https?.certificate, 'SECOND');
+});
+
+test('merging instances with HTTPS options undefined', t => {
+ const instanceA = got.extend({https: {
+ rejectUnauthorized: true,
+ certificate: 'FIRST'
+ }});
+ const instanceB = got.extend({https: {
+ certificate: undefined
+ }});
+
+ const merged = instanceA.extend(instanceB);
+
+ t.true(merged.defaults.options.https?.rejectUnauthorized);
+ t.is(merged.defaults.options.https?.certificate, undefined);
+});
+
+test('accepts options for promise API', t => {
+ got.extend({
+ hooks: {
+ beforeRequest: [
+ (options: NormalizedOptions): void => {
+ options.responseType = 'buffer';
+ }
+ ]
+ }
+ });
+
+ t.pass();
+});
+
+test('merging `prefixUrl`', t => {
+ const prefixUrl = 'http://example.com/';
+
+ const instanceA = got.extend({headers: {unicorn: 'rainbow'}});
+ const instanceB = got.extend({prefixUrl});
+ const mergedAonB = instanceB.extend(instanceA);
+ const mergedBonA = instanceA.extend(instanceB);
+
+ t.is(mergedAonB.defaults.options.prefixUrl, '');
+ t.is(mergedBonA.defaults.options.prefixUrl, prefixUrl);
+
+ t.is(instanceB.extend({}).defaults.options.prefixUrl, prefixUrl);
+ t.is(instanceB.extend({prefixUrl: undefined}).defaults.options.prefixUrl, prefixUrl);
+});
diff --git a/test/normalize-arguments.ts b/test/normalize-arguments.ts
new file mode 100644
index 000000000..a620e741b
--- /dev/null
+++ b/test/normalize-arguments.ts
@@ -0,0 +1,118 @@
+import {URL, URLSearchParams} from 'url';
+import test from 'ava';
+import got from '../source';
+
+test('should merge options replacing responseType', t => {
+ const responseType = 'json';
+ const options = got.mergeOptions(got.defaults.options, {
+ responseType
+ });
+
+ t.is(options.responseType, responseType);
+});
+
+test('no duplicated searchParams values', t => {
+ const options = got.mergeOptions(got.defaults.options, {
+ searchParams: 'string=true&noDuplication=true'
+ }, {
+ searchParams: new URLSearchParams({
+ instance: 'true',
+ noDuplication: 'true'
+ })
+ });
+
+ t.is(options.searchParams?.get('string'), 'true');
+ t.is(options.searchParams?.get('instance'), 'true');
+ t.is(options.searchParams?.getAll('noDuplication').length, 1);
+});
+
+test('should copy non-numerable properties', t => {
+ const options = {
+ json: {hello: '123'}
+ };
+
+ const merged = got.mergeOptions(got.defaults.options, options);
+ const mergedTwice = got.mergeOptions(got.defaults.options, merged);
+
+ t.is(mergedTwice.json, options.json);
+});
+
+test('should replace URLs', t => {
+ const options = got.mergeOptions({
+ url: new URL('http://localhost:41285'),
+ searchParams: new URLSearchParams('page=0')
+ }, {
+ url: 'http://localhost:41285/?page=1',
+ searchParams: undefined
+ });
+
+ const otherOptions = got.mergeOptions({
+ url: new URL('http://localhost:41285'),
+ searchParams: {
+ page: 0
+ }
+ }, {
+ url: 'http://localhost:41285/?page=1',
+ searchParams: undefined
+ });
+
+ t.is(options.url.href, 'http://localhost:41285/?page=1');
+ t.is(otherOptions.url.href, 'http://localhost:41285/?page=1');
+});
+
+test('should get username and password from the URL', t => {
+ const options = got.mergeOptions({
+ url: 'http://user:pass@localhost:41285'
+ });
+
+ t.is(options.username, 'user');
+ t.is(options.password, 'pass');
+});
+
+test('should get username and password from the options', t => {
+ const options = got.mergeOptions({
+ url: 'http://user:pass@localhost:41285',
+ username: 'user_OPT',
+ password: 'pass_OPT'
+ });
+
+ t.is(options.username, 'user_OPT');
+ t.is(options.password, 'pass_OPT');
+});
+
+test('should get username and password from the merged options', t => {
+ const options = got.mergeOptions(
+ {
+ url: 'http://user:pass@localhost:41285'
+ },
+ {
+ username: 'user_OPT_MERGE',
+ password: 'pass_OPT_MERGE'
+ }
+ );
+
+ t.is(options.username, 'user_OPT_MERGE');
+ t.is(options.password, 'pass_OPT_MERGE');
+});
+
+test('null value in search params means empty', t => {
+ const options = got.mergeOptions({
+ url: new URL('http://localhost'),
+ searchParams: {
+ foo: null
+ }
+ });
+
+ t.is(options.url.href, 'http://localhost/?foo=');
+});
+
+test('undefined value in search params means it does not exist', t => {
+ const options = got.mergeOptions({
+ url: new URL('http://localhost'),
+ searchParams: {
+ foo: undefined
+ }
+ });
+
+ t.is(options.url.href, 'http://localhost/');
+});
diff --git a/test/pagination.ts b/test/pagination.ts
new file mode 100644
index 000000000..804708792
--- /dev/null
+++ b/test/pagination.ts
@@ -0,0 +1,690 @@
+import {URL} from 'url';
+import test from 'ava';
+import delay = require('delay');
+import getStream = require('get-stream');
+import got, {Response} from '../source';
+import withServer, {withBodyParsingServer} from './helpers/with-server';
+import {ExtendedHttpTestServer} from './helpers/create-http-test-server';
+
+const thrower = (): any => {
+ throw new Error('This should not be called');
+};
+
+const resetPagination = {
+ paginate: undefined,
+ transform: undefined,
+ filter: undefined,
+ shouldContinue: undefined
+};
+
+const attachHandler = (server: ExtendedHttpTestServer, count: number): void => {
+ server.get('/', (request, response) => {
+ const searchParameters = new URLSearchParams(request.url.split('?')[1]);
+ const page = Number(searchParameters.get('page')) || 1;
+
+ if (page < count) {
+ response.setHeader('link', `<${server.url}/?page=${page + 1}>; rel="next"`);
+ }
+
+ response.end(`[${page <= count ? page : ''}]`);
+ });
+};
+
+test('the link header has no next value', withServer, async (t, server, got) => {
+ const items = [1];
+
+ server.get('/', (_request, response) => {
+ response.setHeader('link', '; rel="prev"');
+ response.end(JSON.stringify(items));
+ });
+
+ const received = await got.paginate.all('');
+ t.deepEqual(received, items);
+});
+
+test('retrieves all elements', withServer, async (t, server, got) => {
+ attachHandler(server, 2);
+
+ const result = await got.paginate.all('');
+
+ t.deepEqual(result, [1, 2]);
+});
+
+test('retrieves all elements with JSON responseType', withServer, async (t, server, got) => {
+ attachHandler(server, 2);
+
+ const result = await got.extend({
+ responseType: 'json'
+ }).paginate.all('');
+
+ t.deepEqual(result, [1, 2]);
+});
+
+test('points to defaults when extending Got without custom `pagination`', withServer, async (t, server, got) => {
+ attachHandler(server, 2);
+
+ const result = await got.extend().paginate.all('');
+
+ t.deepEqual(result, [1, 2]);
+});
+
+test('pagination options can be extended', withServer, async (t, server, got) => {
+ attachHandler(server, 2);
+
+ const result = await got.extend({
+ pagination: {
+ shouldContinue: () => false
+ }
+ }).paginate.all('');
+
+ t.deepEqual(result, []);
+});
+
+test('filters elements', withServer, async (t, server, got) => {
+ attachHandler(server, 3);
+
+ const result = await got.paginate.all({
+ pagination: {
+ filter: (element: number, allItems: number[], currentItems: number[]) => {
+ t.true(Array.isArray(allItems));
+ t.true(Array.isArray(currentItems));
+
+ return element !== 2;
+ }
+ }
+ });
+
+ t.deepEqual(result, [1, 3]);
+});
+
+test('parses elements', withServer, async (t, server, got) => {
+ attachHandler(server, 100);
+
+ const result = await got.paginate.all('?page=100', {
+ pagination: {
+ transform: response => [response.body.length]
+ }
+ });
+
+ t.deepEqual(result, [5]);
+});
+
+test('parses elements - async function', withServer, async (t, server, got) => {
+ attachHandler(server, 100);
+
+ const result = await got.paginate.all('?page=100', {
+ pagination: {
+ transform: async response => [response.body.length]
+ }
+ });
+
+ t.deepEqual(result, [5]);
+});
+
+test('custom paginate function', withServer, async (t, server, got) => {
+ attachHandler(server, 3);
+
+ const result = await got.paginate.all({
+ pagination: {
+ paginate: response => {
+ const url = new URL(response.url);
+
+ if (url.search === '?page=3') {
+ return false;
+ }
+
+ url.search = '?page=3';
+
+ return {url};
+ }
+ }
+ });
+
+ t.deepEqual(result, [1, 3]);
+});
+
+test('custom paginate function using allItems', withServer, async (t, server, got) => {
+ attachHandler(server, 3);
+
+ const result = await got.paginate.all({
+ pagination: {
+ paginate: (_response, allItems: number[]) => {
+ if (allItems.length === 2) {
+ return false;
+ }
+
+ return {path: '/?page=3'};
+ }
+ }
+ });
+
+ t.deepEqual(result, [1, 3]);
+});
+
+test('custom paginate function using currentItems', withServer, async (t, server, got) => {
+ attachHandler(server, 3);
+
+ const result = await got.paginate.all({
+ pagination: {
+ paginate: (_response, _allItems: number[], currentItems: number[]) => {
+ if (currentItems[0] === 3) {
+ return false;
+ }
+
+ return {path: '/?page=3'};
+ }
+ }
+ });
+
+ t.deepEqual(result, [1, 3]);
+});
+
+test('iterator works', withServer, async (t, server, got) => {
+ attachHandler(server, 5);
+
+ const results: number[] = [];
+
+ for await (const item of got.paginate('')) {
+ results.push(item);
+ }
+
+ t.deepEqual(results, [1, 2, 3, 4, 5]);
+});
+
+test('iterator works #2', withServer, async (t, server, got) => {
+ attachHandler(server, 5);
+
+ const results: number[] = [];
+
+ for await (const item of got.paginate.each('')) {
+ results.push(item);
+ }
+
+ t.deepEqual(results, [1, 2, 3, 4, 5]);
+});
+
+test('`shouldContinue` works', withServer, async (t, server, got) => {
+ attachHandler(server, 2);
+
+ const options = {
+ pagination: {
+ shouldContinue: (_item: unknown, allItems: unknown[], currentItems: unknown[]) => {
+ t.true(Array.isArray(allItems));
+ t.true(Array.isArray(currentItems));
+
+ return false;
+ }
+ }
+ };
+
+ const results: number[] = [];
+
+ for await (const item of got.paginate
-
+
- +
+
Huge thanks to for sponsoring Sindre Sorhus!
+
(they love Got too!)
++
+
Example
+ +Using [`bourne`](https://github.com/hapijs/bourne) to prevent prototype pollution: + +```js +const got = require('got'); +const Bourne = require('@hapi/bourne'); + +(async () => { + const parsed = await got('https://example.com', { + parseJson: text => Bourne.parse(text) + }).json(); + + console.log(parsed); +})(); +``` +Examples
+ +Ignore properties starting with `_`: + +```js +const got = require('got'); + +(async () => { + await got.post('https://example.com', { + stringifyJson: object => JSON.stringify(object, (key, value) => { + if (key.startsWith('_')) { + return; + } + + return value; + }), + json: { + some: 'payload', + _ignoreMe: 1234 + } + }); +})(); +``` + +All numbers as strings: + +```js +const got = require('got'); + +(async () => { + await got.post('https://example.com', { + stringifyJson: object => JSON.stringify(object, (key, value) => { + if (typeof value === 'number') { + return value.toString(); + } + + return value; + }), + json: { + some: 'payload', + number: 1 + } + }); +})(); +``` +-Default: `'utf8'` +**Note:** Handlers can be asynchronous. The recommended approach is: -Encoding to be used on `setEncoding` of the response data. If `null`, the body is returned as a Buffer. +```js +const handler = (options, next) => { + if (options.isStream) { + // It's a Stream + return next(options); + } -###### json + // It's a Promise + return (async () => { + try { + const response = await next(options); + response.yourOwnProperty = true; + return response; + } catch (error) { + // Every error will be replaced by this one. + // Before you receive any error here, + // it will be passed to the `beforeError` hooks first. + // Note: this one won't be passed to `beforeError` hook. It's final. + throw new Error('Your very own error.'); + } + })(); +}; -Type: `boolean`
-Default: `false` +const instance = got.extend({handlers: [handler]}); +``` -*This is mutually exclusive with stream mode.* +#### got.extend(...options, ...instances, ...) -Parse response body with `JSON.parse` and set `accept` header to `application/json`. +Merges many instances into a single one: +- options are merged using [`got.mergeOptions()`](#gotmergeoptionsparentoptions-newoptions) (including hooks), +- handlers are stored in an array (you can access them through `instance.defaults.handlers`). -###### query +```js +const a = {headers: {cat: 'meow'}}; +const b = got.extend({ + options: { + headers: { + cow: 'moo' + } + } +}); -Type: `string`, `object`
+// The same as `got.extend(a).extend(b)`. +// Note `a` is options and `b` is an instance. +got.extend(a, b); +//=> {headers: {cat: 'meow', cow: 'moo'}} +``` -Query string object that will be added to the request URL. This will override the query string in `url`. +#### got.mergeOptions(parent, ...sources) -###### timeout +Extends parent options. Avoid using [object spread](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Spread_syntax#Spread_in_object_literals) as it doesn't work recursively: -Type: `number`, `object` +```js +const a = {headers: {cat: 'meow', wolf: ['bark', 'wrrr']}}; +const b = {headers: {cow: 'moo', wolf: ['auuu']}}; -Milliseconds to wait for a server to send response headers before aborting request with `ETIMEDOUT` error. +{...a, ...b} // => {headers: {cow: 'moo', wolf: ['auuu']}} +got.mergeOptions(a, b) // => {headers: {cat: 'meow', cow: 'moo', wolf: ['auuu']}} +``` -Option accepts `object` with separate `connect` and `socket` fields for connection and socket inactivity timeouts. +**Note:** Only Got options are merged! Custom user options should be defined via [`options.context`](#context). -###### retries +Options are deeply merged to a new object. The value of each key is determined as follows: -Type: `number`, `function`
-Default: `5` +- If the new property is not defined, the old value is used. +- If the new property is explicitly set to `undefined`: + - If the parent property is a plain `object`, the parent value is deeply cloned. + - Otherwise, `undefined` is used. +- If the parent value is an instance of `URLSearchParams`: + - If the new value is a `string`, an `object` or an instance of `URLSearchParams`, a new `URLSearchParams` instance is created. The values are merged using [`urlSearchParams.append(key, value)`](https://developer.mozilla.org/en-US/docs/Web/API/URLSearchParams/append). The keys defined in the new value override the keys defined in the parent value. Please note that `null` values point to an empty string and `undefined` values will exclude the entry. + - Otherwise, the only available value is `undefined`. +- If the new property is a plain `object`: + - If the parent property is a plain `object` too, both values are merged recursively into a new `object`. + - Otherwise, only the new value is deeply cloned. +- If the new property is an `Array`, it overwrites the old one with a deep clone of the new property. +- Properties that are not enumerable, such as `context`, `body`, `json`, and `form`, will not be merged. +- Otherwise, the new value is assigned to the key. -Number of request retries when network errors happens. Delays between retries counts with function `1000 * Math.pow(2, retry) + Math.random() * 100`, where `retry` is attempt number (starts from 0). +```js +const a = {json: {cat: 'meow'}}; +const b = {json: {cow: 'moo'}}; -Option accepts `function` with `retry` and `error` arguments. Function must return delay in milliseconds (`0` return value cancels retry). +got.mergeOptions(a, b); +//=> {json: {cow: 'moo'}} +``` -**Note:** if `retries` is `number`, `ENOTFOUND` and `ENETUNREACH` error will not be retried (see full list in [`is-retry-allowed`](https://github.com/floatdrop/is-retry-allowed/blob/master/index.js#L12) module). +#### got.defaults -###### followRedirect +Type: `object` -Type: `boolean`
-Default: `true` +The Got defaults used in that instance. -Defines if redirect responses should be followed automatically. +##### [options](#options) +##### handlers -#### Streams +Type: `Function[]`\ +Default: `[]` -#### got.stream(url, [options]) +An array of functions. You execute them directly by calling `got()`. They are some sort of "global hooks" - these functions are called first. The last handler (*it's hidden*) is either [`asPromise`](source/core/as-promise/index.ts) or [`asStream`](source/core/index.ts), depending on the `options.isStream` property. -`stream` method will return Duplex stream with additional events: +Each handler takes two arguments: -##### .on('request', request) +###### [options](#options) -`request` event to get the request object of the request. +###### next() -**Tip**: You can use `request` event to abort request: +Returns a `Promise` or a `Stream` depending on [`options.isStream`](#isstream). ```js -got.stream('github.com') - .on('request', req => setTimeout(() => req.abort(), 50)); +const settings = { + handlers: [ + (options, next) => { + if (options.isStream) { + // It's a Stream, so we can perform stream-specific actions on it + return next(options) + .on('request', request => { + setTimeout(() => { + request.abort(); + }, 50); + }); + } + + // It's a Promise + return next(options); + } + ], + options: got.mergeOptions(got.defaults.options, { + responseType: 'json' + }) +}; + +const jsonGot = got.extend(settings); ``` -##### .on('response', response) +##### mutableDefaults + +Type: `boolean`\ +Default: `false` -`response` event to get the response object of the final request. +A read-only boolean describing whether the defaults are mutable or not. If set to `true`, you can [update headers over time](#hooksafterresponse), for example, update an access token when it expires. -##### .on('redirect', response, nextOptions) +## Types -`redirect` event to get the response object of a redirect. The second argument is options for the next request to the redirect location. +Got exports some handy TypeScript types and interfaces. See the type definition for all the exported types. -##### .on('error', error, body, response) +### Got -`error` event emitted in case of protocol error (like `ENOTFOUND` etc.) or status error (4xx or 5xx). The second argument is the body of the server response in case of status error. The third argument is response object. +TypeScript will automatically infer types for Got instances, but in case you want to define something like dependencies, you can import the available types directly from Got. -#### got.get(url, [options]) -#### got.post(url, [options]) -#### got.put(url, [options]) -#### got.patch(url, [options]) -#### got.head(url, [options]) -#### got.delete(url, [options]) +```ts +import {GotRequestFunction} from 'got'; -Sets `options.method` to the method name and makes a request. +interface Dependencies { + readonly post: GotRequestFunction +} +``` +### Hooks -## Errors +When writing hooks, you can refer to their types to keep your interfaces consistent. + +```ts +import {BeforeRequestHook} from 'got'; + +const addAccessToken = (accessToken: string): BeforeRequestHook => options => { + options.path = `${options.path}?access_token=${accessToken}`; +} +``` -Each error contains (if available) `statusCode`, `statusMessage`, `host`, `hostname`, `method` and `path` properties to make debugging easier. +## Errors -In Promise mode, the `response` is attached to the error. +Each error contains an `options` property which are the options Got used to create a request - just to make debugging easier.\ +Additionaly, the errors may have `request` (Got Stream) and `response` (Got Response) properties depending on which phase of the request failed. #### got.RequestError -When a request fails. Contains a `code` property with error class code, like `ECONNREFUSED`. +When a request fails. Contains a `code` property with error class code, like `ECONNREFUSED`. If there is no specific code supplied, `code` defaults to `ERR_GOT_REQUEST_ERROR`. All the errors below inherit this one. + +#### got.CacheError + +When a cache method fails, for example, if the database goes down or there's a filesystem error. Contains a `code` property with `ERR_CACHE_ACCESS` or a more specific failure code. #### got.ReadError -When reading from response stream fails. +When reading from response stream fails. Contains a `code` property with `ERR_READING_RESPONSE_STREAM` or a more specific failure code. #### got.ParseError -When `json` option is enabled and `JSON.parse` fails. +When server response code is 2xx, and parsing body fails. Includes a `response` property. Contains a `code` property with `ERR_BODY_PARSE_FAILURE` or a more specific failure code. + +#### got.UploadError + +When the request body is a stream and an error occurs while reading from that stream. Contains a `code` property with `ERR_UPLOAD` or a more specific failure code. #### got.HTTPError -When server response code is not 2xx. Contains `statusCode` and `statusMessage`. +When the server response code is not 2xx nor 3xx if `options.followRedirect` is `true`, but always except for 304. Includes a `response` property. Contains a `code` property with `ERR_NON_2XX_3XX_RESPONSE` or a more specific failure code. + #### got.MaxRedirectsError -When server redirects you more than 10 times. +When the server redirects you more than ten times. Includes a `response` property. Contains a `code` property with `ERR_TOO_MANY_REDIRECTS`. +#### got.UnsupportedProtocolError -## Proxies +When given an unsupported protocol. Contains a `code` property with `ERR_UNSUPPORTED_PROTOCOL`. + +#### got.TimeoutError + +When the request is aborted due to a [timeout](#timeout). Includes an `event` and `timings` property. Contains a `code` property with `ETIMEDOUT`. + +#### got.CancelError + +When the request is aborted with `.cancel()`. Contains a `code` property with `ERR_CANCELED`. -You can use the [`tunnel`](https://github.com/koichik/node-tunnel) module with the `agent` option to work with proxies: +## Aborting the request + +The promise returned by Got has a [`.cancel()`](https://github.com/sindresorhus/p-cancelable) method which when called, aborts the request. + +```js +(async () => { + const request = got(url, options); + + // … + + // In another part of the code + if (something) { + request.cancel(); + } + + // … + + try { + await request; + } catch (error) { + if (request.isCanceled) { // Or `error instanceof got.CancelError` + // Handle cancelation + } + + // Handle other errors + } +})(); +``` + +When using hooks, simply throw an error to abort the request. ```js const got = require('got'); -const tunnel = require('tunnel'); -got('todomvc.com', { - agent: tunnel.httpOverHttp({ - proxy: { - host: 'localhost' +(async () => { + const request = got(url, { + hooks: { + beforeRequest: [ + () => { + throw new Error('Oops. Request canceled.'); + } + ] } - }) -}); + }); + + try { + await request; + } catch (error) { + // … + } +})(); ``` +To abort the Got Stream request, just call `stream.destroy()`. -## Cookies +```js +const got = require('got'); + +const stream = got.stream(url); +stream.destroy(); +``` -You can use the [`cookie`](https://github.com/jshttp/cookie) module to include cookies in a request: + +## Cache + +Got implements [RFC 7234](https://httpwg.org/specs/rfc7234.html) compliant HTTP caching which works out of the box in-memory and is easily pluggable with a wide range of storage adapters. Fresh cache entries are served directly from the cache, and stale cache entries are revalidated with `If-None-Match`/`If-Modified-Since` headers. You can read more about the underlying cache behavior in the [`cacheable-request` documentation](https://github.com/lukechilds/cacheable-request). For DNS cache, Got uses [`cacheable-lookup`](https://github.com/szmarczak/cacheable-lookup). + +You can use the JavaScript `Map` type as an in-memory cache: ```js const got = require('got'); -const cookie = require('cookie'); -got('google.com', { - headers: { - cookie: cookie.serialize('foo', 'bar') +const map = new Map(); + +(async () => { + let response = await got('https://sindresorhus.com', {cache: map}); + console.log(response.isFromCache); + //=> false + + response = await got('https://sindresorhus.com', {cache: map}); + console.log(response.isFromCache); + //=> true +})(); +``` + +Got uses [Keyv](https://github.com/lukechilds/keyv) internally to support a wide range of storage adapters. For something more scalable you could use an [official Keyv storage adapter](https://github.com/lukechilds/keyv#official-storage-adapters): + +``` +$ npm install @keyv/redis +``` + +```js +const got = require('got'); +const KeyvRedis = require('@keyv/redis'); + +const redis = new KeyvRedis('redis://user:pass@localhost:6379'); + +got('https://sindresorhus.com', {cache: redis}); +``` + +Got supports anything that follows the Map API, so it's easy to write your own storage adapter or use a third-party solution. + +For example, the following are all valid storage adapters: + +```js +const storageAdapter = new Map(); +// Or +const storageAdapter = require('./my-storage-adapter'); +// Or +const QuickLRU = require('quick-lru'); +const storageAdapter = new QuickLRU({maxSize: 1000}); + +got('https://sindresorhus.com', {cache: storageAdapter}); +``` + +View the [Keyv docs](https://github.com/lukechilds/keyv) for more information on how to use storage adapters. + +## Proxies + +You can use the [`tunnel`](https://github.com/koichik/node-tunnel) package with the `agent` option to work with proxies: + +```js +const got = require('got'); +const tunnel = require('tunnel'); + +got('https://sindresorhus.com', { + agent: { + https: tunnel.httpsOverHttp({ + proxy: { + host: 'localhost' + } + }) + } +}); +``` + +Otherwise, you can use the [`hpagent`](https://github.com/delvedor/hpagent) package, which keeps the internal sockets alive to be reused. + +```js +const got = require('got'); +const {HttpsProxyAgent} = require('hpagent'); + +got('https://sindresorhus.com', { + agent: { + https: new HttpsProxyAgent({ + keepAlive: true, + keepAliveMsecs: 1000, + maxSockets: 256, + maxFreeSockets: 256, + scheduling: 'lifo', + proxy: 'https://localhost:8080' + }) } }); ``` +Alternatively, use [`global-agent`](https://github.com/gajus/global-agent) to configure a global proxy for all HTTP/HTTPS traffic in your program. + +Read the [`http2-wrapper`](https://github.com/szmarczak/http2-wrapper/#proxy-support) docs to learn about proxying for HTTP/2. + +## Cookies + +You can use the [`tough-cookie`](https://github.com/salesforce/tough-cookie) package: + +```js +const {promisify} = require('util'); +const got = require('got'); +const {CookieJar} = require('tough-cookie'); + +(async () => { + const cookieJar = new CookieJar(); + const setCookie = promisify(cookieJar.setCookie.bind(cookieJar)); + + await setCookie('foo=bar', 'https://example.com'); + await got('https://example.com', {cookieJar}); +})(); +``` ## Form data -You can use the [`form-data`](https://github.com/form-data/form-data) module to create POST request with form data: +You can use the [`form-data`](https://github.com/form-data/form-data) package to create POST request with form data: ```js const fs = require('fs'); const got = require('got'); const FormData = require('form-data'); + const form = new FormData(); form.append('my_file', fs.createReadStream('/foo/bar.jpg')); -got.post('google.com', { +got.post('https://example.com', { body: form }); ``` - ## OAuth -You can use the [`oauth-1.0a`](https://github.com/ddo/oauth-1.0a) module to create a signed OAuth request: +You can use the [`oauth-1.0a`](https://github.com/ddo/oauth-1.0a) package to create a signed OAuth request: ```js const got = require('got'); @@ -280,56 +1967,511 @@ const url = 'https://api.twitter.com/1.1/statuses/home_timeline.json'; got(url, { headers: oauth.toHeader(oauth.authorize({url, method: 'GET'}, token)), - json: true + responseType: 'json' }); ``` - ## Unix Domain Sockets Requests can also be sent via [unix domain sockets](http://serverfault.com/questions/124517/whats-the-difference-between-unix-socket-and-tcp-ip-socket). Use the following URL scheme: `PROTOCOL://unix:SOCKET:PATH`. - `PROTOCOL` - `http` or `https` *(optional)* -- `SOCKET` - absolute path to a unix domain socket, e.g. `/var/run/docker.sock` -- `PATH` - request path, e.g. `/v2/keys` +- `SOCKET` - Absolute path to a unix domain socket, for example: `/var/run/docker.sock` +- `PATH` - Request path, for example: `/v2/keys` ```js +const got = require('got'); + got('http://unix:/var/run/docker.sock:/containers/json'); -// or without protocol (http by default) +// Or without protocol (HTTP by default) got('unix:/var/run/docker.sock:/containers/json'); ``` +## AWS + +Requests to AWS services need to have their headers signed. This can be accomplished by using the [`got4aws`](https://www.npmjs.com/package/got4aws) package. This is an example for querying an ["API Gateway"](https://docs.aws.amazon.com/apigateway/api-reference/signing-requests/) with a signed request. + +```js +const got4aws = require('got4aws');; + +const awsClient = got4aws(); + +const response = await awsClient('https://
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
++ +> Segment is a happy user of Got! Got powers the main backend API that our app talks to. It's used by our in-house RPC client that we use to communicate with all microservices. +> +> — Vadim Demedes + +> Antora, a static site generator for creating documentation sites, uses Got to download the UI bundle. In Antora, the UI bundle (aka theme) is maintained as a separate project. That project exports the UI as a zip file we call the UI bundle. The main site generator downloads that UI from a URL using Got and streams it to vinyl-zip to extract the files. Those files go on to be used to create the HTML pages and supporting assets. +> +> — Dan Allen + +> GetVoIP is happily using Got in production. One of the unique capabilities of Got is the ability to handle Unix sockets which enables us to build a full control interfaces for our docker stack. +> +> — Daniel Kalen + +> We're using Got inside of Exoframe to handle all the communication between CLI and server. Exoframe is a self-hosted tool that allows simple one-command deployments using Docker. +> +> — Tim Ermilov + +> Karaoke Mugen uses Got to fetch content updates from its online server. +> +> — Axel Terizaki + +> Renovate uses Got, gh-got and gl-got to send millions of queries per day to GitHub, GitLab, npmjs, PyPi, Packagist, Docker Hub, Terraform, CircleCI, and more. +> +> — Rhys Arkins + +> Resistbot uses Got to communicate from the API frontend where all correspondence ingresses to the officials lookup database in back. +> +> — Chris Erickson + +> Natural Cycles is using Got to communicate with all kinds of 3rd-party REST APIs (over 9000!). +> +> — Kirill Groshkov + +> Microlink is a cloud browser as an API service that uses Got widely as the main HTTP client, serving ~22M requests a month, every time a network call needs to be performed. +> +> — Kiko Beats + +> We’re using Got at Radity. Thanks for such an amazing work! +> +> — Mirzayev Farid + +## For enterprise + +Available as part of the Tidelift Subscription. + +The maintainers of `got` and thousands of other packages are working with Tidelift to deliver commercial support and maintenance for the open source dependencies you use to build your applications. Save time, reduce risk, and improve code health, while paying the maintainers of the exact dependencies you use. [Learn more.](https://tidelift.com/subscription/pkg/npm-got?utm_source=npm-got&utm_medium=referral&utm_campaign=enterprise&utm_term=repo) diff --git a/source/as-promise/create-rejection.ts b/source/as-promise/create-rejection.ts new file mode 100644 index 000000000..358e1fe76 --- /dev/null +++ b/source/as-promise/create-rejection.ts @@ -0,0 +1,31 @@ +import {CancelableRequest, BeforeErrorHook, RequestError} from './types'; + +export default function createRejection(error: Error, ...beforeErrorGroups: Array