Node.js implementation of a proxy server (think Squid) with support for SSL, authentication, upstream proxy chaining, custom HTTP responses and measuring traffic statistics. The authentication and proxy chaining configuration is defined in code and can be dynamic. Note that the proxy server only supports Basic authentication (see Proxy-Authorization for details).
For example, this package is useful if you need to use proxies with authentication
in the headless Chrome web browser, because it doesn't accept proxy URLs such as http://username:password@proxy.example.com:8080.
With this library, you can set up a local proxy server without any password
that will forward requests to the upstream proxy with password.
The package is used for this exact purpose by the Apify web scraping platform.
To learn more about the rationale behind this package, read How to make headless Chrome and Puppeteer use a proxy server with authentication.
const ProxyChain = require('proxy-chain');
const server = new ProxyChain.Server({ port: 8000 });
server.listen(() => {
console.log(`Proxy server is listening on port ${8000}`);
});const ProxyChain = require('proxy-chain');
const server = new ProxyChain.Server({
// Port where the server will listen. By default 8000.
port: 8000,
// Enables verbose logging
verbose: true,
// Custom user-defined function to authenticate incoming proxy requests,
// and optionally provide the URL to chained upstream proxy.
// The function must return an object (or promise resolving to the object) with the following signature:
// { requestAuthentication: Boolean, upstreamProxyUrl: String }
// If the function is not defined or is null, the server runs in simple mode.
// Note that the function takes a single argument with the following properties:
// * request - An instance of http.IncomingMessage class with information about the client request
// (which is either HTTP CONNECT for SSL protocol, or other HTTP request)
// * username - Username parsed from the Proxy-Authorization header. Might be empty string.
// * password - Password parsed from the Proxy-Authorization header. Might be empty string.
// * hostname - Hostname of the target server
// * port - Port of the target server
// * isHttp - If true, this is a HTTP request, otherwise it's a HTTP CONNECT tunnel for SSL
// or other protocols
// * connectionId - Unique ID of the HTTP connection. It can be used to obtain traffic statistics.
prepareRequestFunction: ({ request, username, password, hostname, port, isHttp, connectionId }) => {
return {
// If set to true, the client is sent HTTP 407 resposne with the Proxy-Authenticate header set,
// requiring Basic authentication. Here you can verify user credentials.
requestAuthentication: username !== 'bob' || password !== 'TopSecret',
// Sets up an upstream HTTP proxy to which all the requests are forwarded.
// If null, the proxy works in direct mode, i.e. the connection is forwarded directly
// to the target server. This field is ignored if "requestAuthentication" is true.
// The username and password should be URI-encoded, in case it contains some special characters.
// See `parseUrl()` function for details.
upstreamProxyUrl: `http://username:password@proxy.example.com:3128`,
// If "requestAuthentication" is true, you can use the following property
// to define a custom error message to return to the client instead of the default "Proxy credentials required"
failMsg: 'Bad username or password, please try again.',
};
},
});
server.listen(() => {
console.log(`Proxy server is listening on port ${server.port}`);
});
// Emitted when HTTP connection is closed
server.on('connectionClosed', ({ connectionId, stats }) => {
console.log(`Connection ${connectionId} closed`);
console.dir(stats);
});
// Emitted when HTTP request fails
server.on('requestFailed', ({ request, error }) => {
console.log(`Request ${request.url} failed`);
console.error(error);
});To return a custom HTTP response to indicate an error to the client,
you can throw the RequestError from inside of the prepareRequestFunction function.
The class constructor has the following parameters: RequestError(body, statusCode, headers).
By default, the response will have Content-Type: text/plain; charset=utf-8.
const ProxyChain = require('proxy-chain');
const server = new ProxyChain.Server({
prepareRequestFunction: ({ request, username, password, hostname, port, isHttp, connectionId }) => {
if (username !== 'bob') {
throw new ProxyChain.RequestError('Only Bob can use this proxy!', 400);
}
},
});To get traffic statistics for a certain HTTP connection, you can use:
const stats = server.getConnectionStats(connectionId);
console.dir(stats);The resulting object looks like:
{
// Number of bytes sent to client
srcTxBytes: Number,
// Number of bytes received from client
srcRxBytes: Number,
// Number of bytes sent to target server (proxy or website)
trgTxBytes: Number,
// Number of bytes received from target server (proxy or website)
trgRxBytes: Number,
}If the underlying sockets were closed, the corresponding values will be null,
rather than 0.
Custom responses allow you to override the response to a HTTP requests to the proxy, without contacting any target host. For example, this is useful if you want to provide a HTTP proxy-style interface to an external API or respond with some custom page to certain requests. Note that this feature is only available for HTTP connections. That's because HTTPS connections cannot be intercepted without access to the target host's private key.
To provide a custom response, the result of the prepareRequestFunction function must
define the customResponseFunction property, which contains a function that generates the custom response.
The function is passed no parameters and it must return an object (or a promise resolving to an object)
with the following properties:
{
// Optional HTTP status code of the response. By default it is 200.
statusCode: 200,
// Optional HTTP headers of the response
headers: {
'X-My-Header': 'bla bla',
}
// Optional string with the body of the HTTP response
body: 'My custom response',
// Optional encoding of the body. If not provided, defaults to 'UTF-8'
encoding: 'UTF-8',
}Here is a simple example:
const ProxyChain = require('proxy-chain');
const server = new ProxyChain.Server({
port: 8000,
prepareRequestFunction: ({ request, username, password, hostname, port, isHttp }) => {
return {
customResponseFunction: () => {
return {
statusCode: 200,
body: `My custom response to ${request.url}`,
};
},
};
},
});
server.listen(() => {
console.log(`Proxy server is listening on port ${server.port}`);
});To shut down the proxy server, call the close([destroyConnections], [callback]) function. For example:
server.close(true, () => {
console.log('Proxy server was closed.');
});The closeConnections parameter indicates whether pending proxy connections should be forcibly closed.
If it's false, the function will wait until all connections are closed, which can take a long time.
If the callback parameter is omitted, the function returns a promise.
Some upstream proxy providers might include valuable debugging information in the CONNECT response headers when establishing the proxy tunnel, for they may not modify future data in the tunneled connection.
The proxy server would emit a tunnelConnectResponded event for exposing such information, where
the parameter types of the event callback are described in Node.js's documentation. Example:
server.on('tunnelConnectResponded', ({ response, socket, head }) => {
console.log(`CONNECT response headers received: ${response.headers}`);
});Alternatively a helper function may be used:
listenConnectAnonymizedProxy(anonymizedProxyUrl, ({ response, socket, head }) => {
console.log(`CONNECT response headers received: ${response.headers}`);
});The package also provides several utility functions.
Parses and validates a HTTP proxy URL. If the proxy requires authentication, then the function starts an open local proxy server that forwards to the proxy. The port is chosen randomly.
The function takes an optional callback that receives the anonymous proxy URL. If no callback is supplied, the function returns a promise that resolves to a String with anonymous proxy URL or the original URL if it was already anonymous.
The following example shows how you can use a proxy with authentication from headless Chrome and Puppeteer. For details, read this blog post.
const puppeteer = require('puppeteer');
const proxyChain = require('proxy-chain');
(async() => {
const oldProxyUrl = 'http://bob:password123@proxy.example.com:8000';
const newProxyUrl = await proxyChain.anonymizeProxy(oldProxyUrl);
// Prints something like "http://127.0.0.1:45678"
console.log(newProxyUrl);
const browser = await puppeteer.launch({
args: [`--proxy-server=${newProxyUrl}`],
});
// Do your magic here...
const page = await browser.newPage();
await page.goto('https://www.example.com');
await page.screenshot({ path: 'example.png' });
await browser.close();
// Clean up
await proxyChain.closeAnonymizedProxy(newProxyUrl, true);
})();Closes anonymous proxy previously started by anonymizeProxy().
If proxy was not found or was already closed, the function has no effect
and its result is false. Otherwise the result is true.
The closeConnections parameter indicates whether pending proxy connections are forcibly closed.
If it's false, the function will wait until all connections are closed, which can take a long time.
The function takes an optional callback that receives the result Boolean from the function. If callback is not provided, the function returns a promise instead.
Creates a TCP tunnel to targetHost that goes through a HTTP proxy server
specified by the proxyUrl parameter.
The optional options parameter is an object with the following properties:
port: Number- Enables specifying the local port to listen at. By default0, which means a random port will be selected.hostname: String- Local hostname to listen at. By defaultlocalhost.verbose: Boolean- Iftrue, the functions logs a lot. By defaultfalse.
The result of the function is a local endpoint in a form of hostname:port.
All TCP connections made to the local endpoint will be tunneled through the proxy to the target host and port.
For example, this is useful if you want to access a certain service from a specific IP address.
The tunnel should be eventually closed by calling the closeTunnel() function.
The createTunnel() function accepts an optional Node.js-style callback that receives the path to the local endpoint.
If no callback is supplied, the function returns a promise that resolves to a String with
the path to the local endpoint.
For more information, read this blog post.
Example:
const host = await createTunnel('http://bob:pass123@proxy.example.com:8000', 'service.example.com:356');
// Prints something like "localhost:56836"
console.log(host);Closes tunnel previously started by createTunnel().
The result value is false if the tunnel was not found or was already closed, otherwise it is true.
The closeConnections parameter indicates whether pending connections are forcibly closed.
If it's false, the function will wait until all connections are closed, which can take a long time.
The function takes an optional callback that receives the result of the function. If the callback is not provided, the function returns a promise instead.
Allows to configure a callback on the anonymized proxy URL for the CONNECT response headers. See the above section Accessing the CONNECT response headers for proxy tunneling for details.
An utility function for parsing URLs.
It parses the URL using Node.js' new URL(url) and adds the following features:
- The result is a vanilla JavaScript object
portfield is casted to number / null from stringpathfield is added (pathname + search)- both username and password is URI-decoded if possible (if not, the function keeps the username and password as is)
authfield is added, and it contains username + ":" + password, or an empty string.
Note that port is returned even if it is a default port for http(s) and few other protocols. This differs from new URL(url) where port is null when default.
If the URL is invalid, the function throws an error.
The username and password parsing should make it possible to parse proxy URLs containing
special characters, such as http://user:pass:wrd@proxy.example.com
or http://us%35er:passwrd@proxy.example.com. The parsing is done on a best-effort basis.
The safest way is to always URI-encode username and password before constructing
the URL, according to RFC 3986.
Note that compared to the old implementation using url.parse(), the new function:
- is unable to distinguish empty password and missing password
- password and username are empty string if not present (or empty)
- we are able to parse IPv6
Takes a URL and hides the password from it. For example:
// Prints 'http://bob:<redacted>@example.com'
console.log(redactUrl('http://bob:pass123@example.com'));