Repository navigation
Make it possible to use Fetch with proxies or other agents #42814
Description
Activity
@nodejs/undici
Note: the same applies to
undici's mocking ability and common userland modules that achieve mocking for node'shttp/httpsmodulesnock. Undici has the affordance, but it is not exposed in Node.js and so node's globalfetchcannot be mocked.fetch in Node.js is experimental, all of this is planned for when it exits experimental. Could you check out undici and see if it matches your expectations?
On a different note, would it be easier to detect if the global
fetchproperty was non-enumerable?Could you check out undici and see if it matches your expectations?
Undici's dispatchers & setGlobalDispatcher on the core request methods (request/stream/etc) do work great for me, yes.
Undici's fetch doesn't have its own
dispatcheroption available though (nodejs/undici#1350), which is a bit inconvenient (I'm mostly setting a global dispatcher anyway, so in my case it's not a big issue).It would be a little more useful if it directly supported agents, or there was a dispatcher<->agent compatibility wrapper, since there's so many existing node.js agent implementations that could be reused, but again not a big deal.
On a different note, would it be easier to detect if the global fetch property was non-enumerable?
I don't think enumerability matters here, most checks are just
if (!global.fetch) { addPolyfill() }e.g. in isometric-fetch and cross-fetch.Those libraries are quite widely used (5 million + 8 million installs a week via npm, plus other similar libs & inline checks elsewhere). Everybody using those with Node 18 is automatically using this new experimental implementation everywhere now.
That's how I ran into this myself: my test code had the same inline if-no-fetch-then-ponyfill check, and when I ran my tests in node 18 they all failed because they switched to node's fetch and started ignoring the configured HTTP proxy.
Reacted by NorLz@nodejs/tsc did we expose fetch too early?
Reacted by Jordan Harband, Hu Langmin, yqz945 and David TranReacted by Sukka@nodejs/tsc did we expose fetch too early?
Not sure. It's difficult to say if we would have gotten useful feedback with the API still behind a flag.
@pimterry note that you can opt out of node's fetch with
--no-experimental-fetch.Reacted by Tim Perry and Jon Layton@nodejs/tsc did we expose fetch too early?
The problem isn't exposed fetch. It's that it doesnt utilize node's global agents due to its use of undici. There's a whole ecosystem of mocking and proxying libraries that relied on the fact that regardless of what http client / library you used in node, it always boiled down to http/s.request and its use of global agents
Reacted by Tim Perry, Tobias Nießen, Nathan P. Bombana, Domantas Jurkus and Sukkafetch in Node.js is experimental, all of this is planned for when it exits experimental
In hindsight, a global that implements a Web Platform API that's actively being polyfilled probably shouldn't be exposed when still experimental / unstable. That being said
It's difficult to say if we would have gotten useful feedback with the API still behind a flag.
Exposing it via a
node:fetchmodule first would probably allow for more feedback to trickle in. WebCrypto API was also not exposed as global and was first added to thecryptomodule as an export while experimental.Reacted by Jordan Harband, lathesky and Muthu KumarReacted by Brad Isbellhttps://github.com/orgs/nodejs/teams/tsc did we expose fetch too early?
Maybe. But I'm not sure what waiting even more would have made any difference? The same things break. The polyfills are still using node streams and http global agents so the primary breakage would be the same.
Reacted by Szymon MarczakI think we should seriously consider exposing
node:undici, or possiblynode:http-next. What do you think?Reacted by Ethan Arrowood, Tadas Varanauskas, Joshua Daniel Pratt Nielsen, Daniel Bankhead, joe matune, Owen Buckley, Phil Ricketts, Erik Rasmussen, Sukka and lo-joeReacted by joe matune, Owen Buckley, Maciek Lamberski, Sukka and lo-joeI like the idea!
This isn’t hindsight, to be clear; the risk was brought up many times and intentionally gambled.
Reacted by Sukka24 remaining items
Ah, that's interesting! I think the point largely stands though, since adding the dependency does duplicate the vast majority of Undici's other code -
lib/fetchandlib/llhttpalone are more than 50% of the size of the Undici package, and are both included in node's bundle. Needing to install & import all that code that's already present is suboptimal.I briefly had a go at creating an Undici fork that contained only ProxyAgent & setGlobalDispatcher, but Undici's agent.js depends on client.js which is a full HTTP client implementation, so there's no way to avoid this that way.
On the flip side though, that overlap suggests that ProxyAgent & setGlobalDispatcher would probably not significantly increase the size of Undici in Node. Looking at the Undici.js bundle (I assume that's the right place?) I can see that the entirety of
setGlobalDispatcher, and the Dispatcher, DispatcherBase and Agent classes are already included there. It's only ProxyAgent itself that's missing, which is tiny.I've just done a quick test on
mainin Undici: exposing ProxyAgent & setGlobalDispatcher explicitly increases the Undici bundle size from 334.2kb to 336.0kb (+1.8kb / +0.5%).Could these APIs be included and exposed in future? Agents for HTTP are a very core API that it would be useful to have usable out of the box, the equivalent functionality is usable OOTB for the legacy
httpmodule APIs, and 2kb is not a significant jump in bundle size for this functionality.Reacted by aksjer and RichardI would recommend opening up a separate issue about this topic and bringing it to the TSC. I don't think the whole of Undici has the stability guarantees needed to be part of the Node.js LTS cycle yet.
- added 8 commits that reference this issue
on Aug 1, 2025 For any lost souls, this is my understanding of how to use custom certs for
node:http,node:https, andundici.fetch:import { globalAgent } from "node:https"; import { getCACertificates } from "node:tls"; import { Agent, setGlobalDispatcher } from "undici"; const caCerts = [ ...getCACertificates("system"), ...getCACertificates("bundled"), ]; // Get node:http and node:https to use both the system CA certs and the bundled // (Mozilla) CA certs. // // node-fetch (not to be confused with node:fetch) is based on node:http and // node:https, so this should handle that as well. // // - https://github.com/electron/electron/issues/45674 globalAgent.options.ca = caCerts; // Get node:fetch to do the same by reconfiguring the underlying `undici` // library (as node:fetch is not written on top of node:http and node:https). // - https://github.com/nodejs/undici#undicisetglobaldispatcherdispatcher const agent = new Agent({ connect: { cert: caCerts } }); setGlobalDispatcher(agent); // Now all fetch() usages imported from undici from this point onwards will use // our custom certs. Though be aware that fetch() usages imported from // node:fetch will not be affected by this reconfiguration. In other words: // // ```js // import { fetch } from "undici"; // // // This will use the custom certs we set on `globalAgent.options.ca`. // await fetch("https://bbc.co.uk"); // ``` // // ```js // import { fetch } from "node:fetch"; // // // This will *not* be affected by the custom certs we set on // // `globalAgent.options.ca`. It will continue to use the (Mozilla) CA certs // // that come bundled with Node.js. // await fetch("https://bbc.co.uk"); // ```
At the time of writing, there is no
node:undicilibrary, so you do have to install undici separately to get this level of customisation. There is TSC support for exposing such a thing in future, it just lacks a champion for now.Also, if you have to rely on native
fetchin combination with theAgentfromundici, it might make sense to have a unit test like that to avoid any potential subtle compatibility issues because of version mismatch:it('verify that we use exactly the same `undici` version at both the application and runtime levels', async () => { expect(process.versions.undici).toEqual(require('undici/package.json').version); });
Reacted by Pedro Pedruzzi
What is the problem this feature will solve?
The new fetch API as implemented cannot be used with an HTTP proxy, which is required for connectivity in many environments.
For normal HTTP this is implemented via agents, but there's no way to use any agents with this fetch API. Many of us working in environments with proxies use libraries like global-agent which set node's
globalAgentto a proxy agent based on the system settings to automatically configure all libraries, but that doesn't work with fetch either.Notably, this means the docs are wrong right now, when they say:
This agent is not used for HTTP client requests if you use the fetch API.
Node's fetch implementation comes from Undici, and although Undici doesn't offer an explicit way to set this per fetch request (see nodejs/undici#1350) it does offer an agent-equivalent
dispatcheroption on all other request methods, and asetGlobalDispatchermethod to configure a dispatcher globally (like node'sglobalAgent) which does work for fetch.That means it is possible to use proxies with Undici's fetch right now, but not in Node as this isn't exposed anywhere (AFAICT).
What is the feature you are proposing to solve the problem?
Since they're very closely related, it seems like it would be sensible to aim to move everything to either agents or dispatchers for all HTTP APIs in future. While fetch is experimental though it seems reasonable to me to implement this only with Undici's existing dispatchers for now and pick one direction or the other to commonize later.
What alternatives have you considered?
As far as I can tell, there's currently no alternative or workaround available to use proxies with fetch in Node. If you need to use an HTTP proxy for connectivity, the current fetch API is unusable.
This is particularly bad because some libraries that support both browsers & node will use the fetch global automatically when available or
node-fetchotherwise (which useshttpinternally) for their requests. Although it used to be possible to use these libraries in a proxy environment by using global-agent or passing an agent explicitly, it's now impossible to use these libraries at all, because they use the new fetch global which ignores all agent configuration.