http.Agent free sockets maintain reference to initial ClientRequest object until the socket is destroyed/closed #10133
Description
- Version: 6.9.1 (but appears to have been present since 6.0)
- Platform: Linux aa8dd06572b6 4.8.10-moby deps: update openssl to 1.0.1j #1 SMP Mon Nov 21 22:02:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
- Subsystem: http
This issue applies to an http.Agent using keepalive requests, where sockets are returned to a <hostname>:<port> freeSockets list after a request is complete.
In 6.0 (specifically commit 9bee03a), a change was made to enable asynchronous createConnection. As part of this, the callback that is passed to Agent.prototype.createSocket closes over the ClientRequest object that is prompting the creation of the socket.
Subsequently, the newly created socket has three listeners added, onFree, onClose and onAgentRemove. Because these are declared inside createConnection ( 9bee03a#diff-5f7fb0850412c6be189faeddea6c5359R194 ), the context for each one of these handlers points to a prior context that includes a reference to the cb parameter.
So long as the socket stays open, it will prevent GC of the initiating ClientRequest, even though the ClientRequest and response (IncomingMessage) have long finished being processed. In addition, common http request libraries such as request will often aggregate the entire response data on the IncomingMessage object. Since each ClientRequest object references the corresponding IncomingMessage response object, this results in each response body being prevented from garbage collection.
These objects will be retained until the freeSocket is eventually closed or destroyed (i.e. the listeners are removed).
This data can be seen in the following heap snapshot taken from a simple application which opens 10 sockets, requests 1MB of data over each socket, and then returns 9 of the sockets to the freeSockets list.
This data will remain in the heap, uncollectable, until the socket is closed or destroyed.
Gist containing sample code that produces the above heapSnapshot: https://gist.github.com/evantorrie/5746cbd5db6fbb65673c54dfa6b2337e