You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We deployed Node 8.0 on our servers, but found an error when a large number of concurrent requests came.
Stack:
TypeError: Cannot read property 'asyncReset' of null
at Agent.addRequest (_http_agent.js:170:19)
at new ClientRequest (_http_client.js:271:16)
at Object.request (http.js:39:10)
I checked the source file _http_agent.js. Line 170 is socket._handle.asyncReset();. I think it's better to check whehter socket._handle is null before use.
You are most likely using a custom agent somewhere in your code. Any change you could check for that? We would always like to understand the cause of the error, rather than just making a blind fix. Preventing asyncReset() from being called by checking socket._handle may cause other issues.
@AndreasMadsen Unfortunately I haven't found a better approach than to check if socket._handle exists in the case of passing a custom Agent. We have to do something similar in lib/timers.js because custom timer objects can be passed in (and we do that quite a bit in core)`.
Though we may be able to get around this using a combination of my final proposal at the bottom of #13548 (comment) and what we do in lib/timers.js. Which is we assign a new asyncId to the custom object on async_id_symbol and use that. It may have a strange timing issue about when to run init() but I think that's something we can get around.
I'm seeing the same error, while using an instantiated agent:
@mnutt we've been rolling patches around this area during the last month (#13348, #13092, and soon #14026 and #13839). Are you still seeing this error in node@8.1.3?
@AndreasMadsen Unfortunately I only see this on my production environment when the number of concurrent users is high, and I can't put a nightly build on it..
2017-07-20T11:31:56-0700 <error> RouteLoader.js:60 () message Cannot read property 'asyncReset' of null error TypeError: Cannot read property 'asyncReset' of null
at Agent.addRequest (_http_agent.js:171:19)
at new ClientRequest (_http_client.js:272:16)
at Object.request (http.js:39:10)
at Request.send (/Users/robert/Repositories/ry-openrtb-exchange/rtbexchange/server/modules/Bid/Request.js:88:11)
at /Users/robert/Repositories/ry-openrtb-exchange/rtbexchange/server/modules/Dsp/Manager.js:172:29
at Utilities.ProcessorRunner.completefn (/Users/robert/Repositories/ry-openrtb-exchange/rtbexchange/server/interfaces/Utilities.js:147:9)
at /Users/robert/Repositories/ry-openrtb-exchange/rtbexchange/node_modules/async/internal/once.js:12:16
at iteratorCallback (/Users/robert/Repositories/ry-openrtb-exchange/rtbexchange/node_modules/async/eachOf.js:60:13)
at /Users/robert/Repositories/ry-openrtb-exchange/rtbexchange/node_modules/async/internal/onlyOnce.js:12:16
I did not see it when I changed back to version 7.10.1, it was in all the versions of 8.. that I tried.
I don't believe I saw this error until I added node-redis, I'm not sure if that is a red-herring or not.
So for context: node v8 added a new layer of tracing (i.e. Async Hooks). This layer is active even if the user does not use it, to enable ad hoc interrogating the system state.
Although this new layer has been meticulously designed and tested, there are a few open edge cases (one of them is custom implementations of HTTP Agent). node@8.2.0 included several bug fixes in this layer (node@8.2.1 was fast tracked, and we managed to release it 1 day after 8.2.0 to solve two new bugs in Async Hooks). Meanwhile we converged on a design change that should minimize (hopefully eliminate) errors experienced by users who don't actually activate the Async Hooks.
Thank you for your patience, and your support by reporting these issues! 🎩
Is there any workaround to ignore this issue in version 8.1.x? Our production environment is having version 8.1.x and migrating it to a higher version may cause other implications in our codebase.
Please suggest a remedy for version 8.1.x. Thanks in advance.
Activity
AndreasMadsen commentedon Jun 8, 2017
You are most likely using a custom agent somewhere in your code. Any change you could check for that? We would always like to understand the cause of the error, rather than just making a blind fix. Preventing
asyncReset()from being called by checkingsocket._handlemay cause other issues./cc @nodejs/async_hooks
trevnorris commentedon Jun 8, 2017
@AndreasMadsen Unfortunately I haven't found a better approach than to check if
socket._handleexists in the case of passing a customAgent. We have to do something similar inlib/timers.jsbecause custom timer objects can be passed in (and we do that quite a bit in core)`.Though we may be able to get around this using a combination of my final proposal at the bottom of #13548 (comment) and what we do in
lib/timers.js. Which is we assign a newasyncIdto the custom object onasync_id_symboland use that. It may have a strange timing issue about when to runinit()but I think that's something we can get around.mnutt commentedon Jul 2, 2017
I'm seeing the same error, while using an instantiated agent:
Would we see a difference in this approach vs just modifying the limits of the default global agent?
refack commentedon Jul 2, 2017
@mnutt we've been rolling patches around this area during the last month (#13348, #13092, and soon #14026 and #13839). Are you still seeing this error in
node@8.1.3?mnutt commentedon Jul 3, 2017
Sorry, I should have mentioned that yes, it was node 8.1.3.
adiulici commentedon Jul 17, 2017
I'm seeing the same issue on node v8.1.14. Unfortunately it's being thrown inside a dependency (new relic) so I cannot offer much of a detail.
AndreasMadsen commentedon Jul 17, 2017
@aashil could you try the nightly build, most of the
async_hooksfixes are inv8.2.0which hasn't been released yet (#13744).adiulici commentedon Jul 17, 2017
@AndreasMadsen Unfortunately I only see this on my production environment when the number of concurrent users is high, and I can't put a nightly build on it..
refack commentedon Jul 19, 2017
@adiulici node@8.2.0 was just released some relevant bug fixes are in it. If you are comfortable you could try it.
rwlaschin commentedon Jul 20, 2017
HI. I'm seeing this issue as well 8.1.4
I'm using a custom agent
const keepAliveAgent = new http.Agent({ keepAlive: true });
I'll try 8.2.0 and see if the issue is resolved.
rwlaschin commentedon Jul 20, 2017
Unfortunately, no it still happens
It is very frequent.
edit trevnorris: place output in code block
rwlaschin commentedon Jul 20, 2017
I did not see it when I changed back to version 7.10.1, it was in all the versions of 8.. that I tried.
I don't believe I saw this error until I added node-redis, I'm not sure if that is a red-herring or not.
trevnorris commentedon Jul 20, 2017
Sorry for the delay. See the issue. I'll write up a fix tonight.
refack commentedon Jul 21, 2017
So for context:
node v8added a new layer of tracing (i.e.Async Hooks). This layer is active even if the user does not use it, to enable ad hoc interrogating the system state.Although this new layer has been meticulously designed and tested, there are a few open edge cases (one of them is custom implementations of HTTP Agent).
node@8.2.0included several bug fixes in this layer (node@8.2.1was fast tracked, and we managed to release it 1 day after 8.2.0 to solve two new bugs in Async Hooks). Meanwhile we converged on a design change that should minimize (hopefully eliminate) errors experienced by users who don't actually activate the Async Hooks.Thank you for your patience, and your support by reporting these issues! 🎩
trevnorris commentedon Jul 21, 2017
Fix at #14419.
http: check for handle before running asyncReset()
fix: nodejs/node#13539
fix: nodejs/node#13539
SoumyaDey1994 commentedon Dec 11, 2019
Hi,
Is there any workaround to ignore this issue in version 8.1.x? Our production environment is having version 8.1.x and migrating it to a higher version may cause other implications in our codebase.
Please suggest a remedy for version 8.1.x. Thanks in advance.