Repository navigation
SIGNAL USERS READ THIS FIRST: code: 'ERR_INTERNAL_ASSERTION' in internalConnectMultiple #47644
Description
Activity
@tniessen It seems so. I'll take a look soon
I'm also seeing a similar issue with ssh2 after upgrading to Node 20.
Repro steps:
- Connect to an invalid host
- Add an
errorevent handler
client.connect({ host: 'yahoo.com', port: 22, username: 'bob', password: 'secret' }) // Adding this line crashes the process. client.on('error', () => {})
seeing this issue as well on production
Error [ERR_INTERNAL_ASSERTION]: This is caused by either a bug in Node.js or incorrect usage of Node.js internals.
| 2023-04-29T13:32:31.670-04:00 | Please open an issue with this stack trace at https://github.com/nodejs/node/issues
| 2023-04-29T13:32:31.670-04:00 | at new NodeError (node:internal/errors:399:5)
| 2023-04-29T13:32:31.670-04:00 | at assert (node:internal/assert:14:11)
| 2023-04-29T13:32:31.670-04:00 | at internalConnectMultiple (node:net:1106:3)
| 2023-04-29T13:32:31.670-04:00 | at Timeout.internalConnectMultipleTimeout (node:net:1637:3)
| 2023-04-29T13:32:31.670-04:00 | at listOnTimeout (node:internal/timers:575:11)
| 2023-04-29T13:32:31.670-04:00 | at process.processTimers (node:internal/timers:514:7)Our old build from 10 days ago still works, but all new builds seem to run into this issue.
Reacted by Max Holman, Bradley, Warren Halderman and Eliott McKenzieJust a note, subscribing as still present in
v20.1.0:In a net and async/await heavy program:
node:internal/assert:14 throw new ERR_INTERNAL_ASSERTION(message); ^ Error [ERR_INTERNAL_ASSERTION]: This is caused by either a bug in Node.js or incorrect usage of Node.js internals. Please open an issue with this stack trace at https://github.com/nodejs/node/issues at new NodeError (node:internal/errors:399:5) at assert (node:internal/assert:14:11) at internalConnectMultiple (node:net:1107:3) at Timeout.internalConnectMultipleTimeout (node:net:1638:3) at listOnTimeout (node:internal/timers:575:11) at process.processTimers (node:internal/timers:514:7) { code: 'ERR_INTERNAL_ASSERTION' } Node.js v20.1.0This might be fixed by #47860.
Will keep you posted on this.Reacted by Mike Ralphson@kamagatos @yuki12321 The PR above has landed in master. If you can compile Node locally, do you mind checking it if solves your issues as well?
Reacted by Mike RalphsonI now see the following on
master:/Users/mikeralphson/c/node/node[73535]: ../../src/crypto/crypto_tls.cc:1233:static void node::crypto::TLSWrap::GetServername(const FunctionCallbackInfo<v8::Value> &): Assertion `(wrap->ssl_) != nullptr' failed. 1: 0x100bc1cf0 node::Abort() [/Users/mikeralphson/c/node/out/Release/node] 2: 0x100bc1a30 node::PrintCaughtException(v8::Isolate*, v8::Local<v8::Context>, v8::TryCatch const&) [/Users/mikeralphson/c/node/out/Release/node] 3: 0x100d14700 node::crypto::TLSWrap::GetServername(v8::FunctionCallbackInfo<v8::Value> const&) [/Users/mikeralphson/c/node/out/Release/node] 4: 0x100dca0d4 v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, unsigned long*, int) [/Users/mikeralphson/c/node/out/Release/node] 5: 0x100dc9928 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/Users/mikeralphson/c/node/out/Release/node] 6: 0x10164cb24 Builtins_CEntry_Return1_ArgvOnStack_BuiltinExit [/Users/mikeralphson/c/node/out/Release/node] 7: 0x1015c43e4 Builtins_InterpreterEntryTrampoline [/Users/mikeralphson/c/node/out/Release/node] 8: 0x10664d2cc 9: 0x1015c250c Builtins_JSEntryTrampoline [/Users/mikeralphson/c/node/out/Release/node] 10: 0x1015c21f4 Builtins_JSEntry [/Users/mikeralphson/c/node/out/Release/node] 11: 0x100eb082c v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/Users/mikeralphson/c/node/out/Release/node] 12: 0x100eb00a8 v8::internal::Execution::Call(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, int, v8::internal::Handle<v8::internal::Object>*) [/Users/mikeralphson/c/node/out/Release/node] 13: 0x100d77e80 v8::Function::Call(v8::Local<v8::Context>, v8::Local<v8::Value>, int, v8::Local<v8::Value>*) [/Users/mikeralphson/c/node/out/Release/node] 14: 0x100afdf84 node::InternalMakeCallback(node::Environment*, v8::Local<v8::Object>, v8::Local<v8::Object>, v8::Local<v8::Function>, int, v8::Local<v8::Value>*, node::async_context) [/Users/mikeralphson/c/node/out/Release/node] 15: 0x100b120b8 node::AsyncWrap::MakeCallback(v8::Local<v8::Function>, int, v8::Local<v8::Value>*) [/Users/mikeralphson/c/node/out/Release/node] 16: 0x100b292dc node::ConnectionWrap<node::TCPWrap, uv_tcp_s>::AfterConnect(uv_connect_s*, int) [/Users/mikeralphson/c/node/out/Release/node] 17: 0x100c917b0 node::MakeLibuvRequestCallback<uv_connect_s, void (*)(uv_connect_s*, int)>::Wrapper(uv_connect_s*, int) [/Users/mikeralphson/c/node/out/Release/node] 18: 0x1015acf94 uv__stream_io [/Users/mikeralphson/c/node/out/Release/node] 19: 0x1015b5650 uv__io_poll [/Users/mikeralphson/c/node/out/Release/node] 20: 0x1015a2e94 uv_run [/Users/mikeralphson/c/node/out/Release/node] 21: 0x100afe7d0 node::SpinEventLoopInternal(node::Environment*) [/Users/mikeralphson/c/node/out/Release/node] 22: 0x100c03504 node::NodeMainInstance::Run() [/Users/mikeralphson/c/node/out/Release/node] 23: 0x100b899d0 node::LoadSnapshotDataAndRun(node::SnapshotData const**, node::InitializationResultImpl const*) [/Users/mikeralphson/c/node/out/Release/node] 24: 0x100b89c24 node::Start(int, char**) [/Users/mikeralphson/c/node/out/Release/node] 25: 0x1a2ad3f28 start [/usr/lib/dyld] zsh: abort@MikeRalphson Can you please provide a repro file, along with the
digquery of all the hosts you are trying to reach?Reacted by Mike RalphsonThe same internal assertion is causing CI to fail on Fedora machines, see #48000. However, it is not the
ERR_INTERNAL_ASSERTIONthat this issue was about originally.Side note: the main branch is called
main. We intentionally abandoned the previous branch name.Reacted by Mike RalphsonThis randomly started happening on my production build after a deployment. I'm using lts-alpine image in my docker file, has there been a recent change that could be causing this?
@brandon-beacher It's the network family auto selection which was enabled by default in 20.0.0.
I fixed this issue (already merged in main) and once I'll fix some other problems it should go out on 20.2.0 or 20.3.0.Reacted by Mike RalphsonThis is still an issue in Node.js 20.2.0. As a workaround, you can try restoring the more predictable pre-20 behavior by setting this environment variable:
NODE_OPTIONS="--no-network-family-autoselection"Reacted by silverwind, Brian, Sagnik Pradhan, Furkan Baytekin, Filip Vranjes, Marek Pilczuk, Mariano and Ihor RohovetsReacted by Furkan Baytekin- changed the title
[-]code: 'ERR_INTERNAL_ASSERTION'[/-][+]code: 'ERR_INTERNAL_ASSERTION' in internalConnectMultiple[/+]on Jun 2, 2023 40 remaining items
- added a commit that references this issue
on Aug 14, 2023 Issue still occurring for node 20.5.1 with the same stacktrace.
Randomly starts occurring and keeps happening for multiple hours after which it randomly stopped.
Node is running as a kubernetes service and not using workers or websockets (like #48763).
Using the docker imagenode:20.5.1@Rand0mF In order to address this, I need to know which host your node app is connecting to and how your local (or Kubernetes system) is resolving it. Can you provide me such info?
@ShogunPanda Unfortunately I don't know.. The app is doing many queries to all kinds of hosts. The error logs don't provide any useful information for that, that's the only thing which is printed. Reverse engineering is difficult as it seems to occur completely at random, I didn't yet find a way to reproduce it.
Error [ERR_INTERNAL_ASSERTION]: This is caused by either a bug in Node.js or incorrect usage of Node.js internals. Please open an issue with this stack trace at https://github.com/nodejs/node/issues at new NodeError (node:internal/errors:405:5) at assert (node:internal/assert:14:11) at internalConnectMultiple (node:net:1118:3) at Timeout.internalConnectMultipleTimeout (node:net:1687:3) at listOnTimeout (node:internal/timers:575:11) at process.processTimers (node:internal/timers:514:7)Reacted by Antonio Marino, Ainsley Clark and miguelVery sporadic error which seems to be difficult to debug locally as @Rand0mF mentioned.
export NODE_OPTIONS=--no-network-family-autoselectionworks.
Allv20.xversions are producing the same error.Reacted by João Pimentel Ferreira@ainsleyclark Is the same for you? Can you provide a list of which hosts the production system is connecting to and how these are resolved by the production system DNS servers?
I'm seeing lots of these errors too, on Node v20.8.0. Similar to other reports, it seems to be very intermittent and only happening under load. I can't confirm the addresses involved either unfortunately (software is running on end user machines, I just see exception reports).
@ShogunPanda
list of hosts will still be difficult, here's the dig output from within the container forcluster-external hosts
dig google.com ; <<>> DiG 9.18.19 <<>> google.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 7886 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;google.com. IN A ;; ANSWER SECTION: google.com. 30 IN A 142.250.185.142 ;; Query time: 2 msec ;; SERVER: 10.251.0.10#53(10.251.0.10) (UDP) ;; WHEN: Thu Nov 16 14:03:26 UTC 2023 ;; MSG SIZE rcvd: 55cluster-internal hosts
dig hello-world.default.svc.cluster.local ; <<>> DiG 9.18.19 <<>> hello-world.default.svc.cluster.local ;; global options: +cmd ;; Got answer: ;; WARNING: .local is reserved for Multicast DNS ;; You are currently testing what happens when an mDNS query is leaked to DNS ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 13182 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;hello-world.default.svc.cluster.local. IN A ;; ANSWER SECTION: hello-world.default.svc.cluster.local. 30 IN A 10.251.3.203 ;; Query time: 8 msec ;; SERVER: 10.251.0.10#53(10.251.0.10) (UDP) ;; WHEN: Thu Nov 16 14:08:01 UTC 2023 ;; MSG SIZE rcvd: 71any other flags which would be helpful?
Nope, that should be it. I will keep you posted.
Still experiencing the issue with Node 20^. Switching to Node 18 solves the problem for me.
This should be solved by #51045. Will keep you as soon as this reaches 20.x.
Reacted by MarianoAny updates on when this is going to be fixed for 20.x @ShogunPanda ?
It has been for a long time @guillenotfound https://nodejs.org/en/blog/release/v20.12.0#new-connection-attempt-events.
Latest node release'sv21.7.1...@guillenotfound It has been release in 20.12.0.
- added a commit that references this issue
on Apr 7, 2026
Version
20.0.0
Platform
Darwin MacBookPro.local 22.3.0 Darwin Kernel Version 22.3.0: Mon Jan 30 20:42:11 PST 2023; root:xnu-8792.81.3~2/RELEASE_X86_64 x86_64
Subsystem
No response
What steps will reproduce the bug?
I ran the Storybook installation on Next.js 13.3.0 and this is what I see.
How often does it reproduce? Is there a required condition?
No response
What is the expected behavior? Why is that the expected behavior?
No response
What do you see instead?
Additional information
No response