Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove StreamBase templating #25142

Closed
wants to merge 8 commits into from
Closed

Conversation

maclover7
Copy link
Contributor

@maclover7 maclover7 commented Dec 19, 2018

First attempt at trying to remove the templating in shared StreamBase methods. This approach casts directly to StreamBase via an internal field.

Unfortunately, I keep hitting linker errors when trying to run make -j8, and would appreciate any help in figuring out where they're coming from:

duplicate symbol __ZN4node10StreamBase15GetBytesWrittenERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(js_stream.o)
duplicate symbol __ZN4node10StreamBase11GetExternalERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(js_stream.o)
duplicate symbol __ZN4node10StreamBase12GetBytesReadERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(js_stream.o)
duplicate symbol __ZN4node10StreamBase5GetFDERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(js_stream.o)
duplicate symbol __ZN4node10StreamBase15GetBytesWrittenERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(node_file.o)
duplicate symbol __ZN4node10StreamBase11GetExternalERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(node_file.o)
duplicate symbol __ZN4node10StreamBase12GetBytesReadERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(node_file.o)
duplicate symbol __ZN4node10StreamBase5GetFDERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(node_file.o)
duplicate symbol __ZN4node10StreamBase15GetBytesWrittenERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(node_http_parser_llhttp.o)
duplicate symbol __ZN4node10StreamBase11GetExternalERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(node_http_parser_llhttp.o)
duplicate symbol __ZN4node10StreamBase12GetBytesReadERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(node_http_parser_llhttp.o)
duplicate symbol __ZN4node10StreamBase5GetFDERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(node_http_parser_llhttp.o)
duplicate symbol __ZN4node10StreamBase15GetBytesWrittenERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(node_http_parser_traditional.o)
duplicate symbol __ZN4node10StreamBase11GetExternalERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(node_http_parser_traditional.o)
duplicate symbol __ZN4node10StreamBase12GetBytesReadERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(node_http_parser_traditional.o)
duplicate symbol __ZN4node10StreamBase5GetFDERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(node_http_parser_traditional.o)
duplicate symbol __ZN4node10StreamBase15GetBytesWrittenERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(node_http2.o)
duplicate symbol __ZN4node10StreamBase11GetExternalERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(node_http2.o)
duplicate symbol __ZN4node10StreamBase12GetBytesReadERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(node_http2.o)
duplicate symbol __ZN4node10StreamBase5GetFDERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(node_http2.o)
duplicate symbol __ZN4node10StreamBase15GetBytesWrittenERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(pipe_wrap.o)
duplicate symbol __ZN4node10StreamBase11GetExternalERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(pipe_wrap.o)
duplicate symbol __ZN4node10StreamBase12GetBytesReadERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(pipe_wrap.o)
duplicate symbol __ZN4node10StreamBase5GetFDERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(pipe_wrap.o)
duplicate symbol __ZN4node10StreamBase15GetBytesWrittenERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(process_wrap.o)
duplicate symbol __ZN4node10StreamBase11GetExternalERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(process_wrap.o)
duplicate symbol __ZN4node10StreamBase12GetBytesReadERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(process_wrap.o)
duplicate symbol __ZN4node10StreamBase5GetFDERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(process_wrap.o)
duplicate symbol __ZN4node10StreamBase15GetBytesWrittenERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(stream_base.o)
duplicate symbol __ZN4node10StreamBase11GetExternalERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(stream_base.o)
duplicate symbol __ZN4node10StreamBase12GetBytesReadERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(stream_base.o)
duplicate symbol __ZN4node10StreamBase5GetFDERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(stream_base.o)
duplicate symbol __ZN4node10StreamBase15GetBytesWrittenERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(stream_pipe.o)
duplicate symbol __ZN4node10StreamBase11GetExternalERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(stream_pipe.o)
duplicate symbol __ZN4node10StreamBase12GetBytesReadERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(stream_pipe.o)
duplicate symbol __ZN4node10StreamBase5GetFDERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(stream_pipe.o)
duplicate symbol __ZN4node10StreamBase15GetBytesWrittenERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(stream_wrap.o)
duplicate symbol __ZN4node10StreamBase11GetExternalERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(stream_wrap.o)
duplicate symbol __ZN4node10StreamBase12GetBytesReadERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(stream_wrap.o)
duplicate symbol __ZN4node10StreamBase5GetFDERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(stream_wrap.o)
duplicate symbol __ZN4node10StreamBase15GetBytesWrittenERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(tcp_wrap.o)
duplicate symbol __ZN4node10StreamBase11GetExternalERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(tcp_wrap.o)
duplicate symbol __ZN4node10StreamBase12GetBytesReadERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(tcp_wrap.o)
duplicate symbol __ZN4node10StreamBase5GetFDERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(tcp_wrap.o)
duplicate symbol __ZN4node10StreamBase15GetBytesWrittenERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(tty_wrap.o)
duplicate symbol __ZN4node10StreamBase11GetExternalERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(tty_wrap.o)
duplicate symbol __ZN4node10StreamBase12GetBytesReadERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(tty_wrap.o)
duplicate symbol __ZN4node10StreamBase5GetFDERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(tty_wrap.o)
duplicate symbol __ZN4node10StreamBase15GetBytesWrittenERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(tls_wrap.o)
duplicate symbol __ZN4node10StreamBase11GetExternalERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(tls_wrap.o)
duplicate symbol __ZN4node10StreamBase12GetBytesReadERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(tls_wrap.o)
duplicate symbol __ZN4node10StreamBase5GetFDERKN2v820FunctionCallbackInfoINS1_5ValueEEE in:
    /Users/jon/code/nodejs/node/out/Release/libnode.a(connection_wrap.o)
    /Users/jon/code/nodejs/node/out/Release/libnode.a(tls_wrap.o)
ld: 52 duplicate symbols for architecture x86_64
Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. labels Dec 19, 2018
Copy link
Member

@addaleax addaleax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

… Nice!

src/stream_base-inl.h Outdated Show resolved Hide resolved
#define STREAM_BASE_METHOD(func, str) \
do { \
Local<FunctionTemplate> func_templ = \
env->NewFunctionTemplate(func, \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw, we use 4 spaces of indentation here – and maybe it makes sense to turn this macro into a function?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tips: use CLANG_FORMAT_START=master make format-cpp to auto format, or replace master with some other commit that you are branching from

@maclover7
Copy link
Contributor Author

updated @addaleax @joyeecheung, PTAL

Copy link
Member

@addaleax addaleax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still LGTM :)

@maclover7
Copy link
Contributor Author

maclover7 commented Dec 19, 2018

@maclover7
Copy link
Contributor Author

Looks like there is an intermittent timeout issue on some platforms with test-tls-socket-close, hoping common.platformTimeout can fix...

@addaleax
Copy link
Member

addaleax commented Jan 7, 2019

@addaleax addaleax added author ready PRs that have at least one approval, no pending requests for changes, and a CI started. and removed author ready PRs that have at least one approval, no pending requests for changes, and a CI started. labels Jan 7, 2019
@addaleax
Copy link
Member

addaleax commented Jan 9, 2019

@maclover7 It looks like the issue is that some streams, in particular libuv streams, delete the associated C++ object before the JS object is garbage collected. They set the first internal field to nullptr, but they don’t know about the second internal field.

This would be a (but maybe not the best?) fix:

diff --git a/src/stream_base-inl.h b/src/stream_base-inl.h
index 1842962192db..d672fc5a0dae 100644
--- a/src/stream_base-inl.h
+++ b/src/stream_base-inl.h
@@ -274,6 +274,9 @@ inline void StreamBase::AttachToObject(v8::Local<v8::Object> obj) {
 }
 
 inline StreamBase* StreamBase::FromObject(v8::Local<v8::Object> obj) {
+  if (obj->GetAlignedPointerFromInternalField(0) == nullptr)
+    return nullptr;
+
   return static_cast<StreamBase*>(
       obj->GetAlignedPointerFromInternalField(kStreamBaseField));
 }

@addaleax
Copy link
Member

Ping @maclover7 – barring better ideas, I can also apply the patch above and push to this PR for you?

@addaleax
Copy link
Member

addaleax commented Mar 7, 2019

Hm … ping @maclover7 again? :)

@addaleax
Copy link
Member

addaleax commented Mar 7, 2019

I’ve rebased + pushed the suggested change… @jasnell Can you confirm your approval?

CI: https://ci.nodejs.org/job/node-test-pull-request/21304/

@addaleax
Copy link
Member

addaleax commented Mar 7, 2019

@addaleax
Copy link
Member

addaleax commented Mar 8, 2019

Landed in 4697e1b 🎉

@addaleax addaleax closed this Mar 8, 2019
addaleax pushed a commit that referenced this pull request Mar 8, 2019
PR-URL: #25142
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
addaleax added a commit to addaleax/node that referenced this pull request Mar 8, 2019
Since 4697e1b, it is no longer
necessary to use `v8::External`s to pass `StreamBase` instances
to native functions.

Refs: nodejs#25142
danbev pushed a commit that referenced this pull request Mar 11, 2019
Since 4697e1b, it is no longer
necessary to use `v8::External`s to pass `StreamBase` instances
to native functions.

PR-URL: #26510
Refs: #25142
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
BridgeAR pushed a commit to BridgeAR/node that referenced this pull request Mar 12, 2019
PR-URL: nodejs#25142
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
BridgeAR pushed a commit that referenced this pull request Mar 13, 2019
Since 4697e1b, it is no longer
necessary to use `v8::External`s to pass `StreamBase` instances
to native functions.

PR-URL: #26510
Refs: #25142
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
BridgeAR pushed a commit that referenced this pull request Mar 14, 2019
PR-URL: #25142
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
BridgeAR pushed a commit that referenced this pull request Mar 14, 2019
Since 4697e1b, it is no longer
necessary to use `v8::External`s to pass `StreamBase` instances
to native functions.

PR-URL: #26510
Refs: #25142
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
@maclover7 maclover7 deleted the jm-streambase branch March 16, 2019 02:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants