Export module for Node when MODULARIZE=1 #5239

phraemer · 2017-05-23T16:27:41Z

As mentioned in this issue #5216

Basically if you turn on modularisation the function cannot be called if you require(...) it in a node environment.

This PR adds environment detection code if MODULARIZE=1 and exports the module.

Some tests failed but I don't believe as a result of these changes

======================================================================
ERROR: test_dylink_zlib (test_core.asm1)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/Users/james/Documents/mine/emscripten/emscripten/tests/test_core.py", line 3767, in test_dylink_zlib
    force_c=True)
  File "/Users/james/Documents/mine/emscripten/emscripten/tests/test_core.py", line 3062, in dylink_test
    shutil.move('liblib.cpp.o.' + side_suffix, 'liblib.so')
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 130, in copy2
    copyfile(src, dst)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: 'liblib.cpp.o.js'

======================================================================
ERROR: test_dylink_zlib (test_core.asm2)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/Users/james/Documents/mine/emscripten/emscripten/tests/test_core.py", line 3767, in test_dylink_zlib
    force_c=True)
  File "/Users/james/Documents/mine/emscripten/emscripten/tests/test_core.py", line 3062, in dylink_test
    shutil.move('liblib.cpp.o.' + side_suffix, 'liblib.so')
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 130, in copy2
    copyfile(src, dst)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: 'liblib.cpp.o.js'

======================================================================
ERROR: test_dylink_zlib (test_core.asm2g)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/Users/james/Documents/mine/emscripten/emscripten/tests/test_core.py", line 3767, in test_dylink_zlib
    force_c=True)
  File "/Users/james/Documents/mine/emscripten/emscripten/tests/test_core.py", line 3062, in dylink_test
    shutil.move('liblib.cpp.o.' + side_suffix, 'liblib.so')
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 130, in copy2
    copyfile(src, dst)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: 'liblib.cpp.o.js'

======================================================================
ERROR: test_dylink_zlib (test_core.asm2i)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/Users/james/Documents/mine/emscripten/emscripten/tests/test_core.py", line 3767, in test_dylink_zlib
    force_c=True)
  File "/Users/james/Documents/mine/emscripten/emscripten/tests/test_core.py", line 3062, in dylink_test
    shutil.move('liblib.cpp.o.' + side_suffix, 'liblib.so')
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 130, in copy2
    copyfile(src, dst)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: 'liblib.cpp.o.js'

======================================================================
ERROR: test_dylink_zlib (test_core.asm2nn)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/Users/james/Documents/mine/emscripten/emscripten/tests/test_core.py", line 3767, in test_dylink_zlib
    force_c=True)
  File "/Users/james/Documents/mine/emscripten/emscripten/tests/test_core.py", line 3062, in dylink_test
    shutil.move('liblib.cpp.o.' + side_suffix, 'liblib.so')
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 130, in copy2
    copyfile(src, dst)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: 'liblib.cpp.o.js'

======================================================================
ERROR: test_dylink_zlib (test_core.asm3)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/Users/james/Documents/mine/emscripten/emscripten/tests/test_core.py", line 3767, in test_dylink_zlib
    force_c=True)
  File "/Users/james/Documents/mine/emscripten/emscripten/tests/test_core.py", line 3062, in dylink_test
    shutil.move('liblib.cpp.o.' + side_suffix, 'liblib.so')
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 130, in copy2
    copyfile(src, dst)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: 'liblib.cpp.o.js'

======================================================================
ERROR: test_dylink_zlib (test_core.default)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/Users/james/Documents/mine/emscripten/emscripten/tests/test_core.py", line 3767, in test_dylink_zlib
    force_c=True)
  File "/Users/james/Documents/mine/emscripten/emscripten/tests/test_core.py", line 3062, in dylink_test
    shutil.move('liblib.cpp.o.' + side_suffix, 'liblib.so')
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 130, in copy2
    copyfile(src, dst)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: 'liblib.cpp.o.js'

----------------------------------------------------------------------
Ran 3816 tests in 5690.772s

FAILED (errors=7)

kripken · 2017-05-23T18:20:02Z

src/detectNode.js

@@ -0,0 +1,32 @@
+var isNodeEnvironment = function() {
+    // Same as in shell.js


indentation should be 2 spaces

kripken · 2017-05-23T18:21:45Z

src/detectNode.js

+    // 2) We could be the application main() thread proxied to worker. (with Emscripten -s PROXY_TO_WORKER=1) (ENVIRONMENT_IS_WORKER == true, ENVIRONMENT_IS_PTHREAD == false)
+    // 3) We could be an application pthread running in a worker. (ENVIRONMENT_IS_WORKER == true and ENVIRONMENT_IS_PTHREAD == true)
+
+    if (Module['ENVIRONMENT']) {


Module may not exist here, first because we may have a different export name, second because we just have the modularize function, and it's parameter is the Module object, but it will only be called later.

I think all we can do is the code path that does not use Module. I think that should be enough here?

kripken · 2017-05-23T18:22:42Z

Thanks, overall looks good. A few comments above. Also, we should add a test for this (may be a test for node exporting already, where we can add to that test or add a new test alongside it).

bvibber · 2017-05-23T22:24:36Z

I think this would be handy for code using webpack/browserify/etc as well, but I think they would fail the runtime check for isNodeEnvironment() by matching positive for being a web or worker environment. Maybe test for 'require' and 'module', rather than for 'process' and not-being-web?

phraemer · 2017-05-24T09:00:44Z

@Brion a good suggestion. For this PR would you mind if I keep it as similar to existing code first? I.e. what's in shell.js.

We could do a second PR later and make it even more general.

Perhaps this is relevant https://github.com/umdjs/umd/blob/master/templates/returnExportsGlobal.js

phraemer · 2017-05-24T09:22:38Z

@kripken thanks for the review! Hope it's good now.

kripken · 2017-05-24T18:16:03Z

Looks good, just still thinking about the issue that @Brion mentioned. Is the issue also an identical existing issue with shell.js? In that case let's merge this and figure that out later. Or is it a separate issue, in which case we should address it here first?

I lean to thinking of it as a separate issue, as shell.js tries to identify the actual environment, while here we just need to know if there is a module object we need to export on.

phraemer · 2017-05-24T18:25:47Z

Yeah it's the same in my opinion but definitely worth doing later. I.e. in a separate PR.

(edit for clarity)

curiousdannii · 2017-05-26T01:27:09Z

This partially addresses #5093. I wouldn't say it fixes it though, because won't it not work if a browserified module is used in a browser? I think it should do as UMD does: if (typeof module === 'object' && module.exports). That code pattern is already so common that I think it's proved reliable. No need to complicate matters.

Also in Electron it will probably act as both a browser and node.

Added authorship Not passing this Simplify since Module may not exist. Fix tabs.

phraemer · 2017-05-26T10:09:38Z

OK yeah it does make sense. Updated.

kripken · 2017-05-26T17:56:35Z

Great, thanks.

kripken reviewed May 23, 2017

View reviewed changes

kripken mentioned this pull request May 24, 2017

empty Module object when imported in ReactNative #5241

Closed

James Swift added 3 commits May 26, 2017 11:46

Export module in Node when MODULARIZE=1

0405b42

Added authorship Not passing this Simplify since Module may not exist. Fix tabs.

Test NodeJS export if MODULARIZE=1

a0a779b

The UMD style detection is more general and simpler.

cea9225

Merge branch 'incoming' into jamesswift/modularize-node

c13f77a

kripken merged commit 0b09280 into emscripten-core:incoming May 26, 2017

curiousdannii mentioned this pull request May 26, 2017

MODULARIZE can't be used in Node #5093

Closed

kripken mentioned this pull request May 30, 2017

export Module even if ENVIRONMENT_IS_NODE is false #5255

Closed

LucasThompson mentioned this pull request Jun 28, 2017

UMD module wrapper is not robustly generated and is likely wrong piper-audio/piper-vamp-js#1

Open

cpitclaudel mentioned this pull request Mar 24, 2018

--wrap-with-fun doesn't export to node ocsigen/js_of_ocaml#653

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Export module for Node when MODULARIZE=1 #5239

Export module for Node when MODULARIZE=1 #5239

Uh oh!

phraemer commented May 23, 2017

Uh oh!

kripken May 23, 2017

Uh oh!

kripken May 23, 2017

Uh oh!

kripken commented May 23, 2017

Uh oh!

bvibber commented May 23, 2017

Uh oh!

phraemer commented May 24, 2017 •

edited

Loading

Uh oh!

phraemer commented May 24, 2017

Uh oh!

kripken commented May 24, 2017

Uh oh!

phraemer commented May 24, 2017 •

edited

Loading

Uh oh!

curiousdannii commented May 26, 2017 •

edited

Loading

Uh oh!

phraemer commented May 26, 2017

Uh oh!

kripken commented May 26, 2017

Uh oh!

Uh oh!

		@@ -0,0 +1,32 @@
		var isNodeEnvironment = function() {
		// Same as in shell.js

Export module for Node when MODULARIZE=1 #5239

Export module for Node when MODULARIZE=1 #5239

Uh oh!

Conversation

phraemer commented May 23, 2017

Uh oh!

kripken May 23, 2017

Choose a reason for hiding this comment

Uh oh!

kripken May 23, 2017

Choose a reason for hiding this comment

Uh oh!

kripken commented May 23, 2017

Uh oh!

bvibber commented May 23, 2017

Uh oh!

phraemer commented May 24, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

phraemer commented May 24, 2017

Uh oh!

kripken commented May 24, 2017

Uh oh!

phraemer commented May 24, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

curiousdannii commented May 26, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

phraemer commented May 26, 2017

Uh oh!

kripken commented May 26, 2017

Uh oh!

Uh oh!

phraemer commented May 24, 2017 •

edited

Loading

phraemer commented May 24, 2017 •

edited

Loading

curiousdannii commented May 26, 2017 •

edited

Loading