Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove python_selector.py in favor of run_python.sh #10729

Merged
merged 1 commit into from
Mar 20, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion ChangeLog.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,11 @@ See docs/process.md for how version tagging works.

Current Trunk
-------------
- Program entry points without extensions are now shell scripts rather than
python programs. See #10729. This means that `python emcc` no longer works.
However `emcc`, `emcc.py` and `python emcc.py` all continue to work.
The reason for this change is that `#!/usr/bin/env python` is no longer
portable since the python symlink was dropped from Ubuntu 20.04.

v1.39.11: 03/20/2020
--------------------
Expand All @@ -27,7 +32,7 @@ v1.39.11: 03/20/2020
https://libbsd.freedesktop.org/.
- Change the meaning of `ASYNCIFY_IMPORTS`: it now contains only new imports
you add, and does not need to contain the list of default system imports like
``emscripten_sleep``. There is no harm in providing them, though, so this
`emscripten_sleep`. There is no harm in providing them, though, so this
is not a breaking change.
- Enable DWARF support: When compiling with `-g`, normal DWARF emitting happens,
and when linking with `-g` we preserve that and update it. This is a change
Expand Down
15 changes: 0 additions & 15 deletions em++

This file was deleted.

1 change: 1 addition & 0 deletions em++
16 changes: 7 additions & 9 deletions em++.py
Original file line number Diff line number Diff line change
@@ -1,19 +1,17 @@
#!/usr/bin/env python
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is em++.py special among the tools? seems like it's getting different treatment in this PR than the others and it's not obvious why.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because em++ was using the python_selector in a slightly different way. It was using it to redirect to emcc.py as well as to choose the python version.

With this change, once python is running we assume its the correct version, we never try to re-invoke python. So this change means that em++ now runs emcc directly, without re-invoking python.

#!/usr/bin/env python3
# Copyright 2011 The Emscripten Authors. All rights reserved.
# Emscripten is available under two separate licenses, the MIT license and the
# University of Illinois/NCSA Open Source License. Both these licenses can be
# found in the LICENSE file.

# This script should work in python 2 *or* 3. It loads the main code using
# python_selector, which may pick a different version.
# It also tells emcc.py that we want C++ and not C by default

import os
import sys
from tools import python_selector
import emcc

sys.argv += ['--emscripten-cxx']

emcc = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'emcc')
if __name__ == '__main__':
python_selector.run(emcc)
try:
sys.exit(emcc.run(sys.argv))
except KeyboardInterrupt:
emcc.logger.warning('KeyboardInterrupt')
sys.exit(1)
9 changes: 0 additions & 9 deletions emar

This file was deleted.

1 change: 1 addition & 0 deletions emar
11 changes: 0 additions & 11 deletions emcc

This file was deleted.

1 change: 1 addition & 0 deletions emcc
9 changes: 0 additions & 9 deletions emcmake

This file was deleted.

1 change: 1 addition & 0 deletions emcmake
9 changes: 0 additions & 9 deletions emconfigure

This file was deleted.

1 change: 1 addition & 0 deletions emconfigure
9 changes: 0 additions & 9 deletions emmake

This file was deleted.

1 change: 1 addition & 0 deletions emmake
11 changes: 0 additions & 11 deletions emranlib

This file was deleted.

1 change: 1 addition & 0 deletions emranlib
9 changes: 0 additions & 9 deletions emrun

This file was deleted.

1 change: 1 addition & 0 deletions emrun
11 changes: 0 additions & 11 deletions emscons

This file was deleted.

1 change: 1 addition & 0 deletions emscons
6 changes: 3 additions & 3 deletions tests/test_browser.py
Original file line number Diff line number Diff line change
Expand Up @@ -2464,12 +2464,12 @@ def test_mmap_file(self):
def test_emrun_info(self):
if not has_browser():
self.skipTest('need a browser')
result = run_process([PYTHON, path_from_root('emrun'), '--system_info', '--browser_info'], stdout=PIPE).stdout
result = run_process([PYTHON, path_from_root('emrun.py'), '--system_info', '--browser_info'], stdout=PIPE).stdout
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this change necessary? if it is that seems like a breaking change. if it isn't then why?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes this change is a slightly breaking change. You can no longer run python emcc (because emcc is now itself a shell script that runs python). You can still run ./emcc.py directly or python emcc.py if you like.

I think this is the best approach. The python_selector method where python re-invokes itself is kind of nasty. If I run python2 emcc I kind of expect it to run under python2 not invoke a python3 subprocess to run itself.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sgtm. is there some way we can show a clear error on python emcc / emranlib / etc., like keeping the files around but only containing a print of an error? or if that's not possible, maybe a comment?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not deleting those files. They are now symbolic links to run_python.sh. So they exist by they are shell scripts.

I'm not sure we can have python <some_shell_script> do anything useful... but let me see...

assert 'CPU' in result
assert 'Browser' in result
assert 'Traceback' not in result

result = run_process([PYTHON, path_from_root('emrun'), '--list_browsers'], stdout=PIPE).stdout
result = run_process([PYTHON, path_from_root('emrun.py'), '--list_browsers'], stdout=PIPE).stdout
assert 'Traceback' not in result

# Deliberately named as test_zzz_emrun to make this test the last one
Expand All @@ -2484,7 +2484,7 @@ def test_zzz_emrun(self):
# and the browser will not close as part of the test, pinning down the cwd on Windows and it wouldn't be possible to delete it. Therefore switch away from that directory
# before launching.
os.chdir(path_from_root())
args_base = [PYTHON, path_from_root('emrun'), '--timeout', '30', '--safe_firefox_profile', '--kill_exit', '--port', '6939', '--verbose', '--log_stdout', os.path.join(outdir, 'stdout.txt'), '--log_stderr', os.path.join(outdir, 'stderr.txt')]
args_base = [PYTHON, path_from_root('emrun.py'), '--timeout', '30', '--safe_firefox_profile', '--kill_exit', '--port', '6939', '--verbose', '--log_stdout', os.path.join(outdir, 'stdout.txt'), '--log_stderr', os.path.join(outdir, 'stderr.txt')]
if EMTEST_BROWSER is not None:
# If EMTEST_BROWSER carried command line arguments to pass to the browser,
# (e.g. "firefox -profile /path/to/foo") those can't be passed via emrun,
Expand Down
28 changes: 12 additions & 16 deletions tests/test_other.py
Original file line number Diff line number Diff line change
Expand Up @@ -6094,27 +6094,27 @@ def check(what, args, fail=True, expect=''):
assert ('is a helper for' in output.stderr) == fail
assert ('Typical usage' in output.stderr) == fail
self.assertContained(expect, output.stdout)
check('emmake', [])
check('emconfigure', [])
check('emmake', ['--version'])
check('emconfigure', ['--version'])
check('emmake', ['make'], fail=False)
check('emconfigure', ['configure'], fail=False)
check('emconfigure', ['./configure'], fail=False)
check('emcmake', ['cmake'], fail=False)
check('emmake.py', [])
check('emconfigure.py', [])
check('emmake.py', ['--version'])
check('emconfigure.py', ['--version'])
check('emmake.py', ['make'], fail=False)
check('emconfigure.py', ['configure'], fail=False)
check('emconfigure.py', ['./configure'], fail=False)
check('emcmake.py', ['cmake'], fail=False)

create_test_file('test.py', '''
import os
print(os.environ.get('CROSS_COMPILE'))
''')
check('emconfigure', [PYTHON, 'test.py'], expect=path_from_root('em'), fail=False)
check('emmake', [PYTHON, 'test.py'], expect=path_from_root('em'), fail=False)
check('emconfigure.py', [PYTHON, 'test.py'], expect=path_from_root('em'), fail=False)
check('emmake.py', [PYTHON, 'test.py'], expect=path_from_root('em'), fail=False)

create_test_file('test.py', '''
import os
print(os.environ.get('NM'))
''')
check('emconfigure', [PYTHON, 'test.py'], expect=shared.LLVM_NM, fail=False)
check('emconfigure.py', [PYTHON, 'test.py'], expect=shared.LLVM_NM, fail=False)

@no_windows('This test is broken, https://github.com/emscripten-core/emscripten/issues/8872')
def test_emmake_python(self):
Expand All @@ -6135,7 +6135,7 @@ def test_sdl2_config(self):
out = run_process([PYTHON, path_from_root('system', 'bin', 'sdl2-config')] + args, stdout=PIPE, stderr=PIPE).stdout
assert expected in out, out
print('via emmake')
out = run_process([PYTHON, path_from_root('emmake'), 'sdl2-config'] + args, stdout=PIPE, stderr=PIPE).stdout
out = run_process([PYTHON, path_from_root('emmake.py'), 'sdl2-config'] + args, stdout=PIPE, stderr=PIPE).stdout
assert expected in out, out

def test_module_onexit(self):
Expand Down Expand Up @@ -7253,10 +7253,6 @@ def run(python):
has = Building.which(python) is not None
print(python, has)
if has:
print(' checking emcc...')
run_process([python, trim_py_suffix(EMCC), '--version'], stdout=PIPE)
print(' checking em++...')
run_process([python, trim_py_suffix(EMXX), '--version'], stdout=PIPE)
print(' checking emcc.py...')
run_process([python, EMCC, '--version'], stdout=PIPE)
print(' checking em++.py...')
Expand Down
4 changes: 2 additions & 2 deletions tests/test_sockets.py
Original file line number Diff line number Diff line change
Expand Up @@ -284,8 +284,8 @@ def test_enet(self):
shared.try_delete('enet')
shutil.copytree(path_from_root('tests', 'enet'), 'enet')
with chdir('enet'):
run_process([PYTHON, path_from_root('emconfigure'), './configure'])
run_process([PYTHON, path_from_root('emmake'), 'make'])
run_process([PYTHON, path_from_root('emconfigure.py'), './configure'])
run_process([PYTHON, path_from_root('emmake.py'), 'make'])
enet = [self.in_dir('enet', '.libs', 'libenet.a'), '-I' + path_from_root('tests', 'enet', 'include')]

for harness in [
Expand Down
9 changes: 0 additions & 9 deletions tools/emdump

This file was deleted.

1 change: 1 addition & 0 deletions tools/emdump
24 changes: 0 additions & 24 deletions tools/python_selector.py

This file was deleted.

26 changes: 26 additions & 0 deletions tools/run_python.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
#!/bin/sh
# Copyright 2011 The Emscripten Authors. All rights reserved.
# Emscripten is available under two separate licenses, the MIT license and the
# University of Illinois/NCSA Open Source License. Both these licenses can be
# found in the LICENSE file.
#
# Entpy point for running python scripts on UNIX systems.

if [ -z $PYTHON ]; then
PYTHON=$(which python3)
fi

if [ -z $PYTHON ]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that it's possible for Python to be installed in a path with spaces, which breaks this line: #11005

PYTHON=$(which python)
fi

if [ -z $PYTHON ]; then
PYTHON=$(which python2)
fi

if [ -z $PYTHON ]; then
print 'unable to find python in $PATH'
exit 1
fi

exec $PYTHON "$0.py" "$@"
6 changes: 2 additions & 4 deletions tools/shared.py
Original file line number Diff line number Diff line change
Expand Up @@ -710,7 +710,7 @@ def replace_or_append_suffix(filename, new_suffix):
EMCC = path_from_root('emcc.py')
EMXX = path_from_root('em++.py')
EMAR = path_from_root('emar.py')
EMRANLIB = path_from_root('emranlib')
EMRANLIB = path_from_root('emranlib.py')
EMCONFIG = path_from_root('em-config')
EMLINK = path_from_root('emlink.py')
EMCONFIGURE = path_from_root('emconfigure.py')
Expand Down Expand Up @@ -1538,9 +1538,7 @@ def nop(arg):
# not to, as some configure scripts expect e.g. CC to be a literal executable
# (but "python emcc.py" is not a file that exists).
# note that we point to emcc etc. here, without a suffix, instead of to
# emcc.py etc. The unsuffixed versions have the python_selector logic that can
# pick the right version as needed (which is not crucial right now as we support
# both 2 and 3, but eventually we may be 3-only).
# emcc.py etc.
env['CC'] = quote(unsuffixed(EMCC)) if not WINDOWS else 'python %s' % quote(EMCC)
env['CXX'] = quote(unsuffixed(EMXX)) if not WINDOWS else 'python %s' % quote(EMXX)
env['AR'] = quote(unsuffixed(EMAR)) if not WINDOWS else 'python %s' % quote(EMAR)
Expand Down