Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade utility scripts to Python 3 #1056

Closed
wkozaczuk opened this issue Oct 5, 2019 · 4 comments
Closed

Upgrade utility scripts to Python 3 #1056

wkozaczuk opened this issue Oct 5, 2019 · 4 comments

Comments

@wkozaczuk
Copy link
Collaborator

Given the Python end-of-life scheduled for Jan 1st, 2020, we should upgrade at least the key scripts (setup.py, module.py, test.py, etc) under script to be Python 3 compatible.

Please see https://docs.python.org/3/howto/pyporting.html

@lgoldstein
Copy link
Contributor

I second this - I recently installed Ubuntu 18 and it comes with Python 3 and not 2 (I can install Python 2 as well, but IMO it would not be advisable).

@BassMatt
Copy link

Hi!

I submitted a patch with a few changes to all the files listed in scripts/README to update them to python3.

I was wondering if the files in scripts/osv/, and scripts/trace.py were in scope? I spent a little bit of time updating them but they seemed to do some more complicated things with prof.py, so I'm looking for a bit of guidance first..

@wkozaczuk
Copy link
Collaborator Author

@BassMatt thanks for the patch. Please address the minor issues others pointed on the mailing list and we will apply it.
Regarding what is in scope I think that both scripts/trace.py and scripts/osv/* are. Though I would leave the prof.py for now as I am not sure how much used it used. In general, the most important ones are scripts/run.py and others used by it directly or indirectly as well as all the python files used by script/build (module.py, etc).

wkozaczuk pushed a commit to wkozaczuk/osv that referenced this issue Feb 12, 2020
Main scripts in scripts/ folder updated to use Python3

I went through the scripts detailed in scripts/README and updated them to use Python3. I used the Python "Future" module to provide suggestions, then manually went through and applied the changes. The "Future" module gives suggestions to allow for cross-compatibility between Python2/3, but since it was expressed that only Python3 needed to be supported, I left all that out.

The issue is detailed here:
cloudius-systems#1056
Message-Id: <20191212200248.12048-1-bassmatt0515@gmail.com>
@wkozaczuk
Copy link
Collaborator Author

wkozaczuk commented Feb 17, 2020

Following python files have been identified as 'in use' or deemed as possibly useful in the near future:

./modules/httpserver-api/json2code.py
./modules/httpserver-api/tests/api/testenv.py 
./modules/httpserver-api/tests/api/testfile.py
./modules/httpserver-api/tests/api/testfs.py 
./modules/httpserver-api/tests/api/testjolokia.py
./modules/httpserver-api/tests/api/testjvm.py
./modules/httpserver-api/tests/api/testnetwork.py
./modules/httpserver-api/tests/api/testos.py 
./modules/httpserver-api/tests/api/testtrace.py 
./modules/httpserver-api/tests/basetest.py
./modules/httpserver-api/tests/testhttpserver-api.py
./modules/httpserver-html5-gui/tests/static/teststatic.py
./modules/httpserver-html5-gui/tests/testhttpserver.py
./modules/openjdk8-from-host/module.py
./scripts/export_manifest.py
./scripts/firecracker.py
./scripts/freq.py
./scripts/gen-rofs-img.py
./scripts/imgedit.py
./scripts/libosv.py
./scripts/loader.py
./scripts/manifest_common.py
./scripts/memory_analyzer.py
./scripts/metadata.py
./scripts/mkbootfs.py
./scripts/nbd_client.py
./scripts/osv/client.py
./scripts/osv/debug.py
./scripts/osv/modules/api.py
./scripts/osv/modules/filemap.py
./scripts/osv/modules/resolve.py
./scripts/osv/prof.py
./scripts/osv/trace.py
./scripts/osv/tree.py
./scripts/prof.py
./scripts/run.py
./scripts/setup.py
./scripts/tester.py
./scripts/test.py
./scripts/tests/test_app.py
./scripts/tests/test_app_with_test_script.py
./scripts/tests/test_http_app_with_curl_and_ab.py
./scripts/tests/testing.py
./scripts/tests/test_net.py
./scripts/tests/test_tracing.py
./scripts/top.py
./scripts/trace.py
./scripts/upload_manifest.py
./tests/tst-tcp-hash-cli.py

wkozaczuk added a commit that referenced this issue Feb 19, 2020
This patch incorporates changes from original patch
submitted by Matt Bass as well as extra changes made
by Waldemar Kozaczuk to upgrade python scripts from version
2 to version 3.

Some trivial changes are the result of automated upgrade tools like
"Future" module or 2to3. Others involving proper handling of binary and string
data especially when using subprocess module have been manually made. Same goes
for any code that tried to compare non-uniform data which python 3 is more
strict.

Most of the scripts have been tested but there maybe be scenarios/flows
which have been missed.

Ref #1056

Signed-off-by: Matt Bass <bassmatt0515@gmail.com>
Signed-off-by: Waldemar Kozaczuk <jwkozaczuk@gmail.com>
wkozaczuk added a commit that referenced this issue Feb 21, 2020
This is the second batch of the python 3 upgrade changes.
This one focuces on httpserver API unit tests and most changes
revolve around changes to urllib.* modules.

The scripts have been tested by runnin following tests in
the root of the modules/httpserver-api directory:

- make check-http
- make check-ssl

Ref #1056

Signed-off-by: Waldemar Kozaczuk <jwkozaczuk@gmail.com>
@nyh nyh closed this as completed in 228190a Feb 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants