Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

freeze.py: Don't hardcode Python extension paths #3031

Merged
merged 1 commit into from
Nov 17, 2019

Conversation

ferdnyc
Copy link
Contributor

@ferdnyc ferdnyc commented Oct 5, 2019

This is a start at making freeze.py reusable and more resilient, by removing the hardcoded paths to Python extension modules (PyQt5 components). Instead, each needed module is imported in turn, and sys.modules.get(module_name).__spec__.origin is used as the path to the .so file.

Also, some indentation is reduced by replacing positive conditionals in the loop with negative conditionals and continue, then outdenting the previous contents.

@ferdnyc
Copy link
Contributor Author

ferdnyc commented Oct 5, 2019

BTW, I'm pretty sure two of the paths were typoed, which may explain some of the troubles with freeze.py finding dependencies. Note how the PyQt5.QtGui and PyQt5.QtDBus extensions have "34dm" in their filenames, instead of "34m":

openshot-qt/freeze.py

Lines 203 to 212 in 7c8925b

for library in [os.path.join(libopenshot_path, "libopenshot.so"),
"/usr/lib/python3/dist-packages/PyQt5/QtWebKit.cpython-34m-x86_64-linux-gnu.so",
"/usr/lib/python3/dist-packages/PyQt5/QtSvg.cpython-34m-x86_64-linux-gnu.so",
"/usr/lib/python3/dist-packages/PyQt5/QtWebKitWidgets.cpython-34m-x86_64-linux-gnu.so",
"/usr/lib/python3/dist-packages/PyQt5/QtWidgets.cpython-34m-x86_64-linux-gnu.so",
"/usr/lib/python3/dist-packages/PyQt5/QtCore.cpython-34m-x86_64-linux-gnu.so",
"/usr/lib/python3/dist-packages/PyQt5/QtGui.cpython-34dm-x86_64-linux-gnu.so",
"/usr/lib/python3/dist-packages/PyQt5/QtDBus.cpython-34dm-x86_64-linux-gnu.so",
"/usr/lib/x86_64-linux-gnu/qt5/plugins/platforms/libqxcb.so",
"/usr/local/lib/libresvg.so"]:

It's possible that's correct, but it seems unlikely. Regardless, this new code will protect against any issues there, and allow upgrading Python itself and/or PyQt5 without breaking the freeze script, by retrieving the correct filenames direct from Python.

@ferdnyc
Copy link
Contributor Author

ferdnyc commented Nov 8, 2019

BTW, I'm pretty sure two of the paths were typoed

(Ooh, or maybe the 34dm libraries are debug versions? That sounds plausible — though they still shouldn't be the ones included in the packaging, I'd think.)

@jonoomph
Copy link
Member

As long as this still works, I'm good with it! Hoping this has been tested, but gonna merge and see what happens.

@jonoomph jonoomph merged commit 2804309 into OpenShot:develop Nov 17, 2019
@ferdnyc
Copy link
Contributor Author

ferdnyc commented Nov 19, 2019

As long as this still works, I'm good with it! Hoping this has been tested, but gonna merge and see what happens.

I'll be honest: changes to freeze.py and (especially) installer/build-server.py often aren't tested when I submit them, for the simple reason that the existing code currently contains a ton of hardcoded stuff that only applies on the GitLab builder hosts. (I mean, build-server.py uses paths in /home/ubuntu/, which unsurprisingly enough are not valid on my Fedora system.)

But, at lot of these PRs are attempts at changing that, and making the code more generic so that it can be run in other places. This one is a perfect example, since previously the Python module freezing would fail on any system that didn't use the exact same filenames. (i.e. not even most other Ubuntu installs, where the version numbers would be different.) All of which is to say, the concept behind these changes was tested locally, in developing the new code. But the changes themselves were not tested end-to-end, because I'm still not able to run the full AppImage creation process locally — at all! (A situation I'm slowly working towards improving, I hope.)

I guess what I'm saying is: No, very often these PRs aren't tested, but that's because they modify code that is itself inherently untestable. But a liberal merge-and-revert-if-necessary approach to them is probably the only way that situation's ever going to improve, so I'd encourage taking that approach to them. I'm always happy to fix what I do break, when I break things. ("Move fast and break stuff", wasn't that Facebook's internal motto? Not that I'd want to pattern myself after them, but it's hard to deny they must've been on to something...)

Some of my PRs can't ever carry a promise that they won't break anything. (Not an honest one, anyway. I could lie and say, "Suuuure, I guarantee the code will work, complete confidence!" I'm just not that guy.) Sometimes, in fact, the whole point of them is to break things. (In the way you sometimes have to re-break a fractured bone, in order to set it properly.) But ultimately, worst-comes-to-worst... that's exactly why version control exists, right? 👼

@ferdnyc ferdnyc deleted the freeze-hardcoding branch November 19, 2019 22:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants