-
Notifications
You must be signed in to change notification settings - Fork 414
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Sources for windows AIO #1444
Sources for windows AIO #1444
Conversation
Excellent idea! We should get our Windows maintainer @prculley to check this. Perhaps we could even automatically generate the AIO as part of the release process? |
Great idea. @jmichault Any chance the share.tgz could be created on the fly, looks like it contains existing icons etc. Next would be to automate the Apple Build |
Maybe a good test of installer generators would be a scheduled broad 5.2 Beta test? (Only for GitHub distribution, not for inclusion on the Wiki download page or wiki rollover.) |
I've looked this over and it appears to work. I had to modify it slightly to utilize my directory layout, but that should not affect the functionality. Might be a good idea to document the expected and resulting dir layout. I was initially a bit concerned that the various patches I had applied to the older MINGW files were not getting used. But as near as I can tell the things that I had patched seem to have been fixed over that last several years. I generated an experimental AIO for the 5.1.6 Gramps. To do this I had to fix the gramps/setup.py slightly to deal with bdist_wheel which was fixed in master, but not in maintenance/gramps51. It appears to install correctly and runs the limited tests I made correctly. I will probably spend some more time with this in the near future. If/when this PR is accepted, we will want to create a newer updated wiki article to help lead folks in running it. |
Could there be a item moved in the Windows install? The Examples would be FAR better located in the Gramps User Directory. This would also allow the Examples to be updated via the Addon Manager And could the installer also set the GRAMPSHOME and GRAMPSDIR https://www.gramps-project.org/wiki/index.php/Gramps_5.1_Wiki_Manual_-_Command_Line#GRAMPSHOME |
Don't set |
@jmichault The Windows build is failing. I have merged the aio directory, but not the workflow for now. |
Rebased. |
Regarding build failure; Perhaps someone who knows more about package management can chime in. |
@prculley The latest python3 is at https://repo.msys2.org/mingw/mingw64/mingw-w64-x86_64-python-3.11.4-4-any.pkg.tar.zst and there are packages going back to 3.9.7. Dunno what you mean by virtual: 23M compressed seems like it would be pretty complete. That has nothing to do with the build errors:
That breaks the What I don't get is why they fail: https://repo.msys2.org/mingw/mingw64/mingw-w64-x86_64-python-bsddb3-6.2.9-4-any.pkg.tar.zst exists, but on my VM:
So it worked two weeks ago, but doesn't now. I suspect that's due to my having done |
Virtual packages in pacman are just placeholders. A package could require a "python3" package, but the dependency could actually be fulfilled by a "python" package. I expect that they have finally completed a transition where "python" packages have moved from "python2" to "python3". Also, we may not need all the dependencies in v5.2 since we can now attempt to install addon prerequisites using PyPI. I have been unable to test this for Windows though. |
While looking at the Windows with MSYS2 wiki page, the thought occurred that maybe inviting more people (like everyone who edited that wiki page) to this conversation might help? Changing the participation impetus from 'pull' to 'push' might be necessary. https://www.gramps-project.org/wiki/index.php/Gramps_for_Windows_with_MSYS2 |
I've updated the package names and made a few changes to get it to build. There are still a few issues that need further investigation:
|
|
That's OK for CI but if you or @prculley are using aio/build.sh to build the AIO you need to get bsddb3 from PyPI instead (just like asyncio) and uncomment the line or users won't be able to load their old BerkeleyDB databases. |
@jralls I loaded the gspell package, but I didn't know what entry to put in the missing dll list. I'll give PyPI a try for bsddb3. |
@Nick-Hall Jumping in the conversation here: Based on contents of the gspell pacakge being installed, the DLL should be |
I can't get bsddb3 installed using pip. The
An |
I just looked into this a bit, and I found that bsddb3 doesn't work for Python v10. See https://www.jcea.es/programacion/pybsddb.htm So for upgrade purposes we may have to switch to the berkeleydb. Quoting the author "Take note that upgrading to berkeleydb is easy but not transparent. Notably, keys and values are bytes in berkeleydb lib, while in ‘bsddb3’ they are strings." |
I'm using python-bsddb 6.2.9 with python 3.11 without any problems. |
I get the same error with berkeleydb as I do with bsddb3. |
You need |
@jralls The environment variables only seem the be read for a posix install. For Windows, it expects everything in a fixed directory. |
That's annoying. It doesn't look like there's a good solution that will make it work with pip. How about
bypassing the removal of pybsddb3 from the package management system? Maybe for long term safety we should copy that tarball to sourceforge and install it from there instead of repo.msys2.org. |
With the
|
Implying that it didn't end up on sys.path. Did it perhaps install in the wrong python version (i.e. /mingw64/lib/python3.10 instead of /mingw64/lib/python3.11)? |
The build now works and the GrampsAIO is available as an artifact. @jralls It was installing to 3.10 rather than 3.11, so I just copied over the files and it seems to work. I'm not using any icons from the The only thing that I am still extracting from It would be useful if someone with a copy of Windows could install the AIO to check that it actually works. |
error returned: "Fatal error in launcher: U" |
It looks like pip wasn't installed properly. |
Net search says that is a partial message. Should be "Fatal error in launcher: Unable to create process using pip" |
Try |
hmmm. python isn't in the path either. Searching again. But it will probably need a full path to pip |
Since Windows is pretty particular about modifying anything in the "Program Files" without admin rights, I tried running Gramps with admin rights. I now get a different "Module installation failed" We may need to rethink the whole idea of using pip to install, and go back to making the installation in the addons directory (where admin rights are not needed). This is what was done in the 5.1.x versions of the lifeline charts. |
@prculley Did you remember to change "sys.executable" to "python.exe"? |
That maybe OK for Windows, but we probably wouldn't want to do that for Mac or Linux. |
@Nick-Hall Looks like the aio is using the embedded python for Windows? If so the following might be relevant:
Some stackoverflow answers seems to indicate various ways to include pip but are not definitive! |
I see that we are actually trying to run "python -m pip install lifeline_chart" which cannot work on Windows since we don't have python.exe present. |
OK. So we do have to take a different approach on Windows. I'm actually happy to disable the auto-install functionality in Linux. It is easy to click the "Requires" button and install any requirements manually. I'm not sure what we want to do with the Mac bundle. |
I just installed Python 3.8.1 and re-tried python -m pip install svgwrite it installed. but warned about using pip 19.2.3 when 23.2.1 is available |
However, Gramps does not recognize svgwrite in the installed location under Python38 |
I tried patching the code to run pip.exe instead of python.exe. The error I get is |
it is a partial message: https://bobbyhadz.com/blog/python-fatal-error-in-launcher-unable-to-create-process-using-pip-exe "Fatal error in launcher: U"...nable to create process using pip The Python "Fatal error in launcher: Unable to create process using pip" occurs for multiple reasons:
|
It looks like pip.exe is trying to open python.exe and failing... (found with Process monitor). |
IIUC you're trying to make it feasible for addons contributors to have any random dependency and still allow it to work with an AIO bundle. Neither provides the python executable directly, so shelling out to pip won't work. Importing pip--after all, it's a python module--and calling its API from the add-on manager might work. Adding stuff to C:\Program Files outside of an installer is probably not a great idea, and adding stuff to a macOS app bundle voids its signature and may cause it to not launch. The best solution to that is to tell pip to do a user install next to the plugins and to add that location to If that doesn't work then you'll have to fall back to the pre-5.2 approach of providing a reasonable set of dependencies in the AIO and making it clear to add-on contributors that if their addons have other dependencies it will mean that they're available only to Linux users and the very few users on Windows and macOS capable of and willing to manage a from-scratch build of Gramps. |
I meant that it's not in the drop down menu, when no tree is loaded. That menu only has one entry, named something like Preferences (Voorkeuren in Dutch). The icon is visible, and works OK. |
When I do that, it works for the Python version that I installed in Windows:
Gramps doesn't see it, because it has no idea about this package location, and only cares about the one in its own installation folder. PullRequest# 1569 : Make the Addon Manager available without a tree loaded |
I ran some tests in W10, installing svgwrite in Python for Windows, in Python for MSYS, and by letting the 5.1.6 LifeLines plug-in have its way, and saw that, apart from Python for MSYS, they all installed svgwrite inside AppData, meaning in user space, which is the proper way to do this, IMHO. Even when I run pip in Linux, without sudo, it does that, by putting it inside ~/.local/lib/python-3.9/site-packages. To me, this suggests that we should do the same in Gramps 5.2, meaning that we create a folder named GrampsAIO64-5.2.0 or something similar inside AppData\Local\Packages, and let Gramps pip store all packages there. |
I defined
It installs the modules, but doesn't recognise them until I restart Gramps. |
Using pip in the AIO seems like a non-starter. pip requires python.exe, which doesn't exist in the embedded python created by cx_freeze. I looked into importing pip, the developers say you should NOT do this, they even changed some things recently to break the import hack some folks were doing. Attempts to install Python and pip to the system from some other source, doesn't work for several reasons, like not caring unless it is in its own installation folder, and having to be the exact pip/python as the version used to create the AIO . |
I'm thinking that if we want to try to support prerequisites installation, we are going to have to do it like the 5.1.x lifelinecharts gpr code did it. I think it would make some sense to provide a couple of utility calls in Gramps, perhaps with appropriately adjusted code from the lifelinecharts, and storing the loaded modules in the correct user space (instead of with each addon). Not as good a pip, as the addon would be required to deal with extended prerequisites. |
Here is a Windows AIO for testing. As Paul points out it may not work. It is really doing the same thing as the old Life line charts code. The module is downloaded to a directory under USER_PLUGINS/lib which I have appended to We could replace the pip code with something similar to download a PyPI package and extract it into a target directory. |
And Paul is right ...
I checked the ZIP file, to be sure, and the certificate wasn't there. |
But on the other hand, it properly posted that error to the Error Report dialog instead of just posting to the console. |
The link that Paul gave suggests using "python -m pip" which was my first approach. I was aware that my last attempt was not recommended. The If someone can write a function |
Currently, the gramps repository lacks sources to create the Windows AIO installer.
This pull request adds: