Automate windows builds

[Cimbali]

This a major pain that has been brought up in #107 and #108 most recently, but earliest attempts date have existed. Basically building on windows is an immensely painful process. I believed to have documented this already somewhere but it appears not, so I’m creating an issue to track progress on this. I haven’t looked at this in 6+ months so I’m taking a look at things to see if they got un-stuck.

The core issues are:

• Pympress depends on gtk3 and poppler, which we need to either build or obtain in binary form, and redistribute.
• Distributing a python package as binary requires freezing it, from which we use cx_Freeze (though other alternatives

Basically the 2 main paths to build pympress for windows are either using Linux toolchains on Windows (i.e. MinGW/Msys and the related) or Microsoft toolchains (i.e. MSVC). Those are not compatible. So a system’s Python for Windows is the latter, while a python installed inside msys2 for example is the former.

The current/legacy technique relies on PyGObjectWin32, which uses the Microsoft way, with all of the packages pre-compiled, so it’s a rather pain-free mechanism. However this seems to be unmaintained and rather outdated by now. Also I do not particularly wish to boot an old machine up or have to keep a 32bit and 64bit python 3.4 on a Windows VM, this really does not scale.

Testing all of this via appveyor scripts or RDP is very slow, however this is as far as I’ve gotten along both paths:

1. Going the msys2 way gives a functional pympress, so msys2 is currently the best way to install pympress on a windows system, but we can’t use it (yet?) to build installers automatically.

   • most (all?) dependencies are pre-compiled and distributed as “wheels” over pip. All install without problems.
   • The cx_Freeze package (and any other freezing package I tried) requires compilation, as no wheel is (was?) available, and this breaks. The cx_Freeze package’s current master (marcelotduarte/cx_Freeze@29ed764) can be installed from source using pip successfully.
   • The correct list of things (typically DLLs) to be included in the frozen package needs to be adjusted.
     The following warning message is also slightly worrying:

     *** WARNING *** unable to create version resource
     install pywin32 extensions first

2. Going the MSVC way, we have to compile everything. Gtk+3 and cx_Freeze both work after some tweaking, but I can’t manage to get or install Poppler.

   • Gtk+ can be obtained from the gvsbuild project
   • From this you can then build pygobject and pycairo and test applications using Gtk seem to work
   • I’ve managed to install poppler from conda-forge, however the typelib (the bindings needed for the introspection) seem to be missing. So from gi.repository import Poppler breaks. Maybe by getting the recipe and building poppler ourselves with that recipe we could add the typelib?
   • cx_Freeze’s current master (marcelotduarte/cx_Freeze@29ed764) can be installed, but I have not tested its functionality

[Cimbali]

Builds using msys2 are here: https://ci.appveyor.com/project/Cimbali/pympress

I’ve managed to test the 64-bit no-vlc version successfully. Now waiting for all the different settings to build (~10 minutes each), and then we can test the other ones.

[ComFreek]

Awesome! Where can I find the new binaries, though? They don't seem to be at https://github.com/Cimbali/pympress/releases/tag/v1.4.0.

[FriedrichFroebel]

@ComFreek They seem to be available as build artifacts at AppVeyor only at the moment, for example at https://ci.appveyor.com/project/Cimbali/pympress/build/job/3fyiql1t0rqcy0nc/artifacts.

[Cimbali]

There still are some problems. The VLC binaries are 1GB for example.

[Cimbali]

Sticking with non-vlc binaries as the system’s VLC is detected without problems. Not the windows way I suppose, but it also has its advantages in disk space for the user and so on.

[ComFreek]

@FriedrichFroebel Ah!

Can I count on the binaries being shipped with every GitHub release? As in https://github.com/Cimbali/pympress/releases. I do not intend to update my Choco package manually. Instead, I've got an automated solution querying the GitHub API for the releases and inferring the download links from the filenames of the uploaded assets. I've seen that the asset names for v1.4.0 and v1.2.0 differ. Would it be possible to stick to one naming convention so that my automation does not break?

This would be not set in stone. Changing the naming convention again would just require shortly pinging me before hand and adjusting my automation.

Not the windows way I suppose

As a Windows user, I find this totally fine. (Yes, I am probably biased by being a programmer and fan of automation 😄)

[Cimbali]

The automated build & deployment to github releases and current naming schemes are here to stay. It’spympress-v{version}-{architecture}.msi and architecture will be one of the msys names, either x86_64 for 64 bits or i686 for 32 bits.

Maybe if some day we have non-mingw builds there’ll be a need to differentiate those, but I have no plans to work on that in the future. I’ve tested the mingw binaries, they work, and that’s good enough for me. Building for windows is really a most painful process.