[RFC] CMake buildsystem QoL improvements ?

[LebedevRI]

Hello.
To preface, i will understand if this isn't something that is wanted for the project..

I'd like to try using halide in some my future toy project.
As it stands now, halide is largely not packaged in distributions,
certainly not in debian; and this is a dealbreaker for me :/

After looking around, i understand why that might be the case - the cmake build system
is, uhm, making some very unusual and interesting choices.

The question is: is there any interest in trying to straighten out the cmake build,
make it more likely to be packaged, and easier to use in cmake-based projects?

Right now i see 3 main issues:

1. There should be a switch to OPTIONALLY link to shared LLVM libraries (trivial, i already have a patch)
2. The installation destinations need to be fixed, the current install directory structure is not ideal. (Likely simple. Should the defaults (that are likely in-parity with other build systems?) be fixed, or should the defaults remain?)
3. A proper Halide-Config.cmake magic needs to be added, so it can be found with simple find_package(halide).

[abadams]

These sorts of improvements are generally welcome, provided they don't break existing usage. @steven-johnson is the person who knows the most about the existing cmake-based build. I generally avoid it and just use the makefiles.

[BachiLi]

A related issue is that the Python bindings also need a CMake file (especially important for Windows users).

[steven-johnson]

Improvements of the sort you describe would be quite welcome. We have historically used CMake primarily for Windows testing, but as you can see, it often lags behind; getting it to 100% parity with Makefile would be a highly desirable goal in my opinion. Contributions in this vein would be great, especially because none of the core contributors use CMake much outside of this project and thus things like the installation options are easy for us to overlook.

[LebedevRI]

provided they don't break existing usage

Yeah, that's the fun part i suppose.
Do i understand correctly that if one needs to opt-in into new behavior explicitly,
for example via a CMake option that you'd pass when building Halide itself,
then the existing usage would not be considered broken?

[zvookin]

We're totally into helping to make stuff like this better. The main thing I'll add to the discussion so far is that future success rides on making sure the things you care about are covered by tests somehow so that if anything breaks it gets caught before it gets checked in.

I'd also like to see us have regular packaging for various environments as part of our release process.

[LebedevRI]

provided they don't break existing usage

Yeah, that's the fun part i suppose.
Do i understand correctly that if one needs to opt-in into new behavior explicitly,
for example via a CMake option that you'd pass when building Halide itself,
then the existing usage would not be considered broken?

I.e. what i'm saying is, would it be acceptable to diverge into two incompatible layouts:

• the currently-distributed tarball with current halide_config.cmake and it's usage scenarios
• and a proper CMake config, proper installation directories, etc

via a CMake option at Halide configuration time?
Or should the halide_config.cmake continue working even when the installation paths are fixed?

I'd also like to see us have regular packaging for various environments as part of our release process.

Can be done. As the end result i do plan to end up with debian package files,
so at the very least we can rope https://build.opensuse.org/ into producing
continuous builds for debian-like distros. After that it should be trivial to also
add packages for suse/redhat. So that should cover reasonable chunk of usebase :)
A webhook will need to be configured in github repo settings to auto-trigger builds.

So don't worry about that part.

The main thing I'll add to the discussion so far is that future success rides on making sure the things you care about are covered by tests somehow so that if anything breaks it gets caught before it gets checked in.

I'm sorry, i'm not sure how this could look.

[steven-johnson]

I suspect (without evidence) that the current tarball + usage has limited usage, because (as you've pointed out) it has various serious drawbacks. We'd want to take a straw poll on the mailing list, Gitter room, etc, to see if there are users with nontrivial dependence on the status quo, but I really don't want to fork our build system even more than it already is; making breaking changes to CMake that are a net improvement to people used to that ecosystem is a win IMHO.

[abadams]

By not breaking the status quo I really just meant that it has to still be tested and work to the same extent that it currently does on all the platforms we support. I.e. don't break the build bots.

[alexreinking]

I am also very interested in improving and modernizing the CMake build. In particular, I don't see any reason we couldn't depend on a very recent version of CMake (like 3.13, the version available by default in Debian Buster) instead of 3.3. The main advantage here being that there are many new features available only in 3.12+ that would greatly improve the simplicity and maintainability of our build.

Adding Halide to Nuget (for Windows), Homebrew (for macOS), a Debian/Ubuntu-compatible PPA, and Pip/Conda (for everyone else), would absolutely ease the installation burden for new users.

[steven-johnson]

I would be in favor of such a move as well.

…

On Mon, Oct 14, 2019, 7:29 PM Alex Reinking ***@***.***> wrote: I am also very interested in improving and modernizing the CMake build. In particular, I don't see any reason we couldn't depend on a very recent version of CMake (like 3.13, the version available by default in Debian Buster) instead of 3.3. Adding Halide to Nuget (for Windows), Homebrew (for macOS), a Debian/Ubuntu-compatible PPA, and Pip/Conda (for everyone else), would absolutely ease the installation burden for new users. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4284?email_source=notifications&email_token=AACQJ65VBZVXXCSLZQ6DKT3QOUTJHA5CNFSM4I3PAAHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBHGGMY#issuecomment-542008115>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACQJ62YFRJ7RALLTB7HZ4LQOUTJHANCNFSM4I3PAAHA> .

[alexreinking]

Most of this has been handled in #4644 -- still need to create deb packages.

[LebedevRI]

Oh, nice!
I did the initial deb packaging work, let me see if i can resurrect it.

[LebedevRI]

Whoa, huge congrats to everyone involved!

[alexreinking]

Thanks @LebedevRI -- I took the lead on this and @steven-johnson was invaluable getting our build infrastructure up to date and finding bugs in my work.