Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issues with "Linux Cross-Compile" (Debian WSL) #7526

Open
1 task done
bratpeki opened this issue Sep 29, 2024 · 40 comments
Open
1 task done

Issues with "Linux Cross-Compile" (Debian WSL) #7526

bratpeki opened this issue Sep 29, 2024 · 40 comments
Labels

Comments

@bratpeki
Copy link
Contributor

bratpeki commented Sep 29, 2024

System Information

Windows 10

LMMS Version(s)

Git

Most Recent Working Version

No response

Bug Summary

Following https://github.com/LMMS/lmms/wiki/dependencies-windows#linux-cross-compile to compile on Windows is not straightforward.

I set up Debian using WSL2 on my machine.

The sudo add-apt-repository ppa:tobydox/mingw-w64 command gives the following result:

image

Expected Behaviour

The command gives no errors, and one may proceed with the compilation steps.

Steps To Reproduce

  • Get WSL2 and install Debian
  • Run sudo apt update and sudo apt install software-properties-common
  • Run the command in question

Logs

peki@...:~$ sudo add-apt-repository ppa:tobydox/mingw-w64
[sudo] password for peki:
Traceback (most recent call last):
  File "/usr/bin/add-apt-repository", line 362, in <module>
    sys.exit(0 if addaptrepo.main() else 1)
                  ^^^^^^^^^^^^^^^^^
  File "/usr/bin/add-apt-repository", line 345, in main
    shortcut = handler(source, **shortcut_params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/softwareproperties/shortcuts.py", line 40, in shortcut_handler
    return handler(shortcut, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 86, in __init__
    if self.lpppa.publish_debug_symbols:
        ^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 126, in lpppa
    self._lpppa = self.lpteam.getPPAByName(name=self.ppaname)
                  ^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/softwareproperties/ppa.py", line 113, in lpteam
    self._lpteam = self.lp.people(self.teamname)
                   ^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'people'

Screenshots / Minimum Reproducible Project

No response

Please search the issue tracker for existing bug reports before submitting your own.

  • I have searched all existing issues and confirmed that this is not a duplicate.
@bratpeki bratpeki added the bug label Sep 29, 2024
@bratpeki
Copy link
Contributor Author

This could have something to do with versioning (Bullseye, Buster, etc), but I don't know.

@bratpeki bratpeki changed the title Issues with "Linux Cross-Compile" Issues with "Linux Cross-Compile" (Debian WSL) Sep 30, 2024
@bratpeki
Copy link
Contributor Author

bratpeki commented Oct 3, 2024

Solved it with sudo apt-get install python3-launchpadlib. I'm now getting the following issue:

E: The repository 'https://ppa.launchpadcontent.net/tobydox/mingw-w64/ubuntu bookworm Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

@bratpeki
Copy link
Contributor Author

bratpeki commented Oct 3, 2024

Fixed with echo "deb [trusted=yes] http://www.deb-multimedia.org bookworm main" >> /etc/apt/sources.list.

@bratpeki
Copy link
Contributor Author

bratpeki commented Oct 3, 2024

Then running sudo apt-get install gets you:

E: Unable to locate package sdl-mingw-w64
E: Unable to locate package sdl2-mingw-w64
E: Unable to locate package libvorbis-mingw-w64
E: Unable to locate package lame-mingw-w64
E: Unable to locate package fluidsynth-mingw-w64
E: Unable to locate package stk-mingw-w64
E: Unable to locate package glib2-mingw-w64
E: Unable to locate package portaudio-mingw-w64
E: Unable to locate package libsndfile-mingw-w64
E: Unable to locate package fftw-mingw-w64
E: Unable to locate package flac-mingw-w64
E: Unable to locate package fltk-mingw-w64
E: Unable to locate package libgig-mingw-w64
E: Unable to locate package libsamplerate-mingw-w64
E: Unable to locate package libsoundio-mingw-w64
E: Unable to locate package qt5base-mingw-w64

WSL uses Bookworm:

peki@...:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 12 (bookworm)
Release:        12
Codename:       bookworm

@messmerd
Copy link
Member

messmerd commented Oct 3, 2024

Tobydox's PPA is for Ubuntu 20.04 (focal), not Debian 12 (bookworm): https://ppa.launchpadcontent.net/tobydox/mingw-w64/ubuntu/dists/focal/

It looks like @FyiurAmron erroneously changed the wiki article to say "Debian" rather than "Ubuntu" starting with this revision: https://github.com/LMMS/lmms/wiki/Dependencies-Windows/4b141b4f8fb77c9cd46799e2e87f6a9499dcd03a

I just changed it back to "Ubuntu". While you probably could get it to work on Debian, we do not officially support anything other than what our automated builds use, which is Ubuntu 20.04.

@FyiurAmron
Copy link
Contributor

@messmerd FWIW, you're completely right here, I tried to recall why I changed that but couldn't find any rational reason, probably were just editing 3 pages at once and went on a spree and done this due to some brain fart. Maybe I wanted to describe the exact procedure for normal Debian distros in-depth and either forgotten about that or gave up on it, or maybe it was just a completely random mistake. Can't recall now, the last months been hectic for me.

Anyhow, I 100% agree, although technically Ubuntu still is Debian, just repackaged :D (and yes, with the deps either built from source or otherwise repackaged, I see no reason why it wouldn't work, but I also see no reliable way to support people doing that since I'm not using raw Debian on a daily basis anymore and I guess it's an edge case anyhow)

@bratpeki tl;dr those PPAs are a relic from the past, so compiling them libs from source would be your best bet - but while you could do that or even try to configure APT to fetch those for focal and use them (it's possible even if probably useless), it makes no sense on WSL IMVHO - for Win, just go with MSYS, it should work out-of-the-box, especially after the bunch of updates I did for it this year. If it doesn't, ping me, we'll figure something out.

@bratpeki
Copy link
Contributor Author

bratpeki commented Oct 3, 2024

Excellent, thank you both! I'll move the compilation process to MSYS, then!

@FyiurAmron, where can I find relevant documentation, if any?

I'm persuming MSYS2 is focused on using Pacman, so I could get the dependency list off the lmms-git AUR repo, but is there anything on the LMMS Wiki regarding this?

(Update: MSYS' Pacman is definitely not housing the same packages as Arch's Pacman, for anyone learning about it and reading this in the future, LOL)

@FyiurAmron
Copy link
Contributor

FyiurAmron commented Oct 3, 2024

@bratpeki doh! Now I recall, I was in the process of updating those docs when RL stuff struck me. Found some commented out doc blocks hinting on what I wanted to do next, and it was indeed providing docs for MSYS process, and perhaps Debian afterwards. It's quite straightforward for MSYS anyway and strictly follows what's described on https://github.com/LMMS/lmms/wiki/Compiling for other compile paths; I'll post you the exact file I guess was supposed to be the basis of the doc:

# arch is either x86_64 or i686 ; assuming fresh MSYS install

### mandatory platform/toolchain
pacman -S git
pacman -S mingw-w64-x86_64-toolchain
pacman -S mingw-w64-x86_64-cmake
# MSYS2 make, runs as "Unix Makefiles" to simplify the build
pacman -S make
# CMake deps
pacman -S perl-List-MoreUtils
pacman -S perl-XML-Parser

### mandatory 3rd party deps
pacman -S mingw-w64-x86_64-libsamplerate
pacman -S mingw-w64-x86_64-fftw
pacman -S mingw-w64-x86_64-libsndfile
pacman -S mingw-w64-x86_64-qt5-base
# or qt6 etc. for the fork
pacman -S mingw-w64-x86_64-SDL2
# or some other output equivalent to SDL2

### misc deps
# for Windows packaging
pacman -S mingw-w64-x86_64-nsis
# for ZynAddSubFx
pacman -S mingw-w64-x86_64-fltk
# for SF2
pacman -S mingw-w64-x86_64-fluidsynth

### and all the other libs you fancy

After getting the libs installed, prepare the dirs for make (cd lmms && mkdir build && cd build), run make etc. (cmake .. -DCMAKE_INSTALL_PREFIX=../target and cmake --build . is what I had in my shell history) and you should get the binaries in no time (well, that is if you consider ~10-30 minutes "no time" :)

If it works for you, feel free to add this or any other relevant data to the wiki as new section. We had some outdated docs for MSYS in the olden days, but https://github.com/LMMS/lmms/wiki/Dependencies-Windows/_compare/a857e2562de0d13a5387d75db2508c9cf7127641...6cfbc94f040dce913ab456429c1b2b897dad1167 removed them a couple of months ago (not that it was a wrong call by itself, that part was outdated as hell and was also mostly unnecessary anymore).

@bratpeki
Copy link
Contributor Author

bratpeki commented Oct 3, 2024

Wonderful, thank you! I'd help you keep the building Wiki page up-to-date, if necessary, and I believe the cmake commands should be different, to specify the output binary is a Windows one!

@bratpeki
Copy link
Contributor Author

bratpeki commented Oct 3, 2024

:: There are 13 members in group mingw-w64-x86_64-toolchain:
:: Repository mingw64
   1) mingw-w64-x86_64-binutils  2) mingw-w64-x86_64-crt-git  3) mingw-w64-x86_64-gcc  4) mingw-w64-x86_64-gdb
   5) mingw-w64-x86_64-gdb-multiarch  6) mingw-w64-x86_64-headers-git  7) mingw-w64-x86_64-libmangle-git
   8) mingw-w64-x86_64-libwinpthread-git  9) mingw-w64-x86_64-make  10) mingw-w64-x86_64-pkgconf
   11) mingw-w64-x86_64-tools-git  12) mingw-w64-x86_64-winpthreads-git  13) mingw-w64-x86_64-winstorecompat-git

I believe I should either choose 1 or 3, looking into it now.

@FyiurAmron
Copy link
Contributor

I believe the cmake commands should be different, to specify the output binary is a Windows one!

@bratpeki From what I recall, MSYS's cmake can only output Win32/64 binary, so no need to instruct it to do so. The commands should work as-is (maybe you'll need to tell CMake to use Unix buildfiles instead of Ninjas etc., but YMMV there, I just didn't have any luck with Ninja builds, some random errors started happening).

@FyiurAmron
Copy link
Contributor

FyiurAmron commented Oct 3, 2024

I believe I should either choose 1 or 3, looking into it now.

You'll need all 13 of them IIRC. Some are optional-ish (gdb e.g.), but most of them are strictly required, and there is no harm in having them all anyway.

@bratpeki
Copy link
Contributor Author

bratpeki commented Oct 3, 2024

After finnicking around with my antivirus software, I am pleased to announced I've compiled it! The instructions were straight-to-the point, with the exception of having to restart!

@bratpeki
Copy link
Contributor Author

bratpeki commented Oct 3, 2024

When attempting to run from the ...\msys64\home\...\lmms\build, lmms.exe reports missing libs, namely libgcc_s_seh-1.dll and libfftw3f-3.dll. As I recall, running from the build dir is possible on Linux, is that not the case on Windows?

@FyiurAmron
Copy link
Contributor

FyiurAmron commented Oct 4, 2024

@bratpeki

After finnicking around with my antivirus software, I am pleased to announced I've compiled it! The instructions were straight-to-the point, with the exception of having to restart!

TBH, I never had to do that for MSYS, but obviously YMMV here, I'm still on old-ish MSYS version on Win7, maybe Win10 version changed something, maybe you were just unlucky with your particular setup for some reason.

When attempting to run from the ...\msys64\home\...\lmms\build, lmms.exe reports missing libs, namely libgcc_s_seh-1.dll and libfftw3f-3.dll. As I recall, running from the build dir is possible on Linux, is that not the case on Windows?

It should be possible on both. However, to do that on Windows you'd have to have a dir with those DLLs in a shared location (i.e. on Windows PATH) for this to work without having the DLLs bundled in the same dir. That's one way to handle it, althout some files required for "normal" LMMS work might be missing anyway (I'm not using this one personally for that reason).

Another way is to use CMake to create an NSIS build to get a complete package with all the DLLs bundled. AFAIK you'll get all the needed files in build\_CPack_Packages\win64\NSIS\lmms-1.3.0-alpha-mingw-win64 (exe, DLLs, all the rest), and you'll get the installer for free as well. This is fool-proof, you basically get a portable-ish install this way, same files that you'd get by unpacking the installer manually. The only downside is that the dir will be regenerated with each build-with-package, so you probably need to copy this somewhere.

Yet another approach is to just copy the exe from build to an existing 1.3.0-alpha install location and either overwrite the old one or rename one of them. This one should be fool-proof as well, drag-n-drop or one-liner is needed only. Renaming the exes allows you to compare the different build versions side-by-side as well too, so it's a 10/10 approach in my book.

You might also mix the above solutions in any way you fancy, obviously. Do what suits you best.

@Rossmaxx
Copy link
Contributor

Rossmaxx commented Oct 4, 2024

Or you can just cmake --install . --prefix "../target/"?

@FyiurAmron
Copy link
Contributor

@Rossmaxx a very good point. So, in total, that's at least 4 reliable methods to handle this. Should be enough :D

@bratpeki
Copy link
Contributor Author

bratpeki commented Oct 4, 2024

Thank you both!

This works like a charm with an out of the box MSYS2 install.

I think I could write up an addition to the PR, as this seems like a very easy way to get building up and running on Windows!

I'll give Ross' command and packaging a shot, and let you know how it goes.

@bratpeki
Copy link
Contributor Author

bratpeki commented Oct 5, 2024

Another way is to use CMake to create an NSIS build [...] you'll get all the needed files in build\_CPack_Packages\win64\NSIS\lmms-1.3.0-alpha-mingw-win64

Works like a charm!

@bratpeki
Copy link
Contributor Author

bratpeki commented Oct 5, 2024

I have utilized the following build rules, and I'm just sending them to have them saved and formatted, as well as present my progress:

  1. NSIS packaging
mkdir build
cd build
cmake .. -DCMAKE_INSTALL_PREFIX=../target
cmake --build . --target package

This makes using Ninja by default and installs the files to build\_CPack_Packages\.... I would also persume it installs to target, but the compilation process is a bit long on my machine, so I'll verify it later!

  1. MSYS Makefiles + installing to target
mkdir build
cd build
cmake -G "MSYS Makefiles" ..
make
cmake --install . --prefix "../target/"

@Rossmaxx
Copy link
Contributor

Rossmaxx commented Oct 7, 2024

So can we close?

@bratpeki
Copy link
Contributor Author

bratpeki commented Oct 7, 2024

Just let me make a proposed edit to the Wiki and we can close!

@tresf
Copy link
Member

tresf commented Oct 8, 2024

Excellent, thank you both! I'll move the compilation process to MSYS, then!

Morbid curiosity, but why not use an Ubuntu WSL instance?

@bratpeki
Copy link
Contributor Author

bratpeki commented Oct 8, 2024

I am also intending to try an Ubuntu WSL, since I made an issue of trying to compile on Debian, which isn't officially supported. I realized my mistake only recently, and forgot to write about it. Thank you for commenting, that reminded me to write!

Also relevant:

It looks like FyiurAmron erroneously changed the wiki article to say "Debian" rather than "Ubuntu" starting with this revision [...]

@bratpeki
Copy link
Contributor Author

bratpeki commented Oct 8, 2024

Also, @FyiurAmron generally recommended MSYS, saying that the PPAs are "a relic of the past", so I'm intending to provide both compilation methods, if compiling with the PPAs is still possible. If I have trouble we can't overcome, MSYS has proved to be ridiculously simple!

@tresf
Copy link
Member

tresf commented Oct 8, 2024

I am also intending to try an Ubuntu WSL, since I made an issue of trying to compile on Debian, which isn't officially supported. I realized my mistake only recently, and forgot to write about it. Thank you for commenting, that reminded me to write!

This may be a bit outdated (6 years old) but here's how I did it last: https://gist.github.com/tresf/3c739a739b56d8dc0679c3f2f85c349d

@tresf
Copy link
Member

tresf commented Oct 8, 2024

PPAs are "a relic of the past"

For cross-compiling, I'm unaware of an equivalent to Toby's PPA. I think cross-compiling is a nice option however we may decide to remove if if keeping the PPAs around is too much work.

@bratpeki
Copy link
Contributor Author

bratpeki commented Oct 8, 2024

MSYS2:

### MANDATORY PLATFORM/TOOLCHAIN

pacman -S git
pacman -S mingw-w64-x86_64-toolchain
pacman -S mingw-w64-x86_64-cmake

# MSYS2 make, runs as "Unix Makefiles" to simplify the build
pacman -S make

# CMake
pacman -S perl-List-MoreUtils
pacman -S perl-XML-Parser

### MANDATORY 3RD-PARTY DEPENDENCIES

pacman -S mingw-w64-x86_64-libsamplerate
pacman -S mingw-w64-x86_64-fftw
pacman -S mingw-w64-x86_64-libsndfile
pacman -S mingw-w64-x86_64-qt5-base
pacman -S mingw-w64-x86_64-SDL2

### OTHER DEPENDENCIES

# Windows packaging
pacman -S mingw-w64-x86_64-nsis

# ZynAddSubFx
pacman -S mingw-w64-x86_64-fltk

# SF2
pacman -S mingw-w64-x86_64-fluidsynth

mkdir build
cd build

# Default CMake building
cmake ..
cmake --build . --target package

# You can also use Makefiles:
# cmake -G "MSYS Makefiles" ..
# make
# cmake --install . --prefix "../target/"

# You can find the build in lmms\build\_CPack_Packages\

I'll look at what @tresf sent as well.

@bratpeki
Copy link
Contributor Author

bratpeki commented Oct 9, 2024

WSL Ubuntu 20.04.6 LTS off the Windows store fails the big sudo apt-get install command off here, with this error:

Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package sdl-mingw-w64

More info:

peki@...:~$ sudo add-apt-repository ppa:tobydox/mingw-w64
[sudo] password for peki:
Libraries packaged for the mingw-w64 target (Windows runtime)
More info: https://launchpad.net/~tobydox/+archive/ubuntu/mingw-w64
Press [ENTER] to continue or Ctrl-c to cancel adding it.
Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease
Hit:2 http://ppa.launchpad.net/tobydox/mingw-w64/ubuntu focal InRelease
Hit:3 http://archive.ubuntu.com/ubuntu focal InRelease
Hit:4 http://archive.ubuntu.com/ubuntu focal-updates InRelease
Hit:5 http://archive.ubuntu.com/ubuntu focal-backports InRelease
Reading package lists... Done

peki@...:~$ sudo apt-get update
Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease
Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease
Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease
Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease
Hit:5 http://ppa.launchpad.net/tobydox/mingw-w64/ubuntu focal InRelease
Reading package lists... Done

peki@...:~$ sudo apt-get install \
> git cmake nsis libxml-parser-perl liblist-moreutils-perl mingw-w64-tools \
> gcc-mingw-w64-i686 gcc-mingw-w64-x86-64 g++-mingw-w64-i686 g++-mingw-w64-x86-64 \
> sdl-mingw-w64 sdl2-mingw-w64 libvorbis-mingw-w64 lame-mingw-w64 \
> fluidsynth-mingw-w64 stk-mingw-w64 glib2-mingw-w64 portaudio-mingw-w64 \
> libsndfile-mingw-w64 fftw-mingw-w64 flac-mingw-w64 fltk-mingw-w64 \
> libgig-mingw-w64 libsamplerate-mingw-w64 libz-mingw-w64-dev \
> binutils-mingw-w64 gcc-mingw-w64 libsoundio-mingw-w64 \
> qt5base-mingw-w64 qttools5-dev-tools
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package sdl-mingw-w64

@tresf
Copy link
Member

tresf commented Oct 10, 2024

WSL Ubuntu 20.04.6 LTS off the Windows store fails the big sudo apt-get install command off here, with this error:

I believe you need to force the PPAs.https://github.com/LMMS/lmms-ci-docker/blob/a812fb1290942ca0b53b92d4e32ff07370629cad/linux.mingw/Dockerfile#L9-L10

@bratpeki
Copy link
Contributor Author

The error arises from a missing package: E: Unable to locate package sdl-mingw-w64.

A quick search shows very little about the package.

@Rossmaxx
Copy link
Contributor

The sdl mingw package should be a part of Toby's ppa

@bratpeki
Copy link
Contributor Author

bratpeki commented Oct 13, 2024

It does seem to be: https://launchpad.net/~tobydox/+archive/ubuntu/mingw-w64

I'll try building in the current state.

@messmerd
Copy link
Member

messmerd commented Oct 13, 2024

@bratpeki You don't need SDL1 anymore. With #7259, we stopped using SDL1 for automated builds. And since #7443, support for SDL1 has been removed entirely.

I will update the wiki to remove sdl-mingw-w64 from the list of packages you need.

@bratpeki
Copy link
Contributor Author

Excellent! Please notify me when you update it.

@messmerd
Copy link
Member

I already did it shortly after my previous comment: https://github.com/LMMS/lmms/wiki/Dependencies-Windows/_history

@bratpeki
Copy link
Contributor Author

bratpeki commented Oct 14, 2024

I'm attempting to build in Ubuntu 20.04.6:

Optional plugins
----------------
* Lv2 plugins                 : not found, install it or set PKG_CONFIG_PATH appropriately
* SUIL for plugin UIs         : not found, install it or set PKG_CONFIG_PATH appropriately
* ZynAddSubFX instrument      : OK
* Carla Patchbay & Rack       : OK (weak linking enabled)
* SoundFont2 player           : OK (FluidSynth version < 2: per-note panning unsupported)
* Sid instrument              : OK
* Stk Mallets                 : OK
* VST plugin host             : OK
  * 32-bit Windows host       : OK
  * 64-bit Windows host       : OK
* CALF LADSPA plugins         : OK
* CAPS LADSPA plugins         : OK
* CMT LADSPA plugins          : OK
* TAP LADSPA plugins          : OK
* SWH LADSPA plugins          : OK
* GIG player                  : not found, libgig needed for decoding .gig files

Running sudo apt-get install libgig-dev gigtools did not seem to fix the issue.

@messmerd
Copy link
Member

Seems like it should work if you have libgig-mingw-w64?

@bratpeki
Copy link
Contributor Author

bratpeki commented Oct 15, 2024

libgig-mingw-w64 is already the newest version (4.0.0-4.1).

How very odd... I believe I had issues pulling the repo earlier, I'll try to do this clean and report back.

@bratpeki
Copy link
Contributor Author

Yup, even after running these, and then the wall of commands. I get the same * GIG player : not found, libgig needed for decoding .gig files. I've appended the log files.
CMakeError.log
CMakeOutput.log

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants