Skip to content

How to generate GStreamer binaries for CI

Jump to bottom
Fernando Jiménez Moreno edited this page Jun 16, 2020 · 13 revisions

Servo Travis builds run on Ubuntu Xenial, which has very old GStreamer packages (1.8, while we need at least 1.14), so we need to build a more recent version of these packages for this platform. To do so, the recommended process (on Linux) is the following:

schroot is used to handle chroot environments, it does the bind mounting of /proc, /sys and all that repetitive stuff that seals the isolation of the chrooted environment.

The debootstrap's variant used is buildd because it installs the build-essential package.

$ sudo mkdir -p /srv/chroot/gst-xenial64
$ cd /srv/chroot/gst-xenial64
$ sudo debootstrap --arch=amd64 --variant=buildd xenial ./gst-xenial64/ http://archive.ubuntu.com/ubuntu
$ sudo vim /etc/schroot/chroot.d

This is the used schroot configuration. Please, adapt it to your needs.

[gst]
description=Ubuntu Xenial 64-bit for GStreamer
directory=/srv/chroot/gst-xenial64
type=directory
users=<user>
root-users=<user>
profile=default
setup.fstab=default/my-home.fstab

Overriding the fstab default for a custom one to set home directory of to a clean directory.

$ mkdir -p ~/home-gst
$ sudo vim /etc/schroot/default/my-home.fstab
# fstab: static file system information for chroots.
# Note that the mount point will be prefixed by the chroot path
# (CHROOT_PATH)
#
#                
/proc           /proc           none    rw,bind         0       0
/sys            /sys            none    rw,bind         0       0
/dev            /dev            none    rw,bind         0       0
/dev/pts        /dev/pts        none    rw,bind         0       0
/home           /home           none    rw,bind         0       0
/home/<user>/home-gst /home/<user>   none    rw,bind 0       0
/tmp            /tmp            none    rw,bind         0       0

configure chroot environment

We will get into the chroot environment as super user in order to add the required packages. For that pupose we must add the universe repository in apt.

  • glib compilation requires: autotools-dev gnome-pkg-tools libtool libffi-dev libelf-dev libpcre3-dev desktop-file-utils libselinux1-dev libgamin-dev dbus dbus-x11 shared-mime-info libxml2-utils
  • Python compilation requires: libssl-dev libreadline-dev libsqlite3-dev
  • GStreamer compilation requires: bison flex yasm python3-pip libasound2-dev libbz2-dev libcap-dev libdrm-dev libegl1-mesa-dev libfaad-dev libgl1-mesa-dev libgles2-mesa-dev libgmp-dev libgsl0-dev libjpeg-dev libmms-dev libmpg123-dev libogg-dev libopus-dev liborc-0.4-dev libpango1.0-dev libpng-dev libpulse-dev librtmp-dev libtheora-dev libtwolame-dev libvorbis-dev libvpx-dev libwebp-dev pkg-config unzip zlib1g-dev
  • And for general build environment: language-pack-en ccache git curl ninja-build
$ schroot --user root --chroot gst
(gst)# sed -i "s/main$/main universe/g" /etc/apt/sources.list
(gst)# apt update
(gst)# apt upgrade
(gst)# apt --no-install-recommends --no-install-suggests install \
autotools-dev gnome-pkg-tools libtool libffi-dev libelf-dev \
libpcre3-dev desktop-file-utils libselinux1-dev libgamin-dev dbus \
dbus-x11 shared-mime-info libxml2-utils \
libssl-dev libreadline-dev libsqlite3-dev \ 
language-pack-en ccache git curl bison flex yasm python3-pip \
libasound2-dev libbz2-dev libcap-dev libdrm-dev libegl1-mesa-dev \
libfaad-dev libgl1-mesa-dev libgles2-mesa-dev libgmp-dev libgsl0-dev \
libjpeg-dev libmms-dev libmpg123-dev libogg-dev libopus-dev \
liborc-0.4-dev libpango1.0-dev libpng-dev libpulse-dev librtmp-dev \
libtheora-dev libtwolame-dev libvorbis-dev libvpx-dev libwebp-dev \
pkg-config unzip zlib1g-dev

Finally we create our installation prefix. In this case /opt/gst to avoid the contamination of /usr/local and we set the ownership of that directory to , so they could deploy on it. And exit.

(gst)# mkdir -p /opt/gst
(gst)# chown <user> /opt/gst
(gst)# exit

compile ffmpeg 4.1

Now, let's login again, this time as the unprivileged , to build the bundle, starting with ffmpeg. Notice the usage of ccache and the out-of-source building just as recommendations.

$ schroot --chroot gst
(gst)$ git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg
(gst)$ cd ffmpeg
(gst)$ git checkout -b work n4.1.1
(gst)$ mkdir build
(gst)$ cd build
(gst)$ ../configure --disable-static --enable-shared \
--disable-programs --enable-pic --disable-doc --prefix=/opt/gst 
(gst)$ PATH=/usr/lib/ccache/:${PATH} make -j8 install

compile glib 2.54

(gst)$ cd ~
(gst)$ git clone https://gitlab.gnome.org/GNOME/glib.git
(gst)$ cd glib
(gst)$ git checkout -b work origin/glib-2-54
(gst)$ mkdir mybuild
(gst)$ cd mybuild
(gst)$ ../autogen.sh --prefix=/opt/gst
(gst)$ PATH=/usr/lib/ccache/:${PATH} make -j8 install

install Python 3.5.2

Pyenv is a project that allows the automation of installing and executing, in the user home directory, multiple versions of Python.

(gst)$ curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash
(gst)$ ~/.pyenv/bin/pyenv install 3.5.2

Install meson 0.50

We will install the last available version of meson in the user home directory, that is why PATH is extended and exported.

(gst)$ cd ~
(gst)$ ~/.pyenv/versions/3.5.2/bin/pip3 install --user meson
(gst)$ export PATH=${HOME}/.local/bin:${PATH}

build GStreamer 1.16.2

PKG_CONFIG_PATH is exported to expose the compiled versions of ffmpeg and glib. Notice that the libraries are installed in /opt/lib in order to avoid the dispersion of pkg-config files.

(gst)$ cd ~/
(gst)$ export PKG_CONFIG_PATH=/opt/gst/lib/pkgconfig/
(gst)$ git clone https://gitlab.freedesktop.org/gstreamer/gst-build.git
(gst)$ cd gst-build
(gst)$ git checkout -b work origin/1.16
(gst)$ git checkout 1.16.2
(gst)$ meson -Dpython=disabled -Dlibav=enabled -Dugly=disabled -Dbad=enabled -Ddevtools=disabled -Drtsp_server=disabled -Domx=disabled -Dvaapi=disabled -Dsharp=disabled -Dintrospection=disabled -Dgst-plugins-bad:iqa=disabled -Dges=disabled -Dlibnice=enabled -Dgst-plugins-bad:webrtc=enabled -Dlibnice:crypto-library=openssl -Dlibnice:tests=disabled --prefix=/opt/gst build --libdir=lib
(gst)$ cd subprojects/gst-plugins-base && git cherry-pick 5f19e83f14bf29a31b76684181396080aa228113 ; cd ~/gst-build
(gst)$ ninja -C build install

The cherry-pick is to include a fix that is supposed to be backported soon. You only need to do it once, further reconfigures will not require redoing the cherry-pick.

You may get an error about os.terminal_size(fallback) throwing a ValueError. Just edit ~/.pyenv/versions/3.5.0/lib/python3.5/shutil.py and add a , ValueError to the except block after that. This is https://bugs.python.org/issue24966

test!

(gst)$ cd ~/
(gst)$ LD_LIBRARY_PATH=/opt/gst/lib \
GST_PLUGIN_SYSTEM_PATH=/opt/gst/lib/gstreamer-1.0/ \
/opt/gst/bin/gst-inspect-1.0

And the list of available elemente shall be shown.

archive the bundle

(gst)$ cd ~/
(gst)$ tar zpcvf gstreamer-1.16-x86_64-linux-gnu.tar.gz -C /opt ./gst

update your .travis.yml

These are the packages you shall add to run this generated GStreamer bundle:

  • libasound2-plugins
  • libfaad2
  • libfftw3-single3
  • libjack-jackd2-0
  • libmms0
  • libmpg123-0
  • libopus0
  • liborc-0.4-0
  • libpulsedsp
  • libsamplerate0
  • libspeexdsp1
  • libtdb1
  • libtheora0
  • libtwolame0
  • libwayland-egl1-mesa
  • libwebp5
  • libwebrtc-audio-processing-0
  • liborc-0.4-dev
  • pulseaudio
  • pulseaudio-utils
  • libvpx3
  • librtmp1

And this is the before_install and before_script targets:

      before_install:
        - curl -L http://server.example/gstreamer-1.14-x86_64-linux-gnu.tar.gz | tar xz
        - sed -i "s;prefix=/opt/gst;prefix=$PWD/gst;g" $PWD/gst/lib/pkgconfig/*.pc
        - export PKG_CONFIG_PATH=$PWD/gst/lib/pkgconfig
        - export GST_PLUGIN_SYSTEM_PATH=$PWD/gst/lib/gstreamer-1.0
        - export GST_PLUGIN_SCANNER=$PWD/gst/libexec/gstreamer-1.0/gst-plugin-scanner
        - export PATH=$PATH:$PWD/gst/bin
        - export LD_LIBRARY_PATH=$PWD/gst/lib:$LD_LIBRARY_PATH

      before_script:
        - pulseaudio --start
        - gst-inspect-1.0 | grep Total
Clone this wiki locally