RFE: make the python bindings available via pip

[vasanthaganeshk]

No description provided.

[pcmoore]

I will consider it, but I consider this to be extremely low priority at the moment (that can always change).

[pcmoore]

For future reference, Python documentation on packaging/distributing code:

• https://packaging.python.org/distributing

[maxking]

Thanks @pcmoore for the comments on the PRs. I was hoping to help out on packaging the python bindings and started looking into it last night.

Some of the quick fixes were adding the MANIFEST.in and update the setup.py files. I was able to get the updated setup.py to work with pip install, but it sure needs more work to actually work with the development version (I haven't looked into the failure of #217 yet, but maybe soon).

IMO, it would make sense to carve out the Python bindings to a separate repo for a better dev/test/release workflow. I have worked on Python projects, but not too much on Cythonized ones, but I am happy to help out in whatever capacity I can.

I was hoping to help out on setting up the docs after fixing #61 (which probably would be simplified if we move this to a new repo, although, idk how sphinx the Python documentation tool handles cython files).

[drakenclimber]

From @pcmoore's comment in Issue #216:

We basically need to decide if we want to continue to support the Python bindings as part of the main libseccomp source distribution or if we want to split them out like we do for the golang bindings.

I like the idea of splitting the python bindings out of the main libseccomp package, and I also agree that it's fairly low priority for me. If someone else is interested in picking up the work, I would gladly help review it, though :).

[rhatdan]

BTW I agree that the python bindings should be in a separate repo. Although I am not volunteering. :^)

[maxking]

If someone with permissions to create a new repo under the seccomp org can set it up, I can pull the Python bindings out of the repo and create a PR.

There are couple of changes that I foresee:

• Use Python native build tooling, setup.py build should be enough
• Version numbers can be hardcoded in setup.py or be picked up from git tags instead (using setuptools-scm package) if need be.
• Dynamic binding to main libseccomp instead of static binding. (Already proposed in RFE: dynamically link the Python bindings against the main libseccomp library #217)

I haven't looked too much into test harness to see if there is a good way to port them out to use native python tests via pytest or nose2 style testing frameworks.

Finally, some additional documentation to build distribution packages both source and binary distributions.

[pcmoore]

I'm going to tentatively mark this for v2.6, which really just means "sometime after we get v2.5 out the door". Getting v2.5 finished is our priority right now, we can revisit this issue later.

[ghost]

Making libseccomp pip-installable would be very helpful for me as well. Here is my amateur work around using make/poetry/docker (and patching setup.py) if anybody else is trying to figure it out. I also tweaked/set build parameters so it wouldn't require sudo.

build-seccomp/setup.py (slightly modified /src/python/setup.py):

# OLD IMPORTS
# -----------------------------------------
# import os 
# 
# from distutils.core import setup
# from distutils.extension import Extension
# from Cython.Distutils import build_ext
# -----------------------------------------

# Replace with these NEW IMPORTS
# -----------------------------------------
import os

from setuptools import setup, Extension
from Cython.Distutils import build_ext
# -----------------------------------------

# rest of the file unmodified...

Makefile:

.ONESHELL: install-libseccomp-native install-libseccomp-deps install-libseccomp build-whl-libseccomp

SECCOMP_BUILD_ENV = VERSION_RELEASE="0.0.0" \
                    CPPFLAGS="-I ../../include " \
                    CFLAGS="-Wall -g -O2" \
                    LDFLAGS="-Wl,-z -Wl,relro "
WHEEL_DIR = $(CURDIR)/dist

install-libseccomp-native:
ifeq (,$(wildcard libseccomp/.)) # If already cloned/built ignore
	git clone https://github.com/seccomp/libseccomp.git --branch v2.4.3 --depth 1
	cp build-seccomp/setup.py libseccomp/src/python/setup.py
	cd libseccomp
	./autogen.sh
	./configure --prefix=$(CURDIR)/libseccomp/build-aux
	make install
endif

install-deps:
	poetry install --no-root

# `make install-libseccomp` will install seccomp library into poetry virtual environment.
# this is a good option for local development
install-libseccomp: install-libseccomp-native install-deps
	cd libseccomp/src/python
	export $(SECCOMP_BUILD_ENV)
	poetry run python setup.py install

# `make build-whl-libseccomp` will create a .whl file for the seccomp library
# this is a good option for generating a build artifact in CI/CD or Docker
build-whl-libseccomp: install-libseccomp-native install-deps
	cd libseccomp/src/python
	export $(SECCOMP_BUILD_ENV)
	poetry run python setup.py bdist_wheel --dist-dir=$(WHEEL_DIR)

build-docker:
	docker build -t libseccomp-python .

run-docker:
	docker run --env SOME_VAR_1=42 --env SOME_VAR_2="Hello, World!" libseccomp-python

Dockerfile (I have an equivalent but uglier/more cache friendly version in my application):

# ============= BUILD STAGE ============= #
FROM python:3.8.3-buster as builder

# Install poetry
RUN pip install "poetry==1.0.5"

# Build/install wheels
WORKDIR /app
COPY pyproject.toml poetry.lock Makefile ./
COPY build-seccomp ./build-seccomp
RUN make build-whl-libseccomp

# ========== FINAL IMAGE STAGE ========== #
FROM python:3.8.3-slim-buster

# Install application
COPY --from=builder /app/dist /app/dist
WORKDIR /app
RUN pip install --no-index --find-links dist seccomp
COPY app ./app

# Configure
ENV SOME_VAR_1=
ENV SOME_VAR_2=

# Run the application
CMD ["python", "app/main.py"]

pyproject.toml:

[tool.poetry]
name = "libseccomp-demo"
version = "0.1.0"
description = "Building libseccomp for Python projects."
authors = ["Han Solo"]

[tool.poetry.dependencies]
python = "^3.8"

[tool.poetry.dev-dependencies]
cython = "^0.29.19"
wheel = "^0.34.2"

[build-system]
requires = ["poetry==1.0.5"]
build-backend = "poetry.masonry.api"

app/main.py:

import os
import seccomp

print(os.environ['SOME_VAR_1'])
print(os.environ['SOME_VAR_2'])

[pcmoore]

I just wanted to share a "thank you" for all the help on this. @drakenclimber and myself are still busy trying to get the v2.5.0 release finished so this may not see much attention from us until that is done, but I can promise you we are not forgetting about this :)

[pcmoore]

In an effort to get v2.6.0 out sooner than later, I'm going to suggest we push this out to v2.7.0; if you have any concerns or objections please drop a comment.