cython-ep2008.txt

==============================================
The Cython Compiler for C-Extensions in Python
==============================================

Dr. Stefan Behnel
-----------------

.. class:: center

  http://www.cython.org/

  cython-dev@codespeak.net

.. footer:: Dr. Stefan Behnel, EuroPython 2008, Vilnius/Lietuva

.. include:: <s5defs.txt>


About myself
============

* Passionate Python developer since 2002

  * after Basic, Logo, Pascal, Prolog, Scheme, Java, ...

* CS studies in Germany, Ireland, France

* PhD in distributed systems in 2007

  * Language design for self-organising systems

  * Darmstadt University of Technologies, Germany

* Current occupations:

  * Employed by Senacor Technologies AG, Germany

    * IT transformations, SOA design, Java-Development, ...

  * »lxml« OpenSource XML toolkit for Python

    * http://codespeak.net/lxml/

  * Cython


What is Cython?
===============

Cython is

* an Open-Source project

  * https://cython.org/

* a Python compiler (almost)

  * an enhanced, optimising fork of Pyrex

* an extended Python language for

  * writing fast Python extension modules

  * interfacing Python with C libraries


A little bit of history
=======================

* April 2002: release of Pyrex 0.1 by Greg Ewing

  * Greg considers Pyrex a language in design phase

  * Pyrex became a key language for many projects

    * over the years, many people patched their Pyrex

    * not all patches were answered by Greg (not in time)

* minor forks and enhanced branches followed

  * March 2006: my fork of Pyrex for »lxml« XML toolkit

  * November 2006: »SageX« fork of Pyrex

    * by Robert Bradshaw, William Stein (Univ. Seattle, USA)

    * context: »Sage«, a free mathematics software package

* 28th July 2007: official Cython launch

  * integration of lxml's Pyrex fork into SageX

  * the rest is in http://hg.cython.org/cython-devel/


Major Cython Developers
=======================

* Robert Bradshaw and Stefan Behnel

  * lead developers

* Greg Ewing

  * main developer and maintainer of Pyrex

  * we happily share code and discuss ideas

* Dag Sverre Seljebotn

  * Google Summer-of-Code developer

  * NumPy integration, many ideas and enhancements

* many, *many* others - see

  * https://cython.org/

  * the mailing list archives of Cython and Pyrex


How to use Cython
=================

* you write Python code

  * Cython translates it into C code

  * your C compiler builds a shared library for CPython

  * you import your module

* Cython has support for

  * compile-time includes/imports

    * with dependency tracking

  * distutils

    * *optionally* compile Python code from setup.py!


Example: compiling Python code
==============================

.. sourcecode:: bash

    $ cat worker.py

.. sourcecode:: python

    class HardWorker(object):
        u"Almost Sisyphus"
        def __init__(self, task):
            self.task = task

        def work_hard(self):
            for i in range(100):
	        self.task()

.. sourcecode:: bash

    $ cython worker.py

* translates to 842 line `.c file <ep2008/worker.c>`_ (Cython 0.9.8)

  * lots of portability #define's

  * tons of helpful C comments with Python code snippets

  * a lot of code that you don't want to write yourself


Portable Code
=============

* Cython compiler generates C code that compiles

  * with all major compilers (C and C++)

  * on all major platforms

  * in Python 2.3 to 3.0 beta1

* Cython language syntax follows Python 2.6

  * optional Python 3 syntax support is on TODO list

    * get involved to get it quicker!

\... the fastest way to port Python 2 code to Py3 ;-)


Python 2 feature support
========================

* most of Python 2 syntax is supported

  * top-level classes and functions

  * exceptions

  * object operations, arithmetic, ...

* plus some Py3/2.6 features:

  * keyword-only arguments

  * unicode literals via ``__future__`` import

* in recent developer branch:

  * ``with`` statement

  * closures (i.e. local classes and functions) are close!


Speed
=====

Cython generates very efficient C code

* according to PyBench:

  * conditions and loops run 2-8x faster than in Py2.5

  * most benchmarks run 30%-80% faster

  * overall more than 30% faster for plain Python code

* optional type declarations

  * let Cython generate plain C instead of C-API calls

  * make code several times faster than the above


Type declarations
=================

* »cdef« keyword declares

  * local variables with C types

  * functions with C signatures

  * classes as builtin extension types

* Example::

    def stupid_lower_case(char* s):
        cdef Py_ssize_t size, i

	size = len(s)
        for i in range(size):
            if s[i] >= 'A' and s[i] <= 'Z':
                s[i] += 'a' - 'A'
        return s


Why is this stupid?
===================

Ask Cython!

.. sourcecode:: bash

    $ cat stupidlowercase.py

::

    def stupid_lower_case(char* s):
        cdef Py_ssize_t size, i

	size = len(s)
        for i in range(size):
            if s[i] >= 'A' and s[i] <= 'Z':
                s[i] += 'a' - 'A'
        return s

.. sourcecode:: bash

    $ cython --annotate stupidlowercase.py

=> `stupidlowercase.html <ep2008/stupidlowercase.html>`_


Calling C functions
===================

* Example::

    cdef extern from "Python.h":
        # copied from the Python C-API docs:
        object PyUnicode_DecodeASCII(
            char* s, Py_ssize_t size, char* errors)

    cdef extern from "string.h":
        int strlen(char *s)


    cdef slightly_better_lower_case(char* s):
        cdef Py_ssize_t i, size = strlen(s)
        for i in range(size):
            if s[i] >= 'A' and s[i] <= 'Z':
                s[i] += 'a' - 'A'
	return PyUnicode_DecodeASCII(s, size, NULL)


Features in work
================

* Dynamic classes and functions with closures

  .. sourcecode:: python

    def factory(a,b):
        def closure_function(c):
            return a+b+c
        return closure_function

* Native support for new ``buffer`` protocol

  * part of NumPy integration by Dag Seljebotn

  .. sourcecode:: python

    def inplace_negative_grayscale_image(
                      ndarray[unsigned char, 2] image):
        cdef int i, j
        for i in range(image.shape[0]):
            for j in range(image.shape[1]):
                arr[i, j] = 255 - arr[i, j]


Huge pile of great ideas
========================

* Cython Enhancement Proposals (CEPs)

  * http://wiki.cython.org/enhancements

* native pickle support for extension classes

* meta-programming facilities

* type inference strategies

* compile-time assertions for optimisations

* object-like C-array handling

* improved C++ integration

* ...


Conclusion
==========

* Cython is a tool for

  * efficiently translating Python code to C

  * easily interfacing to external C libraries

* Use it to

  * speed up existing Python modules

    * concentrate on optimisations, not rewrites!

  * write C extensions for CPython

    * don't change the language just to get fast code!

  * wrap C libraries *in Python*

    * concentrate on the mapping, not the glue!


... but Cython is also
======================

* a great project

* a very open playground for great ideas!


Cython
======

  **Cython**

  **C-Extensions in Python**

  \... use it, and join the project!

  https://cython.org/