Skip to main content

Cython Basics: Compile Python to C

Cython is a compiler that translates Python (and Cython-annotated code) into C, which then compiles to machine code. Unlike Numba's JIT, Cython is an ahead-of-time compiler: you write code, run the Cython compiler explicitly, and get a compiled .so (Linux) or .pyd (Windows) shared library. It's the foundation for SciPy, scikit-learn, and pandas' performance-critical paths—battle-tested for two decades. Getting started requires only installing Cython and creating a .pyx file instead of .py. Your Python code compiles unchanged and usually runs 5–10× faster just from removing bytecode interpretation.

What Is Cython?​

Cython is a language that sits between Python and C. A Cython file (.pyx extension) looks like Python but can include C type hints (cdef int x = 5). The Cython compiler reads your .pyx file, generates equivalent C code, and hands it to a C compiler (gcc, clang, MSVC) to produce a compiled module. You import and use it like any Python module—no callbacks to C needed. Cython removes the bytecode interpretation layer entirely: your code becomes machine instructions directly.

Cython's killer feature is incremental adoption. Valid Python is valid Cython; you add type annotations gradually to hot functions, leaving the rest in pure Python. This hybrid approach is why Cython dominates production codebases—you don't need to rewrite everything.

Install Cython and Create Your First .pyx File​

Install Cython via pip:

pip install cython

Create a new directory and add a simple module:

# fibonacci.pyx
def fib_python(int n):
    """Compute the nth Fibonacci number."""
    if n < 2:
        return n
    a, b = 0, 1
    for i in range(2, n + 1):
        a, b = b, a + b
    return b

The function signature def fib_python(int n) tells Cython that n is a C int. Without that hint, Cython treats it as a Python object. Inside the function, Cython compiles arithmetic to native code.

Build and Import the Module​

Cython uses setuptools to compile. Create a setup.py in the same directory:

# setup.py
from setuptools import setup
from Cython.Build import cythonize

setup(
    name="fibonacci",
    ext_modules=cythonize("fibonacci.pyx")
)

Compile in-place:

python setup.py build_ext --inplace

This generates fibonacci.c (the C source) and fibonacci.cpython*.so (or .pyd on Windows)—a compiled module you import normally:

# test_fibonacci.py
from fibonacci import fib_python

result = fib_python(100)
print(result)  # Output: 354224848179261915075

That single int annotation sped up Fibonacci ~3–5× by eliminating Python object overhead. No C knowledge required.

Benchmarking: Python vs Cython​

Let's measure the impact. Pure Python version:

# fib_pure.py
def fib_python(n):
    """Pure Python, no type hints."""
    if n < 2:
        return n
    a, b = 0, 1
    for i in range(2, n + 1):
        a, b = b, a + b
    return b

import timeit
t = timeit.timeit(lambda: fib_python(100), number=100_000)
print(f"Pure Python: {t:.3f}s")

Output:

Pure Python: 2.456s

Now the Cython version (with int type hint):

# test_cython_fib.py
from fibonacci import fib_python
import timeit

t = timeit.timeit(lambda: fib_python(100), number=100_000)
print(f"Cython: {t:.3f}s")

Output:

Cython: 0.412s

Cython is 6× faster. The speedup comes from:

  1. Eliminating bytecode interpretation (machine code runs directly)
  2. Skipping Python object creation for the return value (C long int is used directly)
  3. Native integer arithmetic (no dynamic dispatch to int.__add__)

Understanding the Compilation Pipeline​

When you run python setup.py build_ext --inplace, here's what happens:

  1. Cython compiler reads fibonacci.pyx
  2. Generates C code (~500 lines for a 10-line .pyx file) that handles type checking, reference counting, and exception unwinding
  3. C compiler (gcc/clang/MSVC) compiles the C to object code with -O2 or -O3 optimizations
  4. Linker creates a shared object (.so/.pyd)

You can inspect the generated C:

cython fibonacci.pyx
cat fibonacci.c | head -100

You'll see boilerplate for Python API calls, but the hot loop is pure C.

Common Pitfalls When Starting​

Forgetting to rebuild after edits: Cython compiles once. If you edit fibonacci.pyx, you must re-run setup.py build_ext --inplace. The old .so is cached and won't update. Always rebuild after changes.

Mixing types in loops: Even if you declare int x, if you assign a Python object to x later, Cython falls back to Python semantics. Keep types consistent.

Not benchmarking: Don't assume type hints make code faster. Measure with timeit before and after. Sometimes the speedup is 1.2×, sometimes 50×—it depends on what the code does.

Key Takeaways​

  • Cython is an ahead-of-time compiler that turns Python into C code and native libraries.
  • A .pyx file looks like Python but accepts C type hints; valid Python is valid Cython.
  • Add a setup.py with cythonize() and run build_ext --inplace to compile.
  • Even simple type hints like def f(int x) deliver 5–10× speedups for numerical code.
  • Cython is a foundation layer in SciPy, pandas, and scikit-learn—proven at scale.

Frequently Asked Questions​

Do I need a C compiler to use Cython?​

Yes. On Linux, install gcc or clang (apt-get install build-essential). On macOS, install Xcode Command Line Tools (xcode-select --install). On Windows, install Microsoft C++ Build Tools or Visual Studio Community (free). Cython needs a C compiler to turn its generated C code into binaries.

Can I import a Cython .pyx file directly without compiling?​

No, Cython always compiles to C first. Use pyximport for development-only direct imports (it auto-compiles in the background), but production always uses explicit compilation via setup.py.

What's the difference between def and cdef in Cython?​

def fib(int n) is a Python function callable from Python with C types inside. cdef int fib(int n) is a C function not callable from Python; it's only for internal use. For a function you call from Python, use def.

Does Cython release the GIL?​

Not by default. To parallelize with the GIL released, add cdef functions and annotate with nogil. This is covered in Article 6 but requires more setup.

Is Cython code still Python?​

Yes and no. Cython is a superset of Python: all Python code is valid Cython, but Cython code with C type hints is not valid Python. You must use the Cython compiler; a standard Python interpreter cannot run .pyx files.

Further Reading​