Roland's homepage

My random knot in the Web

From python script to executable with cython

Recently I saw that you could convert python scripts into executable with the help of cython, and I had to try that.

Script

The source is a simple script to list directories in the order they were modified.

import os

results = []
for e in os.scandir():
    if not e.is_dir():
        continue
    if e.name.startswith("."):
        continue
    results.append((e.stat().st_mtime, e.name))
results.sort(key=lambda x: -x[0])
for _, name in results:
    print(name)

Environment

These tests were done on FreeBSD 13.1-RELEASE-p3 on the amd64 platform.

From Python to C

This command makes a C program file from a Python script.

cython --embed -3 -o dirmod.c dirmod.py

Compiling the C code

The code is compiled as follows:

clang -Oz -I /usr/local/include/python3.9/ -L/usr/local/lib/ \
-o dirmod dirmod.c -lpython3.9
strip dirmod

Output

The relative size of the files:

du -h *
 41K    dirmod
265K    dirmod.c
4.5K    dirmod.py

The produced executable is dynamically linked:

ldd dirmod
dirmod:
     libpython3.9.so.1.0 => /usr/local/lib/libpython3.9.so.1.0 (0x822c54000)
     libc.so.7 => /lib/libc.so.7 (0x8216dd000)
     libcrypt.so.5 => /lib/libcrypt.so.5 (0x822a4d000)
     libintl.so.8 => /usr/local/lib/libintl.so.8 (0x8232d5000)
     libdl.so.1 => /usr/lib/libdl.so.1 (0x8239c6000)
     libutil.so.9 => /lib/libutil.so.9 (0x8241c6000)
     libm.so.5 => /lib/libm.so.5 (0x825208000)
     libthr.so.3 => /lib/libthr.so.3 (0x825a46000)

As can be seen, this program depends on the Python shared library.

Compiling a static executable

To compile a static executable, the compiler invocation is slightly different. This is based on the output of /usr/local/lib/python3.9/config-3.9/python-config.py:

clang -Oz -static -I /usr/local/include/python3.9/ -L/usr/local/lib/ \
-L/usr/local/lib/python3.9/config-3.9/ \
-o dirmod-static dirmod.c \
-lpython3.9 -lcrypt -lintl -lutil -lpthread -lm -lpthread -ldl
strip dirmod-static

Output

The relative size of the files:

du -h *
 41K    dirmod
4.3M    dirmod-static
265K    dirmod.c
4.5K    dirmod.py

The executable is statically linked:

file dirmod-static
dirmod-static: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD),
statically linked, for FreeBSD 13.1, FreeBSD-style, stripped

For comments, please send me an e-mail.

Related articles

←  On Python speed Profiling with pyinstrument  →