Sysconfig data

The sysconfig module yields information about paths on the system Python is installed on.

To discover those, I wrote the following script;

#!/usr/bin/env python
"""Print the “scripts” and “data” paths for the “_home” and “_user” schemes."""
import sysconfig as sc
import os

for scheme in (nm for nm in sc.get_scheme_names() if os.name in nm):
    print(f"{scheme}:")
    for path in ("scripts", "data"):
        print(f"* {path}:", sc.get_path(path, scheme))

On FreeBSD UNIX (an example of a “posix” system):

posix_home:
* scripts: /usr/local/bin
* data: /usr/local
posix_prefix:
* scripts: /usr/local/bin
* data: /usr/local
posix_user:
* scripts: /home/rsmith/.local/bin
* data: /home/rsmith/.local

Usually, installing in /usr/local requires root privileges. The ~/.local/bin folder structure will have to be created if it doesn’t exist. And it has to be added to the user’s $PATH.

On ms-windows (which is called nt according to os.name):

nt:
* scripts: C:\_LocalData\Python3\Scripts
* data: C:\_LocalData\Python3
nt_user:
* scripts: C:\Users\Roland Smith\AppData\Roaming\Python\Python39\Scripts
* data: C:\Users\Roland Smith\AppData\Roaming\Python

The first scheme nt might, depending how Python was installed. In this case, _LocalData is a directory where I have write access to. The second scheme, nt_user does not require administrator privileges.

I don’t have access to a macOS machine, so that wasn’t tested.

Installation scheme

Based on the data from sysconfig, the installation is done as follows;

• On posix systems, install using the posix_user scheme.
• On nt systems, first try the nt scheme, then nt_user. The reason for this is that it’s easier to tell people that the script will be installed in the Scripts directory of their Python installation.

The windows problem

On ms-windows, Python generally comes with two programs;

• There is python.exe for scripts running in a terminal. This program is generally associated [1] with .py-files.
• And there is pythonw.exe for scripts that run a GUI and don’t need a terminal window. This is associated with .pyw-files.

On posix operating systems, installed scripts generally do not have an extension, and there is no pythonw.

So on ms-windows, scripts that use a GUI need to have another extension when they are installed. Since there is a wide variety of GUI toolkits, auto-detection of a GUI was deemed inpractical. So in the simple setup script, a program is characterized by a 2-tuple of (original name, ms-windows extension). For example;

scripts = [("cmdline.py", ".py"), ("tk-gui.py", ".pyw")]

This avoids the whole GUI toolkit detection problem.

The module problem

Installing scripts that use their own modules has its own set of problems;

• Different scripts could use the same module name, leading to conflicts.
• If such a script is updated or removed, old modules might remain.

To combat this problem, I decided to wrap these applications up in a python zip application. So the whole application is contained in a single file that is easy to remove or update. Python has been able to run programs from zip-files since 2.6. The first time I saw it in the wild was with youtube-dl. For the first (unpublished) versions of the self-contained setup script, I used the zip program to create the zip-files. But since this is not always available, I switched to using zipfile.PyZipFile since this is part of the python standard library.

A project directory for self-contained script(s) could look like this:

project/
├── console.py
├── gui.py
├── foo
│   ├── __init__.py
│   ├── baz.py
│   ├── core.py
│   ├── utils.py
│   └── version.py
╰── bar
    ├── __init__.py
    └── extra.py

In this case, this is an application that has both a console and a GUI interface, both of which use the functionality in the module(s). The modules are subdirectories of the project.

Of course a program that uses modules can also use a GUI. Therefore the definition of the scripts variable is a little different;

scripts = [
    ("foo", "foo", "console.py", ".py"),
    ("foo-gui", ["foo", "bar"], "gui.py", ".pyw"),
]

First is the name of the application. Second is the name of the module(s) that it uses. (This can be a string or an iterable of strings.) Third the name of the main file. This is copied into the archive as __main__.pyc. And last is the extension of the zipped program for use on ms-windows.