Updating Python to 3.5
This article documents the process I followed to update Python 3.4 to 3.5 on my FreeBSD machines.
Introduction
The goal is to replace Python 3.4 and all the packages that depend on it by
Python 3.5. Normally I would use portmaster for updating but at the time of
writing portmaster can’t deal with installing a port for two different Python
versions. During an update, it will e.g. suggest to update py34-nose-1.3.6
to py27-nose-1.3.7
, probably because Python 2.7 is still the default
version. This is however not what I want.
So fo the moment my solution is to do the updates by hand, because then I can
specifiy I want to build a port for Python 3.5 by setting the
PYTHON_VERSION
variable in the make
command.
Note
In the following code fragments, an ellipsis “…” denotes that some lines of output have been removed for brevity.
Investigate and backup
First priority was to see which Python 3.4 packages were installed:
> pkg info -x py34
py34-Jinja2-2.8
py34-MarkupSafe-0.23
py34-Wand-0.4.1
py34-cairo-1.10.0_3
py34-certifi-2015.04.28
py34-gobject3-3.16.2_1
py34-ipython-3.2.1
py34-jsonschema-2.4.0
py34-mistune-0.5
py34-nose-1.3.7
py34-numpy-1.9.2,1
py34-pep8-1.5.7
py34-pexpect-3.3
py34-pillow-2.9.0
py34-pip-7.0.3
py34-pycrypto-2.6.1_1
py34-pytz-2015.4,1
py34-pyzmq-14.7.0
py34-requests-2.7.0
py34-setuptools34-17.0
py34-sqlite3-3.4.3_6
py34-tornado-4.2.1
Just to be sure, backup packages for all these ports were created:
> pkg create py34-Jinja2-2.8 py34-MarkupSafe-0.23 \
py34-Wand-0.4.1 py34-cairo-1.10.0_3 py34-certifi-2015.04.28 \
py34-gobject3-3.16.2_1 py34-ipython-3.2.1 py34-jsonschema-2.4.0 \
py34-mistune-0.5 py34-nose-1.3.7 py34-numpy-1.9.2,1 py34-pep8-1.5.7 \
py34-pexpect-3.3 py34-pillow-2.9.0 py34-pip-7.0.3 py34-pycrypto-2.6.1_1 \
py34-pytz-2015.4,1 py34-pyzmq-14.7.0 py34-requests-2.7.0 \
py34-setuptools34-17.0 py34-sqlite3-3.4.3_6 py34-tornado-4.2.1
Creating package for py34-Jinja2-2.8
…
Creating package for py34-tornado-4.2.1
Looking at the Makefiles of all of these ports, I checked that they are compatible with Python 3.5. Upon concluding that, I removed the 3.4 packages:
# pkg delete -f py34-Jinja2-2.8 py34-MarkupSafe-0.23 \
? py34-Wand-0.4.1 py34-cairo-1.10.0_3 py34-certifi-2015.04.28 \
? py34-gobject3-3.16.2_1 py34-ipython-3.2.1 py34-jsonschema-2.4.0 \
? py34-mistune-0.5 py34-nose-1.3.7 py34-numpy-1.9.2,1 py34-pep8-1.5.7 \
? py34-pexpect-3.3 py34-pillow-2.9.0 py34-pip-7.0.3 py34-pycrypto-2.6.1_1 \
? py34-pytz-2015.4,1 py34-pyzmq-14.7.0 py34-requests-2.7.0 \
? py34-setuptools34-17.0 py34-sqlite3-3.4.3_6 py34-tornado-4.2.1
Updating database digests format: 100%
Checking integrity... done (0 conflicting)
Deinstallation has been requested for the following 22 packages (of 0 packages in the universe):
Installed packages to be REMOVED:
py34-Jinja2-2.8
…
py34-tornado-4.2.1
The operation will free 69 MiB.
Proceed with deinstalling packages? [y/N]: y
…
At this point I used pip-3.4 list
to check which Python modules were
installed outside of the ports tree:
> pip-3.4 list
You are using pip version 7.0.3, however version 7.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
icalendar (3.9.1)
line-profiler (1.0)
python-magic (0.4.6)
As an aside, Trying to run the “upgrade” command as suggested by pip failed. Don’t try that!
Configuration
In /etc/make.conf
, I changed the default Python 3 version to 3.5:
DEFAULT_VERSIONS= ruby=2.1 python3=3.5
In my vimrc
, I set the paths for the python checkers:
let g:syntastic_python_python_exec='/usr/local/bin/python3'
let g:syntastic_python_pep8_exec='/usr/local/bin/pep8'
Note that I removed the Python2 version of pep8.
Installing Python 3.5
First I installed the new Python 3.5:
# cd /usr/ports/lang/python35/
# make PYTHON_VERSION=python3.5 install clean
The build process make me choose options:
┌────────────────────────────── python35-3.5.0 ────────────────────────────────┐ │ ┌──────────────────────────────────────────────────────────────────────────┐ │ │ │+[ ] DEBUG Build with debugging support │ │ │ │+[x] IPV6 IPv6 protocol support │ │ │ │+[x] LIBFFI Use libffi from ports instead of bundled version │ │ │ │+[x] NLS Enable gettext support for the locale module │ │ │ │+[x] PYMALLOC Enable specialized mallocs │ │ │ │+[x] THREADS Threading support │ │ │ │+[x] TSC Enable processor timestamp counter profiling │ │ │ │───────────────────────── Hash Algorithm (PEP-456) ───────────────────────│ │ │ │+( ) FNV Modified Fowler-Noll-Vo Algorithm │ │ │ │+(*) SIPHASH SipHash24 Algorithm │ │ │ └──────────────────────────────────────────────────────────────────────────┘ │ ├──────────────────────────────────────────────────────────────────────────────┤ │ < OK > <Cancel> │ └──────────────────────────────────────────────────────────────────────────────┘
The only non-default options I chose were TSC and SIPHASH.
After that the build and install proceeded normally. The only thing that struck me was a significant amount of “unreachable code” warnings when compiling. The program seemed to work normally.
After that I deleted the python34-3.4.3_1
program:
# pkg delete python34-3.4.3_1
Updating database digests format: 100%
Checking integrity... done (0 conflicting)
Deinstallation has been requested for the following 2 packages (of 0 packages in the universe):
Installed packages to be REMOVED:
python34-3.4.3_1
python3-3_3
The operation will free 78 MiB.
Proceed with deinstalling packages? [y/N]: y
[1/2] Deinstalling python3-3_3...
[1/2] Deleting files for python3-3_3: 100%
[2/2] Deinstalling python34-3.4.3_1...
[2/2] Deleting files for python34-3.4.3_1: 100%
There were some manually installed modules left so I had to remove the Python 3.4 directory by hand:
# rm -rf /usr/local/lib/python3.4
And I re-installed the lang/python3
port:
# cd /usr/ports/lang/python3
# make install clean
===> python3-3_3 depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by python3-3_3 for building
===> Extracting for python3-3_3
===> Patching for python3-3_3
===> Configuring for python3-3_3
===> Staging for python3-3_3
===> python3-3_3 depends on file: /usr/local/bin/python3.5 - found
===> python3-3_3 depends on file: /usr/local/bin/python3.5 - found
===> Generating temporary packing list
/bin/ln -sf python3.5 /usr/ports/lang/python3/work/stage/usr/local/bin/python3
/bin/ln -sf 2to3-3.5 /usr/ports/lang/python3/work/stage/usr/local/bin/2to3-3
/bin/ln -sf idle3.5 /usr/ports/lang/python3/work/stage/usr/local/bin/idle3
/bin/ln -sf pydoc3.5 /usr/ports/lang/python3/work/stage/usr/local/bin/pydoc3
/bin/ln -sf python3.5-config /usr/ports/lang/python3/work/stage/usr/local/bin/python3-config
/bin/ln -sf python-3.5.pc /usr/ports/lang/python3/work/stage/usr/local/libdata/pkgconfig/python3.pc
====> Compressing man pages (compress-man)
===> Installing for python3-3_3
===> Checking if python3 already installed
===> Registering installation for python3-3_3
Installing python3-3_3...
===> Cleaning for python3-3_3
Installing the Python 3 ports
Starting with the most important ones:
# cd /usr/ports/math/py-numpy/
# make PYTHON_VERSION=python3.5 install clean
# cd /usr/ports/graphics/py-wand/
# make PYTHON_VERSION=python3.5 install clean
cd /usr/ports/databases/py-sqlite3
# make PYTHON_VERSION=python3.5 install clean
# cd /usr/ports/devel/py-nose
# make PYTHON_VERSION=python3.5 install clean
# cd /usr/ports/devel/py-pytz
# make PYTHON_VERSION=python3.5 install clean
# cd /usr/ports/devel/pep8
# make PYTHON_VERSION=python3.5 install clean
# cd /usr/ports/www/py-requests
# make PYTHON_VERSION=python3.5 install clean
# cd /usr/ports/devel/ipython/
# make PYTHON_VERSION=python3.5 install clean
Installing IPython also takes care of pexpect
, tornado
, certifi
,
Jinja2
, MarkupSafe
, jsonschema
, mistune
and pyzmq
.
More ports:
# cd /usr/ports/graphics/py-pillow/
# make PYTHON_VERSION=python3.5 install clean
# cd /usr/ports/security/py-pycrypto
# make PYTHON_VERSION=python3.5 install clean
# cd /usr/ports/devel/py-pip
# make PYTHON_VERSION=python3.5 install clean
The graphics/py3-cairo
port produces an error, so I’ll skip that for now.
Also I haven’t used gobject, so I’ll skip that as well.
Python 3 modules that are not available from ports
Some of the modules I use are not in the FreeBSD ports tree. Those I installed
by hand. For this I used pip
because these are pure Python modules.
# pip-3.5 install python-magic
# pip-3.5 install line_profiler
# pip-3.5 install icalendar
For comments, please send me an e-mail.
Related articles
- Profiling Python scripts(6): auto-orient
- Profiling with pyinstrument
- From python script to executable with cython
- On Python speed
- Python 3.11 speed comparison with 3.9