Updating the ports tree

On 2016-12-25, python 3.6 landed in the ports tree. The ports tree was updated as follows.

# svnlite update /usr/ports

There were no relevant notices in /usr/ports/UPDATING, so I could continue with the update.

Listing installed python 3.5 ports and modules

First, I determined which python 3.5 ports I had installed.

> pkg info -ox py35

This lists the package and its origin, like this:

py35-Jinja2-2.8                devel/py-Jinja2
py35-MarkupSafe-0.23           textproc/py-MarkupSafe
py35-Wand-0.4.4                graphics/py-wand
py35-bottle-0.12.7             www/py-bottle
py35-cairo-1.10.0_3            graphics/py3-cairo

This list is saved to a file. Next I determined which modules were installed according to pip to also find stuff that I had installed without a port. This was done with the command:

> pip-3.5 list --format=columns

The output of this was appended to the above mentioned file.

Updating python itself

To prevent issues, I removed the python 3.5 packages before updating python.

# pkg delete -x py35

Next, I modified /etc/make.conf to set python 3.6 as the default version.

-DEFAULT_VERSIONS= ruby=2.1 python3=3.5 ghostscript=agpl ssl=libressl perl5=5.24
+DEFAULT_VERSIONS= ruby=2.1 python3=3.6 ghostscript=agpl ssl=libressl perl5=5.24

To update the port, I used portmaster to update the meta-port.

# portmaster python3-3_3

I choose the same port options for 3.6 as for 3.5. In addition to the default options, I enabled the TSC option and choose the SipHash24 hash algorithm. The portmaster run builds and installs python 3.6. After a successful installation of python 3.6, I deleted the old python 3.5 installation.

# pkg delete python35-3.5.2

Note that this leaves the installed modules that are not ports in /usr/local/lib/python3.5/site-packages/. This directory should be completely removed.

During the compilation of the port, I noticed a warning that the configure option --with-tsc was not recognized. Looking at the output of configure --help, it seems that this option is gone. I submitted problem report (“PR”) 215580 to remove it from the port.

Re-installing the python ports

Working from the above mentioned list of ports, I re-installed them. For each port, this went like this.

# cd /usr/port/<category>/<port>
# make PYTHON_VERSION=python3.6 install clean

Generally I start with ports of modules that are important to me like numpy, ipython, and matplotlib.

The matplotlib port deserves special mention. At the moment this article was written, this port is marked as python 2.7 only. However, with the correct back end (TkAgg) it works fine with python 3. In November 2016 I submitted problem report 214853 about this. Unfortunately this PR is without a complete working patch because it requires some voodoo that is currently beyond me.

So in the meantime I locally modified the port Makefile to remove the python 2.7 restriction, and compile and install this port with only the TkAgg back end. To prevent problems, I lock this package after it has been successfully installed.

Re-installing python modules

Some modules that I use are not available as ports. Those I generally install from source. The reason for doing this instead of using pip is that some very interesting modules like binwalk and pyknon are not available in ports.

After unpacking the source code, and changing to the source code directory, installation is generally done with.

# python3 setup.py install