Roland's homepage

My random knot in the Web

Using nullfs and unionfs for the ports tree in a jail

When running a virtual server in a jail, one would like to avoid replication of files as much as practical. One candidate for sharing would be the ports tree in /usr/ports. One could just make a symbolic link from /usr/ports in the jail's tree to the one on the host. But what if multiple jails were to be run sharing the same /usr/ports? You could end up with multiple jails trying to rebuild the same port at the same time. That would probably not work as intended.

So we use a combination of a nullfs and a unionfs to create copies of /usr/ports for each jail without writing in ports tree on the host.

The mount_nullfs command is used to create a virtual copy of /usr/ports in the filesystem of the jail:

slackbox# cd /usr/local/var/jails/192.168.0.100/usr
slackbox# mount_nullfs /usr/ports/ ports/

Now we create an empty directory in the filesystem tree of the jail, and use that as an overlay for the virtual ports tree using mount_unionfs;

slackbox# mkdir tmp/foo
slackbox# mount_unionfs -o noatime tmp/foo ports/

The affect of this is that whenever a file is written in the overlayed copy of the ports tree, that file is actually written in the tree under /usr/local/var/jails/192.168.0.100/usr/tmp/foo! So you can compile ports in the ports tree in the jail without writing files in the host's ports tree!

To make use of this, the four commands above should be run before starting a jail. The start-up sequence of the jail now becomes;

slackbox# cd /usr/local/var/jails/192.168.0.100/usr
slackbox# mount_nullfs /usr/ports/ ports/
slackbox# mkdir tmp/foo
slackbox# mount_unionfs -o noatime tmp/foo ports/
slackbox# /etc/rc.d/jail onestart server
Configuring jails:.
Starting jails: server.erewhon.net.

When you are finished with the jail, you should do the following to take everything down;

slackbox# /etc/rc.d/jail onestop server
slackbox# umount /usr/local/var/jails/192.168.0.100/usr/ports
slackbox# umount /usr/local/var/jails/192.168.0.100/usr/ports

And yes, the umount command does need to be run twice: once for the unionfs, and once for the nullfs!

To save space, one can now remove the contents of the unionfs;

slackbox# cd /usr/local/var/jails/192.168.0.100/usr
slackbox# rm -rf tmp/foo/*

Start the jail with the nullfs and unionfs mounts in place. Updating the ports in the jail is then done as follows;

slackbox# cd /usr/local/var/jails/192.168.0.100/usr
slackbox# mkdir tmp/foo
slackbox# mount_nullfs /usr/ports/ ports/
slackbox# mount_unionfs -o noatime tmp/foo ports/
slackbox# /etc/rc.d/jail onestart server
Configuring jails:.
Starting jails: server.erewhon.net.
slackbox# exit
slackbox:~> ssh server
server:~> su
server# portmaster -ai

Afterwards, close the jail as usual.