Attempting a conky replacement in Python (part 1)
My window manager (i3) has a small bar on the bottom of the screen. I use conky to put a single line of (text) information there.
It looks like this:
Ext: 34.1KiB/910B Int: 0B/0B | Mail: 51 | RAM: 35% | CPU: 4%, 49°C | Mon 2019-07-01 21:20:17
This is refreshed every second. Conky is a pretty substantial program that can do a lot more than I ask to do. Recently it added Lua as an embedded programming language. So my question was; can I write a simple replacement in Python 3?
This program uses FreeBSD specific commands like sysctl and netstat.
(Well, OS X is based on a FreeBSD userland. so they might exist there as well.
But you probably won’t be running i3 on OS X. :-)
Simple implementation
My first implementation is pretty straightforward. It calls a number of functions, and
joins the outputs with |.
items = [network(), mail(), memory(), cpu(), date()]
print(' | '.join(items))
Let’s see how fast this runs using IPython.
Network
In [1]: import os
...: import statistics as stat
...: import subprocess as sp
...: import time
In [2]: # Global data
...: netdata = {'age0': (0, 0, 0), 'rl0': (0, 0, 0)}
...: lastmail = {'count': -1, 'time': 0}
...: cpusage = {'used': 0, 'total': 0}
Out[2]: {'used': 0, 'total': 0}
In [3]: def netstat(interface):
...: """Return network statistics for named interface."""
...: lines = sp.check_output(['netstat', '-b', '-n', '-I', interface],
...: encoding='ascii').splitlines()
...: tm = time.time()
...: if len(lines) == 1:
...: return None
...: items = lines[1].split()
...: ibytes = int(items[7])
...: obytes = int(items[10])
...: return ibytes, obytes, tm
...:
In [4]: %timeit netstat('age0')
79.9 ms ± 488 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [5]: def network():
...: global netdata
...: ifaces = {'Ext': 'age0', 'Int': 'rl0'}
...: items = []
...: for desc, i in ifaces.items():
...: data = netstat(i)
...: if data:
...: dt = data[2] - netdata[i][2]
...: delta_in = int((data[0] - netdata[i][0])/dt)
...: delta_out = int((data[1] - netdata[i][1])/dt)
...: netdata[i] = data
...: items.append(f'{desc}: {delta_in}B/{delta_out}B')
...: else:
...: items.append(f'{desc}: 0B/0B')
...: return ' '.join(items)
...:
In [6]: %timeit network()
160 ms ± 638 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
It is clear that almost all of the time in network() is spent in
netstat(). I suspect that most of that time is spent calling the
subprocess. Let’s check that.
In [7]: %timeit sp.check_output(['netstat', '-b', '-n', '-I', 'age0'], encoding='ascii').splitlines()
79.3 ms ± 403 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
Almost all of the time in netstat is indeed spent running the subprocess.
Initially I used regular expressions for parsing the mbox file. This
turned out to be very slow. So I quickly changed that to looking at the
beginning of each string.
In [8]: def mail():
...: mboxname = '/home/rsmith/Mail/received'
...: global lastmail
...: newtime = os.stat(mboxname).st_mtime
...: if newtime > lastmail['time']:
...: with open(mboxname) as mbox:
...: lines = mbox.readlines()
...: # This is faster than regex; see above.
...: read = len([ln for ln in lines if ln.startswith('Status: R')])
...: total = len([ln for ln in lines if ln.startswith('From ')])
...: mailcount = total - read
...: lastmail = {'count': mailcount, 'time': newtime}
...: return f'Mail: {mailcount}'
...:
In [9]: %timeit -n 1 -r 1 mail()
333 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)
Since this takes around 1/3 of a second, we only do this if the modification
time of the mail file is newer than the last time we checked. Still, this is
pretty slow. I should probably look at using mmap.
Memory
In [10]: def vmstats():
...: """Return FreeBSD's vm stats."""
...: rv = {}
...: for ln in sp.check_output(['sysctl', 'vm.stats.vm'], encoding='ascii').splitlines():
...: name, count = ln.split()
...: rv[name[14:-1]] = int(count)
...: return rv
...:
In [11]: %timeit vmstats()
80.5 ms ± 278 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [12]: def memory():
...: stats = vmstats()
...: memmax = stats['page_count']
...: mem = (memmax - stats['free_count'] - stats['inactive_count'] - stats['cache_count'])
...: free = int(100 * mem / memmax)
...: return f'RAM: {free}%'
...:
In [13]: %timeit memory()
81.9 ms ± 620 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [14]: %timeit sp.check_output(['sysctl', 'vm.stats.vm'], encoding='ascii').splitlines()
80.3 ms ± 83.7 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
As expected based on the previous items, most of the time is spent in the
subprocess.check_output call.
CPU load and temperature
In [15]: def cpu():
...: global cpusage
...: items = [f'dev.cpu.{n}.temperature' for n in range(4)] + ['kern.cp_time']
...: lines = sp.check_output(['sysctl'] + items, encoding='ascii').splitlines()
...: temps = [float(ln.split()[1][:-1]) for ln in lines[:-1]]
...: T = round(stat.mean(temps))
...: states = [int(j) for j in lines[-1].split()[1:]]
...: # According to /usr/include/sys/resource.h, these are:
...: # USER, NICE, SYS, INT, IDLE
...: total = sum(states)
...: used = total - states[-1]
...: frac = int((used - cpusage['used']) / (total - cpusage['total']) * 100)
...: cpusage = {'used': used, 'total': total}
...: return f'CPU: {frac}%, {T}°C'
...:
In [16]: %timeit cpu()
82.2 ms ± 169 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [18]: %timeit sp.check_output(['sysctl'] + items, encoding='ascii').splitlines()
83.4 ms ± 2.92 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
Here again we see that most of the time is spent calling the subprocess.
Date and time
This is trivial, and very fast.
In [19]: def date():
...: return time.strftime('%a %Y-%m-%d %H:%M:%S')
...:
In [20]: %timeit date()
4.79 µs ± 19 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
Total
Let’s run the complete program and see how long it takes.
> python3 statusline-i3.py
Ext: 4B/0B Int: 0B/0B | Mail: 51 | RAM: 36% | CPU: 4%, 54°C | Mon 2019-07-01 22:05:45
DEBUG: runtime = 0.62 s
Ext: 0B/0B Int: 0B/0B | Mail: 51 | RAM: 36% | CPU: 10%, 53°C | Mon 2019-07-01 22:05:46
DEBUG: runtime = 0.29 s
Ext: 0B/0B Int: 0B/0B | Mail: 51 | RAM: 36% | CPU: 7%, 54°C | Mon 2019-07-01 22:05:47
DEBUG: runtime = 0.29 s
Ext: 0B/0B Int: 0B/0B | Mail: 51 | RAM: 36% | CPU: 6%, 53°C | Mon 2019-07-01 22:05:48
DEBUG: runtime = 0.28 s
Ext: 0B/0B Int: 0B/0B | Mail: 51 | RAM: 36% | CPU: 7%, 53°C | Mon 2019-07-01 22:05:49
DEBUG: runtime = 0.29 s
Ext: 0B/0B Int: 0B/0B | Mail: 51 | RAM: 36% | CPU: 7%, 53°C | Mon 2019-07-01 22:05:50
DEBUG: runtime = 0.29 s
Ext: 0B/0B Int: 0B/0B | Mail: 51 | RAM: 36% | CPU: 7%, 53°C | Mon 2019-07-01 22:05:51
DEBUG: runtime = 0.29 s
Ext: 0B/0B Int: 0B/0B | Mail: 51 | RAM: 36% | CPU: 6%, 52°C | Mon 2019-07-01 22:05:52
DEBUG: runtime = 0.29 s
Here, “runtime” is the (clock, not CPU) time that one iteration takes. After startup, the program stabilizes at approximately 0.3 s per iteration. The rest of the time (up to the 1 second refresh time) it sleeps. One could say that this program is fast enough, since it has more than enough time to do its job.
The CPU usage is too high for my taste, though. A simple status line should not require 7% of CPU time!
Conclusions for now
Parsing the mbox file takes the most time by far. But this is only done intermittently.
The rest of the time is mostly spent calling external applications lite
netstat and sysctl.
In part 2 we’re going to make this faster.
For comments, please send me an e-mail.
Related articles
- Have Python log to syslog
- Attempting a conky replacement in Python (part 2)
- Setting the Razer ornata chroma color from userspace
- Updating python3 to 3.6
- Switching IPython to Python 3