Matt Daubneys Blog

Maybe 6 months ago I started going to Reading Hackspace. For those of you unaware, wikipedia describes a hackspace as “a location where people with common interests, often in computers,technology, science, digital or electronic art (but also in many other realms) can meet, socialise and/or collaborate.” In the past 6 months of hanging out with the Hackspace people, I’ve probably learnt more than in the previous 12.

Reading Hackspace has recently aquired a physical space to work in, and in order to help things along I offered to build the RFID based door system. As with all systems of this manner, the spec was changed a couple of times along the way, and probably will be changed again in the future.

The door system is composed of three parts:

1. An Arduino based card reader connected to an electronic lock
2. An MQTT server (namely the epic Mosquitto)
3. A python script that uses a SQL database to authenticate users running on an Ubuntu server.

A full write up is in progress on the Reading Hackspace wiki . Here’s a quick pic of the reader screwed to the door.

And a video of the system working!

I’ve been meaning to post this for a little while, but now seems as good a time as any

My little team in the office has expanded since I started as the only developer. With two other devs on board managing the bzr commits has meant a little extra overhead to make sure I know when new revisions have been pushed. Thinking there had to be a better way then getting people to email me when they do a commit and push to the main branch I went digging through the bzr docs.

It turns out I’ve been “doing it wrong™”. The bzr repo was setup so that people could connect to it using sftp as that was a quick easy way to get things rolling when it was needed. Apparently bzr has an inbuilt “smart server” that can run scripts on certain hooks when certain events take place. This looked like the way to go!

First thing was setting up the smart server. I threw Apache onto the dev box, install mod-wsgi (because it’s so much better than mod-python) and started reading through the instructions. About an hour of screaming and poking I got the system running  as a smart server, meaning I could push using bzr+http instead of sftp. Now came the difficult part.

It seems that only very specific events can be hooked into on the server side. This wasn’t immediately obvious from the bzr docs, but a little shouting, throwing things at the monitor and emptying nerf after nerf at the keyboard eventually got me to the hook I wanted specifically.

Now I had the ability to hook into things with bzr, but where could I send the events that it was generating? EMail was a bit dull, so I went back to MQTT, with the thought of commits could now light a lamp in the office when they happen The code for the server side bzr plugin is below. You just need to drop it into your .plugins directory for your smart server (ensuring you set this up in you wsgi configuration).

from bzrlib import branch
import mosquitto as mqtt
import os

mqttServerIp = "192.168.0.250"

def post_push_hook(push_result):
    branchFolder = [x for x in str(push_result.branch.__dict__['_base']).replace("//","/").split("/") if x]
    connectSendDisconnect("new branch revision: "+str(push_result.new_revno),branchFolder[1])

def connectSendDisconnect(msg, branchName):
    mqttc = mqtt.Mosquitto("bzrlib"+str(os.getpid()))
    mqttc.connect(mqttServerIp, 1883, 60, True)
    mqttc.publish("/code/"+branchName, msg, 1, False)
    mqttc.loop()
    mqttc.disconnect()

branch.Branch.hooks.install_named_hook('post_change_branch_tip', post_push_hook, "My post_push hook")

Magic! Now a message will be sent on the “/code/branchName” topic every time a commit happens Using some borrowed python magic from http://chemicaloliver.net/programming/first-steps-using-python-and-mqtt/ I’ve made it integrate into the default Ubuntu notifications system so a nice little box pops up informing me of a commit and the new revision number of the branch Doubles aces!

My fight against distractions has carried on further. This brief episode is brought to you by the theory of “scratching your own itch”.

There are a lot of things that annoy me with various tasks I perform every day. This past week I’ve been taking note of each one, and I count 78 in total. Each one of those 78 things could probably be fixed very easily, but because I’m generally running around like a headless chicken trying to get things done, I don’t even think about fixing them, I just put up with them and carry on. A fair few of these are things that take a while to fix, like the office being short of physical space because of all the orders coming in/out at the moment. Other issues are easy to fix, like the fact that it takes me 30 minutes to an hour to do a quick analysis of some testing data. This is a problem I can solve easily with a smattering of python and a little concentration. In fact, I’m intending to tackle this on Sunday by going into the office when it’s quiet, and just making this problem vanish.

A few other problems on that list of 78 can also be solved with code, some easily, some not so. Some can be solved by a few simple changes in my work routine, like the fact I constantly go to do a task, get pulled away by something else, and then forget what I was going to do. Keeping a simple log book of what I’m doing during the day would solve this easily. It would be better if this was digitized in some form, but for now a simple notebook will probably make a huge difference. This distraction on its own has probably caused me to lose my train of thought more times this week than any other on the list.

So how id my fight going? I’m much more aware of what causes me to be distracted after the past week or so. It’s taking me time to come to terms with each of those distractions and to deal with them, but ultimately, I am becoming more productive in a given period of time. The next big thing I’m going to have to tackle is project methodology. Since I don’t even have a passing familiarity with any of the standard methodologies, this is going to take some research, some thought and  a lot of conversations in order to find the best one to suite the needs of my team at work. As always, any pertinant reading material suggestions are always welcome.

I have been told many many times that moving ssh to a different port (i.e. other than 22) makes your machine more secure. I do see some wisdom in this, however, I’ve decided to put it to the test. I have been using Denyhosts to stop brute force attacks on my ssh servers for some time now, and on my most recent server, the attacks per day are fairly regular, as seen in the graph.

Plot of Deny Hosts Blocks per Day

At the end of June I will stop using port 22 and start using another random port. I’ll then collect data for 3 months and at the end of september do another blog post showing the difference. I also have another server that I will repeat this experiment on, but that one will be 3 months behind.

Hopefully then I will have a nice sturdy scientific answer as to how much more protection moving ssh to a different port gives

The code I used to generate this graph is given below for reference.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77

import os
import matplotlib
import datetime
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
import matplotlib.mlab as mlab
matplotlib.use('Agg')
 
 
datelist = {}
rootdir = './'
 
def plot_all():
	#first convert dict to a set of x values and a set of y values
	keys = datelist.keys()
	keys.sort()
	values = []
	times = []
	for key in keys:
		values.append(datelist[key])
	#now convert the keys into time format
		times.append(datetime.datetime.strptime(key, "%Y-%m-%d"))
 
	#now we're ready to plot with matplotlib
	months   = mdates.MonthLocator()  # every month
	days	= mdates.DayLocator()
	yearsFmt = mdates.DateFormatter('%b')
	dayFmt = mdates.DateFormatter('%d')
 
	fig = plt.figure()
	ax = fig.add_subplot(111)
	dates = range(times[0].toordinal(), times[-1].toordinal()) 
	ax.bar(times,values,width=1)
 
	ax.set_xlabel('Date')
	ax.set_ylabel('Number of Hosts Denied')
 
	#ax.plot(times, values)
	#ax.xaxis.set_major_formatter(yearsFmt)
	#ax.xaxis.set_major_locator(months)
	#ax.xaxis.set_minor_formatter(dayFmt)
	#ax.xaxis.set_minor_locator(days)
	ax.format_xdata = mdates.DateFormatter('%Y-%m-%d')
	fig.autofmt_xdate()
        fig.savefig("plot.png")
 
def countup(file):
	f = open(file, 'r')
	for line in f.readlines():
		#split by spaces to get the date
		line = line.split(" ")
		#now see if this is already in the list
		newline = False	
		for part in line:
			if part == "new":
				newline = True
			else:
				continue
		if newline == True:
			n = 0
			if line[0] in datelist:
				datelist[line[0]] = datelist[line[0]] + 1
			else:
				datelist[line[0]] = 1
	f.close()
 
for subdir, dirs, files in os.walk(rootdir):
	for file in files:
		if not file[-2:] == "py" and file.split(".")[0] == "denyhosts":
			countup(rootdir + file)
 
keys = datelist.keys()
keys.sort()
 
for key in keys:
	print("%s, %s" % (key,datelist[key]))
plot_all()