Matt Daubneys Blog

It’s been etched into my diary now, so I’m going to both Lug Radio Live 2009 and Oggcamp. Both of these will be quite exciting for me as they’ll be the first large scale linux events I’ve been too, however judging from the general community populace, both should be quite friendly and cool!

I’ll be travelling up to wolverhampton from the Newbury/Reading area, so if anyones heading up from that area as well, let me know and see if we can organise some kind of lift share. Hope to see a lot of you there!

I’ve been playing with a velleman k8055 USB experimentation boad and think I’ve finally come up with a use for this first board. I’m going to attempt to automate a few things in my room using a Linksys NSLU2.

The first thing to attempt is a nice simple one and does not involve touching any household electrics. A simple motor arrangement on the curtain rail to open the curtains in the morning and force me to get the hell out of bed!

The second thing I intend to do with it is a little more complex and will require some more research. I intend to somehow replace the dimmer switch in my room using the analgue output from the board. If anyone has any good references on what I need to read up to do this, let me know!

Well, now that Uni has finished, I need a job. Having been doing a Physics degree at Swansea, I’ve more or less come to the conclusion that I don’t really want to go into a Physics field, and would rather move across into a computer networking position. This comes with a few problems. Firstly, I have no professional experience with this kind of work, I do have lots of amateur experience, having networked my house and helped administrate the University Computer Societies network. I have been exploring the option of doing a CCNA to give me a kick start into a career, but these are expensive.

Why does this mean anyone should hire me? Well, I’m incredibly eager to learn. Having no professional experience means I will have to learn fast, and I fully intend to do so. I’m more or less convinced I want to do a CCNA, to the point where I’m willing to fund it myself and just need to find the funds to do so. I have heaps of experience with Linux servers, I’ve administered several for friends over the years and attempt to help people fix their Ubuntu machines in #ubuntu-uk and on the ubuntu-uk mailing list. I also have experience mending windows machines, again largley for friends and family. I am incredibly enthusiastic and really want to get on and work, having been sat on my bum for the past 2 weeks just applying for jobs, I’m now reaching the point where I just want to get on and work. I’m reliable, hard working and eager to get involved.

My CV can be found here in pdf and here in MS Word format, so if anyone has any opportunities going, please let me know!

I have been told many many times that moving ssh to a different port (i.e. other than 22) makes your machine more secure. I do see some wisdom in this, however, I’ve decided to put it to the test. I have been using Denyhosts to stop brute force attacks on my ssh servers for some time now, and on my most recent server, the attacks per day are fairly regular, as seen in the graph.

Plot of Deny Hosts Blocks per Day

At the end of June I will stop using port 22 and start using another random port. I’ll then collect data for 3 months and at the end of september do another blog post showing the difference. I also have another server that I will repeat this experiment on, but that one will be 3 months behind.

Hopefully then I will have a nice sturdy scientific answer as to how much more protection moving ssh to a different port gives

The code I used to generate this graph is given below for reference.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77

import os
import matplotlib
import datetime
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
import matplotlib.mlab as mlab
matplotlib.use('Agg')
 
 
datelist = {}
rootdir = './'
 
def plot_all():
	#first convert dict to a set of x values and a set of y values
	keys = datelist.keys()
	keys.sort()
	values = []
	times = []
	for key in keys:
		values.append(datelist[key])
	#now convert the keys into time format
		times.append(datetime.datetime.strptime(key, "%Y-%m-%d"))
 
	#now we're ready to plot with matplotlib
	months   = mdates.MonthLocator()  # every month
	days	= mdates.DayLocator()
	yearsFmt = mdates.DateFormatter('%b')
	dayFmt = mdates.DateFormatter('%d')
 
	fig = plt.figure()
	ax = fig.add_subplot(111)
	dates = range(times[0].toordinal(), times[-1].toordinal()) 
	ax.bar(times,values,width=1)
 
	ax.set_xlabel('Date')
	ax.set_ylabel('Number of Hosts Denied')
 
	#ax.plot(times, values)
	#ax.xaxis.set_major_formatter(yearsFmt)
	#ax.xaxis.set_major_locator(months)
	#ax.xaxis.set_minor_formatter(dayFmt)
	#ax.xaxis.set_minor_locator(days)
	ax.format_xdata = mdates.DateFormatter('%Y-%m-%d')
	fig.autofmt_xdate()
        fig.savefig("plot.png")
 
def countup(file):
	f = open(file, 'r')
	for line in f.readlines():
		#split by spaces to get the date
		line = line.split(" ")
		#now see if this is already in the list
		newline = False	
		for part in line:
			if part == "new":
				newline = True
			else:
				continue
		if newline == True:
			n = 0
			if line[0] in datelist:
				datelist[line[0]] = datelist[line[0]] + 1
			else:
				datelist[line[0]] = 1
	f.close()
 
for subdir, dirs, files in os.walk(rootdir):
	for file in files:
		if not file[-2:] == "py" and file.split(".")[0] == "denyhosts":
			countup(rootdir + file)
 
keys = datelist.keys()
keys.sort()
 
for key in keys:
	print("%s, %s" % (key,datelist[key]))
plot_all()

Every so often people ask for help and are not sure where the problem lies. When diagnosing problems in Ubuntu, and some of these tips should apply to other linux distros as well, there are a few places you should look at.

System Log Files

Linux distributions have fantastic logging capabilities built in. If there is a hardware fault it will more than likely appear in the system log.  The most useful log files are kept in /var/log/ and can be accessed directly or through the Log File Viewer (System -> Administration -> Log File Viewer.)

System Log Viewer

The one thats most useful is the dmesg log. This can be found in the log viewer or on the command line by typing dmesg.  There’s normally lots of information here and depending on what the fault is with depends on what’s not working. If you have a non-functional USB peripheral, like a USB wifi card or a printer, a look at what dmesg says before it’s plugged in and after it’s plugged in can supply you with some information to start hitting google, the mailing lists or IRC with. Sometimes it will tell you what needs to be done, for example I have stuck a dodgy memory stick in one machine which didn’t automount, dmesg quite happily told me the stick had errors and to check it. Fantastic!

The Tops

Is something slowing your machine down to a crawl? There are a few tools that can help here, largley from the “top” family of applications. Starting with the ones that are graphical, you can use the System Monitor (System -> Administration -> System Monitor) and have a look under the processes tab to see which process is eating the most cpu. Easy peasy.

System Monitor Tool

The next one is top itself. Open a terminal and type “top”. This gives us a nice list of running processes and how much cpu/memory it’s using and its pid (process id). You can kill processes by hitting k and then typing in the pid.

top

The main reason I introduce people to top is because it is installed by default on most distros. A better version is htop, which while still in the terminal, is much easier to read.

htop

As you can see, it gives you all the info top gives you, but in a slightly nicer, more intuitive layout. What if somethings really giving your hard drive a bad time? Want to find out what process is doing that, simply install iotop (sudo apt-get install iotop).

iotop

This gives a list of the processes running on the machines, who’s running it and how much io it’s using.

Other Useful Things to Know

Along with these diagnosis tools, there are also a few tools which can identify what hardware is connected to your machine. These are all in the terminal, and are “lspci”, “lsusb” and “lsmod”. These will give you a list of hardware connected to a pci like adapter, a usb like adapter and the modules loaded into your running kernel. If you’re asking for help somewhere, you will more than likely be asked for the information from one of these.

Rounding Up

The purpose of giving access to all this information is because diagnosis is half way to a fix. If you know what is causing the problem you know where to start looking, and what to start google-ing for. Hopefully this post will be helpful to someone I’ll write a few more in the future going into a bit more depth into each of the commands listed, but a useful place to find more info on them is in their man pages, in a terminal simply run “man <command>” where<command> is the command you’re running.

I’ve discovered recently that I don’t sleep well if I sit on the internet too late at night. Here is my solution.

I’d been intending to learn Lua for a little while now, so this entire project was written in lua, with some bash scripts to call stuff. Essentially what it does, is every minute it checks the time. If the hour is greater than or equal too that set in /etc/bedtimer then it uses iptables to close all the ports I use that go out to the internet. This allows me to continue using my local network stuff (except ssh) if I had things like coursework to do. The server part continues to run every minute after that, incase I try and override it (bad me). There is also a gnome bit that is loaded when I log in. This part simply uses the ubuntu notify-send thing to warn me when I have 15 minutes before the net shuts off, and when it shuts off.

The bit that does all the work is loaded at boot using an initscript. I have yet to reboot and test this, as it’s my first initscript, but it looks ok

Hopefully this will force me to actually stop wasting time on the net in the evenings and actually go read a book or do something to wind down at the end of the day.

The files can be found here. There is a shell script to install it all, but it’s rubbish and may not work on your machine! BE WARNED!

Last night I listened to the excellent Ubuntu-UK Podcast and something mentioned pipped my interest and sent the dusty cogs in my head whirring. Someone wrote in and asked about how to get involved with the community.

This is something that has intrigued me before and I think I’d like to put fingers to keyboard and etch out a few ideas as to how I see a possible solution. This is a bit of a mind dump so you may have to bear with me.

Becoming part of any community can be difficult initially. Everyone feels a little bit of an outsider when they start and it can take a little while to fully integrate with a community. This is what I see as Problem #1.

Problem #2 is trying to find your place. Some technical people feel happy commiting patches and so on to various OSS projects, these people I think just need to find their itch that needs scratching. The large problem is people who are not necessarily technical and want to get involved but don’t know their way around the community yet, or are uncertain where they want to sit.

Problem #1 is a very personal thing. There is no solution, as far as I can see, that doesn’t come from the individual themselves. In order to integrate yourself with a community you sometimes just have to throw yourself into it. Certainly the other members of the community can help by being inclusive in discussions, not being purposely obtuse and by being aware of that unsettling feeling of finding your place, but without the original individual just getting involved a bit, I’m not sure what else could be done.

Problem #2 is where other community members can be more helpful. There are some people, such as popey and a few others, who have been around a little while such that they are in a very good place to be helpful, and try to be so. Maybe what’s needed here is a list of people who can be contacted, either individually or as part of a forum (irc chatrooms, the ubuntu forums, mailing lists, etc) who might be able to point people in the correct direction. Yesterday when I brought this up on IRC I think I used the phrase “Community Ambassadors” but I think in many ways that’s wrong. What’s needed (from my point of view, I’m open to discussion on this) is essentially just an easy way for people who want to get involved to be able to contact these people to get some pointers. The thing that makes the community is the people and a website with ideas on is a bit impersonal sometimes can’t answer peoples questions or just bring the overwhelming scale of what’s available to get involved in down to a manageable viewpoint.

As I can see it, I don’t think this problem has an instant solution, but may need to be pecked at repeatedly until a best fit solution is found. Hopefully this will generate some discussion! Feel free to contact me here, find me on jabber at daubers@jabber.org or blog your viewpoint too. Hopefully we can generate some common ground and generate an action plan to go forward.

My knowledge of linux is sadly lacking, but every day I improve on it. Since I had a few days free over the easter holidays I thought I’d try and improve this further by installing Linux From Scratch. This is essentially building up a linux system from it’s base packages and takes a LONG time. This to me seemed like a good idea, it would exponentially increase my knowledge of what makes linux tick, and what depends on what.

I started on Monday night by reading through all of the LFS documentation. As bedtime reading goes, it’s actually quite interesting, and it seemed like a good idea to get an overview of what needs doing before I started.

Luckily for me my laptop has a fairly large HD (320GB) so finding some space on a partition was quite easy. A quick boot into a livecd and resizing my /home partition created a nice little 10GB partition for LFS to go in. That was at 9am in the morning. By 10:00am I had downloaded all the required programs in the mounted partition as directed by the book and started to build them.  In the first day, I managed to build the initial toolchain and got into the chroot and got to chapter 6.15 in the LFS PDF. The longest thing to build was probably either glibc or GCC, which took just over an hour, but did give me time to cook some tortillas from scratch for lunch

The second day, it took me from about 10 in the morning till roughly 3 in the afternoon to finish installing everything.  Was a bit quicker than I expected, but also a lot more involved.

I did hit a few roadbumps, I didn’t have gawk installed on the host system, so at one point something didn’t compile properly and freaked out a bit. A simple sudo aptitude install gawk fixed that though. When I was recompiling glibc all of the tests failed, it took me a few minutes to realise that the reason this was happening was because I’d forgotten to run “make”.. oops.

The last major bump was that when I chose which options to compile with the kernel, I forgot to add the drivers for my ethernet card and my wireless card, so when I booted the machine I had no network! A quick scan through the options and a recompile sorted that out (to include the b44 driver)

What have I learnt from this? Two things mainly.

1. Where everything should live on a linux system and why its there
2. Package Managers are beyond fantastic. They save so much time and hassle!

Tomorrow I intend to try and make my LFS install useful by installing openssh initially, wget and possible x.org and gnome. That may be a bit ambitious at the moment though!

While installing some software with aptitude last night I noticed this “0% [Connecting to gb.archive.ubuntu.com (2a01:450:10:1::10)]“, gb.archive.ubuntu.com is on ipv6! Hooray for the future that should be now!

Recently I’ve been fighting off depression and it’s made me take stock of what I do quite a lot. Sometimes it’s necessary for me to stop what I’m doing and often it’s because I’ve looked at what I’m doing and become a bit confused. When ever I do this I end up looking at what I’m doing and asking “How is this me?”. It seems like a silly question sometimes, but life is consistantly changing, sometimes for the better and sometimes for the worse, so it’s necessary now and again to stop and define what things really mean.

Who am I? I think it’s safe to sat that what you do and how you act defines you, so I shall pick a few things on what I do and how I act to try and define myself.

Physics

How is this me? I’ve always felt that everything has a reason that can be understood. When told that the universe has been created I always ponder if this was true, how was it created? What where it’s initial ingredients? How can you set off a system with a few simple rules to create such a complex system? What are these rules and can they be replicated. I’ve always felt that this is part of me, and the more I understand of how the world we live in works, the more I see that it’s important to find a place in it.

I always want to understand why something has happened, how something has gone wrong. These in some ways drive me to do things. As I’ve struggled recently with the depression I feel that I’ve been losing this drive a bit and become too accepting of facts and not questioning things enough. This is me.

Linux and OSS

How is this me? While the idea of financial gain is nice, it somehow seems unnerving. In the land of software, it sometimes seems as if it has got to a point of being all about financial gain, and not what can be done.

Linux and OSS are not just about freedom, they are about being able to push what you can do and do it faster, more intelligently to save effort. A lot of proprietry software seems to think that the software is the important factor, but it’s what the user can do with it that’s more important. If a piece of software can’t do something I need to do, then why shouldn’t I be able to add that functionality? If I needed that functionality chances are someone else will do someday to, so I can give that back to the people who originally made the software. While this doesn’t make me monetarily better off, somewhere it will make someones life easier. This is a net saving in frustration and seems to be worth far more than a few pounds extra at the end of the month.

Linux is also about a community, and more of life should be about this. I have found that the UK Ubuntu community is made up of some of the nice, most helpful people I’ve talked too. Some of whome I have started to regard as friends, and hope to make that so once Uni has finished by taking part more and giving more back to those who have given me so much. Much of the philosophy of this I try and take forward to other parts of my life, not just software.

Photography

How is this me? It’s one of the things that makes me stop and look at the world. Sometimes days can get so busy with repetitive, thoughtless tasks that I don’t stop, look or think. When I’m out with a camera I’m always looking, at the way the grass bends with the wind, the way the sun shines across the waves in the bay, the orange glow of the sun rising behind Port Talbot. Small things that seem to be more interesting when you think about what they are, how they happen and what it all really means. Somehow being behind a camera really makes me look enough to appreciate these things the way they should be.

Friends, past and present

Life is incredibly short on cosmological timescales. People come and go, but everyone you meet leaves a mark on you somehow. People I used to be close to have slowly moved away, some people I haven’t seen in a long time. Some friendships are worth more to me than others, but all are individually valuable. These people in a way have formed who am I, what I believe is rightor wrong and what I feel is important in the world. In many ways, bits of everyone I have met are me.

My future wife

In many ways she has shaped the current me more than a lot of other factors, and that isn’t a bad thing. I have known people I would give up a lot to help, but she is the one person I would give up everything for. She is the reason I look forward to a settled married life. There’s no need for piles of money in that future, just enough to get by and her by my side. In a way she is the little piece of the world I have been looking for, and feel like I’ve almost found somewhere I want to be.

There are many other things that are me, but these few I currently feel are the most prominant. So now, people of the interweb, I challenge you to find out just who are you?