2010 in review

The stats helper monkeys at WordPress.com mulled over how this blog did in 2010, and here’s a high level summary of its overall blog health:

Healthy blog!

The Blog-Health-o-Meter™ reads Fresher than ever.

Crunchy numbers

Featured image

About 3 million people visit the Taj Mahal every year. This blog was viewed about 29,000 times in 2010. If it were the Taj Mahal, it would take about 4 days for that many people to see it.

In 2010, there were 2 new posts, growing the total archive of this blog to 61 posts.

The busiest day of the year was March 9th with 150 views. The most popular post that day was Snmpd filling up /var/log/messages.

Where did they come from?

The top referring sites in 2010 were experts-exchange.com, google.com, ifreestores.com, linuxquestions.org, and forosdelweb.com.

Some visitors came searching, mostly for cisco anyconnect mac, rsync exclude file, snmpd logging, telnet escape, and kde dbus-send save session.

Attractions in 2010

These are the posts and pages that got the most views in 2010.


Snmpd filling up /var/log/messages February 2008


Mac OS X Snow Leopard & Cisco AnyConnect VPN August 2009


KDE4 Save Current Session – DBUS to the Rescue April 2008


Escape from Telnet August 2007


Using Exclude Files in rsync – watch those spaces! May 2008


Linux – For all your hardware driver needs

I’ve recently bought myself a new desktop PC ( more of which in another post perhaps). The plan is to give my old desktop to my dad and he uses Windows rather than Linux. ( One day maybe I’ll convert him, I’m sure he’d be amenable to giving it a try).
I wiped the partitions on my old Desktop’s hard drive and booted from my XP install CD.
Unfortunately when it tried to install it said there were no hard drives detected. I thought maybe I needed to reset the SATA RAID JBOD but this made no difference.I booted from an Ubuntu live CD and the hard drive was detected no problem.It transpires that my ASUS motherboard with its Uli SATA RAID controller is not supported by the standard windows install CD. You have to get the drivers on a floppy disk and press F6 during the install to be able to include the drivers off the floppy.

Now, as is the way with a lot of desktops, there is no floppy drive fitted, however I managed to track down an old USB floppy drive. The next challenge was to find a floppy disk, this proved even harder but I found one in the back of the drawer. So I put the drivers on the disk and booted the Windows and duly pressed F6.

All to no avail as the USB floppy drive was not detected and a check of the Microsoft knowledge base confirmed only a couple are supported.

I thought this was game over at this point but I came across a product called nlite that enables you to roll your own Windows installations and customise them, including adding in additional drivers. You can then create an ISO that you can burn to disk.

This got be running and the tellytubby green hill is now showing nicely on my old PC.

Two things spring to mind about this.

Firstly nlite is a cool little programme and is at least free as in beer.

Secondly it shows the power of open source. Linux is able to detect my hardrive from the installation CD no problem presumably because the community decided this driver was important enough to be included or maybe because the way the kernel works the standard drivers just work better with a wider range of hardware.

Either way score another one for the penguin.

Reflections on PyCon UK 2007

Now the dust has settled I thought I would just say a few words about my experience of PyCon UK, my first conference where I have been involved in helping out with the organisation and running on the day(out of only 4 conferences I have ever gone to) .

Firstly, overall I think this was an excellent conference, the number and range of talks was amazing and it all ran pretty smoothly. With the exception of the comedy take away dinner at the pub on the Friday night (Zeth still hasn’t recovered from the stress of that I think) I was only aware of one other problem: No one was available to chair the last session in the room I was just vacating. (I did the intro and left them to it as I had to do some other stuff but it seemed to go ok and it was lunch after that. )

I looked through the feedback forms as we were sorting them out at the end of the conference and I did not see any major criticisms of the conference as a whole.

This is a remarkable achievement due in no small part of the work of the committee (of which I was not a part in case you think I am being immodest). The drive and commitment over several months paid dividends and though all the committee played their parts special thanks must go to Zeth and John for being the drivers and enablers for the conference.

I had a great time and enjoyed all the set up and chairing sessions (well I like the sound of my own voice). However it was really tiring and on Monday at work I was fit for nothing. Next time I will definitely book the Monday off to recover.

It does make a difference to how you perceive a conference if you are involved in the running of it. Seeing 200 people milling about and knowing you’re are helping them have a good conference is a great feeling. One thing I found was that when I wasn’t session chairing I didn’t actually want to go to any other talks I would rather chill out a bit and take stock. ( Well that and do some live blogging.) However cunningly I had chosen to chair sessions where at least one of the topics was something of interest to me. I wonder if I can get away with that next time.

This was also the first conference where I had been able to attend the social events and that was great fun and definitely something I will try and do at future conferences.

I would recommend anyone thinking of helping out at a community driven conference to give it a go. It’s hard but very rewarding work.

PyCon UK 2007 Day 2 Part 2

[References added 18th September]

OK a little behind with the updates but as I am now officially on holiday from work I should get this all ship shape in the next couple of days. On with the rest of day 2.

A Pythonista’s Year at Kew

Julius Welby gave us an hour long talk on his experiences of using python at Kew Gardens to clean up the 1.7 Million data entries in the International Plant Name Index (IPNI). Why did the data need cleaning? Well a lot of the entries in the database go back to hand written lines in a ledger 100 years ago which have then been typed, ocr scanned and then copied to other formats like CD Rom. So there is a lot of scope for error. Especially when a lot of the references to other works are in roman numerals. How well will OCR cope with a smudged “iv” or that a clear “lv”?

Now the topic of data cleaning is on the face of it an important but rather dull task, however Julius imbued the whole undertaking with great enthusiasm and had everyone in the audience intrigued as to the approach he took. As he said at the end of about 55 minutes “I’ll inhale now and you can ask some questions”. He was a really dynamic speaker and clearly there was much more to learn about what he had done. He did not have time to show some of the specific techniques in python he used as he was talking about the general issues of data cleaning so it was of less importance. Even so I’m sure there would be interest in seeing some of the specifics. (Next time maybe).

Jackfield, the Web and Python

Stuart Langridge of LUG Radio fame gave a half hour talk about the Jackfield project he has been working on. In essence this a method of having widgets such as those for Apple Macs or Opera run “natively” on a Linux desktop.

In his entertaining and insightful way Stuart showed how he felt there was a perception that web stuff was really cool and desktop stuff a bit dull. Jackfield was in a sense the way to bring thousands of web applications to the Linux desktop so as to keep it “in the game”.

Interestingly Stuart explained how there was 642 lines of python in Jackfield that “just work” and 357 lines of javascript that are the bit that need work and revolve mostly around making firefox (gecko) rendering behave like the khtml based safari. This showed again the power and simplicity of python.

This was a talk I was very much looking forward to and it did not disappoint. Stuart is a very entertaining speaker who gets across some interesting points.

Python in Higher Education

This was a talk by Nick Efford and Tony Jenkins about their experiences of using python as a first language for undergraduates at the University of Leeds. It started with an interesting picture showing two mountains and asking which was harder to climb. One was the Matterhorn the other was Everest. The point was that to someone unskilled in climbing they both look daunting but to an expert the Matterhorn is probably the slightly easier. This analogy was then extended to programming languages. C and C++ had been the mainstay at Leeds but then Java came along and seemingly offered a much easier option. Sadly though for the beginner this was still the difference between the Matterhorn and Everest.

Python however because of its interpreter, clean syntax and great libraries was actually much more of a foot hill to begin with. Nick and Tony explained how they could now use a single language for areas such as natural language processing or distributed computing by the use of popular and well supported libraries for python. So when students moved to new topics the underlying concepts and structures of the language they were to use were already under their belt.

They also showed some examples of the short term projects that the undergrads did in their first semester and the results were excellent ranging from crazy golf with semi-real physics engines to applications involving networking, http and SQL.

It was great to see that python is becoming a language of choice for computer science students and Nick and Tony have been looking at its use in local secondary schools as well.

Keynote – Laura Creighton

I had seen Laura around the conference and heard her contributions to a few talks and discussions from the floor without realising she was to give this key note.

Laura was heavily involved in campaigning at the European Union Parliament to ensure software patents were not made enforcible in the EU. Her talk was about the approach she needed to take to make this happen. In related topics she covered the problems caused by engineering becoming more about maths than actually doing things and trying things and also how the population as a whole was becoming less and less “creative” in the sense of actually making things. (She excluded writers in this so my blog doesn’t count as creative).

A key point was that the decision makers i.e. European politicians were unskilled at creating software and unaware that they were unskilled in it. (Something found as a general psychological principle by Kruger and Dunning ). To this end they thought that writing software was not a creative task and so would not be stifled by patents on just software ideas that did not actually give away any secrets ( the usual “price” or quid pro quo of a patent ) in return for the monopoly the patent gave.

The masterstroke of her campaign seems to have been showing the aides to the politicians how software is actually written by allowing them to sit in on the process as she and her campaigners developed things to help overcome the problems the EU politicians were having with their various proprietary IT systems. ( Especially those from Microsoft).

Laura is clearly one of those people who has both left and right brains heavily in use and you could almost see the ideas fighting to get out as she gave her talk and recognised more connections to other themes she wanted to address us on.

I was almost out of breath at the end of the 90 minutes or so that she spoke. I hope there is a good audio recording of this as it is definitely something you need to listen to more than once to get full value from it.

Bits and Pieces

I didn’t catch many of the lightning talks but the few I did were entertaining and wide ranging including the use of python to help surgeons examine CT and MRI scans to allow better placement of radioactive seeds in the prostate of men with prostate cancer. Not something I expected to hear at a python conference I must admit.

There was a quick 20 minute AGM where the main committee were re-elected unopposed and the partial accounts were shown. Critically the decision was taken to have Pycon UK 2008 at the same venue and over 3 days on the 12th to 14th September 2008. (The Friday 12th will be for all day tutorials ).

There was a raffle with some cool prizes and a prize for the best lightning talk (which I had missed). In general there was much thanking and clapping as an excellent conference drew to a close.

PyConUK 2007 Day 2 Part 1

[References added 18th September]

No longer live from the conference here is the first part of day 2. Again it needs referencing up and I will try and get this done by the end of the week. (Honest. I will. No really.)

Introduction to Functional Programming in Python

After a hearty and lively conference dinner on the Saturday night it was gratifying to see a good turn out at 09:00 on the Sunday for this talk by David Jones of Ravenbrook (One of the sponsors of the conference).

In a half-hour slot David gave a good comparison of the functional and the imperative (object orientated) approach and looked at the built in support that python has for the functional approach. He had a lucid and expressive style and obviously had a good command of the subject.

What was interesting was the way that caching can be used for pre-calculated results. The example he used was the classic Fibonacci series calculator using recursion. Without caching, fib(30) took a second of compute time, with caching this was down to a few ten thousands of a second which was comparable to the imperative for loop equivalent.

Unfortunately half an hour was only enough time to whet the appetite and you could sense the audience was itching to see more and ask more questions. I would have loved to have seen the sorts of real world problems for which the functional approach can be most effective.

Live from PyCon UK 2007 – Part 2

Another quick post from PyCon UK 2007 which I’ll fully reference up later.

[References added 18th September]

Day 1 Part 2

Dealing with Tabular Data

Alex Wilmer gave this half hour beginners’ talk on using python to deal with data in tabular formats, by which he meant everything from CSV text files through HTML to SQL RDBMs. This is a subject close to my heart as I often find myself using ksh, awk and maybe a bit of perl to convert various systems logs to other formats.

In a simple and clear way Alex showed the key tools in python that can be used to work with different types of data. In the half an hour he had he covered a lot of ground really well but as this is something I’m particularly interested in I wish he’d had an hour slot.

Intro to PyQT GUI Programming

This was a two hour tutorial slot given by Marc Summerfield that showed the use of the Python bindings for the QT GUI programming libraries. Most of the people in there had started with the Python for Beginners tutorial from beforehand and carried on into this tutorial. There was a range of programming abilities and familiarity with event driven GUI concepts so Marc had a lot of concepts to explain.

He was able to demonstrate how PyQT enables great GUI development and even how you can extend it to completely new widgets written from scratch but unfortunately in the two hour slot there wasn’t much time to let people actually try the examples on their laptops.

Keynote Speech on OpenID

Simon Willison gave the keynote speech at the closing plenary session of the 1st day. Simon is best known for being the co-creator of the Django web framework but his topic for the keynote was to proselytize the use of OpenID.

OpenID can perhaps best simply be described as single sign-on for the Web. Given the plethora of sites that people have accounts on the task of managing them has started to become a barrier to people using some services. This is because they feel it is just not worth the effort of filling out all the details to sign up and having to either remember a new username and password or compromise their security by using the same username and password on multiple sites. ( Well we all do it don’t we?).

OpenID is a way for a web site to use a url as a username and for that to provide a link to an ID server that can allow you to verify that it is indeed you signing on. This could be as “simple” as you using an OpenID from say live journal so that once you sign in once to live journal other sites can check that to see you are logged in and therefore you allow them to log you on to the other site. Or it could involve an SMS message being sent or the use of a secure ID key fob.

I haven’t explained the foregoing very well at all and Simon Willison was much more articulate at expressing it and much more entertaining. This is one of those areas where the “open” movement are providing a solution to a problme that I’d never even thought there could be a solution to without recourse to some single overarching (and evil) corporation.

As a keynote it really did hit the right note. (Weak pun intended)

……..and so to Dinner.

The conference dinner was hosted at the Novotel and was extremely good. The food was delicious and the service to nigh on 200 of us was excellent. We were entertained by an after dinner speech from Jono Bacon about his experiences of Python for the the Jokosher audio editing project. He claimed it was his first ever after dinner speech and that included being as a member of an audience never mind as the speaker.

It was an excellent speech and had all the key ingredients for after dinner, it was witty, lively, the right length, rooted in the subject at hand and pitched just right for the audience.

Well it is nearly lunch time on day two now so I’m off to the refectory………

Live from PyCon UK 2007 – Part 1

It’s half-way through day one of the UK’s first national python conference PyCon UK. I’ve taken a few minutes out between talks to review things so far. As this is a live post I’ll reference it up later with links to people and products I mention. [ References added 18th September]

I declare an interest as I am part of the crew for the conference helping set things up and chair some sessions.

I should also declare I’m a bit of a fraud in as much as I have only ever written three python programmes. It’s something I want to do more of so this conference will be my catalyst hopefully.

Before the main conference started today we had a pub meet in the evening on Friday. This was held at Bennett’s pub in the centre of Birmingham. There were about 70 people there at its peak.

As organisers we faced our first crisis because the chef at Benetts had “disappeared” just before we arrived so no food was available and we had 20 hungry pythonista’s hoping for food. Zeth and John struck on the idea of buying take out food from the Thai restaurant across the road and ferrying it. Chaos abounded with dishing out the various portions but everyone got fed, crisis averted. As a vegetarian I didn’t expect to spend so much time dishing up food to corpse crunchers half the evening but it’s all hands to the pump.

Day One

The main conference is being held at the Birmingham Conservatoire and we have 200 delegates using 4 rooms for talks and tutorials. Here’s a brief outline of the ones I have heard so far.

Database Magic with SQLAlchemy

This talk was given by Paul Johnston and as the first talk up in one of the rooms he was setting the standard for the day. The talk was quite a technical overview of the use of SQLAlchemy an Object Relationship Mapper that allows python objects to be stored in standard SQL RDBMS . What was particularly interesting was the way relationships between objects could be mapped and then the SQL generated for queries could be done in an “eager” mode that uses outer joins etc. to reduce the number of sub-queries and calls to the database to get a set of data.

Introduction to Django

This was an eagerly anticipated talk from Jason Davies and was well attended. Jason ran through a broad over view of the Model View Template approach of Django and the background to Django. With just slides to view, it lacked a bit of the interactive element to show the speed with which sites can be developed but Jason handled a good 20 minutes of questions at the end and was able to address them all in a reasonable level of detail without overwhelming the mostly beginner audience with technical detail.

He also showed a slide showing the relative performance of Django, Turbo Gears, Symfony and Ruby on Rails. Django comes out really quite well on this.

Ok off to more talks, not sure if I’ll get time to do another live post but we’ll see.