I'm sick of having to deal with Sawmill which, despite my best and not inconsiderable efforts, still takes more than half a day to import three months worth of trimmed web site logs. So I decided to put my money where my mouth is and create my own lighter weight system which would provide the specific information that I am generally called upon to retrieve; namely top pages, referrers and geographical location of viewers.

In about a days worth of work, and a little waiting around, I managed to write a perl module and scripts that can import three months worth of our web site logs into a MySQL database in around 2 hours, roughly 1/7th of the time it takes Sawmill to achieve the same task. Following along with the theme, it's called Beaver - a more organic log processing system.

I have written this post waiting for a referrer report for December to run on Sawmill to compare the results, Beaver took just over 7 seconds to run the same report on the same machine. The verdict: it's close enough that I don't trust Sawmill to be producing the more correct report - I know exactly what my software does.