Squid Log Generated Top 10 List

I am trying to make a homepage for a small intranet.

All of the users on this network authenticate and use the Squid Proxy.

I would like to display a "Top 10" list of the most commonly visited sites integrated into the corner of the homepage that I am designing.

I suspect that a PHP program would work very nicely, but if another language would work, I'd be just as happy. I've searched high and low for a pre-made program to generate a top 10 list, but haven't been successful.

Does anyone know where I could find a simple log analyser to do this?

Any help is greatly appreciated!

If the Squid logfiles are in the CLF (Common Logfile Format) then there will be loads of programs to do this. I would use Perl for this tool, because it is very fast at processing large text files. Plus there is a logfile module written already, greatly simplifying the task of summarising logfiles.

One important thing to bear in mind is that you don't want to process the logfiles for each page hit as this will make things slow. Instead, write a program to run every hour or so which will output an HTML summary (in a table or something) of the "Ten most visited sites". You can then include this generated page on all your webpages using PHP without reprocessing the logfiles each time.


Hope this helps.

Checkout Calamaris, which is a PERL script that does exactly what alexgreg suggests -> http://freshmeat.net/projects/calamaris/

//NoXcuz