Python Programming
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed ForumsProgramming LanguagesPython Programming

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
  #1  
Old January 18th, 2004, 06:55 AM
StuBoy StuBoy is offline
Junior Member
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Jan 2004
Location: UK
Posts: 3 StuBoy User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 0
Question Webserver log script

Hi all i'm new to all this so i apologise. I've looked at tutorials for python and i can't make heads nor tails of them.

I need to make a python program which will summarise an apache web server log file. It should report the total number of successful web server accesses.

Reply With Quote
  #2  
Old January 18th, 2004, 10:27 AM
Scorpions4ever's Avatar
Scorpions4ever Scorpions4ever is offline
Banned ;)
Dev Shed God 6th Plane (7500 - 7999 posts)
 
Join Date: Nov 2001
Location: Glendale, Los Angeles County, California, USA
Posts: 7,588 Scorpions4ever User rank is General (90000 - 100000 Reputation Level)Scorpions4ever User rank is General (90000 - 100000 Reputation Level)Scorpions4ever User rank is General (90000 - 100000 Reputation Level)Scorpions4ever User rank is General (90000 - 100000 Reputation Level)Scorpions4ever User rank is General (90000 - 100000 Reputation Level)Scorpions4ever User rank is General (90000 - 100000 Reputation Level)Scorpions4ever User rank is General (90000 - 100000 Reputation Level)Scorpions4ever User rank is General (90000 - 100000 Reputation Level)Scorpions4ever User rank is General (90000 - 100000 Reputation Level)Scorpions4ever User rank is General (90000 - 100000 Reputation Level)Scorpions4ever User rank is General (90000 - 100000 Reputation Level)Scorpions4ever User rank is General (90000 - 100000 Reputation Level)Scorpions4ever User rank is General (90000 - 100000 Reputation Level)Scorpions4ever User rank is General (90000 - 100000 Reputation Level)Scorpions4ever User rank is General (90000 - 100000 Reputation Level)Scorpions4ever User rank is General (90000 - 100000 Reputation Level) 
Time spent in forums: 1 Month 1 Day 21 h 31 m
Reputation Power: 1001
Adapting some code from the Python Cookbook (original author for this snippet was Mark Nenadov):
Code:
#!/usr/bin/env python 
def CalculateApacheIpHits(logfile_pathname):
    IpHitListing = {}
    # Use readlines(), if xreadlines() doesn't work.
    Contents = open(logfile_pathname, "r").xreadlines()

    # Alternatively, use the following code, if you don't have
    # xreadlines and don't want to slurp the code with readlines. 
    # Comment the "Contents = open..." and "for line in..." and 
    # replace with the lines below:
    #input = open(logfile_pathname, "r") 
    #while 1:
    #    line = input.readline()
    #    if not line: break

    for line in Contents:
        #Split the string to isolate the IP address
        Ip = line.split(" ")[0]

        if 6 < len(Ip) <= 15:
            #Update the hash
            IpHitListing[Ip] = IpHitListing.get(Ip, 0) + 1;

    return IpHitListing

if __name__ == "__main__":
    # Substitute your own log file name here
    hits = CalculateApacheIpHits("/var/log/apache/access_log");
    for ip in hits.keys():
        print ip, "\t\t", hits[ip]

This one only summarises the hits by IP address. However, you can also get it to check the HTTP return code (if you're logging that, of course) and only log if a successful code (i.e. 200, 301, 302 etc.) is returned.
__________________
Up the Irons
What Would Jimi Do? Smash amps. Burn guitar. Take the groupies home.
"Death Before Dishonour, my Friends!!" - Bruce D ickinson, Iron Maiden Aug 20, 2005 @ OzzFest
Down with Sharon Osbourne

Puzzle of the Month solved by Keath and KevinADC, superior perl programmers of the month
Looking for a perl job with kick-*** programmers in a well-known NASDAQ listed tech company with branches in the US and Europe? We're hiring. PM me for details. Requirements

Last edited by Scorpions4ever : January 18th, 2004 at 10:32 AM.

Reply With Quote
  #3  
Old January 18th, 2004, 11:43 AM
StuBoy StuBoy is offline
Junior Member
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Jan 2004
Location: UK
Posts: 3 StuBoy User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 0
Thank you very much....this will help me tremendously. Will apply it monday hopefully.

Words can't explain.

Reply With Quote
  #4  
Old January 18th, 2004, 01:25 PM
netytan's Avatar
netytan netytan is offline
Hello World :)
Dev Shed Frequenter (2500 - 2999 posts)
 
Join Date: Mar 2003
Location: Hull, UK
Posts: 2,536 netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level) 
Time spent in forums: 1 Week 2 Days 18 h 3 m 4 sec
Reputation Power: 63
Send a message via ICQ to netytan Send a message via AIM to netytan Send a message via MSN to netytan Send a message via Yahoo to netytan
Just a small, more up to date example of how this task could be done...

Code:
#!/usr/bin/env python

def hits(path):
	count = {}
	codes = ('200', '301', '302', 'etc')
	for line in file(path, 'r'):
		line = line.split()
		if line[-2] in codes: count[len(count) + 1] = value
	return count


This should do what scorpy was discribing (untested). This actually has a lot of potensial , would be interested in seeing you're finished piece if you're willing to show it.

Note: xreadlines was depreciated in Python 2.3 and replaced with 'for line in file'

Anyway, have fun.

Mark.
__________________
programming language development: www.netytan.com Hula


Reply With Quote
  #5  
Old January 19th, 2004, 10:57 AM
StuBoy StuBoy is offline
Junior Member
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Jan 2004
Location: UK
Posts: 3 StuBoy User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 0
Cool thx

Reply With Quote
Reply

Viewing: Dev Shed ForumsProgramming LanguagesPython Programming > Webserver log script


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump


Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 5 hosted by Hostway
Stay green...Green IT