#1
  1. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2005
    Posts
    65
    Rep Power
    9

    Designing an web access log reader


    Hi

    Iím thinking of designing a classic asp page that can access a web access log file (.log) and need a bit of help / direction getting started.

    I have a web site that I want to get access stats for specific pages. At the moment the stats package I have supplied by my host works for the top 20 pages but wont show me at a glance the bottom 20 out of several thousand

    What I want to do is to be able to search a .log file (which is already on the server) between 2 dates for a specific string (a url), count the number of results and display the results on an asp page.

    Can anybody point me to a tutorial or give me an overview of how this could work?

    I have a fair amount of experience in ASP, PHP, JS and SQL but I donít really know where to start at the moment.

    Thanks
  2. #2
  3. No Profile Picture
    Grumpier old Moderator
    Devshed Supreme Being (6500+ posts)

    Join Date
    Jun 2003
    Posts
    14,424
    Rep Power
    4539
    asp/vbscript doesn't have fancy file handling or file reading functions. And if the log files are compressed you'll have another problem. Basically you'll need to open the log file for reading, most likely using the filesystem object, then read the file from the top looking for your matching strings and doing whatever when you get a matching hit in a log file line.

    Log files tend to be very large, I doubt you'll be happy with the performance of your page. And depending on how your host is set up, you may not be able to access the log file from your asp code at all.

    If you don't need real time maybe download the log files and use a different software technology to scan and extract the data you need. The only reason you'd want to use asp to process the log file is that it's your only choice.
    ======
    Doug G
    ======
    The man who doesn't read good books has no advantage over the man who can't read them.
    --Mark Twain
  4. #3
  5. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2005
    Posts
    65
    Rep Power
    9
    Many thanks for the reply.

    I have had a play with the filesystem object and it will certainly do what I want it to do. Itís a interesting set of functions for ASP that I didnít know existed before and its been an interesting experience setting it up. But, as you say, the performance of scanning and counting a 70meg file makes this method more or less un-workable in practice.
  6. #4
  7. No Profile Picture
    Grumpier old Moderator
    Devshed Supreme Being (6500+ posts)

    Join Date
    Jun 2003
    Posts
    14,424
    Rep Power
    4539
    If it's your server and you have the ability to install COM objects, and if you have a programmer that can write com objects, you could perhaps create a component dll that does the heavy lifting work scanning the log and provide properties you can use by asp. But if you need portability with other servers custom objects can be a problem, they need to be installed on any target server along with all necessary dependencies.
    ======
    Doug G
    ======
    The man who doesn't read good books has no advantage over the man who can't read them.
    --Mark Twain

IMN logo majestic logo threadwatch logo seochat tools logo