#1
  1. No Profile Picture
    d0g1e
    Guest
    Devshed Newbie (0 - 499 posts)
    How can I process every line in a large log file, starting from the bottom without, sucking it into an array? I tried to use seek but I can't get the proper result.

    Any help is appreciated!

    d0g1e -- d0g1e@cyberspace.org
  2. #2
  3. SwollenMember
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2000
    Location
    the master control
    Posts
    264
    Rep Power
    15
    define 'process'. using an array might not be a bad idea if you are reading in one line at a time then doing your logic, then re-initalize the array with the next line. it all depends on what you are trying to do.
  4. #3
  5. No Profile Picture
    d0g1e
    Guest
    Devshed Newbie (0 - 499 posts)
    By process i mean looking for looking for an IP adress in that line etc.

    But the problem with using arrays is ( i think) that the complete file gets stuffed into the memory, and since the log file is 60MB, there would be a lot of swapping.
  6. #4
  7. SwollenMember
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2000
    Location
    the master control
    Posts
    264
    Rep Power
    15
    no dude, you don't store the whole file into an array. rather you read one line at a time into the array. for example if i have a file and i want to read each line of it and store it into an array i can do the following. this is kind of a crude example of how to take the password file and store each section of each line in an array. but note that only one line of elements is 'stored' per pass. do whatever code you need within the while loop.

    #!/usr/local/bin/perl


    open(LOG, "/etc/passwd");

    while (<LOG> ){
    chomp($_);
    @r = split(/:/,$_);
    foreach $element (@r){
    print $element . "n";
    }
    }


  8. #5
  9. No Profile Picture
    d0g1e
    Guest
    Devshed Newbie (0 - 499 posts)
    Thanks, I used arrays the wrong way eg sucking the contents of the log into the array, then doing a while loop with the array and so on.

    Still one question, how do i start from the bottom of the file?
  10. #6
  11. SwollenMember
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2000
    Location
    the master control
    Posts
    264
    Rep Power
    15
    if this is for the apache log files, then use this line before the previous code:

    `cat access_log | sort -r > access_log.2`

  12. #7
  13. No Profile Picture
    d0g1e
    Guest
    Devshed Newbie (0 - 499 posts)
    Is reversing a logfile the only way to start reading from the bottom? I would think perl had some clever command to do this.
  14. #8
  15. SwollenMember
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2000
    Location
    the master control
    Posts
    264
    Rep Power
    15
    yes, you can reverse the values in the array by using the reverse function...

IMN logo majestic logo threadwatch logo seochat tools logo