#1
  1. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2012
    Posts
    86
    Rep Power
    3

    Looking for a specific line in each log file


    Greetings!
    I hope everybody will have good holidays!
    Anyway, I知 scanning a directory for new logs and if found any I知 opening a log and looking for a specific line with the word - Fatal Error.
    If a log has the line I値l scan it for additional lines if a log does not have the specific line "Fatal Error" then start scanning next new log.
    Here is my code, each part of the code works separately but not all together.
    Thanks, testerV
    Code:
    #!/use/bin/perl -w
       use strict;
       use warnings ;
    
                  my $textFile = "C:\\logs_E\\\ErrorLog.txt" ; 
            open (my $textFile_fh,'>',$textFile) or die"Can't open $textFile" ;
    
                  my $dir = 'C:\\Error_Logs';
        opendir(DIR, $dir) or die "Can't open directory $dir";
       
       while ($file = readdir(DIR)) { 
          next unless (-f "$dir/$file");
          next unless ($file =~ m/\.log$/);
                    if (-M $file < 1) { 
                  open (my $file_fh,'<',$file) or die "Can't open $file";      
            while ($line =<$file_fh>)  {          
                    chomp ($line); 
                next until $line =~ /FatalError/ ;
            while ($line =<$file_fh>) {
                    chomp ($line);	   
    	   if ($line =~ /FatalError/) {  
    	      print $textFile_fh "$line \n"; 
                    }
    	 if ($line =~ /Step_1/) {
    	      print "$line\n" ; 
    	       print $textFile_fh "$line\n" ;
                   }                
    	 if ($line =~ /Step_2/) {
    	      print "$line\n" ; 
    	       print $textFile_fh "$line\n" ;
                   } 
    	 if ($line =~ /End_track/) {
    	      print "$line\n" ; 
    	       print $textFile_fh "$line\n" ;
                   }               
                      }                       
                    }
                  }           
                }   
       exit ;
  2. #2
  3. Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Apr 2012
    Location
    spaceBAR Central
    Posts
    229
    Rep Power
    42
    See if the below code works for you:
    Code:
    my $dir = 'C:\\Error_Logs';
    my $file;
    my $found;
    my $line;
    my $log_lines_file  =  'log_lines_file.txt';
    
    open ( LOG_LINES, "+>>$log_lines_file" ) or die("***Error: Could not open log file($log_lines_file).\n");
    
    opendir( DIR, $dir ) or die "Can't open directory $dir";
    while ( $file = readdir( DIR ) ) {
      next unless ( -f "$dir/$file" );
      next unless ( "$dir/$file" =~ m/\.log$/ );
      if ( -M "$dir/$file" < 1 ) {
        $found = ( system( "grep -q FatalError $dir/$file" ) ) ? 0 : 1;
        # If file contains "FatalError", print specific lines from file.
        if ( $found != 0 ) {
          open( FILE, "<$dir/$file" ) or die "Can't open $file";
          while ( $line = <FILE> ) {
            if ( $line =~ /FatalError|Step_1|Step_2|End_track/ ) {
              print "$line";
              print LOG_LINES "$line" ;
            }
          }
          close FILE;
        }
      }
    }
    close LOG_LINES;
    close DIR;
  4. #3
  5. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2012
    Posts
    86
    Rep Power
    3
    Spacebar, thank you for your time!

    I got error :
    'grep' is not recognized as an internal or external command,
    operable program or batch file.

    I forgot to tell I'm using WXP box.

    Thanks again!
    testerV
  6. #4
  7. Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Apr 2012
    Location
    spaceBAR Central
    Posts
    229
    Rep Power
    42
    oops sorry, I use 'cygwin', You can try using the windows 'findstr' command instead:
    Code:
    use strict;
    use warnings;
    
    my $dir = 'C:\\Error_Logs';
    my $file;
    my $found;
    my $line;
    my $log_lines_file  =  'log_lines_file.txt';
    
    open ( LOG_LINES, "+>>$log_lines_file" ) or die("***Error: Could not open log file($log_lines_file).\n");
    
    opendir( DIR, $dir ) or die "Can't open directory $dir";
    while ( $file = readdir( DIR ) ) {
      next unless ( -f "$dir/$file" );
      next unless ( "$dir/$file" =~ m/\.log$/ );
      if ( -M "$dir/$file" < 1 ) {
        $found = ( system( "findstr /M FatalError $dir/$file" ) );
        # If file contains "FatalError", print specific lines from file.
        if ( $found = "$dir/$file" ) {
          open( FILE, "<$dir/$file" ) or die "Can't open $file";
          while ( $line = <FILE> ) {
            if ( $line =~ /FatalError|Step_1|Step_2|End_track/ ) {
              print "$line";
              print LOG_LINES "$line" ;
            }
          }
          close FILE;
        }
      }
    }
    close LOG_LINES;
    close DIR;
  8. #5
  9. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2012
    Posts
    86
    Rep Power
    3
    Seems it is working.I just have no idea what this part of code means:
    ( system( "findstr /M FatalError $dir/$file" ) );
    Thanks man!
    testerV
  10. #6
  11. Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Apr 2012
    Location
    spaceBAR Central
    Posts
    229
    Rep Power
    42
    do a "findstr /?" from a command prompt and look at the '/M' option.
  12. #7
  13. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2012
    Posts
    86
    Rep Power
    3
    Is it possible to grep files for a string?
    Thanks, testerV


    Code:
    #!/use/bin/perl -w
       use strict;
       use warnings ;
    
                  my $textFile = "C:\\logs_E\\\ErrorLog.txt" ; 
            open (my $textFile_fh,'>',$textFile) or die"Can't open $textFile" ;
    
                  my $dir = 'C:\\Error_Logs';
        opendir(DIR, $dir) or die "Can't open directory $dir";
       
       while ($file = readdir(DIR)) { 
          next unless (-f "$dir/$file");
          next unless ($file =~ m/\.log$/);
                    if (-M $file < 1) { 
                    my $found = (grep /FatalError/ $file) ;
                    print "$found\n" ;
                 }
                }
                exit ;
  14. #8
  15. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jun 2012
    Location
    Paris area, France
    Posts
    843
    Rep Power
    496
    Perl's grep internal command looks for a regex within the elements of an array or a list, don't mix it with the Unix grep command which scans a file for matching lines. This is entirely different.
    Last edited by Laurent_R; November 21st, 2012 at 04:08 AM.
  16. #9
  17. No Profile Picture
    Contributing User
    Devshed Intermediate (1500 - 1999 posts)

    Join Date
    Apr 2009
    Posts
    1,968
    Rep Power
    1225
    Vars should be declared in the smallest scope that they require and close to where they are first needed.

    Don't quote "$vars"
    See: perldoc -q What's wrong with always quoting "$vars"

    Instead of having to escape the backslashes in the paths, it would be better/cleaner to use forward slashes.

    When opening file and dir handles, it's best practice to use a lexical var instead of a bareword for the handle.

    In the case of filehandles, it's best practice to use the 3 arg form of open.

    The die statement on failed open calls should include $! which gives the OS reason why it failed.

    Opening a file in read/write mode is rarely needed or appropriate.

    Proper use of vertical and horizontal whitespace and indentation is very important. It makes the code much easier to read, understand its flow, and maintain.

    Here's how I might write this script. (untested)
    Code:
    #!/usr/bin/perl
    
    use strict;
    use warnings;
    
    my $dir        = 'C:/Logs_E';
    my @logfile    = grep( -M $_ < 1, <$dir/*.log>);
    my $error_log  = "$dir/ErrorLog.txt";
    
    open my $error_fh, '>', $error_log or die "can't write to '$error_log' $!";
    
    FILE:
    foreach my $file ( @logfile ) {
    
        open my $fh, '<', $file or do {
            warn "failed to open/read '$file' $!\n";
            next FILE;
        };
    
        while ( my $line = <$fh> ) {
            
            if ( $line =~ /FatalError/ .. eof $fh) {
                if ( $line =~ /FatalError|Step_1|Step_2|End_Track/ ) {
                    print $line;
                    print {$error_fh} $line;
                }
            }
        }
        close $fh;
    }
    
    close $error_log;
    Last edited by FishMonger; November 21st, 2012 at 11:14 AM.
  18. #10
  19. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2012
    Posts
    86
    Rep Power
    3
    Thanks you Spacebar, Lauren and Fish for the code!
    Special thanks for the coaching, it is very, very much appreciated guys!
    Just few more questions, please answer if possible.
    About 船on't quote "$vars". It works for a single $vasr only. Right?
    For multiple $vars one has to use quotes, multiple $vars will be reference as a string. Right?
    What do you mean 前pening a file in read/write mode is rarely needed or appropriate.?
    Thank you guys again, testerV
  20. #11
  21. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jun 2012
    Location
    Paris area, France
    Posts
    843
    Rep Power
    496
    It is just useless to quote vars like here:

    Code:
              print "$line\n";
              print LOG_LINES "$line" ;
    Don't get into the habit to do that, while you can have:

    Perl Code:
              print $line, "$_";
              print LOG_LINES $line ;


    Even something like that:

    Code:
    print "$foo $bar $baz \n";
    can be written:

    Code:
    print join " ", $foo, $bar, $baz, "\n";
    Of, course, there are cases where it makes sense to use and interpolate variables within a string:

    Code:
    print "The current temperature is: $temperature. Is this warm enough? \n";
    You could still print it as a list as before (or use the concatenation operator), but here you are really printing a string, a sentence, it is OK to include a variable in it. However, putting a single variable within a string to print it is useless and counterproductive (and slower).

IMN logo majestic logo threadwatch logo seochat tools logo