#1
  1. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2012
    Posts
    86
    Rep Power
    3

    Problem printing to CSV file


    Hi,
    I have a problem printing to CSV file.
    I am pulling some data from a log to a csv file.
    I did it before but now Iím having a problem.
    My csv file looks like this:
    Code:
    BAK, HAND_ID,CZT  
    H_8_8_8
    ,888324
    ,czt
    H_17_17_17
    ,171711
    ,czt
    It supposed to look like this:
    Code:
    BAK, HAND_ID,CZT  
    H_8_8_8,888324,czt
    H_17_17_17,171711,czt
    Here is my code
    Code:
    #!/use/bin/perl -w
       use strict;
       use warnings ;
     
     
      my ($junk,$bakid,$hand_id,$czt) ;
      my $ErrorLogcsv = "C:/Error_Logs/Filtered/ErrorLog.csv" ; 
          open (my $ErrorLogcsv_fh,'>',$ErrorLogcsv)or die "Can't open $ErrorLogcsv $!" ; 
            print $ErrorLogcsv_fh "BAK, HAND_ID,CZT  \n";
            
      my $Err_Log  =  'C:/Error_Logs/Filtered/ErrorLog.txt';
          open (my $Err_Log_fh,'>',$Err_Log ) or die "Could not open $Err_Log $!" ;
      
      my $new_files = 'C:/Error_Logs/Filtered/22_Logs.log' ;
         open my $new_files_fh,'<', $new_files or die "CAN'T open $new_files $!\n" ;
        
        while ( my $line = <$new_files_fh> )
        {   
             if ($line =~ /BAK_ID/) {
                ($junk,$bakid,) = split/ /, $line ;
                       print $bakid ;
    	     }	    
             if ($line =~ /Hand_ID/) {
                ($junk,$hand_id) = split/ /, $line ;
                        print $hand_id ; 
    	     }
            if ($line =~ /TPL/) {
               ($junk,$czt) = split/ /, $line ;
                       print $czt ;   
                       print $ErrorLogcsv_fh "$bakid,$hand_id,czt\n" ;
    	     }	  	  
    	  }
      close $new_files_fh ;
    Thanks, testerV
  2. #2
  3. !~ /m$/
    Devshed Specialist (4000 - 4499 posts)

    Join Date
    May 2004
    Location
    Reno, NV
    Posts
    4,264
    Rep Power
    1810
    The problem is the use of global values which will not reset for each line of the file.

    Could help more, but need example input for the new_files data.
    Last edited by keath; November 22nd, 2012 at 10:59 AM.
  4. #3
  5. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    May 2007
    Posts
    765
    Rep Power
    929
    Lack of chomp($line) at the start of the while loop is suspicious when extra newlines are in the output.
    sub{*{$::{$_}}{CODE}==$_[0]&& print for(%:: )}->(\&Meh);
  6. #4
  7. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2012
    Posts
    86
    Rep Power
    3
    Thanks guys for looking in to it!
    Here is the Log I scan:

    Code:
    BAK_ID H_8_8_8
    Hand_ID 888324
    TPL CZT888
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    
    BAK_ID H_17_17_17
    Hand_ID 171711
    TPL CZT171717
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    
    BAK_ID C_102x_CS
    Hand_ID 171711
    TPL CZTx_CS
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    I added chomp $line to the code,
    global $vars to print not global any more.
    Now have error:
    Code:
    Global symbol "$bakid" requires explicit package name 
    Global symbol "$hand_id" requires explicit package name
    Here is the code:

    Code:
    #!/use/bin/perl -w
       use strict;
       use warnings ;
     
     
    #  my ($junk,$bakid,$hand_id,$czt) ;
      my $ErrorLogcsv = "C:/Error_Logs/Filtered/ErrorLog.csv" ; 
          open (my $ErrorLogcsv_fh,'>',$ErrorLogcsv)or die "Can't open $ErrorLogcsv $!" ; 
            print $ErrorLogcsv_fh "BAK, HAND_ID,CZT  \n";
            
      my $Err_Log  =  'C:/Error_Logs/Filtered/ErrorLog.txt';
          open (my $Err_Log_fh,'>',$Err_Log ) or die "Could not open $Err_Log $!" ;
      
      my $new_files = 'C:/Error_Logs/Filtered/22_Logs.log' ;
         open my $new_files_fh,'<', $new_files or die "CAN'T open $new_files $!\n" ;
        
        while ( my $line = <$new_files_fh> )  { 
                    chomp $line ; 
                    
             if ($line =~ /BAK_ID/) {
                  
              my ($junk,$bakid,) = split/ /, $line ;
                       print $bakid ;
                       next:
    	     }	    
             if ($line =~ /Hand_ID/) {
             
              my ($junk,$hand_id) = split/ /, $line ;
                        print $hand_id ; 
                        next ;
    	     }
            if ($line =~ /TPL/) {
                 
             my ($junk,$czt) = split/ /, $line ;
                       print $czt ;   
                       print $ErrorLogcsv_fh "$bakid,$hand_id,czt\n" ;
                       next ;
    	     }	  	  
    	  }
      close $new_files_fh ;
  8. #5
  9. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jun 2012
    Location
    Paris area, France
    Posts
    843
    Rep Power
    496
    You are defining $bakid in a too narrow scope, within the "if ($line =~ /BAK_ID/) {...}" block. When you are using it again in the "if ($line =~ /TPL/) {...}" block, you are no longer in the original scope and $bakid has fallen out of scope. Same thing for the $handid variable, it is no longer in scope when you are in the "if ($line =~ /TPL/) {...}" block.

    You need to declare these two variable one level up, at the level of the while block, with something like this:

    Perl Code:
    while ( my $line = <$new_files_fh> )  { 
        my ($bakid, hand_id);
        chomp $line ; 
        # (...)
         if ($line =~ /Hand_ID/) {
                (undef, $hand_id) = split/ /, $line ;
                print $hand_id ; 
                next ;
        }
        # (...)


    Same idea further down in the code for $hand_id.
  10. #6
  11. !~ /m$/
    Devshed Specialist (4000 - 4499 posts)

    Join Date
    May 2004
    Location
    Reno, NV
    Posts
    4,264
    Rep Power
    1810
    The reason I asked for a data sample is because of all of your 'if' statements. Is the data optional? Is it mandatory? Can we count on order?

    Based on what you provided, it appears you want the top three lines in a record, and that records are separated by an empty line. If that is true, one approach would be:

    Code:
    #!/usr/bin/perl
    use strict;
    use warnings;
    
    my $output = 'outfile.csv';
    open my $out_fh, ">", 'output.csv' or die "Can't open output: $!";
    
    $/ = "\n\n";
    
    while (<DATA>) {
      my ($bak_id,$hand_id,$czt) = $_ =~ /BAK_ID (\w+).Hand_ID (\d+).TPL (\w+)/s;
      print $out_fh "$bak_id,$hand_id,$czt\n";
    }
    
    __DATA__
    BAK_ID H_8_8_8
    Hand_ID 888324
    TPL CZT888
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    
    BAK_ID H_17_17_17
    Hand_ID 171711
    TPL CZT171717
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    
    BAK_ID C_102x_CS
    Hand_ID 171711
    TPL CZTx_CS
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    If Hand_ID does not always contain only digits, you will need to change the regex slightly to allow alphanumerics at that location.
  12. #7
  13. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2012
    Posts
    86
    Rep Power
    3
    Hi Laurent, Keath!
    Thanks for your help again!
    I added suggested by Laurent lines to the script

    Code:
     my ($bakid, $hand_id, $czt, undef,);
    The code still does not want to print to SCV log, my code looks like this now;

    Code:
      my $new_files = 'C:/Error_Logs/Filtered/22_Logs.log' ;
         open my $new_files_fh,'<', $new_files or die "CAN'T open $new_files $!\n" ;
        
        while ( my $line = <$new_files_fh> )  { 
                my ($bakid, $hand_id, $czt, $undef,);
                    chomp $line ; 
                    
             if ($line =~ /BAK_ID/) {              
             ($undef,$bakid,) = split/ /, $line ;
                       print $bakid, "\n" ;
                       next:
    	     }	    
             if ($line =~ /Hand_ID/) {
             ($undef,$hand_id) = split/ /, $line ;
                        print $hand_id, "\n"  ;
                        next ;
    	     }
            if ($line =~ /TPL/) {
                 ($undef,$czt) = split/ /, $line ;
                       print $czt,"\n"  ;
                       print $ErrorLogcsv_fh "$bakid,$hand_id,czt\n" ;
                       next ;
    	     }	
    	  }
      close $new_files_fh ;
    CSV output
    Code:
    BAK, HAND_ID,CZT  
    ,,czt
    ,,czt
    ,,czt
    Keath, here is the log structure, I'm interested in lines:
    BAK_ID H_8_8_8
    Hand_ID HAND888324
    and all lines with TPL CZT

    A log can have hundreds of lines "junk" between line "Hand_ID HAND888324" and line "TPL CZT...", also a log could have 0 to 20 lines of "TPL CZT...".

    A log:

    Code:
    BAK_ID H_8_8_8
    Hand_ID HAND888324
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    TPL CZT888 {0} 009-10 and so on
    TPL CZT1019 {01} H98-00 and so on
    TPL CZT0 {00} 000-10-0 and so on
    TPL CZT76 {0} HA8-A! and so on
  14. #8
  15. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jun 2012
    Location
    Paris area, France
    Posts
    843
    Rep Power
    496
    I've just tried your code with your data (just changing a couple of things). I obtain the following output:

    Code:
    H_8_8_8
    HAND888324
    CZT888
    ,,czt
    CZT1019
    ,,czt
    CZT0
    ,,czt
    CZT76
    ,,czt
    If you want the values on the same line, remove the "\n" from the "print $bakid," and "print $hand_id" statements.

    As for the "czt" lines, I still don't understand what you want to do with them. I also don't understand why you are printing them to a file, while you don't print the other $bakid and $hand_in lines to the standard output (the screen). I changed that to run your code.

    As an additional note, undef (without the '$' sigil) is a "predefined undefined" value, you don't need to declare and use $undef. "(undef, $bakid,) = split/ /, $line ;" will do the work, no need for an additional variable.
  16. #9
  17. !~ /m$/
    Devshed Specialist (4000 - 4499 posts)

    Join Date
    May 2004
    Location
    Reno, NV
    Posts
    4,264
    Rep Power
    1810
    An additional regex could be added to accommodate your new data:

    Code:
    $/ = "\n\n";
    
    while (<$in_fh>) {
      my ($bak_id,$hand_id) = $_ =~ /BAK_ID (\w+).Hand_ID (\w+)/s;
      my @tpl = $_ =~ /^TPL (CZT\w+)/mg;
    
      print $out_fh
        "$bak_id,$hand_id,",
        join(',', @tpl),
        "\n";
    }
    Another method would be to split the lines and examine the first field for the type you are interested in.
  18. #10
  19. !~ /m$/
    Devshed Specialist (4000 - 4499 posts)

    Join Date
    May 2004
    Location
    Reno, NV
    Posts
    4,264
    Rep Power
    1810
    Here's an example of splitting and using the first field to mark the data. The common and important thing in both examples is changing the record separator ($/) at the beginning of the script. Will make working with your data easier.

    Following that, you can decide what parsing method is more reliable.

    Code:
    #!/usr/bin/perl
    use strict;
    use warnings;
    
    my $output = 'outfile.csv';
    open my $out_fh, ">", 'output.csv' or die "Can't open output: $!";
    
    $/ = "\n\n";
    
    while (<DATA>) {
      my @lines = split /\n/;
    
      my %data;
    
      foreach my $line (@lines) {
        my ($type, @field) = split / /, $line;
        if ($type eq 'TPL') {
         push @{$data{TPL}}, $field[0];
        } else {
         $data{$type} = $field[0];
        }
      }
      
      print $out_fh
        "$data{BAK_ID},",
        "$data{Hand_ID},",
        join(',', @{$data{TPL}}),
        "\n";
    
    }
    
    __DATA__
    BAK_ID H_8_8_8
    Hand_ID 888324
    TPL CZT888
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    
    BAK_ID H_17_17_17
    Hand_ID 171711
    TPL CZT171717
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    
    BAK_ID C_102x_CS
    Hand_ID 171711
    TPL CZTx_CS
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    
    BAK_ID H_8_8_8
    Hand_ID HAND888324
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    TPL CZT888 {0} 009-10 and so on
    TPL CZT1019 {01} H98-00 and so on
    TPL CZT0 {00} 000-10-0 and so on
    TPL CZT76 {0} HA8-A! and so on
  20. #11
  21. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2012
    Posts
    86
    Rep Power
    3
    Hi Lauren!
    I have a feeling you have already hate me man!

    I simply want to scan a log:

    Code:
    BAK_ID H_8_8_8
    Hand_ID HAND888324
    TPL CZT888
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    
    BAK_ID H_9_9_9
    Hand_ID HAND999111
    TPL CZT999x0
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    
    BAK_ID H_17_17_17
    Hand_ID HAND_171711
    TPL CZT171717
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    Junk_lines Junk_lines Junk_lines Junk_lines Junk_lines
    I'd like to print three lines to CSV file.
    CSV file header:

    Code:
    BAK_ID,Hand_ID,TPL
    I want to print them in to SCV file in this order :

    Code:
    BAK, HAND_ID,CZT  
    H_8_8_8,HAND888324,CZT888
    H_9_9_9,HAND999111,CZT999x0
    H_17_17_17,HAND_171711,CZT171717
    I removed "\n", from all "print" statements.
    I want all "CZT" lines to be printed to CSV file after the split as is.
    I'd like to print $lines " ...print the other $bakid and $hand_in lines to the standard output (the screen)." to the standard output, to check if the script prints out what I heed and in the format I need but the main point is to print the lines to CSV file.
    I replaced "$undef" with "undef" in my script.

    Now I have errors in the print out screen :
    Code:
    Use of uninitialized value $bakid in concatenation (.) or string at  line 34, <$new_files_fh> .
    Use of uninitialized value $hand_id in concatenation (.) or string at  line 34, <$new_files_fh> 
    Use of uninitialized value $bakid in concatenation (.) or string at  line 34, <$new_files_fh> 
    Use of uninitialized value $hand_id in concatenation (.) or string at  line 34, <$new_files_fh> 
    Use of uninitialized value $bakid in concatenation (.) or string at  line 34, <$new_files_fh> 
    Use of uninitialized value $hand_id in concatenation (.) or string at  line 34, <$new_files_fh> 
    H_8_8_8HAND888324CZT888H_9_9_9HAND999111CZT999x0H_17_17_17HAND_171711CZT171717
    SCV file :
    Code:
    BAK, HAND_ID,CZT  
    ,,CZT888,
    ,,CZT999x0,
    ,,CZT171717,
    This is the script as I used it :
    Code:
    #!/use/bin/perl -w
       use strict;
       use warnings ;
    
    
     
      my $ErrorLogcsv = "C:/Error_Logs/Filtered/ErrorLog.csv" ; 
          open (my $ErrorLogcsv_fh,'>',$ErrorLogcsv)or die "Can't open $ErrorLogcsv $!" ; 
          print $ErrorLogcsv_fh "BAK, HAND_ID,CZT  \n";
            
      my $Err_Log  =  'C:/Error_Logs/Filtered/ErrorLog.txt';
          open (my $Err_Log_fh,'>',$Err_Log ) or die "Could not open $Err_Log $!" ;
      
      my $new_files = 'C:/Error_Logs/Filtered/22_Logs.log' ;
         open my $new_files_fh,'<', $new_files or die "CAN'T open $new_files $!\n" ;
        
        while ( my $line = <$new_files_fh> )  { 
                my ($bakid, $hand_id, $czt,);
                    chomp $line ; 
                    
             if ($line =~ /BAK_ID/) {              
             (undef,$bakid,) = split/ /, $line ;
                       print $bakid ;
                       next:
    	     }	    
             if ($line =~ /Hand_ID/) {
             (undef,$hand_id) = split/ /, $line ;
                        print $hand_id ;  
                        next ;
    	     }
            if ($line =~ /TPL/) {
                 (undef,$czt) = split/ /, $line ;
                       print $czt ;
                       print $ErrorLogcsv_fh "$bakid,$hand_id,$czt,\n" ;
                       next ;
    	     }
    	  }
       close $new_files_fh	;  
       close $ErrorLogcsv_fh ;
       exit ;
  22. #12
  23. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2012
    Posts
    86
    Rep Power
    3
    Finlay it is working! :
    It took just 4 (or 5?) days and all the help I could get from 3 experts.
    But as usual, it was all my fault.
    Thank you guys for all your help! I really appreciate it!!!
    testerV
    Here is the final code:
    Code:
    #!/use/bin/perl -w
       use strict;
       use warnings ;
    
     my ($bakid, $hand_id, $czt,);
     
      my $ErrorLogcsv = "C:/Error_Logs/Filtered/ErrorLog.csv" ; 
          open (my $ErrorLogcsv_fh,'>',$ErrorLogcsv)or die "Can't open $ErrorLogcsv $!" ; 
          print $ErrorLogcsv_fh "BAK, HAND_ID,CZT  \n";
            
      my $Err_Log  =  'C:/Error_Logs/Filtered/ErrorLog.txt';
          open (my $Err_Log_fh,'>',$Err_Log ) or die "Could not open $Err_Log $!" ;
      
      my $new_files = 'C:/Error_Logs/Filtered/22_Logs.log' ;
         open my $new_files_fh,'<', $new_files or die "CAN'T open $new_files $!\n" ;
        
        while ( my $line = <$new_files_fh> )  { 
            #    my ($bakid, $hand_id, $czt,);
                    chomp $line ; 
                    
             if ($line =~ /BAK_ID/) {              
             (undef,$bakid,) = split/ /, $line ;
                       next:
    	     }	    
             if ($line =~ /Hand_ID/) {
             (undef,$hand_id) = split/ /, $line ;
                        next ;
    	     }
            if ($line =~ /TPL/) {
                 (undef,$czt) = split/ /, $line ;
                     print "$bakid,$hand_id,czt\n" ;
                     print $ErrorLogcsv_fh "$hand_id,$bakid,$czt\n" ;
                       next ;
    	     }
    	  }
       close $new_files_fh	;  
       close $ErrorLogcsv_fh ;
       exit ;

IMN logo majestic logo threadwatch logo seochat tools logo