#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2013
    Posts
    3
    Rep Power
    0

    Perl working with files - Perl Equivalent ?


    rimary Factory CTS 1.9.0(46) P1 *Slot 1 CTS 1.10.2(42) P1 Slot 2 CTS 1.9.0(46) P1 loads file information Factory (cmterm-CTS.1-9-0-46R-K9.P1) CTS: CTS.1-9-0-46R-K9.P1.sbn Touch: CTSDEV.1-9-0-46R-K9.P1.SPA Slot 1 (cmterm-CTS.1-9-0-46R-K9.P1) CTS: CTS.1-9-0-46R-K9.P1.sbn Touch: CTSDEV.1-9-0-46R-K9.P1.SPA Slot 2 (cmterm-CTS.1-9-0-46R-K9.P1) CTS: CTS.1-9-0-46R-K9.P1.sbn Touch: CTSDEV.1-9-0-46R-K9.P1.SPA
    Code:
    sed -n "/^left$/,/\*Slot/p" codecs | sed '/Factory/d' | tr '\012' ' '
    Result would be

    left *Slot 1 CTS 1.12.0(26) P1

    Im trying to move to perl and finding it very difficult to work with files. Any help would be appreciated.
  2. #2
  3. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2013
    Location
    /dev/null
    Posts
    163
    Rep Power
    19
    untested:
    Code:
    open FH, "< codecs";
    
    while (<FH>) {
        if (/^left$/../\*Slot/) {
            unless (/Factory/) {
                s/\012//g;
                print;
            }
        }
    }
    
    close(FH);
  4. #3
  5. No Profile Picture
    Contributing User
    Devshed Intermediate (1500 - 1999 posts)

    Join Date
    Apr 2009
    Posts
    1,941
    Rep Power
    1225
    Based on your posted sample data, you can't get that expected result.

    Is all of your data in a single line as your sample suggests, or is it split up on multiple lines like you piped sed command suggests?

    Please explain your parsing needs.
  6. #4
  7. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jun 2012
    Posts
    837
    Rep Power
    496
    Originally Posted by sumncguy

    Im trying to move to perl and finding it very difficult to work with files. Any help would be appreciated.
    This is really not difficult.

    You can use a real program and open the file in a traditional way, the way Noobie1000 has done, although I would suggest a better opening syntax and recommend to check if the program succeeded in opening the file, something like this:

    Perl Code:
    open my $FH, "<", "codecs" or die "Failed to open codecs $!";
    while (<$FH>) { # ...


    Or, for simple once-only actions you could use a Perl one-liner, where you don't even have to worry about opening and closing files and looping on the contents. For example, for what Noobie1000 has proposed, it could be rewritten as this command to be run from the Unix or Linux prompt:

    Code:
    perl -ne 'next if /Factory/; s/\012//g; print if /^left$/../\*Slot/;' codecs
    Under Windows, you need to replace the single quotes by double quotes.

    Having said that, I am not quite sure that this is exactly you need. You should really specify your requirement.
  8. #5
  9. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2013
    Posts
    3
    Rep Power
    0
    Sorry using the quote tags makes the output look ugly so.....

    The source file is actually larger. Its is a combination of a network device status commands.



    Code:
    admin@10.10.10.10's password: 
    Command Line Interface is starting up, please wait ...
    verifying connection to left (admin@10.11.112.13)... success
    verifying connection to right (admin@10.11.112.14)... success
    
       Welcome 
    Last login: Thu Aug  8 16:41:58 2013 from 10.11.112.15
    admin:show config phone
    
       Phone Config
    Phone IP                : 10.11.112.16
    Phone Connection        : Uplink
    admin:show config touch
    
       Touch Config
    admin:show version
    primary
       Factory   CTS 1.9.6(2) P1 
      *Slot 1    CTS 1.9.6(2) P1 
       Slot 2    CTS 1.7.1(4864) P1 
      loads file information
        Factory  (cmterm-CTS.1-9-6-2R-K9.P1)
            CTS: CTS.1-9-6-2R-K9.P1.sbn
          Touch: CTSDEV.1-9-6-2R-K9.P1.SPA
        Slot 1   (cmterm-CTS.1-9-6-2R-K9.P1)
            CTS: CTS.1-9-6-2R-K9.P1.sbn
          Touch: CTSDEV.1-9-6-2R-K9.P1.SPA
        Slot 2 
          No loads file
    left
       Factory   CTS 1.9.6(2) P1 
      *Slot 1    CTS 1.9.6(2) P1 
       Slot 2    CTS 1.7.1(4864) P1 
    right
       Factory   CTS 1.9.6(2) P1 
      *Slot 1    CTS 1.9.6(2) P1 
       Slot 2    CTS 1.7.1(4864) P1 
    admin:show status
    
    Host Name    : hostname
    Date         : Wed Nov 20, 2013 12:52:14
    Time Zone    : CET
    Locale       : en_US
    Product Ver  : CTS 1.9.6(2) P1
    System State : Run Normal
    Boot Slot    : 5
    Uptime: 12:52pm  up 94 days, 22:43, load average: 0.61, 0.53, 0.52
    
    Memory Total:        516180K
            Free:         50040K
            Used:        466140K
          Shared:             0K
         Buffers:        262224K
    
                            Total           Free           Used
    Disk/Ram               253871K         54075K        199796K (79%)
    Disk/Logging           107319K         71277K         30501K (30%)
    Disk/Factory            47104K          4387K         42717K (90%)
    Disk/Slot 1             47088K          4371K         42717K (90%)
    Disk/Slot 2             47088K          8705K         38383K (81%)
    admin:show hardware all
    Audio DSP Build ID                 : 01.09.0003
    Audio Base Board ID                : 0xAD
    Audio Base Board FAB Version       : 5
    Audio Base board FW Version        : 7
    Audio Extension Unit ID            : INFO:Audio/Video Expansion Unit is disconnected
    Audio Extension FAB Version        : INFO:Audio/Video Expansion Unit is disconnected
    Audio Extension FW Version         : INFO:Audio/Video Expansion Unit is disconnected
    Audio Clock Source                 : clock is Camera clock
    Audio PCB S/N                      : INFO:Audio Board Rev1
    Audio PCB P/N                      : INFO:Audio Board Rev1
    Audio PCB Rev                      : INFO:Audio Board Rev1
    Camera Center Product ID           : CTS-CAM-GEN1
    Camera Center Hardware             : 1.0
    Camera Center Firmware Version     : 1676:790
    Camera Center Hardware Version     : 4
    Camera Center Firmware Image Date  : Thu Jun 21 14:02:22 EDT 2012
    Mfg Installed Cert                 : CN=CTS-CODEC-PRIM-SEP001DA2389CA
    Locally Significant Cert           : CN=hostname,OU=telepresence,O=sap,C=DEE
    CF_Model                           : zyzuzyzu
    Display_Serial                     : xzyzyzy
    Display_Hardware_ver               : 1.3
    Display_Model                      : CTS-DISP-65-GEN2
    Display_BootCode_Ver               : 11.02
    Display_AppCode_Ver                : 11.0D
    Document Camera Serial Number      : INFO:Document camera is disconnected
    Document Camera Hardware Version   : INFO:Document camera is disconnected
    Document Camera Model              : INFO:Document camera is disconnected
    Video FPGA ID                      : 0xb0
    Video FPGA Revision                : 0x4
    Video FPGA Firmware Image Date     : UTC Fri Jun 10 01:27:31 2011
    Phone midlet status                : active
    Phone midlet version               : TSPM-1.9.1-P1-1S
    HDMI MainRx ID                     : INFO:/usr/local/bin/belSil is missing
    HDMI MainRx Revision               : INFO:/usr/local/bin/belSil is missing
    HDMI AuxRx ID                      : INFO:/usr/local/bin/belSil is missing
    HDMI AuxRx Revision                : INFO:/usr/local/bin/belSil is missing
    HDMI MainTx ID                     : INFO:/usr/local/bin/belSil is missing
    HDMI MainTx Revision               : INFO:/usr/local/bin/belSil is missing
    HDMI AuxTx ID                      : INFO:/usr/local/bin/belSil is missing
    HDMI AuxTx Revision                : INFO:/usr/local/bin/belSil is missing
    OS_Ver                             : CTS 1.9.6(2) P1
    OS_BuildTime                       : 2013-05-13 14:41:42
    OSD Device 1 Build Date            : 2011-4-19 7:54:34
    OSD Device 5 Build Date            : 2011-4-19 7:54:34
    POE Reset is not available
    Aux Control Unit Model             : INFO:ACU disconnected
    Aux Control Unit Version           : INFO:ACU disconnected
    Aux Control UDI Vid                : INFO:ACU disconnected
    Aux Control UDI Pid                : INFO:ACU disconnected
    Aux Control UDI Sn                 : INFO:ACU disconnected
    Presentation Display Model         : INFO: Could not communicate with the presentation display (Check serial cable and power)
    Max Security Setting               : Encrypted
    UBOOT Version                      : 0.0.36
    Hardware Version                   : 0600
    Serial Number                      : 
    Product ID                         : CTS-CODEC-PRIM 
    UI_Device_SW_Ver                   : No UI device is connected
    UI_Device_IP_Address               : No UI device is connected
    UI_Device_Mac_Address              : No UI device is connected
    System_Uptime                      : 94 days, 22:44
    Video encoder Hardware Version     : 0.1.3.0
    Video decoder Hardware Version     : 0.1.3.0
    admin:show upgrade detail
    There are no peripherals being upgraded at this time
    Command succeeded
    primary sub system...
    AutoUpgrade = true
    Peripheral versions:
                        device csum/ver & date     image csum/ver      
    camera               1676:790 (06/21/2012)           1676:790
    audio card               Rev1                            Rev1
    audio extension           N/A                             N/A
    auxiliary ctrl            N/A                             N/A
    graphics driver        0x9659 (04/19/2011)             0x9659
    graphics icons         0xc056 (04/19/2011)             0xc056
    display device          11.0D                           11.0D
    firmware             06/09/11 (06/09/2011)           06/09/11
    
    left sub system...
    There are no peripherals being upgraded at this time
    Command succeeded
    AutoUpgrade = true
    Peripheral versions:
                        device csum/ver & date     image csum/ver      
    camera               1676:790 (06/21/2012)           1676:790
    graphics driver        0x9659 (04/19/2011)             0x9659
    graphics icons         0xc056 (04/19/2011)             0xc056
    display device          11.0D                           11.0D
    firmware             06/09/11 (06/09/2011)           06/09/11
    
    right sub system...
    There are no peripherals being upgraded at this time
    Command succeeded
    AutoUpgrade = true
    Peripheral versions:
                        device csum/ver & date     image csum/ver      
    camera               1676:790 (06/21/2012)           1676:790
    graphics driver        0x9659 (04/19/2011)             0x9659
    graphics icons         0xc056 (04/19/2011)             0xc056
    display device          11.0D                           11.0D
    firmware             06/09/11 (06/09/2011)           06/09/11
    
    admin:
    The interesting piece is under the show status results. Shown below to simplify.

    Code:
    admin:show version
    primary
       Factory   CTS 1.9.6(2) P1 
      *Slot 1    CTS 1.9.6(2) P1 
       Slot 2    CTS 1.7.1(4864) P1 
      loads file information
        Factory  (cmterm-CTS.1-9-6-2R-K9.P1)
            CTS: CTS.1-9-6-2R-K9.P1.sbn
          Touch: CTSDEV.1-9-6-2R-K9.P1.SPA
        Slot 1   (cmterm-CTS.1-9-6-2R-K9.P1)
            CTS: CTS.1-9-6-2R-K9.P1.sbn
          Touch: CTSDEV.1-9-6-2R-K9.P1.SPA
        Slot 2 
          No loads file
    left
       Factory   CTS 1.9.6(2) P1 
       Slot 1    CTS 1.9.6(2) P1 
       *Slot 2    CTS 1.7.1(4864) P1 
    right
       Factory   CTS 1.9.6(2) P1 
      *Slot 1    CTS 1.9.6(2) P1 
       Slot 2    CTS 1.7.1(4864) P1
    What Im trying to arrive at is

    Code:
    primary *Slot 1    CTS 1.9.6(2) P1 
    left   *Slot 2    CTS 1.7.1(4864) P1 
    right *Slot 1    CTS 1.9.6(2) P1 
    and if touch exists 
    Touch: CTSDEV.1-9-6-2R-K9.P1.SPA
    So ..

    find "primary" grab the line showing the *Slotx
    find "left" grab the line showing *Slotx
    find "right" grab the line showing *Slotx

    I will also need to pull some other info out of the results of remaining show commands.

    I thought it might be a good idea to show how Im doing it with Ksh.


    Code:
    #!/bin/ksh
    for x in `ls *.log`
    do
    dos2unix $x 2>/dev/null
    echo "$x Parse show version"
    echo "------------------"
    p=`sed -n '/^primary$/,/\*Slot /p' $x | sed '/Factory/d' | tr '\012' ' '| awk '{print $1" "$5}'`
    l=`sed -n '/^left$/,/\*Slot /p'  $x | sed '/Factory/d' | tr '\012' ' ' | awk '{print $1" "$5}'`
    r=`sed -n '/^right$/,/\*Slot /p' $x | sed '/Factory/d' | tr '\012' ' ' | awk '{print $1" "$5}'`
    t=`grep "Touch:" $x | head -1 | sed 's/  */ /g'`
    echo "$p : $l : $r : $t"
    echo " "
    echo "Parse show status"
    echo "-----------------"
    s=`grep "System State : " $x`
    upt=`grep "System_Uptime" $x` 
    prod=`grep "Product Ver " $x` 
    echo "$s"
    echo "$upt"
    echo "$prod"
    echo " "
    echo "Parse show hardware"
    echo "-------------------"
    mid=`grep "Phone midlet status" $x`
    midver=`grep "Phone midlet version" $x`
    echo "$mid"
    echo "$midver"
    echo " "
    done





    The results :
    10.97.188.92.log Parse show version
    ------------------
    primary 1.9.6(2) : left 1.9.6(2) : right 1.9.6(2) : Touch: CTSDEV.1-9-6-2R-K9.P1.SPA

    Parse show status
    -----------------
    System State : Run Normal
    System_Uptime : 94 days, 22:44
    Product Ver : CTS 1.9.6(2) P1

    Parse show hardware
    -------------------
    Phone midlet status : active
    Phone midlet version : TSPM-1.9.1-P1-1S
  10. #6
  11. No Profile Picture
    Contributing User
    Devshed Intermediate (1500 - 1999 posts)

    Join Date
    Apr 2009
    Posts
    1,941
    Rep Power
    1225
    There are multiple ways to do this in Perl. My approach preference would be to create a separate subroutine for each of the sections that need to be parsed and use a dispatch table to execute them as each section is seen.

    I don't have time right now to complete it, but here's my initial script.

    Code:
    #!/usr/bin/perl
    
    use strict;
    use warnings;
    use Data::Dumper;
    
    my %dispatch = (
        version   => \&parse_version,
        status    => \&parse_status,
        hardware  => \&parse_hardware,
        error     => \&error,
    );
    
    
    $/ = "\nadmin:show ";
    
    my $stats = 'status.txt';
    open my $stats_fh, '<', $stats or die "failed to open '$stats' <$!>";
    
    while (<$stats_fh>) {
        next unless /^(version|status|hardware)\b/;
        chomp;
        my ($section) = /^([a-z]+)\b/;
        my $code_ref = $dispatch{$section} || $dispatch{error};
        $code_ref->($_);
    }
    
    # end of main body of script
    
    # below are the subroutine definitions
    sub parse_version {
        my $version_info = shift;
        my @info;
        
        push @info, "$1 $2" if $version_info =~ /(primary).+?(\*Slot[^\n]+)/ms;
        push @info, "$1 $2" if $version_info =~ /(left).+?(\*Slot[^\n]+)/ms;
        push @info, "$1 $2" if $version_info =~ /(right).+?(\*Slot[^\n]+)/ms;
        push @info, "$1 $2" if $version_info =~ /(Touch:)\s+([^\n]+)/;
        
        print "Parse show version\n", '-' x 20, "\n";
        print join(' : ', @info), "\n\n";
    }
    
    sub parse_status {
        my $status_info = shift;
        my @status;
        
        print "Parse show status\n", '-' x 20, "\n";
        print join(' : ', @status), "\n\n";
    }
    
    sub parse_hardware {
        my $hardware_info = shift;
        my @hardware;
        
        print "Parse show hardware\n", '-' x 20, "\n";
        print join(' : ', @hardware), "\n\n";
    
    }
    
    sub error {
        #print Dumper \@_;
    }
    Using your sample data, here's what it outputs.
    Parse show version
    --------------------
    primary *Slot 1 CTS 1.9.6(1) P1 : left *Slot 1 CTS 1.9.6(2) P1 : right *Slot 1 CTS 1.9.6(3) P1 : Touch: CTSDEV.1-9-6-2R-K9.P1.SPA

    Parse show status
    --------------------


    Parse show hardware
    --------------------
    You have a discrepancy in what you show as what is needed to be extracted so I'm not sure which version you need.
    Last edited by FishMonger; November 20th, 2013 at 10:32 AM.
  12. #7
  13. No Profile Picture
    Contributing User
    Devshed Intermediate (1500 - 1999 posts)

    Join Date
    Apr 2009
    Posts
    1,941
    Rep Power
    1225
    Here's the completed script.
    Code:
    #!/usr/bin/perl
    
    use strict;
    use warnings;
    
    my %dispatch = (
        version   => \&parse_version,
        status    => \&parse_status,
        hardware  => \&parse_hardware,
    );
    
    foreach my $logfile ( glob "*.log" ) {   
        
        print "$logfile\n";
        
        local $/ = "\nadmin:show ";
        
        open my $fh, '<', $logfile or die "failed to open '$logfile' <$!>";
        
        while (<$fh>) {
            next unless /^(version|status|hardware)\b/;
            chomp;
            my ($section) = /^([a-z]+)\b/;
            print "Parse show $section\n", '-' x 20, "\n";
            
            my $code_ref = $dispatch{$section};
            $code_ref->($_);
        }
        print "\n\n";
        close $fh;
    }
    # end of main body of script
    
    # below are the subroutine definitions
    sub parse_version {
        my $version_info = shift;
        my @info;
        
        push @info, "$1 $2" if $version_info =~ /(primary).+?(\*Slot[^\n]+)/ms;
        push @info, "$1 $2" if $version_info =~ /(left).+?(\*Slot[^\n]+)/ms;
        push @info, "$1 $2" if $version_info =~ /(right).+?(\*Slot[^\n]+)/ms;
        push @info, "$1 $2" if $version_info =~ /(Touch:)\s+([^\n]+)/;
        
        print join(' : ', @info), "\n\n\n";
    }
    
    sub parse_status {
        my $status_info = shift;
        
        print "$1\n" if $status_info =~ /(System State : [^\n]+)/;
        print "System_Uptime : $1\n" if $status_info =~ /Uptime:.+?up (.+?), load/;
        print "$1\n" if $status_info =~ /(Product Ver  : [^\n]+)/;
        print "\n\n";
    }
    
    sub parse_hardware {
        my $hardware_info = shift;
        
        print "$1 $2" if $hardware_info =~ /(Phone midlet status)\s+(.+?\n)/;
        print "$1 $2" if $hardware_info =~ /(Phone midlet version)\s+(.+?\n)/;
    }
    Using the dispatch table approach gives you the ability to easily add/remove/change parsing sections without a major rewrite. All you need to do is adjust the regex on line 21 to filter the wanted "show" sections and then add or change the appropriate subroutine.

    Comments on this post

    • sumncguy agrees : not only did Fish help, he wrote the script for me !!
    Last edited by FishMonger; November 20th, 2013 at 11:52 AM.
  14. #8
  15. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2013
    Posts
    3
    Rep Power
    0
    Well thanks alot Fish .. wasnt really looking for you to do the work for me .. just looking for a few pointers on how to get started.

    Yeah .. couple of errors in my shell version .. but just through together quickly as an example.

    Im going to grab your script and try to reverse engineer it .. dont like to just cut and paste. I like to understand what Im using.

    Thanks again man !!
  16. #9
  17. No Profile Picture
    Contributing User
    Devshed Intermediate (1500 - 1999 posts)

    Join Date
    Apr 2009
    Posts
    1,941
    Rep Power
    1225
    I wasn't planing on writing the whole thing, but I was needing a break from my own project and wanted to demonstrate the power and flexibility of dispatch tables.

IMN logo majestic logo threadwatch logo seochat tools logo