#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Feb 2014
    Location
    California
    Posts
    1
    Rep Power
    0

    Cool Basic file open read/write syntax?


    I'm a beginning perl programmer trying to revamp an existing program after our apache server updated their security read/write permissions. The guy that wrote the program is an experienced programmer (retired from our office) so I'm trying to make as few changes as possible to avoid doing any harm.

    The line that's giving me grief is:
    open(INFO, "awk -F, '\$2==\"$stn\" {print \$0}' $infofile |");

    which (I think) errors on open because perl defaults to open with r+w permissions, and I'm calling a dir with only read access.

    How to fix this syntax to open the file with read only access?

    I tried
    open(<INFO, "awk -F, '\$2==\"$stn\" {print \$0}' $infofile |");
    which returns a 'missing closing brackets <>' error, and

    open(INFO, "<", "awk -F, '\$2==\"$stn\" {print \$0}' $infofile |");
    which returns no errors but doesn't read the file....

    Any ideas? Thanks!
  2. #2
  3. No Profile Picture
    Contributing User
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Apr 2009
    Posts
    2,233
    Rep Power
    1298
    That "guy" might be "a good programmer" in shell scripting, certainly not on Perl programming.

    Perl does not default to opening a file in read/write mode. It's default is read mode.

    I'm not an awk programmer, so I can't say exactly what that portion of the statement is doing other than it (awk) is reading the file and the Perl open statement is opening a pipe to capture that output.

    It would be much cleaner and more efficient to leave awk out of the process and let perl do its work.

    Code:
    open (INFO, '<', $infofile) or die "failed to open '$infofile' <$!>;
    or use a lexical var for the filehandle, which is the preferred method.
    Code:
    open (my $INFO, '<', $infofile) or die "failed to open '$infofile' <$!>;
    If you choose to keep the awk statement, then at the very least you need to add error checking on the open call; that's the or die ... portion of the statement I used.
    Last edited by FishMonger; February 28th, 2014 at 04:10 PM.
  4. #3
  5. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jun 2012
    Location
    Paris area, France
    Posts
    846
    Rep Power
    500
    In fact, once you start being conversant with Perl, you almost never ever feel the need to use awk again (even though awk is a nice programming tool, I have used it quite a bit in the late 1990s and very early 2000s, I will not deny that). But Perl can do absolutely everything that awk can do, and Perl usually does it better, with more possibilities, and most of the times quicker. Some people knowing Perl might still occasionally use awk in a shell script, because the syntax is sometimes slightly more concise, but I have personally stopped writing awk scripts ever since I gathered enough command of Perl to be able to do it better in Perl. The only case when I still use awk is when I have to modify or maintain an existing program using awk, I don't think that I created an awk script in the last ten or eleven years.

    But the point that I wanted to get at is that I can understand Perl programmers using awk in a shell script (even though I would not do it). But calling an awk script from Perl, I really don't understand, this is ridiculous.

    Having said all that, let me try to decipher your awk script (despite the fact that I haven't done anything in awk in the last ten years).

    The -F option says that the comma is the field separator. Next, if the second field is equal to $stn, then the whole line is supposed to be printed and apparently serves as input for the file opening action. There, I don't really get it. It would be useful if you could show a section of the input file.

    Comments on this post

    • Azumi93 agrees : I love Perl too!
  6. #4
  7. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2013
    Location
    /dev/null
    Posts
    170
    Rep Power
    23
    Originally Posted by Laurent_R
    But Perl can do absolutely everything that awk can do, and Perl usually does it better, with more possibilities, and most of the times quicker.
    I agree with you over the merits of perl, and I am a big fan of perl myself. Especially, when I want to make my collegues' life miserable by writing cryptic programs. But I wouldn't undermine the power of awk. awk was developed for text processing and it is much quicker than perl in doing so. Initially, over the course of my work, I would cook perl one-liners for everything (since I didn't know awk back then). But as I explored awk and saw that it could process and fetch required data from GiB's of my log files quickly, I gradually minimized using perl (for such tasks), and most of the times a simple awk one-liner would suffice.

    Here's a quick test I ran on my Linux server (with SUSE 11 Enterprise x86_64 on a 24 processor machine with 48 GiB of total memory):

    Code:
    [user@host ~]$ seq 30000000 > output
    [user@host ~]$ time awk '/5$/ {print}' output > output1
    awk '/7$/ {print}' output > output1  4.71s user 0.14s system 84% cpu 5.756 total
    [user@host ~]$ rm output1
    [user@host ~]$ time perl -ne '/5$/ && print' output > output1
    perl -ne '/7$/ && print' output > output1  6.92s user 0.09s system 95% cpu 7.360 total
    Yes, I agree that after experiencing heavy doses of perl, one is so blinded by it's sheer power that he tends to achieve everything in perl (too bad it can't make my coffee :-P, or wait.. has someone written a module for that? Let me check CPAN :-D).
  8. #5
  9. No Profile Picture
    Contributing User
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Apr 2009
    Posts
    2,233
    Rep Power
    1298
    Code:
    [root@099-91-RKB-2 ~]# seq 30000000 > output
    [root@099-91-RKB-2 ~]# time awk '/5$/ {print}' output > output1
    
    real    0m38.382s
    user    0m36.864s
    sys     0m0.529s
    [root@099-91-RKB-2 ~]# rm output1
    [root@099-91-RKB-2 ~]# time perl -ne '/5$/ && print' output > output1
    
    real    0m14.197s
    user    0m12.514s
    sys     0m0.535s
    Using your test, which does not reflect what the OP's code is doing, I prefer the numbers I get from the perl execution.
  10. #6
  11. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Jun 2012
    Location
    Paris area, France
    Posts
    846
    Rep Power
    500
    The good thing yout Perl, noobie1000, is that, not having the seq command on an AIX server, I could generate a file similar to yours using Perl:

    Code:
    perl -e 'print "$_\n" for 1..30000000' > out1
    Now, an important point is that these comparisons depends on the particular implementations of awk

    Now running the comparison on the AIX server:

    Code:
    $time awk '/5$/ {print}' out1 > out2
    
    real    0m11.189s
    user    0m6.930s
    sys     0m0.137s
    
    $time perl -ne '/5$/ && print' out1 > out3
    
    real    0m8.457s
    user    0m5.127s
    sys     0m0.145s
    Same test on a HP-UX server (with a somewhat smaller file, as the server is less powerful:
    Code:
    $ time awk '/5$/ {print}' out1 > out2
    
    real    0m5.98s
    user    0m5.92s
    sys     0m0.03s
    
    $ time perl -ne '/5$/ && print' out1 > out3
    
    real    0m4.64s
    user    0m4.56s
    sys     0m0.04s
    But I have also seen cases where awk would run faster than Perl, at least on simple regexes, less so on more complicated ones, as I said it is really a matter of implementation.

    But my real point on my previous post was that, while awk is a nice and useful little language, calling an awk script from a Perl program really does not make much sens IMHO.

    Comments on this post

    • noobie1000 agrees : Indeed, calling awk from perl doesn't make sense!

IMN logo majestic logo threadwatch logo seochat tools logo