some looping problems with a perl script...

Hi,

I'm trying to create a perl script to convert a couple of hundred modules, each containing lines of ipchains commands, to be used with iptables. I used to have just one big loop with a number of loops inside it to do this, but it got *slightly* complicated, and I decided to split it up into a function that converts the modules, and then one that replaces the old modules with the new modules.

However! I've come across a problem... When I run the script, I find that the script seems to loop the converting stage of the script. It does this for an uncountable number of times (I CAN post the output, if anyone wants... it makes patterns! ), and then moves onto the onto the next module in the directory. I have no idea why it does this!

Here is the script:

#!/usr/bin/perl -w

sub convert {
opendir(BASE_DIR, "/etc/firewall/modules/") or die "Can't open directory: $!";
@modules_base_dir = grep !/^..?z/, readdir BASE_DIR;
close(BASE_DIR);
foreach $priv_pub (@modules_base_dir) {
print "$priv_pubn";
print "-" x 3, ">";
print "n";
opendir(INTERFACE_DIR, "/etc/firewall/modules/$priv_pub") or die "Can't open directory: $!";
@modules_interface_dir = grep !/^..?z/, readdir INTERFACE_DIR;
close(INTERFACE_DIR);
foreach $module_type (@modules_interface_dir) {
print " $module_typen";
print " " , "=" , ">";
print "n";
opendir(FUNCTION_DIR, "/etc/firewall/modules/$priv_pub/$module_type") or die "Can't open directory: $!";
@modules_function_dir = grep !/.tmp$/, readdir FUNCTION_DIR;
close(FUNCTION_DIR);
foreach $module (@modules_function_dir) {
my $output = $module . ".tmp";
open(INFILE,"/etc/firewall/modules/$priv_pub/$module_type/$module") or die "Can't open module: $!n";
open(OUTFILE, ">/etc/firewall/modules/$priv_pub/$module_type/$output") or die "Can't open temp file: $!n";
my $converted = <INFILE>;
while (<INFILE> ) {
print " Converting $module (writing to $output)...";
$converted =~ s/ipchains/iptables/g;
print OUTFILE "$converted";
$converted = <INFILE>;
}
print "Done!nn" if -e "/etc/firewall/modules/$priv_pub/$module_type/$output";
}
}
}
}

sub move {
opendir(MOVE_BASE_DIR, "/etc/firewall/modules/") or die "Can't open directory: $!";
@move_modules_base_dir = grep /.tmp$/, readdir MOVE_BASE_DIR;
close(MOVE_BASE_DIR);
foreach $priv_pub (@move_modules_base_dir) {
print "$priv_pubn";
print "-" x 3, ">";
print "n";
opendir(MOVE_INTERFACE_DIR, "/etc/firewall/modules/$priv_pub") or die "Can't open directory: $!";
@move_modules_interface_dir = grep /.tmp$/, readdir MOVE_INTERFACE_DIR;
close(MOVE_INTERFACE_DIR);
foreach $module_type (@move_modules_interface_dir) {
print " $module_typen";
print " " , "=" , ">";
print "n";
opendir(MOVE_FUNCTION_DIR, "/etc/firewall/modules/$priv_pub/$module_type") or die "Can't open directory: $!";
@move_modules_function_dir = grep /.tmp$/, readdir MOVE_FUNCTION_DIR;
close(MOVE_FUNCTION_DIR);
foreach $module (@move_modules_function_dir) {
my $old_file = $module;
$old_file =~ s/.tmp//;
rename("/etc/firewall/modules/$priv_pub/$module_type/$module", "/etc/firewall/modules/$priv_pub/$module_type/$old_file");
print "Moved $module to $old_filen" if -e "/etc/firewall/modules/$priv_pub/$module_type/$old_file";
}
}
}
}

convert();
move();

---

If anyone could provide any hints or point out flaws, that would be great! I am stumped, and I cannot see any errors, even when I compare the syntax in the script to the syntaxes of the functions O'Reilly book that I have.

Thanks!!!

<BLOCKQUOTE><font size="1" face="Verdana,Arial,Helvetica">code:</font><HR><pre>
my $converted = <INFILE>;
while (<INFILE> ) {
print " Converting $module (writing to $output)...";
$converted =~ s/ipchains/iptables/g;
print OUTFILE "$converted";
$converted = <INFILE>;
}
[/code]

Why not just do this:

<BLOCKQUOTE><font size="1" face="Verdana,Arial,Helvetica">code:</font><HR><pre>
while ($converted = <INFILE> ) {
print " Converting $module (writing to $output)...";
$converted =~ s/ipchains/iptables/g;
print OUTFILE "$converted";
}
[/code]



[This message has been edited by vpopper (edited December 28, 2000).]

<BLOCKQUOTE><font size="1" face="Verdana,Arial,Helvetica">quote:</font><HR>Originally posted by vpopper:
<BLOCKQUOTE><font size="1" face="Verdana,Arial,Helvetica">code:</font><HR><pre>
my $converted = <INFILE>;
while (<INFILE> ) {
print " Converting $module (writing to $output)...";
$converted =~ s/ipchains/iptables/g;
print OUTFILE "$converted";
$converted = <INFILE>;
}
[/code]

Why not just do this:

<BLOCKQUOTE><font size="1" face="Verdana,Arial,Helvetica">code:</font><HR><pre>
while ($converted = <INFILE> ) {
print " Converting $module (writing to $output)...";
$converted =~ s/ipchains/iptables/g;
print OUTFILE "$converted";
}
[/code][/quote]

wouldn't I have to do "my $converted; $converted = <INFILE>;" beforehand though?

I put the while(...) in there to check that <INFILE> was still valid, so it didn't just convert nothing.

thanks

<BLOCKQUOTE><font size="1" face="Verdana,Arial,Helvetica">quote:</font><HR>Originally posted by nebuchadnezzar:
wouldn't I have to do "my $converted; $converted = <INFILE>;" beforehand though?

I put the while(...) in there to check that <INFILE> was still valid, so it didn't just convert nothing.[/quote]

You can make sure you have something to convert by using a regular expression:

<BLOCKQUOTE><font size="1" face="Verdana,Arial,Helvetica">code:</font><HR><pre>
while ($converted = <INFILE> ) {
next unless $converted =~ m{S+};
[/code]

or use a regexp that matches what you're looking for...

Something like this?:

<BLOCKQUOTE><font size="1" face="Verdana,Arial,Helvetica">code:</font><HR><pre>my $converted = <INFILE>;
while ($converted =~ /ipchains/) {
print " Converting $module (writing to $output)...";
$converted =~ s/ipchains/iptables/g;
print OUTFILE "$converted";
} [/code]

If I understand your code correctly, you're looping through lines in the file, converting the line, then writing to another file.

If this is true, the reason that your code is looping uncontrollably is because:

This line reads a line from the infile and stores it in $converted

my $converted = <INFILE>;

This line also reads a line from the infile, but stores it in $_

while (<INFILE> ) {

...and so on.

What you want to do is loop through the lines in the file by reading the line, coverting it, then printing it out. Here's a shortcut:

<BLOCKQUOTE><font size="1" face="Verdana,Arial,Helvetica">code:</font><HR><pre>
while (<INFILE> ) {
chomp;
next unless /ipchains/;
print " Converting $module (writing to $output)...";

s/ipchains/iptables/g;
print OUTFILE "$_n";
}
[/code]

Now the line is read into $_, the line is skipped if it doesn't contain 'ipchains' (is this what you want?), the line is converted and written to the outfile.



[This message has been edited by vpopper (edited December 28, 2000).]

kind of. What I am trying to do is convert about 100 or more files stored within a number of sub-dirs contained in /etc/firewall/modules , and I want to convert every occurence of ipchains into iptables.

I will try what you just posted, though. Thanks.



[This message has been edited by nebuchadnezzar (edited December 28, 2000).]

<BLOCKQUOTE><font size="1" face="Verdana,Arial,Helvetica">quote:</font><HR>Originally posted by nebuchadnezzar:
[B]kind of. What I am trying to do is convert about 100 or more files stored within a number of sub-dirs contained in /etc/firewall/modules , and I want to convert every occurence of ipchains into iptables.[/B/[/quote]

This is untested, but should work:

<BLOCKQUOTE><font size="1" face="Verdana,Arial,Helvetica">code:</font><HR><pre>
#!/usr/bin/perl

require 5.004;
$| = 1;
umask(022); # change if neccessary

use strict;
use vars qw($BASEDIR @FILES);

$BASEDIR = '/etc/firewall/modules';

die "Invalid BASEDIR!n" unless -d $BASEDIR;

CLS();
INVERSE();
print " Conversion in Progress...", " " x 25, "nn";
NORMAL();

# collect files
print "Collecting files............ ";
chdir $BASEDIR or die "Cannot cd to $BASEDIR: $!n";
&traverse($BASEDIR,(stat('.'))[3]);
printf "%d files found.n", scalar(@FILES);

convert(@FILES);

print "n*All Done*n";
exit;

#==========================================================================
# traverse() -

sub traverse {
my ($dir,$nlink) = @_;
my (@filenames,@subdirs);

opendir DIR, '.' or die "cannot opendir $dir: $!n";
@filenames = grep { !/^..?$/ && !(-l $_) } readdir DIR;
closedir DIR;

for (@filenames) {
next unless stat($_) && -r _;
-d _ && push(@subdirs,$_);
-f _ && -s _ && -T _ && push(@FILES,"$dir/$_");
}

# has subdirs
if ($nlink != 2) {
for (@subdirs) {
chdir $_ or die "cannot cd to $_: $!n";
traverse("$dir/$_",(stat('.'))[3]);
chdir '..';
}
}
}

#==========================================================================
# convert() -

sub convert {
my @files = @_;

foreach my $file (sort @files) {
next unless $file =~ m{$BASEDIR/([^/]+)/([^/]+)/(.+)$};
print "Converting: $1 => $2 => $3 ... ";

open IN, "<$BASEDIR/$1/$2/$3"
or die "Cannot open $BASEDIR/$1/$2/$3: $!n";

open OUT, ">$BASEDIR/$1/$2/$3.tmp"
or die "Cannot open $BASEDIR/$1/$2/$3.tmp: $!n";

while (<IN> ) {
chomp;
next unless m{ipchains};
s{ipchains}{iptables}g;
print OUT "$_n";
}

unless (-e "$BASEDIR/$1/$2/$3.tmp") {
print "Failed.n";
next;
}

print "Succeeded.n";
system("/bin/mv $BASEDIR/$1/$2/$3.tmp $BASEDIR/$1/$2/$3");
die "$!n" if $!;
}

return 1;
}

#==========================================================================
# magic terminal controllers (may not work on your system)

sub CLS { print "e[;He[2J"; } # clear screen
sub INVERSE { print "e[7m"; } # inverse screen
sub NORMAL { print "e[0m"; } # normal screen


[/code]

OK... that IS a big difference from the original script...

I have no idea whatsoever about what it does! This is my first script, so I'd aprreciate it if you were able to go through and explain what some of the lines do?

Thanks

ok... I've decided to have a go at it for myself, might be educational...

from what I can see, at the beginning, you chdir to $BASEDIR, which has been set to /etc/firewall/modules, then traverse starts at . (which happens to be $BASEDIR, as set by &traverse($BASEDIR, ...)).

Then... it does the equivalent of 'ls' and stores it in the array @filenames, and from there works out whether the variables stored in @filenames are directories or other types of files using the -r switch, right?

I don't know what $nlink does, but it then goes into @subdirs and chdirs to the directories in @subdirs, and then changes dirs to $dir/$_ , which would be /etc/firewall/modules/$dir/$_ , correct? I have NO idea what the stat thing does...

then, after that, the convert function is called, and because of the <BLOCKQUOTE><font size="1" face="Verdana,Arial,Helvetica">code:</font><HR><pre>my @files = @_ [/code] there, it assigns @files to @FILES, because convert was called using convert(@FILES), am I correct? Then, <IN> and <OUT> are created (I don't know what $1, $2, and $3 do, though).

This is where I get slightly confused:

<BLOCKQUOTE><font size="1" face="Verdana,Arial,Helvetica">code:</font><HR><pre>while (<IN> ) {
chomp;
next unless m{ipchains};
s{ipchains}{iptables}g;
print OUT "$_n";
} [/code]

what does the 'm' in front of {ipchains} do?

also, shouldn't s{ipchains}{iptables}g be s/ipchains/iptables ? (correct me if I'm wrong!)

thanks...

vpopper - I tried the script you posted, and it worked without any errors (running with -w), but it sseems to only copy the iptables commands to the new modules. What would be good is if all the comments in each of the files was copied along with the new iptables commands.

Do you know how I might be able to do this?

Also, what does the stat function do? I noticed that it is used a couple of times, but I have been unable to understand what it does exactly.

Thanks

<BLOCKQUOTE><font size="1" face="Verdana,Arial,Helvetica">quote:</font><HR>Originally posted by nebuchadnezzar:
vpopper - I tried the script you posted, and it worked without any errors (running with -w), but it sseems to only copy the iptables commands to the new modules. What would be good is if all the comments in each of the files was copied along with the new iptables commands.[/quote]

Just take out this line:
next unless m{ipchains};

<BLOCKQUOTE><font size="1" face="Verdana,Arial,Helvetica">quote:</font><HR>Also, what does the stat function do?[/quote]

The stat function returns a 13-element array giving the statistics for a file. Typically used as follows:

($dev,$ino,$mode,$nlink,$uid,$gid,$gid,
$rdev,$size,$atime,$mtime,$ctime,$blksize,
$blocks) = stat($filename);

In my code example, I use stat in the 'traverse' routine to determine the number of directory links. If the number is not equal to 2 (. and ..) then there must be subdirectories. If there are subdirectories, we traverse down the subdirectories in a recursive file search.

Also in 'traverse' we use call stat() on the file in preparation for a number of file tests. We make sure it is not a symbolic link, that it is readable, that it is a regular text file, that it has a positive size value, etc.

Try 'man stat' on your system for more info, or read "Programming Perl".

Hope this helps :-)

<BLOCKQUOTE><font size="1" face="Verdana,Arial,Helvetica">quote:</font><HR>Originally posted by vpopper:
Try 'man stat' on your system for more info, or read "Programming Perl".

Hope this helps :-)

[/quote]

Ok... thanks! I do have the O'Reilly book - Programming Perl, but I haven't read all of it (let alone the first couple of sections), yet.

Thanks for your help. I'll try and adapt what you've posted.