March 14th, 2013, 09:21 AM
Parallel processing of perl scripts
We have a perl script which can be invoked by many shell scripts.the function of the perl script is, it accepts a file as input, removes leading and trailing spaces between fields and writes into an output file. In our case two different shell scripts called this perl script at the same time for different input files and while writing into their respective out files, both the process failed.
Error Message: Error while writing into out file
Kindly help to resolve this. our requirement is that the script should not fail even its invoked n number of times at the same time.
underlined the line where it fails
print "Start text editsinglepipe.pl $timestamp\n";
print " File to edit @ARGV\n";
print " File to write @ARGV\n";
print " open read file $timestamp\n";
open(RFILE,"@ARGV") or die("Unable to open read file");
print " open write file $timestamp\n";
open(WFILE,"+>@ARGV") or die("Unable to open write file");
print " loop through file $timestamp\n";
#loop through each record of the file
while ($line = <RFILE> )
#Split each column in the record
@record = split(/\|/,$line);
#loop threw each record.
foreach $column (@record)
$column =~ s/^\s+//; #removes leading space
$column =~ s/\s+$//; #removes trailing space
$column =~tr/\x80-\xFF//d; # removes non-ascii characters.
$newline = "$newline$column|"; #puts line back together.
#Write to file;
print WFILE "$newline\n";
print " close file $timestamp\n";
$totaltime = $endtimestamp-$starttimestamp;
print " Edited $filecount Record(s) in $totaltime Seconds\n";
print "End text editsinglepipe.pl $timestamp\n";
March 14th, 2013, 10:26 AM
Sounds like the problem is with file access, not running multiple perl instances.
Although you can explore the idea of file locking depending on your operating system, the easier solution would be to use a database to store your records, since they have already been designed to handle multiple users.
March 14th, 2013, 01:26 PM
[QUOTE=keath]Sounds like the problem is with file access, not running multiple perl instances.
Thanks for the reply Keath.
I dont think its a file locking issue.
Example: i am running two instance of same perl script. both creating different files. even though both creates different files, it fails at line underlined above.
March 14th, 2013, 01:31 PM
can Parallel::ForkManager be used for the above issue?
how does this work?