#1
  1. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2008
    Posts
    60
    Rep Power
    6

    Perl forking doubts


    Hi,
    I have 9 keys and corresponding values in a hash. I have to call another script for each key and hence have to call another script 9 times. This can be easily done while looping inside hash.
    But my constraint is I can start only 3 scripts at a time and wait for them to complete before I call the next 3 scripts followed by last 3 scripts.

    My basic testing works as below.

    Code:
    my $count = 0;
    while ( ($k,$v) = each %Info) {	
    	if ($count < 3) {
    		my @info = split(/,/, $v);
    		$R = $info[1];
    		system "perl test.pl $k $R";
    		$count++;
    			
    	} else {
    			
    		##reset the counter
    		print "go to sleep\n";
    		sleep(1);
    		$count=0;
    		system "perl /scripts/test.pl $k $R\n";
    		$count++;
    	}
    			
    }
    How can I modify the above script so that three jobs can be called using "perl /scripts/test.pl $k $R" and wait for all them to complete before I call the next batch? Thanks.
  2. #2
  3. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2008
    Posts
    60
    Rep Power
    6
    Code:
    while(<count>) {
    	my $pid = fork();
    	if ($pid == 0) {
                             system "perl /scripts/test.pl $k $R\n";
    
    		exit(0);
    		} else {
    			# parent
    			push(@processes, $pid);
    		}
    }
    foreach my $proc (@processes) {
    	waitpid( $proc, 0 );
    }
    I tried the above script to fork 1 script at a time and wait for it complete before I call the next step.

    I am bit confused on how to call the scripts 3 times wait for all 3 to complete before I call the next 3 jobs. Thanks.
  4. #3
  5. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    May 2007
    Posts
    765
    Rep Power
    929
    wait blocks until a child process exits (whichever one exits first, unlike waitpid which waits on a specific process).

    exec is typically called in the child after a fork to invoke another process. (system would perform an additional fork & wait itself)

    Here's a simple way to do it in psuedocode:
    Code:
    FOREACH item IN items
       IF process_count > 3
          wait
          process_count--
    
       fork
       IF parent
          process_count++
       ELSE
          exec
    sub{*{$::{$_}}{CODE}==$_[0]&& print for(%:: )}->(\&Meh);
  6. #4
  7. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2008
    Posts
    60
    Rep Power
    6
    Originally Posted by OmegaZero
    wait blocks until a child process exits (whichever one exits first, unlike waitpid which waits on a specific process).

    exec is typically called in the child after a fork to invoke another process. (system would perform an additional fork & wait itself)

    Here's a simple way to do it in psuedocode:
    Code:
    FOREACH item IN items
       IF process_count > 3
          wait
          process_count--
    
       fork
       IF parent
          process_count++
       ELSE
          exec
    Hi Omega,
    Thanks for your skeleton code. But I am still puzzled to convert my code according to your psudo code beacuse I have all the details in a hash. Could you please explain a bit more. Also I cant understand the forking process. Are you checking whether parentid is completed or not? Thanks again.
    Last edited by lilly07; April 2nd, 2012 at 04:02 AM.
  8. #5
  9. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    May 2007
    Posts
    765
    Rep Power
    929
    Not quite sure what you're asking but...

    Originally Posted by lilly07
    But I am still puzzled to convert my code according to your psudo code beacuse I have all the details in a hash.
    Use whatever iterator/looping construct you need to work with each item. You have code that steps through a hash in your first post.

    Originally Posted by lilly07
    Are you checking whether parentid is completed or not?
    I assume you mean PID / child processes. The parent process doesn't need any management. I'm not keeping track of the PIDs since I don't care when any specific child process terminates--just that one has. wait() will check for any child process's termination
    sub{*{$::{$_}}{CODE}==$_[0]&& print for(%:: )}->(\&Meh);

IMN logo majestic logo threadwatch logo seochat tools logo