need help with output redirection

Dev Shed Forums Sponsor:

Get inside! Sample the range of functionality easily built with JMSL Library for Time Series Data Analysis, Heat Maps, Portfolio Optimization, Monte Carlo Simulation, Stock Price Charting and more. Download Now!

October 15th, 2002, 11:36 AM

OB_redemption

Contributing User

Join Date: Jan 2002

Posts: 32

Time spent in forums: < 1 sec
Reputation Power: 7

need help with output redirection

Hi all

I've been trying to figure this out for days but still to no avail and I'm fairly frustrated. If you can help or have any suggestions, please post and help me out

I've been trying to get the output of each command in a pipeline logged into a log file. For example, for the pipeline ls -l|grep sh|wc, I'd need to log the output of ls -l, that of grep sh and also that of wc in the output log file. I've managed to stitch up the commands so that piping works fine but as soon as I try to add the code for redirection to the output log, everything breaks down.

So what I wanna know is: how do you redirect the output of intermediate commands to a file without breaking the pipe that's already there?

Here's my code at the moment:

Code:

    /* duplicate stdin as a copy of read_fd, so that input is redirected from read_fd
       read_fd is the file descriptor of the end of the pipe (of prev command) to get input from */
    if (read_fd)
    {
	dup2(read_fd, STDIN_FILENO);
	close(read_fd);
    }

    /* duplicate stdout as a copy of write_fd, so that output is redirected to write_fd
       write_fd is the file descriptor of the end of the pipe to write to*/
    if (write_fd)
    {
	dup2(write_fd, STDOUT_FILENO);
	close(write_fd);
    }

    if ( logging == TRUE )
    {
	/* open output log, which gets file descriptor of 1, effectively replacing stdout */
	if ( (outlog_fd = open(OUTPUTLOG, O_WRONLY | O_CREAT | O_APPEND)) == -1 )
	    perror("open");

	dup2(outlog_fd, STDOUT_FILENO);
    }

    /* execute command, searching for the executable file in the environment PATH variable if necessary */
    if ( execvp(command_argv[0], command_argv) == -1 )
    {
	printf("%s: command not found\n", command_argv[0]);
	exit(1);
    }

thanks!

October 15th, 2002, 01:24 PM

Analyser

*bounce*

Join Date: Jan 2002

Location: Delft, The Netherlands

Posts: 510

Time spent in forums: 2 Days 20 h 37 m 7 sec
Reputation Power: 9

Ok, here we go again

The *NIX kernel in general has no provisions for duplicating streams. It can handle streams flowing from, say, A to B, but not from A to both B and C. That's just not how the kernel works.

If you want to send data from one end to more than one destination, you have to do it yourself, explicitly copying every byte to all the write fds:

Code:

/* returns 1 on success, 0 on EOF, -1 on error */
int
split_stream (int src_fd, int dest_fd1, int dest_fd2)
{
        char c;
        ssize_t retval;

        if ( (retval = read(src_fd, &c, sizeof(c))) < 0) {
                perror("read");
                return (-1);
        }

        if (retval == 0) {
                /* EOF; caller should handle closing of fds etc */
                return (0);
        }

        while ( (retval = write(dest_fd1, &c, sizeof(c))) == 0) {
                ; /* do nothing */
        }

        if (retval < 0) {
                perror("write");
                return (-1);
        }

        while ( (retval = write(dest_fd2, &c, sizeof(c))) == 0) {
                ; /* do nothing */
        }

        if (retval < 0) {
                perror("write");
                return (-1);
        }

        return (1);
}

Your next question then probably is, where to put this code? Time for another diagram:

Code:

-------------          -------------
| program 1 |  (pipe)  | program 2 |
|           +--------->|           +---> stdout (screen)
|  (ls -l)  |          |    (wc)   |
-------------          -------------

That's basically what things look like now. Output from ls goes through the pipe to wc (or grep, or whatever). What you really want though, is this:

Code:

-------------        ----------        -------------
| program 1 | (pipe) | logger | (pipe) | program 2 |
|           +------->|........+------->|           +---> stdout
|  (ls -l)  |        |    :   |        |    (wc)   |
-------------        -----+----        -------------
                          |
                          v
                    (to logfile)

That means you'll have to create a logger process for every process you want logged. It is very well possible to use one logger process that reads from multiple pipes and copies them all to even more pipes, but I think things are complicated enough for you already

Hopefully this will give you some ideas on how to proceed. If not, I'm sure you'll let us know

__________________
"A poor programmer is he who blames his tools."
http://analyser.oli.tudelft.nl/

October 16th, 2002, 01:05 AM

OB_redemption

Contributing User

Join Date: Jan 2002

Posts: 32

Time spent in forums: < 1 sec
Reputation Power: 7

Heh thanks for helping again Analyser

Now, I've been trying to make sense of what you're saying and I think I've got most of it but I've no idea how to implement it in code. I've tried to, and for a few frustrating hours, but again no results. I'm not good at pipes and using dup2 I guess.

How do I create a logger process to place in between the original 2 processes of a pipe? Do I have to create a new pipe and redirect the input of ls -l (from the above example) to the pipe and tie the pipe to the logger process? How do I do that? Forking the process?

I'm really lost and I really need your advice (you did say to let you know

)

- Joel

October 16th, 2002, 02:32 AM

Analyser

*bounce*

Join Date: Jan 2002

Location: Delft, The Netherlands

Posts: 510

Time spent in forums: 2 Days 20 h 37 m 7 sec
Reputation Power: 9

Quote:

How do I create a logger process to place in between the original 2 processes of a pipe? Do I have to create a new pipe and redirect the input of ls -l (from the above example) to the pipe and tie the pipe to the logger process? How do I do that? Forking the process?

The short version: yes, you'll need an extra pipe, and yes, it'll include another fork() for every logger process.

The long version:

You'll need an additional pipe for every logger process. Just look at the second scheme; The pipe from ls goes into the logger process instead of to wc, and there's a new pipe, going from the logger to wc. You have to set all that up in the parent process.

The thing is, you might not want to rely on a version of tee being available, but the mechanics that it implements (reading from one fd and writing to many) is what you need. So you'll have to roll your own version of tee, basically.

Now, you don't have to write it as a seperate program. You can just stuff the same functionality into a function, and call it from a child process, thus creating a logger.

One note to the scheme: the down-arrow to the logfile is (obviously) not a pipe; the logger just has to open a file to write to.

As always, good luck

Thread Tools	Search this Thread
Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode	Rate This Thread: