Splitting files based on condition

Dev Shed Forums Sponsor:

May 10th, 2005, 01:03 PM

siva111111

Registered User

Join Date: May 2005

Posts: 1

Time spent in forums: 28 m 34 sec
Reputation Power: 0

Splitting files based on condition

I have a ~ delimited file with 8 or 9 columns i.e. in some cases, the 9th column is populated. If the 9th column is not null, then data needs to go to file1, if there are only 8 columns, then file2.

The if statement below does not work - all the data goes to file1.txt. Any ideas will be appreciated. Thank you.

while read line
do
v9=`cut -f9 -d~ $filename`
if [[ -n $v9 ]];then
echo "$line" >> file1.txt
else
echo "$line" >> file2.txt
fi
done < $filename

May 16th, 2005, 05:09 AM

christo

Introspective

Join Date: Nov 2001

Location: London, UK

Posts: 3,297

Time spent in forums: 3 Days 1 h 9 m 17 sec
Reputation Power: 104

well it's probably a bit late, but this is a good way to do it..

create a file called file1 which contains all your original data like this:

Quote:

one~two~three~four~five~six~seven~eight~
one~two~three~four~five~six~seven~eight~nine
one~two~three~four~five~six~seven~eight~
one~two~three~four~five~six~seven~eight~
one~two~three~four~five~six~seven~eight~
one~two~three~four~five~six~seven~eight~
one~two~three~four~five~six~seven~eight~nine
one~two~three~four~five~six~seven~eight~
one~two~three~four~five~six~seven~eight~
one~two~three~four~five~six~seven~eight~nine
one~two~three~four~five~six~seven~eight~nine
one~two~three~four~five~six~seven~eight~
one~two~three~four~five~six~seven~eight~

now create a new file called myprogram.pl containing this:

Code:

#!/usr/bin/perl

$file1="/home/chris/bin/file1";
$file2="/home/chris/bin/file2";
$file3="/home/chris/bin/file3";

open(FILE1,"<$file1");
open(FILE2,">$file2");
open(FILE3,">$file3");

while(<FILE1>){
        chomp;
        @fields=split "~";
        if($fields[8]=~m/.+/){
                print FILE3 "$_\n";
                next;
        }
        print FILE2 "$_\n";
}

make sure you change the path from /home/chris/bin/ to whatever your working directory is. Chmod the script and run it.

It will create two new files, file2 and file3 which contain your results.

hth,

christo

__________________
.
Spiration channels: Free scripts, programming tutorials and articles
Dotcut alerts: Online Press cuttings / news alerts
Clearprop: UK microlight school, wiltshire
Uk dating: UK safe dating with Topdates
About Christo
.
.

May 19th, 2005, 09:46 AM

zlutovsky

Contributing User

Join Date: Dec 2004

Location: Prague, Czech Rep.

Posts: 117

Time spent in forums: 22 h 42 m 44 sec
Reputation Power: 6

Quote:

Originally Posted by christo

well it's probably a bit late, but this is a good way to do it..

create a file called file1 which contains all your original data like this:

now create a new file called myprogram.pl containing this:

Code:

#!/usr/bin/perl

$file1="/home/chris/bin/file1";
$file2="/home/chris/bin/file2";
$file3="/home/chris/bin/file3";

open(FILE1,"<$file1");
open(FILE2,">$file2");
open(FILE3,">$file3");

while(<FILE1>){
        chomp;
        @fields=split "~";
        if($fields[8]=~m/.+/){
                print FILE3 "$_\n";
                next;
        }
        print FILE2 "$_\n";
}

Hi,

perl is a fine tool, but in this simple case the old awk is quite good and efficient, as well. Try this:

:
awk '
BEGIN { FS = "~" }
$9 == "" {print | "cat" ; next}
{print | "cat >&2" }
' <<!
one~two~three~four~five~six~seven~eight~
one~two~three~four~five~six~seven~eight~nine
one~two~three~four~five~six~seven~eight~
one~two~three~four~five~six~seven~eight~
one~two~three~four~five~six~seven~eight~
one~two~three~four~five~six~seven~eight~
one~two~three~four~five~six~seven~eight~nine
one~two~three~four~five~six~seven~eight~
one~two~three~four~five~six~seven~eight~
one~two~three~four~five~six~seven~eight~nine
one~two~three~four~five~six~seven~eight~nine
one~two~three~four~five~six~seven~eight~
!

This script will send all lines with empty field 9 to the stdout and all other lines in stdrerr. You can then redirect them wher ever you need.

Regards

Thread Tools	Search this Thread
Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode	Rate This Thread: