Need help in file column parsing

Dev Shed Forums Sponsor:

February 11th, 2013, 04:37 AM

utpalmtbi

Registered User

Join Date: Feb 2013

Posts: 17

Time spent in forums: 7 h 56 m 29 sec
Reputation Power: 0

Need help in file column parsing

Hello all,

I have a very large file (input.txt) containing 8 columns of data in the following tab separated format:

or30|1 or14|1353 or30 or14 0 0 63.7 98.3
or30|1 or76|2491 or30 or76 0 0 65.1 98.3
or30|1 or39|1309 or30 or39 0 0 72 98.3
or30|1 or64|261 or30 or64 0 0 70.8 98.3
or30|1 or35|1353 or30 or35 0 0 70.3 98.3
or30|1 or60|639 or30 or60 0 0 69.2 98.3
or30|1 or59|1597 or30 or59 0 0 72.2 78.3
or30|1 or56|995 or30 or56 0 0 69.2 58.3
or30|1 or52|852 or30 or52 0 0 89.2 34.3

......................
.................

I want to keep only those lines where last two columns are >= 70 and >= 90 respectively, and paste the result in a output file (output.txt). Based on the above input, the output file should be:

or30|1 or39|1309 or30 or39 0 0 72 98.3
or30|1 or64|261 or30 or64 0 0 70.8 98.3
or30|1 or35|1353 or30 or35 0 0 70.3 98.3

Any guidance is highly appreciated. Thanks in advance...

February 11th, 2013, 04:48 AM

SuperOscar

Contributing User

Join Date: Jul 2007

Location: Joensuu, Finland

Posts: 412

Time spent in forums: 1 Week 7 h 14 m 58 sec
Reputation Power: 65

Untested:

Code:

with open('input.txt', 'r') as input_data:
    with open('output.txt', 'w') as output_data:
        for line in input_data:
            penult, ult = [float(num) for num in line.split('\t')[-2:]]
            if penult >= 70 and ult >= 90:
                output_data.write(line)

__________________
My armada: openSUSE 12.3 (home desktop, laptop, work desktop), Ubuntu 12.04 LTS (mini laptop), Debian GNU/Linux 7.0 (server), Mythbuntu 12.04 LTS (HTPC), Bodhi Linux 2.0 & Windows 7 Ultimate (test desktop), FreeBSD 9.1 (test server)

Last edited by SuperOscar : February 11th, 2013 at 05:12 AM.

February 11th, 2013, 04:59 AM

utpalmtbi

Registered User

Join Date: Feb 2013

Posts: 17

Time spent in forums: 7 h 56 m 29 sec
Reputation Power: 0

thanks

Works fine... thanks SuperOscar..

February 11th, 2013, 05:13 AM

SuperOscar

Contributing User

Join Date: Jul 2007

Location: Joensuu, Finland

Posts: 412

Time spent in forums: 1 Week 7 h 14 m 58 sec
Reputation Power: 65

Quote:

Originally Posted by utpalmtbi

Works fine... thanks SuperOscar..

You’re welcome. I just removed the “buff = []” line from the start of the script; it was from an older idea.

February 11th, 2013, 10:25 AM

b49P23TIvg

Contributing User

Join Date: Aug 2011

Posts: 3,458

Time spent in forums: 1 Month 2 Weeks 4 Days 6 h 26 m 43 sec
Reputation Power: 403

gawk is easier for task

Code:

$ gawk -F"\t" '(70<=$(NF-1))&&(90<=$NF)' /tmp/d.dat
or30|1	or39|1309	or30	or39	0	0	72	98.3
or30|1	or64|261	or30	or64	0	0	70.8	98.3
or30|1	or35|1353	or30	or35	0	0	70.3	98.3

__________________
[code]Code tags[/code] are essential for python code!

Need help in file column parsing

Developer Shed Advertisers and Affiliates