UNIX Help
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed Forums > Operating Systems > UNIX Help
Reload this Page

getline with if condition

Discuss getline with if condition in the UNIX Help forum on Dev Shed.
getline with if condition UNIX Help forum discussing the Unix Operating System and all variants including Irix, Solarix, and AIX. Unix was designed as a true multi-user operating system.

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
« Previous Thread | Next Thread »  
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
Stop making mediocre tutorials.The best tutorials are video! Camtasia Studio makes it easy to create engaging, buzz-building screen videos at any size, in any popular format. Download the free trial!
  #1  
Old September 19th, 2003, 11:08 AM
josephg josephg is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Sep 2003
Location: sydney
Posts: 47 josephg User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 56 m
Reputation Power: 5
Smile getline with if condition
Hi
I am new with shell script programming could some one please help.
How can I get the hostname with his infoline-1 to infoline-n if the total error for the hostname is >0 and then the next hostname-n and infoline-n (Skip the xxxx data and read and print the hostname No of errors and his information

Data file is:
Xxxxxxxxxx
Xxxxxxxxx
Hostname1
Infoline-1
Error=1
Infoline-n
Error=0
xxxxx
Total Error for hostname1 =1

Xxxxxxxxxxx
Xxxxxxxxxxx
Xxxxxxxxxx
Hostname2
Infoline-1
Error=0
Infoline-n
Error=0
Xxxxxxx
Total Error for hostname2 =0
Xxxxxxx
Hostname 3
Infoline-1
error=3
infoline-n
error=1
Total Error for hostname3 =4
Reply With Quote
  #2  
Old September 20th, 2003, 09:01 AM
druuna druuna is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Sep 2003
Posts: 137 druuna User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 5 m 37 sec
Reputation Power: 0
It's not the most elegant script I wrote, but it does what you want (I think):
Code:
#!/bin/bash
#
# infile must be given
# outfile is optional (if none given: host-info.<yymmdd>)

INFILE="$1"
OUTFILE="$2"

# Are input/output file provided:
[[ -z ${INFILE} ]] && echo "No input file given. Quiting" && exit 1
[[ -z ${OUTFILE} ]] && OUTFILE="host-info.`date '+%y%m%d'`"

cat ${INFILE} | \
sed -n '/^Hostname/,/^Total Error/{
s/Total Error for hostname\(.*\)/Total Error for hostname\1\
TAB/
p
}' | \
awk '
BEGIN { FS = "\n" ; RS = "\t\n" ; OFS = "%" ; ORS = "\n" }
$(NF-1) !~ /Total Error for hostname.*=0/ {
  for ( x = 1; x <= NF; ++x ) {
    print $x
  }
}' | \
egrep "^Hostname|^Infoline|^Total|^$" > ${OUTFILE}


IMPORTANT: You will probably copy and paste this, so I have to warn you
for one thing: TAB (line 16) should be replaced by a real tab!!

Using your inputfile example, this will be in the output file:
Code:
Hostname1 
Infoline-1 
Infoline-n 
Total Error for hostname1 =1 

Hostname 3 
Infoline-1 
Total Error for hostname3 =4


Explaining the code:

I used sed to create 'paragraphs' that are more uniform. These paragraphs
start with Hostname.... and end with Total Error....... and I added a line with
a tab in it (which I use with awk later on)

After sed has done it's job, you are left with the following (substitute <TAB> with a real tab):
Code:
Hostname1 
Infoline-1 
Error=1 
Infoline-n 
Error=0 
xxxxx 
Total Error for hostname1 =1 
<TAB>
Hostname2 
Infoline-1 
Error=0 
Infoline-n 
Error=0 
Xxxxxxx 
Total Error for hostname2 =0 
<TAB>
Hostname 3 
Infoline-1 
error=3 
infoline-n 
error=1 
Total Error for hostname3 =4
<TAB>


Although we have an irregular amount of fields, we do have TAB
seperated records and can continue using awk. After setting FS,RS,OFS,
ORS (input/ouput field and record seperators) we check if this record has
errors. If it doesn't: don't show it. The for loop is needed because of the
irregular amount of fields.

After awk has done its job we are left with:
Code:
Hostname1 
Infoline-1 
Error=1 
Infoline-n 
Error=0 
xxxxx 
Total Error for hostname1 =1 

Hostname 3 
Infoline-1 
error=3 
infoline-n 
error=1 
Total Error for hostname3 =4


What's left is to discard the unwanted lines and only show what we want to
see. This is what the egrep line does.
Last edited by druuna : September 20th, 2003 at 09:08 AM.
Reply With Quote
  #3  
Old September 22nd, 2003, 01:32 AM
josephg josephg is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Sep 2003
Location: sydney
Posts: 47 josephg User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 56 m
Reputation Power: 5
Thumbs up getline and if condition
Excellent and thank you,
But the data output display all infolines (Contains Errors and non Errors) for the hostnameX.
What if we want only display the infolines CONTAINS ERRORS only for the hostnameX with Total errors >0 with the same pattern of output or
In Column format
(INFOLINES are text lines Ex. UNIX Programming Environment)

Example

Hostname1
Infonline-2
Error=4
Infoline-3
Error=7
UNIX Programming Environment = 2
OR

HOSTNAME INFOLINE ERRORS

Hostname1 infoline2 4
Infoline3 7
UNIX…… 2

Hostname3 Infoline1 3
Infoline-n 1

Column formatted


Your data output currently producing all Infolines

Hostname1
Infoline-1
Error=1
Infoline-n
Error=0
xxxxx
Total Error for hostname1 =1

Hostname 3
Infoline-1
error=3
infoline-n
error=1
Total Error for hostname3 =4
Reply With Quote
  #4  
Old September 22nd, 2003, 08:09 AM
druuna druuna is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Sep 2003
Posts: 137 druuna User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 5 m 37 sec
Reputation Power: 0
I'm not entirely clear what it is you want. If I understand correctly you want the error amount/number printed after the Infoline.

One other thing, you say that my script produces this:

Hostname1
Infoline-1
Error=1
Infoline-n
Error=0
xxxxx
Total Error for hostname1 =1

I do believe the output is:

Hostname1
Infoline-1
Infoline-n
Total Error for hostname1 =1

Could you post some real examples from your inputfile. I need to know if the xxxxx in your previous posts can be anything at all or are actually messagelines that start with certain keywords.
I.e. Does xxxx always starts with 'UNIX Programming Environment'.

If that is the case:

Code:
#!/bin/bash
#
# infile must be given
# outfile is optional (if none given: host-info.<yymmdd>)

INFILE="$1"
OUTFILE="$2"

# Are input/output file provided:
[[ -z ${INFILE} ]] && echo "No input file given. Quiting" && exit 1
[[ -z ${OUTFILE} ]] && OUTFILE="host-info.`date '+%y%m%d'`"

cat ${INFILE} | \
sed -n '/^Hostname/,/^Total Error/{
s/Total Error for hostname\(.*\)/Total Error for hostname\1\
TAB/
p
}' | \
awk '
BEGIN { FS = "\n" ; RS = "\t\n" ; OFS = "%" ; ORS = "\n" }
$(NF-1) !~ /Total Error for hostname.*=0/ {
  for ( x = 1; x <= NF; ++x ) {
    #print $x
    printf("%s", $x )
  }
printf("\n")
}' | \
sed -e 's/^Hostname/\
Hostname/' -e 's/[Ee]rror=//g' -e 's/line-/line /g' -e 's/ [Ii]nfoline/\
- Infoline/g' -e 's/ UNIX Programming Environment =/\
- UNIX Programming Environment/g' -e 's/Total Error/\
Total Error/' -e 's/=\([0-9]\)/\1/'


If I test this with the following input:

Xxxxxxxxxx
Xxxxxxxxx
Hostname1
Infoline-1
Error=1
Infoline-7
Error=0
UNIX Programming Environment = 5
Total Error for hostname1 =1

Hostname1
Infonline-2
Error=4
Infoline-3
Error=7
UNIX Programming Environment = 2
Total Error for hostname1 =7

Xxxxxxxxxxx
Xxxxxxxxxxx
Xxxxxxxxxx
Hostname2
Infoline-1
Error=0
Infoline-4
Error=0
UNIX Programming Environment = 1
Total Error for hostname2 =0
Xxxxxxx
Hostname 3
Infoline-1
error=3
infoline-8
error=1
Total Error for hostname3 =4

This is the result:

Hostname1
- Infoline 1 1
- Infoline 7 0
- UNIX Programming Environment 5
Total Error for hostname1 1

Hostname1 Infonline 2 4
- Infoline 3 7
- UNIX Programming Environment 2
Total Error for hostname1 7

Hostname 3
- Infoline 1 3
- Infoline 8 1
Total Error for hostname3 4
Reply With Quote
  #5  
Old September 23rd, 2003, 06:35 PM
josephg josephg is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Sep 2003
Location: sydney
Posts: 47 josephg User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 56 m
Reputation Power: 5
Thumbs up limits on size of input file
I have run the script successfully (your first script) on small size of input file only let say less than 10000 bytes.
But when I run it with larger file I get the following errors could you please let me know how can I rectify the problem what I need to modify in your first script to read larger input files, is there any limit size of reading files

The error is
awk: 0602-534 Input line 1) Hostname: Hostname1 cannot be longer than 10,239 bytes
.
The source line number is 2.

Thank you.
Reply With Quote
  #6  
Old September 23rd, 2003, 07:31 PM
josephg josephg is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Sep 2003
Location: sydney
Posts: 47 josephg User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 56 m
Reputation Power: 5
Thumbs up limits on size of input file
I have run the script successfully (your first script) on small size of input file only let say less than 10000 bytes.
But when I run it with larger file I get the following errors could you please let me know how can I rectify the problem what I need to modify in your first script to read larger input files, is there any limit size of reading files

The error is
awk: 0602-534 Input line 1) Hostname: Hostname1 cannot be longer than 10,239 bytes
.
The source line number is 2.

Thank you.
Reply With Quote
  #7  
Old September 24th, 2003, 05:34 AM
druuna druuna is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Sep 2003
Posts: 137 druuna User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 5 m 37 sec
Reputation Power: 0
I cannot reproduce the error you are describing.

Made an inputfile (1.5 Mb large, 113541 lines long) and fed this to the script. It runs perfectly. No errors whatsoever. I also made one of the Hostname lines long (166 chars), also no errors.

3 things come to mind:

1) Your inputfile is different from the example you posted before (that's why I asked for a real example in a previous post). The other reason I asked for (part of) a real file: The example you provide is very irregular, that's one of the reasons why I had to account for things that probably are not present, or more consistent in the real inputfile.

2) Your input file has dos line terminators (do you see ^M at the end of the lines if you open it with vi??) or other unrecognized chars (again, I need a real example to check some of this).

3) Your computer has limited memory. The test with the 1.5 Mb input file was done on a machine with 'only' 64Mb of physical memory. I have encounter sed/awk problems that had to do with machine limits (memory/swap/diskspace). But I'm talking about an input file that was over 2Gb on a machine with 512 Mb physical mem.

I did find one thing (but it is sed related and doesn't produce errors) that could be done better:

In the following two lines (near bottom of script):

=> Hostname/' -e 's/[Ee]rror=//g' -e 's/line-/line /g' -e 's/ [Ii]nfoline/\
Remove the space before [Ii]nfoline.

=> - Infoline/g' -e 's/ UNIX Programming Environment =/\
Remove the space before UNIX.

In the end it should look like this:

Hostname/' -e 's/[Ee]rror=//g' -e 's/line-/line /g' -e 's/[Ii]nfoline/\
- Infoline/g' -e 's/UNIX Programming Environment =/\
Reply With Quote
  #8  
Old September 25th, 2003, 07:42 PM
josephg josephg is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Sep 2003
Location: sydney
Posts: 47 josephg User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 56 m
Reputation Power: 5
error condition
Thanks for your help and follow-up

I have modify your script as bellow and the data file is attached I am running your script on AIX 4.3.3 and the error message is as follow

/tmp> ./your_script.sh datafile
awk: 0602-534 Input line 1) Hostname: bhccms
cannot be longer than 10,239 bytes.
The source line number is 2.


#!/bin/ksh
#
# infile must be given
# outfile is optional (if none given: host-info.<yymmdd> )

INFILE="$1"
OUTFILE="$2"

# Are input/output file provided:
[[ -z ${INFILE} ]] && echo "No input file given. Quiting" && exit 1
[[ -z ${OUTFILE} ]] && OUTFILE="host-info.`date '+%y%m%d'`"

cat ${INFILE} | \
sed -n '/Hostname/,/Total Violations/{
s/Total Violations\(.*\)/Total Violations\1\
TAB/
p
}' | \
awk '
BEGIN { FS = "\n" ; RS = "\t\n" ; OFS = "%" ; ORS = "\n" }
$(NF-1) !~ /"Total Violations = 0"/ {
for ( x = 1; x <= NF; ++x ) {
print $x }
}' | \
egrep "Hostname|Total|^$" > ${OUTFILE}


Thanks
Attached Files
File Type: txt datafile.txt (28.4 KB, 214 views)
Reply With Quote
  #9  
Old September 26th, 2003, 07:59 AM
druuna druuna is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Sep 2003
Posts: 137 druuna User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 5 m 37 sec
Reputation Power: 0
Glad to know what os and version you are using.

There's probably nothing wrong with the script(s), its awk that's behaving odd when using aix 4.3.3 (found this after a google search). I've had problems with awk before (on Stratus FTX, but the problem was different then yours).

I also tested this script on my hp box and it worked fine.

Possible solutions:
- use nawk instead if awk (should be present in /usr/bin),
- ask sysadmin about limits.

There is an awk testscript that you could run to see if awk is working the way it should:

http://www.shelldorado.com/articles/testawk.sh

I haven't used AIX before, so I cannot help you with problems related to this os.
Reply With Quote
  #10  
Old September 30th, 2003, 05:54 AM
josephg josephg is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Sep 2003
Location: sydney
Posts: 47 josephg User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 56 m
Reputation Power: 5
getline
Thank you so much you have been very help full
Reply With Quote
  #11  
Old October 2nd, 2003, 08:59 AM
drgroove's Avatar
drgroove drgroove is offline
pushing envelopes, not pencils
Dev Shed God 2nd Plane (6000 - 6499 posts)
  
Join Date: Feb 2002
Posts: 6,224 drgroove User rank is First Lieutenant (10000 - 20000 Reputation Level)drgroove User rank is First Lieutenant (10000 - 20000 Reputation Level)drgroove User rank is First Lieutenant (10000 - 20000 Reputation Level)drgroove User rank is First Lieutenant (10000 - 20000 Reputation Level)drgroove User rank is First Lieutenant (10000 - 20000 Reputation Level)drgroove User rank is First Lieutenant (10000 - 20000 Reputation Level)drgroove User rank is First Lieutenant (10000 - 20000 Reputation Level)drgroove User rank is First Lieutenant (10000 - 20000 Reputation Level) 
Time spent in forums: 1 Day 4 h 34 m 57 sec
Reputation Power: 174
Moved to the UNIX forum
__________________
Give a person code, and they'll hack for a day; Teach them how to code, and they'll hack forever.
Analyze twice; hack once.
The world's first existential ITIL question: If a change is released into production without a ticket to track it,
was it actually released?

About DrGroove: ITIL-Certified IT Process Engineer - Enterprise Application Architect -
Freelance IT Journalist - Devshed Moderator - Funk Bassist Extraordinaire
Reply With Quote
Reply

Viewing: Dev Shed Forums > Operating Systems > UNIX Help > getline with if condition

« Previous Thread | Next Thread »

Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Linear Mode Linear Mode
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump


Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support |