UNIX Help
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed ForumsOperating SystemsUNIX Help

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
  #1  
Old June 2nd, 2006, 01:55 AM
Verletto Verletto is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: May 2006
Location: Sweden
Posts: 14 Verletto User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 5 h 56 m 46 sec
Reputation Power: 0
Post Can I optimize these commands?

Hello!

I try to optimize some shell code and come across the following two questionmarks:

1) Is there any way of writing the following two commands in one single line?

grep '{PATTERN}' $IN_FILE | sed '{ARG}' > $OUT_FILE_1
grep -v '{PATTERN}' $IN_FILE > $OUT_FILE_2

Without the -v option on the 2nd line I guess I could have written:
grep '{PATTERN}' $IN_FILE > $OUT_FILE_2 | sed '{ARG}' > $OUT_FILE_1

... which leads me to the question if I can actually output after each step of a filter cascade like on this line?


2) I would like to use the grep command on several conditions including boolean logic operators.
The following line uses filter cascade but uses 4 grep processes at run time:

grep -v '{PATTERN_1}' $FILE | grep -v '{PATTERN_2}' | grep -v '{PATTERN_3}' | grep '{PATTERN_4}'

(note the last grep hasn't the -v option)
Is there any possibility of using only one grep command combinating the -v and eventually the -e option?

Thanks for your help!

Reply With Quote
  #2  
Old June 2nd, 2006, 05:09 AM
SimonJM SimonJM is offline
Contributing User
Dev Shed Novice (500 - 999 posts) Click here for more information
 
Join Date: Mar 2006
Posts: 767 SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level) 
Time spent in forums: 2 Weeks 1 Day 23 h 25 m 34 sec
Reputation Power: 336
I do not believe you can do part 1 in just the one command line, not with grep anyway. What about using awk to do it - simple if/else to direct output to one file or another, based on if the search value is found?
The 'cascade' you mention, depending on what you want, may be achieved with the tee command.
The second part you can, depending on implementation shrink down into fewer commands:
grep -v -e '{PATTERN_1}' -e '{PATTERN_2}' -e '{PATTERN_3}' | grep '{PATTERN_4}

Reply With Quote
  #3  
Old June 2nd, 2006, 08:13 AM
Verletto Verletto is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: May 2006
Location: Sweden
Posts: 14 Verletto User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 5 h 56 m 46 sec
Reputation Power: 0
Post

Thank you for your reply, Simon!

Good idea about awk, however I don't see how the sed filter could fit into this solution. The command looks like this:

sed -e 's/;//g' -e 's/(\([0-9]*\))//g'

(removal of the ";" character and of all brackets and their eventual numeric contents).
I don't know if awk can make it as quickly. Each of those filters are not that tough to implement within an awk command, however their combination can be pretty tough compared to the sed solution.

Reply With Quote
  #4  
Old June 2nd, 2006, 11:02 AM
Ehlanna's Avatar
Ehlanna Ehlanna is offline
Not a clue what to put ...
Dev Shed Novice (500 - 999 posts)
 
Join Date: Jan 2006
Location: in front of this keyboard
Posts: 815 Ehlanna User rank is Captain (20000 - 30000 Reputation Level)Ehlanna User rank is Captain (20000 - 30000 Reputation Level)Ehlanna User rank is Captain (20000 - 30000 Reputation Level)Ehlanna User rank is Captain (20000 - 30000 Reputation Level)Ehlanna User rank is Captain (20000 - 30000 Reputation Level)Ehlanna User rank is Captain (20000 - 30000 Reputation Level)Ehlanna User rank is Captain (20000 - 30000 Reputation Level)Ehlanna User rank is Captain (20000 - 30000 Reputation Level)Ehlanna User rank is Captain (20000 - 30000 Reputation Level) 
Time spent in forums: 2 Weeks 2 Days 5 h 43 m 48 sec
Reputation Power: 243
Use gsub in the awk script to do the substitution - for just the lines that match your criteria.
__________________
According to Sod's Law, buttered toast lands butter side down, when dropped.
Per nature, cats always land on their feet.
So, what happens when you strap buttered toast to the back of a cat and throw it out a window?
.

Reply With Quote
  #5  
Old June 5th, 2006, 05:54 AM
zlutovsky zlutovsky is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Dec 2004
Location: Prague, Czech Rep.
Posts: 117 zlutovsky User rank is Corporal (100 - 500 Reputation Level)zlutovsky User rank is Corporal (100 - 500 Reputation Level)zlutovsky User rank is Corporal (100 - 500 Reputation Level)zlutovsky User rank is Corporal (100 - 500 Reputation Level) 
Time spent in forums: 22 h 42 m 44 sec
Reputation Power: 6
Smile

Quote:
Originally Posted by Ehlanna
Use gsub in the awk script to do the substitution - for just the lines that match your criteria.


Try this:
:
awk '
{
gsub(/;|\([0-9]*\)/,"")
print
}
' <<!
XXXX;;; adsf (123) 123
asdfasdf
(034556) ;; AAAAAAAAA
!

This awk script was tested on AIX. It is OK, but it does not mean that it is faster than sed solution. Generally, awk is slower than sed.

Regards

Reply With Quote
  #6  
Old June 5th, 2006, 06:58 AM
Verletto Verletto is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: May 2006
Location: Sweden
Posts: 14 Verletto User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 5 h 56 m 46 sec
Reputation Power: 0
Thank you both for your help But I had already written the following awk line:

awk '{if (match($0,{PATTERN})) {sub(/[(]*[0-9]*[)]*;/,"",$0); print substr($0,9) > "'$OUT_FILE1'"} else {print $0 > "'$OUT_FILE_2'"} }' $IN_FILE

...from the two grep commands below:

grep '{PATTERN}' $IN_FILE | sed -e 's/;//g' -e 's/(\([0-9]*\))//g' > $OUT_FILE_1
grep -v '{PATTERN}' $IN_FILE > $OUT_FILE_2

I still think the two grep/sed commands were a bit more readable!

Quote:
Originally Posted by zlutovsky
Try this:
:
awk '
{
gsub(/;|\([0-9]*\)/,"")
print
}
' <<!
XXXX;;; adsf (123) 123
asdfasdf
(034556) ;; AAAAAAAAA
!

This awk script was tested on AIX. It is OK, but it does not mean that it is faster than sed solution. Generally, awk is slower than sed.

Regards

Reply With Quote
Reply

Viewing: Dev Shed ForumsOperating SystemsUNIX Help > Can I optimize these commands?


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump


Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 1 hosted by Hostway
Stay green...Green IT