UNIX Help
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed ForumsOperating SystemsUNIX Help

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
  #1  
Old August 7th, 2006, 08:07 PM
xxxindigo xxxindigo is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Aug 2006
Posts: 1 xxxindigo User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 5 m 49 sec
Reputation Power: 0
Find command - Performance

I'm trying write a script to find certain files in a Unix system. I have 3 large directories called /Archive1 /Archive2 and Archive3. I have to find a file in each directory. Here is my script:

#!/bin/bash


tt=/usr/tmp/zznineteenzz
arch=/usr/tmp/foo.tomk
dirs="/Archives /Archives2 /Archives3"

for i in `cat $tt`; do

for dd in $dirs; do

for ff in `.dosu find $dd -name $i*Z -print`; do
if [ -f $arch ]
then
echo $ff >> $arch
fi
done
done
done
====

I'm worried about performance issues. Is there a better way? Thanks everyone!

Reply With Quote
  #2  
Old August 8th, 2006, 08:53 AM
jim mcnamara jim mcnamara is offline
......@.........
Dev Shed Beginner (1000 - 1499 posts)
 
Join Date: Jun 2004
Posts: 1,308 jim mcnamara User rank is Sergeant Major (2000 - 5000 Reputation Level)jim mcnamara User rank is Sergeant Major (2000 - 5000 Reputation Level)jim mcnamara User rank is Sergeant Major (2000 - 5000 Reputation Level)jim mcnamara User rank is Sergeant Major (2000 - 5000 Reputation Level)jim mcnamara User rank is Sergeant Major (2000 - 5000 Reputation Level)jim mcnamara User rank is Sergeant Major (2000 - 5000 Reputation Level) 
Time spent in forums: 1 Week 3 Days 5 h 20 m 49 sec
Reputation Power: 48
Code:
for dir in "$dirs"
do
     find "$dir" -type f -print | grep -f /usr/tmp/zznineteenzz 
done > file_listing

This way you read thru each directory only once with find.

Reply With Quote
  #3  
Old August 8th, 2006, 10:09 AM
SimonJM SimonJM is offline
Contributing User
Dev Shed Novice (500 - 999 posts) Click here for more information
 
Join Date: Mar 2006
Posts: 762 SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level) 
Time spent in forums: 2 Weeks 1 Day 22 h 8 m 4 sec
Reputation Power: 336
Not totally sure, but I think you can do find /dir1 /dir2 /dir3 -type f ... etc., so do all 3 dirs at once - man find will ngive a yea/nay.

Reply With Quote
  #4  
Old August 8th, 2006, 11:56 AM
jim mcnamara jim mcnamara is offline
......@.........
Dev Shed Beginner (1000 - 1499 posts)
 
Join Date: Jun 2004
Posts: 1,308 jim mcnamara User rank is Sergeant Major (2000 - 5000 Reputation Level)jim mcnamara User rank is Sergeant Major (2000 - 5000 Reputation Level)jim mcnamara User rank is Sergeant Major (2000 - 5000 Reputation Level)jim mcnamara User rank is Sergeant Major (2000 - 5000 Reputation Level)jim mcnamara User rank is Sergeant Major (2000 - 5000 Reputation Level)jim mcnamara User rank is Sergeant Major (2000 - 5000 Reputation Level) 
Time spent in forums: 1 Week 3 Days 5 h 20 m 49 sec
Reputation Power: 48
Some find implementations may do that. You mentioned "a UNIX system", this will work anywhere.

Do not confuse the number of lines in a script with performance. The critical step here is just reading thru a directory and then grepping against a list of files contained in /usr/tmp/zznineteenzz

Reply With Quote
  #5  
Old August 9th, 2006, 06:03 AM
SimonJM SimonJM is offline
Contributing User
Dev Shed Novice (500 - 999 posts) Click here for more information
 
Join Date: Mar 2006
Posts: 762 SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level)SimonJM User rank is Major (30000 - 40000 Reputation Level) 
Time spent in forums: 2 Weeks 1 Day 22 h 8 m 4 sec
Reputation Power: 336
A little quicker (just one command not in an external loop, o shaving off those all important microseconds! ) and also easier/less to type ...

Reply With Quote
  #6  
Old August 10th, 2006, 02:06 PM
playskool playskool is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Aug 2005
Posts: 48 playskool Negative: is most likely a SPAMMER and a traitor to the cause. 
Time spent in forums: 23 h 5 m 35 sec
Reputation Power: 0
Try using find and xargs.....

find /path/to/ <anyother find options> | xargs grep 'search value'

Reply With Quote
  #7  
Old August 10th, 2006, 02:39 PM
playskool playskool is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Aug 2005
Posts: 48 playskool Negative: is most likely a SPAMMER and a traitor to the cause. 
Time spent in forums: 23 h 5 m 35 sec
Reputation Power: 0
Quote:
Originally Posted by playskool
Try using find and xargs.....

find /path/to/ <anyother find options> | xargs grep 'search value'


ignore this I didn't read it right...thought you were looking to find something in a file.

My bad!

Reply With Quote
  #8  
Old August 10th, 2006, 03:59 PM
jim mcnamara jim mcnamara is offline
......@.........
Dev Shed Beginner (1000 - 1499 posts)
 
Join Date: Jun 2004
Posts: 1,308 jim mcnamara User rank is Sergeant Major (2000 - 5000 Reputation Level)jim mcnamara User rank is Sergeant Major (2000 - 5000 Reputation Level)jim mcnamara User rank is Sergeant Major (2000 - 5000 Reputation Level)jim mcnamara User rank is Sergeant Major (2000 - 5000 Reputation Level)jim mcnamara User rank is Sergeant Major (2000 - 5000 Reputation Level)jim mcnamara User rank is Sergeant Major (2000 - 5000 Reputation Level) 
Time spent in forums: 1 Week 3 Days 5 h 20 m 49 sec
Reputation Power: 48
Since we're beating performance to death here... POSIX find supports a + operator
which improves performance of the -exec clause by having the subprocess command work on a large set of data:
Code:
for dir in "$dirs"
do
     find "$dir" -type f -print  -exec  grep -l /usr/tmp/zznineteenzz {} \+
done > file_listing

So, this is another way to approach the problem.

Reply With Quote
Reply

Viewing: Dev Shed ForumsOperating SystemsUNIX Help > Find command - Performance


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump


Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 3 hosted by Hostway
Stay green...Green IT