|
|||||||||
|
|||||||||
| |||||||||
|
|
|
| |||||||||
 |
|
|
«
Previous Thread
|
Next Thread
»
|
Thread Tools | Search this Thread | Rate Thread | Display Modes |
Dev Shed Forums Sponsor:
|
|
|
|
#1
February 16th, 2006, 11:31 AM
|
||||
|
||||
|
Cat foo|xargs -I {} find {} -exec ... AGHHH!
Hi All,
I have a file 'sqlout' that is a list of 1000 file names. I need to go through a directory structure with 10's of thousands of files and copy only the ones in sqlout to another directory. Code:
cat sqlout|xargs -I [] find . -name [] -exec cp {} /target \;
works but it's awfully slow. I've added some restrictions to find (like -type f and others) but it hasn't helped with the speed. Any magic way that this can happen faster? Tx, Jennifer
|
|
#2
February 17th, 2006, 10:36 AM
|
||||
|
||||
|
Are you sure you are not vastly over-egging the pudding here? Some questions: are all the files in the same source directory? does the sqlout file contain a full path or just name?
What would be taking the time (apart from the actual I/O of the copy) is the 1,000 find commands you are doing. That would be the thing to aim to get rid of if you could.
|
|
#3
February 17th, 2006, 01:37 PM
|
||||
|
||||
|
the files are all in different directories
the filelist is just the file name because I don't know where the files are. :sigh: I guess I'm stuck with the slow way. Thanks for clarifying the problem. Cheers, Jennifer
|
|
#4
February 20th, 2006, 01:53 PM
|
||||
|
||||
|
A bit of a pain, but if that is what you have tthen that is what you have and you have to work with it.
The only thing I can think of is to ensure that (since you are using find .) you are in the lowest level common directory of where the files can be found (assuming and hoping there is one - other than /).
|
|
#5
February 21st, 2006, 10:37 AM
|
|||
|
|||
|
Assuming that you built the sqlout from some criteria, couldn't you just find your way through the directory tree and print the file name?
IE find . | xargs grep slqoutCriteria > outputFile Seems like your are grabbing a filename from your file and finding that file in the directory tree. Like you're not shorting your find. Hope I understand you right....
|
|
#6
February 22nd, 2006, 07:00 AM
|
||||
|
||||
|
To, possibly, improve the speed - cut down the iterations of the find command. Thus, do the comamnd just the once, save the results, then interrogate that file.
So, working with what my assumption of the requirement is, we get: Code:
bd=`pwd`
ts=/tmp/cpy.sh
of=/tmp/fl.tmp
cat /dev/null > $ts
find . -type f > $of
for x in `cat sqlout`
do
grep "\/${x}$" $of | cut -c2- | awk -v b=$bd '{printf("cp %s%s /target\n",b,$1)}' >> $ts
done
sh $ts
|
|
| Viewing: Dev Shed Forums > Operating Systems > UNIX Help > Cat foo|xargs -I {} find {} -exec ... AGHHH! |
| Thread Tools | Search this Thread |
| Display Modes | Rate This Thread |
|
|
 |
|
|
Del.icio.us
Digg
Google
Spurl
Blink
Furl
Simpy
Y! MyWeb
Linear Mode
