|
|||||||||
|
|||||||||
| |||||||||
|
|
|
| |||||||||
 |
|
|
«
Previous Thread
|
Next Thread
»
|
Thread Tools | Search this Thread | Rate Thread | Display Modes |
Dev Shed Forums Sponsor:
|
|
Stop making mediocre tutorials.The best tutorials are video! Camtasia Studio makes it easy to create engaging, buzz-building screen videos at any size, in any popular format. Download the free trial!
|
|
#1
April 24th, 2006, 04:02 PM
|
||||
|
||||
|
Checking for duplicate file names
First off, this is a repost of a post of mine in the linux forum yesterday. I didn't realize that there is a seperate scripts forum. Sorry for the noise.
I need to check a bunch of recursive directories for duplicate file names. I plan on moving them all into one large directory, and don't want to overwrite a file by accident. Is there a way I can check this in bash? I don't know any bash, but I know enough C to print out Hello World. Better yet, is ther a linux command that does this? Thanks, all.
__________________
. . . What is Firefox? . . . . . . What is Linux? . . . . . . . . . . . What is Love?
|
|
#2
April 24th, 2006, 07:30 PM
|
||||
|
||||
|
Interesting problem - without putting too much thought into it (i.e., this is probably a crap solution!) how about something like:
Code:
find /base/dir/path -type f -exec ls -1 {} \; > /a/temp/file
for f in `cat /a/temp/file`
do
fn=`basename $f`
echo "$fn:$f" >> /another/temp/file
done
sort -k1 -t: -u /another/temp.file > /yet/another/tempfile
diff /another/temp/file /yet/another/tempfile
Basically - get a list of files (with full (relative) path and save it. Process that file and get the filename alone and put that, and the full path into another file, which you then sort on the 1st field, asking for the output to be unique (i.e., no duplicates). Diff thte two files and any differences SHOULD be any duplicate filenames. That, at least, is the theory!
|
|
#3
April 24th, 2006, 07:37 PM
|
||||
|
||||
|
Thanks. I can follow it a bit, but I've a long way to go before I write something like that. I'll whip up a test directory and try it. Thank you.
|
|
#4
April 24th, 2006, 07:47 PM
|
||||
|
||||
|
*blinks at the rapid reply* Wow, I didn;t see you looking over my shoulder .... and .. hey get OUT of my bedroom!!!!!
 Good luck with it, let me know if you have any issues - apart from the temp file names I tried to get the syntax as correct as possible but memory is not my ... er, what was I saying ...? You may need to 'quote' the colon in the sort command - sort -k2 -t\: Edit: I made the assumptions: a) you have a songle base dir under which all the subdir and files are found. If there is not an easy or obvious base dir you can repeat the directory search: find /fred/charlie/files /george/bill/files- type f .... b) that no file names will have a space character in them. Last edited by Ehlanna : April 24th, 2006 at 07:50 PM. Reason: additional info
|
|
#5
April 24th, 2006, 08:33 PM
|
||||
|
||||
|
Quote:
Oh, that's a bad assumption. Some of those files are wife-named and eliminating spaces disrupts the delicate linux-wife balance I work so hard to maintain. What would I have to quote in order to make this spouse-proof? (only I will run it, but the spaces must stay  )
|
|
#6
April 25th, 2006, 12:12 PM
|
||||
|
||||
|
Ok a bit more awake now ...
Let's take a gander at this and try to remember *shudder* spaces in names ... Code:
# Set up some temp files
TF1=/tmp/filelist.txt
TF2=/tmp/filesort.txt
TF3=/tmp/filediff.txt
# List files, extract the name from the path, sort and output to file
find /base/dir -type f -exec ls -1 {} \; | awk -F\/ '{printf("\"%s\"\n",$(NF))}' | sort > $TF1
# Sort, removing duplicates
sort -u $TF1 > $TF2
# compare files, any diffs are duplicates
diff $TF1 $TF2 | grep "^< " | cut -c3- > $TF3
#We now have a list of any duplicate filenames, 'framed' in double quotes
|
|
#7
April 25th, 2006, 01:54 PM
|
||||
|
||||
|
Quote:
LMAO, I hope you meant "of" there!!! If not, it depends on what the tons are of ...More serioulsy, I just hope it works - it is a little cack-handed, and I am sure theer are more elegant ways to go: probably chucking an xargs -I {} in somewhere.
|
|
#8
April 25th, 2006, 02:34 PM
|
||||
|
||||
|
Oh yeah, my fingers don't always say what I'd like them too. Well, neither does my mouth for that matter...
I'm going to have to play with this a little later. The more that I want to do something, the more attention I must give to other matters... But thank you very much. I hate asking people to do my work for me, but bash scripting is just something that I can't seem to find the time to learn properly. Know that I do learn from your example and that I am not so lazy! Thank you.
|
|
#9
April 25th, 2006, 05:09 PM
|
||||
|
||||
|
I am pleased to hear that you, and I hope others, are learning - I think I must have the genes of a teacher lurking somewhere and I enjoy passing on knowledge (as well as acquiring it).
|
|
| Viewing: Dev Shed Forums > Web Site Management > Scripts > Checking for duplicate file names |
| Thread Tools | Search this Thread |
| Display Modes | Rate This Thread |
|
|
 |
|
|
Del.icio.us
Digg
Google
Spurl
Blink
Furl
Simpy
Y! MyWeb
Linear Mode
