Too many files hurts performance?

Note: This was originally posted in the MySQL forum http://forums.devshed.com/t106677/s.html

For every record in my database I am storing two files named after the records primary key. When a user submits a query, the files are called for each record that matches his query. I will be having several million records and the files will range from 5kb to 2mg.

This is working fine on a small scale (5,000 records), but I read this from http://lists.mysql.com/java/900
So I am trying a directory structure using the first four digits of the primary key. Ex: 1234 would go into /1/2/3/4/ and 12345 would go into /2/3/4/5/. I am currently using Windows XP but plan to use FreeBSD 4.X or 5.X.

How does FreeBSD handle several files in a directory (like 10,000-100,000)? Am I better off breaking the files up by digits, or is this directory structure inefficient?

Thank you,
Brandon

It's not about directory alone but partition, if you run out of inode, creating more subdirs and relocate files under it doesn't change your inode capacity.

I'm relatively new to Unix and have researched Inode since FreeBSD's post, but I still am a little unsure of the inodes default setting and how to set it larger.

I’m using FreeBSD’s default partition. I have a pretty minimal install, only using the command line interface and just a couple programs (MySQL, Java SDK). I plan to add anywhere from 500 million to 1 billion files. Will the default partition for a 100gig hard drive have a large enough inode capacity?

Also, does it matter how many files are in a given directory (assuming it’s under the inode limit) when locating a given file? I am splitting them up as mentioned earlier so there won’t be too many files in one directory, but if it doesn’t matter then I will put all the files in one.

Thanks again,
Brandon

>> but I still am a little unsure of the inodes default setting and how to set it larger.
See the -i option for newfs. Online man page at: http://www.freebsd.org/cgi/man.cgi?...ASE&format=html

Thank you Scorpions4Ever, I had no idea what command to search for. As for my second question:

"does it matter how many files are in a given directory (assuming it’s under the inode limit) when locating a given file? I am splitting them up as mentioned earlier so there won’t be too many files in one directory, but if it doesn’t matter then I will put all the files in one."