Efficient application logging design in multi-threaded apps

In an application that has to do a tremendous amount of logging to disk, we are trying to understand techniques for multi-threaded apps.

In our first design, each thread would call a logging function which blocks other threads from calling until it's complete. This slows down not only the thread that is writing but all others waiting to write.

So we implement a logging queue in memory where threads would lock a resource just long enough to copy the logging information to the memory table and then continue on. A seperate logging thread would cycle, locking that same resource long enough to copy a message out of the table, and then it could take it's time writing to disk. The problem now is that the other threads enter new messages faster than the logger thread and it gets way behind and as the queue grows we consume large amounts of memory.

Does anyone have any thoughts on an efficient design or know where to look for articles on it?

Tx for your thoughts!

I would recommend using a syncronized logging method that will spawn a new thread for each log activity and assign a low priority to that thread to perform it's job. This way the only lag in this method will be the spawn of a new thread, which should be insignifigant. Hopefully this will remove delay that could be apparent to calling threads.

Cheers,
Cypher

log4j from apache is really fast.

If you are talking serious performance issues, you might want to have this on a seperate box.

Your application could send the log over a web service (or remoting if .Net, or RMI if Java using binary formatter).

On the server, it would place the log message in a queueing component so the comsumer of the service doesn't have to wait for it to be processed before returning to them.

It just means your infrastructure team has an extra box to take care of. It's really not that hard, and it's a scalable solution, since you have a new layer there for logging.

Regards,
Eli