October 3rd, 2011, 02:19 PM
Message Queue Implementation Question - C programming
I have to build a Message Queue System for the University, like Amazon SQS. Whe have n queues, p producers and q consumers. The idea: for example, we have 5 producers and 4 consumers for the queue 1. Messages are delivered from the queue to any consumer when they ask for them. On that moment, the message is setted as "unvisible" in order not to be transferred to other(s) consumer(s). Each consumer has x seconds to process the message. If that time is exceeded, the message is marked "visible" and re-enters to the head of the queue. If the message is processed before x seconds, it is deleted (the queue receives an ACK signal). Doing so, messages are never lost (system requirement).
Messages are in plain text and others like pdf, xml, etc. But they are small (no longer than 100KB). MIME is suggested, and the use of meta-information. The programming language should be C, and should use UNIX domain sockets (Linux operating system). Also, there will be an administrator who will manage the queues (creation, destruction, etc), using other IPC protocols like pipes and/or signals
My question is, what kind of implementation would you suggest for and why (linked list, circular buffer, etc)?