Emails - Cron or no Cron

Hi, i have built a custom site using Codeigniter.

My site allows users to post a story to the site... then other users can comment on the story.

There is an option under the users Settings to choose whether you want to be notified by email if:

A - MY STORY
[ ] a user comments on my story

B - MY COMMENTS ON OTHER USER'S STORIES
[ ] another user comments on a story that i have commented on


I don't think i need any special script or service to handle email A, as it is just one email to one user, however Email B is a different story as 1000 users might have commented on a story and then 1000 users might need to receive an email each time a new user comments on the same story they have commented on, so:

1 - is there a 3rd party service that i should use in conjunction with codeigniter to handle these bulk emails

2 - should i process all of these batch emails using a cron... if so can someone recommend the best way to run this process

Thanks in advance for your help...

You can run a script in the background using proc_open or shell_exec.

This person has the exact same situation as you do.

Cool, some people mentioned using SendGrid

I go for the mail queue option. A cron runs looking at the mail queue.

The main app logic just populates the mail queue.

this way the expensive task of sending out the emails runs in a separate process the the user visiting the site and commenting

Rhytz's solution will also make it a separate process. I would probably go this route since you won't need extra tables to hold the queue, however, either will work well.

Ok, thanks... so are you saying there is no need to go with a 3rd party email service provider

No need, but you might want to consider using PHPMailer just for sending your normal emails.

There's no need to...it depends on your requirements.

Part of what we do where we work is to design, build and broadcast email marketing campaigns for our clients. Some of our clients have lists with only a few thousand email addresses on it, others have hundreds of thousands in their lists.

This started to take a toll on our dedicated box which is was primarily being used to host client's websites and transactional mail

when we got to a stage where our server was sending out too many bulk emails that it interfered with transactional mail we started to look for an alternative.

We considered sendgrid, but then figured out we could just do what sendgrid do - but a lot cheaper, and just for us - so we set up a series of VPSs with postfix installed to run as a relay server, locked it down to our dedicated server's IP (where our email marketing web app runs from) and there we had it - our own private network of relay servers, each one costing about £10 month for hosting and bandwidth. I recken each server is capable of sending out 5 million emails per week at full capacity (add a few £ for extra bandwidth) and each server can be resized (currently £0.01/hour for 256Mb server; could be £0.02 for 512Mb, doubling each time up to 32 GB ram)

If a new client comes on board with huge lists then we create a new VPS just for them, so IP reputation can be contained / limited and as the general volume grows we can resize the VPS as necessary.

We use PHPMailer's SMTP mode to connect to our remote VPSs

Sending out 1000 emails takes a few minutes using a mysql based mail queue (looping over the list and sending our through PHPMailer). Our cron sends out 300 / minute. However, these are full HTML marketing emails. For notication emails like devshed send out you could probably increase this limit quite a bit.

The main software we use is actually 3rd party and partly closed source but when I rebuild it for the new version of our software I will not be limiting the number of emails sent out on the cron. Instead the script will have a time limit on it, and each time the script runs it will grap a few records, send these out, time the sending and keep an eye on the remaining time, then die gracefully before the script time limit - meaning that email delivery is not halted part way through delivery and also be "multi-threaded" by having the time limit greater than the cron interval

I recommend using a queue+cron over a background process because it allows you to throttle outgoing mail more easily and is more robust. Sending mail is an expensive task, so if you're sending out more than one about email in a request you need to use one method or the other.

There are two primary problems with using a background task:

1) If you spawn a new background task every time situation B occurs, you might end up with many backgrounds tasks all trying to send thousands of emails at once. You could bog down the server easily.

2) If your background task has sent 500 of 1000 emails and your server crashes, you lose the last 500 emails. With a queue+cron, you don't.


If you're going to be sending thousands of emails I recommend that you do go with a mail server provider like SendGrid. What Northie said in his last post is true, but most companies are not big enough that it is economical for them to do what Northie has done. Correctly sysadmin'ing a mail server is a lot of work.

Thanks guys, i think SendGrid is my best option, thanks...