0
\$\begingroup\$

I need to build a application which sends a lot of emails. These emails are based on a table. Basic idea is, I grab a bunch of records from database. Create MailMessage (System.Net.Mail) and send it. Since there are a lot of mails to be sent, I am thinking about developing a multi-threaded application.

    public void Run()
    {
        var mailItems = MailItemDataProvider.GetLiveMailItems();
        if (mailItems.Count > 0)
        {
            var maxThreads = 5;
            var totalItems = mailItems.Count;
            int skipper = totalItems / (maxThreads - 1);
            if (skipper == 0)
                skipper = totalItems;

            for (var i = 0; i < totalItems; i += skipper)
            {
                var itemsLot = mailItems.Skip(i).Take(skipper).ToList();
                Task.Factory.StartNew(() => ProcessMails(itemsLot, ThreadPriority.Highest));
            }
        }
    }

It basically grabs a thousand records from the database and divide them into five chunks. The chunks are passed to the mail processor. The above function is triggered from a timer (Timer_Elapsed event) configured to run every minute. This is actually part of Windows Service. Mail processor is like this:

    protected void ProcessMails(IEnumerable<MailItem> mailItems)
    {
        foreach (var mailItem in mailItems)
        {
            bool errorFound;
            var mail = CreateMailMessage(mailItem, out errorFound); 
            // returns (System.Net.Mail) MailMessage object

            if (!errorFound)
            {
                if (Mailer.SendEmail(mail))
                {
                    mailItem.StatusId = Status.Sent;
                }
                else
                {
                    mailItem.StatusId = Status.Failed;
                    mailItem.FailMessage = "Error sending email.";
                }
            }
            else
            {
                mailItem.StatusId = Status.Failed;
            }
        }
        UpdateMailItems(mailItems); 
        // this updates mail sent status in the database
    }

In my test environment, it is working fine. I have tested for 100,000 emails. Following are my concerns in the function that is processed by the threads:

  1. What will happen when SMTP slows down (for some unknown reason)? I start up five threads, but it will add up five more before the first five complete. How many threads can my server handle? (I know it depends on actual resources available, but practically, how will it perform?)
  2. I have database update inside the process. Is that good idea?
  3. Is there a chance that some threads will never finish? Do I need to make sure that threads are completed? How do I detect the ever running threads and kill them?

I am new to multi threading. Please review my code. Any comments or suggestion are welcome.

\$\endgroup\$
  • \$\begingroup\$ Why do you need to send so many emails? \$\endgroup\$ – Lstor Jun 28 '13 at 2:06
  • \$\begingroup\$ It's a business requirement. \$\endgroup\$ – Romi Jun 28 '13 at 3:36
  • \$\begingroup\$ "It's a business requirement." - Is your business the spam business scnr ;-) \$\endgroup\$ – frugi Jun 28 '13 at 6:37
  • \$\begingroup\$ No intention of spamming, honestly. \$\endgroup\$ – Romi Jun 28 '13 at 6:57
3
\$\begingroup\$

Instead of rolling your own version of a parallel for each loop, you should use Parallel.ForEach(). It will make your code simpler, avoid the possibility of bugs in your Run() method and you will still be able to configure the number of threads used (through ParallelOptions).


I start up five threads, but it will add up five more before the first five complete.

If I understand your code correctly, the only way this could happen is if you called Run() for a second time. If that's the case, it might make sense to use the producer-consumer pattern using BlockingCollection<T> and GetConsumingEnumerable() (or GetConsumingPartitioner() if you decide to switch to Parallel.ForEach()).

How many threads can my server handle?

There is no way for us to answer that. You will have to perform your own measurements.

I have database update inside the process. Is that good idea?

I don't see why would that cause any significant issues.

Is there a chance that some threads will never finish?

There is no way for us to know that without seeing the code you didn't show.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.