QueueBackgroundWorkItem to reliably schedule and run background processes in ASP.NET

QueueBackgroundWorkItem to reliably schedule and run background processes in ASP.NET

Rate This
  • Comments 20

Stack Overflow is loaded with questions on how to reliably run a resource-intensive process on a background thread. See  so0, so1, so2, so3, so4, so5, so6, so7, so8, so9, so10 . Examples of long-running tasks include sending email, image processing, and generating a PDF file. When Phil Haack was a program manager on the ASP.NET MVC team, he wrote the definitive blog on the inherent unreliability of running background tasks on ASP.NET. While Phil’s blog is a good read, there are now three supported approaches to launching long-running process on ASP.NET:

  1. Cloud Services worker role is an environment in which you can run code. It’s basically a computer, really. You run whatever code you want (EXE, BAT, PS1, NodeJS, .NET, etc.)  An Azure worker role provides the most industrial strength and scalable solution to this problem.  For an excellent tutorial with this approach, see Tom Dykstra’s Get Started with Azure Cloud Services and ASP.NET.
  2. WebJobs (including the WebJobs SDK) are a way in Azure to run scheduled tasks or tasks that trigger on demand (given various types of triggers). Apps specifically written with the Azure Jobs SDK can be used to run code in any environment, including a local computer, Azure Web Site, Azure Worker Role, Azure VM, etc.  Although you can run them anywhere, they run most efficiently within Azure. For more information see Azure WebJobs - Recommended Resources.
  3. QueueBackgroundWorkItem (QBWI). This was specifically added to enable ASP.NET apps to reliably run short-lived background tasks. (With some limitations explained at the end of this blog.)    As of today, you can’t use QBWI on an Azure Web Site or Cloud Services web role because QBWI requires .Net 4.5.2. We hope to have Azure Web/Cloud running .Net 4.5.2 soon.

In addition to the three supported approaches above, the open source HangFire package allows you to run background tasks.

QueueBackgroundWorkItem overview

QBWI schedules a task which can run in the background, independent of any request. This differs from a normal ThreadPool work item in that ASP.NET automatically keeps track of how many work items registered through this API are currently running, and the ASP.NET runtime will try to delay AppDomain shutdown until these work items have finished executing.

QueueBackgroundWorkItem API

[SecurityPermission(SecurityAction.LinkDemand, Unrestricted =true)]
public static void QueueBackgroundWorkItem(Action<CancellationToken> workItem);

Takes a void-returning callback; the work item will be considered finished when the callback returns.

[SecurityPermission(SecurityAction.LinkDemand, Unrestricted = true)] 
public static void QueueBackgroundWorkItem(Func<CancellationToken, Task> workItem);

Takes a Task returning callback; the work item will be considered finished when the returned Task transitions to a terminal state.

Send email with attachment using QBWI

To use QBWI (QueueBackgroundWorkItem) in Visual Studio, you’ll need to install .Net 4.5.2, then install the .Net 4.5.2 Developer Pack. For my sample I created an MVC app and used SendGrid to send an email with a large jpg attachment. To use QBWI, you’ll need to right click the project in solution explore and select Properties. Select the Application tab on the left, then select .Net Framework 4.5.2 in the Target Framework dropdown. If you don’t see 4.5.2, you didn’t install the .Net 4.5.2 Developer Pack or you don’t have .Net 4.5.2 installed.

netVers

The following code sends email with an image file attached:

 public ActionResult SendEmail2([Bind(Include = "Name,Email")] User user)
 {
    if (ModelState.IsValid)
    {
       HostingEnvironment.QueueBackgroundWorkItem(ct => SendMailAsync(user.Email));
       return RedirectToAction("Index", "Home");
    }

    return View(user);
 }   

 private async Task SendMailAsync(string email)
 {
    var myMessage = new SendGridMessage();

    myMessage.From = new MailAddress("Rick@Contoso.com");
    myMessage.AddTo(email);
    myMessage.Subject = "Using QueueBackgroundWorkItem";

    //Add the HTML and Text bodies
    myMessage.Html = "<p>Check out my new blog at "
          + "<a href=\"http://blogs.msdn.com/b/webdev/\">"
          + "http://blogs.msdn.com/b/webdev/</a></p>";
    myMessage.Text = "Check out my new blog at http://blogs.msdn.com/b/webdev/";

    using (var attachmentFS = new FileStream(GH.FilePath, FileMode.Open))
    {
       myMessage.AddAttachment(attachmentFS, "My Cool File.jpg");
    }

    var credentials = new NetworkCredential(
       ConfigurationManager.AppSettings["mailAccount"],
       ConfigurationManager.AppSettings["mailPassword"]
       );

    // Create a Web transport for sending email.
    var transportWeb = new Web(credentials);

    if (transportWeb != null)
       await transportWeb.DeliverAsync(myMessage);
 }

I set the account and password on the Configure tab in the Azure portal to keep my credentials secure.

conTab

Using the cancellation token

You can drop the following code in a new MVC app to test the cancellation token:

public class HomeController : Controller
{
   static int logCount = 0;
   public ActionResult Index()
   {
      return View();
   }

   public ActionResult About()
   {
      ViewBag.Message = "Your application description page.";
      HostingEnvironment.QueueBackgroundWorkItem(ct => workItemAction1(ct, "About"));
      return View();
   }

   public ActionResult Contact()
   {
      ViewBag.Message = "Your contact page.";
      string info = "Headers: " + HttpContext.Request.Headers.AllKeys.ToString()
              + "  URI: " + HttpContext.Request.Url.AbsoluteUri.ToString()
              + " User: " + HttpContext.User.ToString();

      HostingEnvironment.QueueBackgroundWorkItem(ct => workItem1Async(ct, "Contact"));
      return View();
   }

   private void workItemAction1(CancellationToken ct, string msg)
   {
      logCount++;
      int currentLogCount = logCount;

      ct = addLog(ct, currentLogCount, msg);
   }

   private async Task<CancellationToken> workItem1Async(CancellationToken ct, string msg)
   {
      logCount++;
      int currentLogCount = logCount;

      await addLogAsync(ct, currentLogCount, msg);
      return ct;
   }

   private CancellationToken addLog(CancellationToken ct, int currentLogCount, string msg)
   {
      Trace.TraceInformation(msg);

      for (int i = 0; i < 5; i++)
      {
         if (ct.IsCancellationRequested)
         {
            Trace.TraceWarning(string.Format("{0} - signaled cancellation", 
               DateTime.Now.ToLongTimeString()));
            break;
         }
         Trace.TraceInformation(string.Format("{0} - logcount:{1}", 
            DateTime.Now.ToLongTimeString(), currentLogCount));
         Thread.Sleep(6000);
      }
      return ct;
   }

   private async Task<CancellationToken> addLogAsync(
      CancellationToken ct, int currentLogCount, string msg)
   {

      try
      {
         for (int i = 0; i < 5; i++)
         {
            if (ct.IsCancellationRequested)
            {
               Trace.TraceWarning(string.Format("{0} - signaled cancellation : msg {1}",
                  DateTime.Now.ToLongTimeString(), msg));
               break;
            }
            Trace.TraceInformation(string.Format("{0} - msg:{1} - logcount:{2}",
               DateTime.Now.Second.ToString(), msg, currentLogCount));

            // "Simulate" this operation took a long time, but was able to run without
            // blocking the calling thread (i.e., it's doing I/O operations which are async)
            // We use Task.Delay rather than Thread.Sleep, because Task.Delay returns
            // the thread immediately back to the thread-pool, whereas Thread.Sleep blocks it.
            // Task.Delay is essentially the asynchronous version of Thread.Sleep:
            await Task.Delay(2000, ct);
         }
      }
      catch (TaskCanceledException tce)
      {
         Trace.TraceError("Caught TaskCanceledException - signaled cancellation " + tce.Message);
      }
      return ct;
   }
}

Hit F5 to debug the app, then click on the About or Contact link. Right click on the IIS Express icon in the task notification area and select Exit.

iisX

The visual studio output window shows the task is canceled.

out

QueueBackgroundWorkItem limitations

  • The QBWI API cannot be called outside of an ASP.NET-managed AppDomain.
  • The AppDomain shutdown can only be delayed 90 seconds (It's actually the minimum of HttpRuntimeSection.ShutdownTimeout and processModel shutdownTimeLimit). If you have so many items queued that they can’t be completed in 90 seconds, the ASP.NET runtime will unload the AppDomain without waiting for the work items to finish.
  • The caller's ExecutionContext is not flowed to the work item. For example, the code that you run in the background thread doesn’t have access to commonly used context properties. If you need HttpContext information, copy the values you care about to a state object or inside a closure and pass it in to the background worker.  Don’t pass the HttpContext instance itself, as it’s not a thread-safe object and even simple property getters (like HttpContext.Request.Url) might throw.
  • Scheduled work items are not guaranteed to ever execute, once the app pool starts to shut down, QueueBackgroundWorkItem calls will not be honored.
  • The provided CancellationToken will be signaled when the application is shutting down. The work item should make every effort to honor this token.  If a work item does not honor this token and continues executing, the ASP.NET runtime will unload the AppDomain without waiting for the work item to finish.
  • We don’t guarantee that background work items will ever get invoked or will run to completion.  For instance, if we believe a background work item is misbehaving, we’ll kill it.  And if the w3wp.exe process crashes, all background work items are obviously dead.  If you need reliability, you should use Azure’s built-in scheduling functions.

Special thanks to @LeviBroderick  who not only wrote the QBWI code, but helped me with this post.

Follow me ( @RickAndMSFT )   on twitter where I have a no spam guarantee of quality tweets.

Leave a Comment
  • Please add 8 and 7 and type the answer here:
  • Post
  • Cool.

    Is there a downloadable sample?

    Thanks.

  • What to do when you need in the "AddLog" message access to the HttpContext?

    Thanks

  • Why all this when you can just set up nservice bus.

  • Thanks for documenting this so clearly!

    BTW, your SendEmail example does not use QueueBackgroundWorkItem.

  • @Stephen Cleary   BTW, your SendEmail example does not use QueueBackgroundWorkItem.

    Fixed, thanks for pointing that out. I accidently pasted the version I tested on Azure, which doesn't currently support QBWI

    @paul van bladel  - Sample

    I'll get working on one, stay tuned.

  • @mike   Why all this when you can just set up nservice bus.

    This is vastly simpler than service bus and doesn't require Azure. Even my Cloud services sample cited doesn't use service bus.

  • @paul van bladel   What to do when you need in the "AddLog" message access to the HttpContext?

    Good question, I added that to the limitations section.

  • Nice post

  • the example shows the usage of the cancellationtoken when the app is shut down.

    What's the way to proceed to cancel the task externally (e.g. via another call to the controller) ?

    Thanks.

    paul.

  • If you want to forget about all problems related to background processing inside ASP.NET applications, look at http://hangfire.io – the simplest, reliable and transparent solution that works even on shared hosting.

  • @odinserj  - thanks, I added HangFire to the article.

  • Great post......Thank you.

  • Thanks for the article.  Those are some serious limitations IMO.

  • "The AppDomain shutdown can only be delayed 90 seconds (It's actually the minimum of HttpRuntimeSection.ShutdownTimeout and processModel shutdownTimeLimit). If you have so many items queued that they can’t be completed in 30 seconds, the ASP.NET runtime will unload the AppDomain without waiting for the work items to finish. "

    It should be 90 all around ?

  • @Andrei Ignat  90 seconds

    Thanks! I'll fix that

Page 1 of 2 (20 items) 12