Phone + Cloud Series: Polling Stock Quotes with an Azure Worker Role

Article
12/20/2010

Introduction

This is a pretty common task in any financial app. Though usually you’d only poll stocks every 15 minutes, it makes sense to put it in a worker role so that you can send push notifications to your users when a stock goes through the roof, or plummets, or whatever you choose.

Azure worker roles are unique in that they are an accepted infinite loop. Most programs have a defined start and end, but a worker role allows you to always do some kind of background processing, making it perfect for an application like this. Worker roles, however, are not cheap. This is why we’ll just test it in the Azure development fabric instead of real Azure. This worker role is the glue in the application; once you turn it on, things start working!

Let’s begin by creating our Azure project. We’ll need a worker role as well as a WCF service Web Role which the end-user devices will use to register their push notification address. This is an externally available endpoint that you can use to invoke server behavior without using a web interface, much like an XML web service.

Creating the Azure Project and Roles

The first thing to do is go to File | New Project | Cloud. If you don’t have the Azure tools installed, the singular selection will allow you to do so. Otherwise, select Windows Azure Project. I named mine AzureStockAlertService.
The next screen gives you an option to add roles. We want a WCF Service Web Role and a Worker Role. I named the WCF service PushRegService and the Worker Role StockMonitor. Click OK and the projects will be created.

Again, the purpose of these services: PushRegService is something the phone can call remotely using WCF. Since only the phone actually knows what its Push Notification URI is, this is a mechanism to “register” it with our Azure service, by way of entering the Push Notification URI into our SQL Azure database. StockMonitor is responsible for polling stocks and telling the Microsoft Push Notification Service to send notifications to the users who have requested them who have stock activity happening.
Visual Studio creates all of the role definitions and projects for you. I used to think code generation was for babies; now I’m not so sure.

Next, we want to create a new project in this solution that contains logic that both of these services need to be able to access. First of all, our object-relational mappings (LINQ to SQL or LINQ to Entity Framework, for example) and the push notification library. Barring good architectural practices for now, we’ll just stuff these shared things into a new project called Utilities.

Creating the shared Utilities project

The purpose of this library is to provide a central location for our data interface (LINQ to SQL, for example) and the push notifications stuff that I stole from an App Hub sample for push notifications in XNA. Both the WCF role and the worker role will reference this project.

Right-click on the solution in Solution Explorer and go to Add | New Project. Under Visual C# choose Class Library. Name it Utilities. Delete the eponymous Class1.cs class.
Download that sample I talked about a second ago and inspect the code. I’ve used two files from this project.

Create a new folder called PushNotifications in the Utilities project and add the code below. You can start with the code from the sample if you want; basically all I have done is renamed some namespaces and “made sure it all compiled.”

In the PushNotifications folder in the Utilities project, add a new class file:
PushNotificationSender.cs

    1: //----------------------------------------------------------------------------- 
    2: // PushNotificationSender.cs
    3: // 
    4: // Microsoft XNA Community Game Platform
    5: // Copyright (C) Microsoft Corporation. All rights reserved. 
    6: //----------------------------------------------------------------------------- 
    7:   
    8: using System; 
    9: using System.Text; 
   10: using System.Net; 
   11: using System.IO; 
   12:   
   13: namespace Utilities.PushNotifications
   14: { 
   15:     /// <summary> 
   16:     /// A utility class for sending the three different types of push notifications. 
   17:     /// </summary> 
   18:     public class PushNotificationSender
   19:     { 
   20:         public enum NotificationType
   21:         { 
   22:             Tile = 1, 
   23:             Toast = 2, 
   24:             Raw = 3
   25:         } 
   26:   
   27:         public delegate void SendCompletedEventHandler(PushNotificationCallbackArgs args); 
   28:         public event SendCompletedEventHandler NotificationSendCompleted; 
   29:   
   30:         public const string MESSAGE_ID_HEADER = "X-MessageID"; 
   31:         public const string NOTIFICATION_CLASS_HEADER = "X-NotificationClass"; 
   32:         public const string NOTIFICATION_STATUS_HEADER = "X-NotificationStatus"; 
   33:         public const string DEVICE_CONNECTION_STATUS_HEADER = "X-DeviceConnectionStatus"; 
   34:         public const string SUBSCRIPTION_STATUS_HEADER = "X-SubscriptionStatus"; 
   35:         public const string WINDOWSPHONE_TARGET_HEADER = "X-WindowsPhone-Target"; 
   36:         public const int MAX_PAYLOAD_LENGTH = 1024; 
   37:   
   38:   
   39:         /// <summary> 
   40:         /// Sends a raw notification, which is just a byte payload. 
   41:         /// </summary> 
   42:         public void SendRawNotification(Uri deviceUri, byte[] payload) 
   43:         { 
   44:             SendNotificationByType(deviceUri, payload, NotificationType.Raw); 
   45:         } 
   46:   
   47:   
   48:         /// <summary> 
   49:         /// Sends a tile notification, which is a title, a count, and an image URI. 
   50:         /// </summary> 
   51:         public void SendTileNotification(Uri deviceUri, string title, int count, string backgroundImage) 
   52:         { 
   53:             // Malformed push notifications cause exceptions to be thrown on the receiving end
   54:             // so make sure we have valid data to start with. 
   55:             if (string.IsNullOrEmpty(title)) 
   56:             { 
   57:                 throw new InvalidOperationException("Tile notifications require title text"); 
   58:             } 
   59:   
   60:             // Set up the XML
   61:             string msg = 
   62:                 "<?xml version=\"1.0\" encoding=\"utf-8\"?>" + 
   63:                 "<wp:Notification xmlns:wp=\"WPNotification\">" + 
   64:                     "<wp:Tile>"; 
   65:             if (!string.IsNullOrEmpty(backgroundImage)) 
   66:             { 
   67:                 msg += "<wp:BackgroundImage>" + backgroundImage + "</wp:BackgroundImage>"; 
   68:             } 
   69:             msg += 
   70:         "<wp:Count>" + count.ToString() + "</wp:Count>" + 
   71:         "<wp:Title>" + title + "</wp:Title>" + 
   72:     "</wp:Tile>" + 
   73: "</wp:Notification>"; 
   74:   
   75:             byte[] payload = new UTF8Encoding().GetBytes(msg); 
   76:   
   77:             SendNotificationByType(deviceUri, payload, NotificationType.Tile); 
   78:         } 
   79:   
   80:   
   81:         /// <summary> 
   82:         /// Sends a toast notification, which is two lines of text. 
   83:         /// </summary> 
   84:         public void SendToastNotification(Uri deviceUri, string text1, string text2) 
   85:         { 
   86:             // Malformed push notifications cause exceptions to be thrown on the receiving end
   87:             // so make sure we have valid data to start with. 
   88:             if (string.IsNullOrEmpty(text1) && string.IsNullOrEmpty(text2)) 
   89:             { 
   90:                 throw new InvalidOperationException("toast notifications must have at least 1 valid string"); 
   91:             } 
   92:   
   93:             // Set up the XML            
   94:             string msg = 
   95:                 "<?xml version=\"1.0\" encoding=\"utf-8\"?>" + 
   96:                 "<wp:Notification xmlns:wp=\"WPNotification\">" + 
   97:                   "<wp:Toast>" + 
   98:                      "<wp:Text1>" + text1 + "</wp:Text1>" + 
   99:                      "<wp:Text2>" + text2 + "</wp:Text2>" + 
  100:                   "</wp:Toast>" + 
  101:                 "</wp:Notification>"; 
  102:   
  103:             byte[] payload = new UTF8Encoding().GetBytes(msg); 
  104:   
  105:             SendNotificationByType(deviceUri, payload, NotificationType.Toast); 
  106:         } 
  107:   
  108:   
  109:         /// <summary> 
  110:         /// helper function to set up the request headers based on type and send the notification payload. 
  111:         /// </summary> 
  112:         private void SendNotificationByType(Uri channelUri, byte[] payload, NotificationType notificationType) 
  113:         { 
  114:             // Check the length of the payload and reject it if too long. 
  115:             if (payload.Length > MAX_PAYLOAD_LENGTH) 
  116:                 throw new ArgumentOutOfRangeException("Payload is too long. Maximum payload size shouldn't exceed " + MAX_PAYLOAD_LENGTH.ToString() + " bytes"); 
  117:   
  118:             try
  119:             { 
  120:                 // Create and initialize the request object. 
  121:                 HttpWebRequest request = (HttpWebRequest)WebRequest.Create(channelUri); 
  122:                 request.Method = WebRequestMethods.Http.Post; 
  123:                 request.ContentLength = payload.Length; 
  124:                 request.Headers[MESSAGE_ID_HEADER] = Guid.NewGuid().ToString(); 
  125:   
  126:                 // Each type of push notification uses a different code in its X-NotificationClass
  127:                 // header to specify its delivery priority.  The three priorities are: 
  128:   
  129:                 //     Realtime.  The notification is delivered as soon as possible. 
  130:                 //     Priority.  The notification is delivered within 450 seconds. 
  131:                 //     Regular.   The notification is delivered within 900 seconds. 
  132:   
  133:                 //          Realtime    Priority    Regular
  134:                 // Raw    3-10        13-20       23-31
  135:                 // Tile   1           11          21
  136:                 // Toast  2           12          22
  137:   
  138:                 switch (notificationType) 
  139:                 { 
  140:                     case NotificationType.Tile: 
  141:                         // the notification type for a tile notification is "token". 
  142:                         request.Headers[WINDOWSPHONE_TARGET_HEADER] = "token"; 
  143:                         request.ContentType = "text/xml"; 
  144:                         // Request real-time delivery for tile notifications. 
  145:                         request.Headers[NOTIFICATION_CLASS_HEADER] = "1"; 
  146:                         break; 
  147:   
  148:                     case NotificationType.Toast: 
  149:                         request.Headers[WINDOWSPHONE_TARGET_HEADER] = "toast"; 
  150:                         request.ContentType = "text/xml"; 
  151:                         // Request real-time delivery for toast notifications. 
  152:                         request.Headers[NOTIFICATION_CLASS_HEADER] = "2"; 
  153:                         break; 
  154:   
  155:                     case NotificationType.Raw: 
  156:                         // Request real-time delivery for raw notifications. 
  157:                         request.Headers[NOTIFICATION_CLASS_HEADER] = "3"; 
  158:                         break; 
  159:   
  160:                     default: 
  161:                         throw new ArgumentException("Unknown notification type", "notificationType"); 
  162:   
  163:                 } 
  164:   
  165:                 Stream requestStream = request.GetRequestStream(); 
  166:   
  167:                 requestStream.Write(payload, 0, payload.Length); 
  168:                 requestStream.Close(); 
  169:   
  170:                 HttpWebResponse response = request.GetResponse() as HttpWebResponse; 
  171:   
  172:                 if (NotificationSendCompleted != null && response != null) 
  173:                 { 
  174:                     PushNotificationCallbackArgs args = new PushNotificationCallbackArgs(notificationType, (HttpWebResponse)response); 
  175:                     NotificationSendCompleted(args); 
  176:                 } 
  177:             } 
  178:             catch (WebException ex) 
  179:             { 
  180:                 // Notify the caller on exception as well. 
  181:                 if (NotificationSendCompleted != null) 
  182:                 { 
  183:                     if (null != ex.Response) 
  184:                     { 
  185:                         PushNotificationCallbackArgs args = new PushNotificationCallbackArgs(notificationType, (HttpWebResponse)ex.Response); 
  186:                         NotificationSendCompleted(args); 
  187:                     } 
  188:                 } 
  189:             } 
  190:         } 
  191:     } 
  192: }

And in the same folder, add another class file: PushNotificationCallbackArgs.cs

    1: //-----------------------------------------------------------------------------
    2: // PushNotificationCallbackArgs.cs
    3: //
    4: // Microsoft XNA Community Game Platform
    5: // Copyright (C) Microsoft Corporation. All rights reserved.
    6: //-----------------------------------------------------------------------------
    7:  
    8: using System;
    9: using System.Net;
   10:  
   11: namespace Utilities.PushNotifications
   12: {
   13:     /// <summary>
   14:     /// A wrapper class for the status of a sent push notification.
   15:     /// </summary>
   16:     public class PushNotificationCallbackArgs
   17:     {
   18:         public PushNotificationCallbackArgs(PushNotificationSender.NotificationType notificationType, HttpWebResponse response)
   19:         {
   20:             this.Timestamp = DateTimeOffset.Now;
   21:             this.NotificationType = notificationType;
   22:  
   23:             if (null != response)
   24:             {
   25:                 this.MessageId = response.Headers[PushNotificationSender.MESSAGE_ID_HEADER];
   26:                 this.ChannelUri = response.ResponseUri.ToString();
   27:                 this.StatusCode = response.StatusCode;
   28:                 this.NotificationStatus = response.Headers[PushNotificationSender.NOTIFICATION_STATUS_HEADER];
   29:                 this.DeviceConnectionStatus = response.Headers[PushNotificationSender.DEVICE_CONNECTION_STATUS_HEADER];
   30:                 this.SubscriptionStatus = response.Headers[PushNotificationSender.SUBSCRIPTION_STATUS_HEADER];
   31:             }
   32:         }
   33:  
   34:         public DateTimeOffset Timestamp { get; private set; }
   35:         public string MessageId { get; private set; }
   36:         public string ChannelUri { get; private set; }
   37:         public PushNotificationSender.NotificationType NotificationType { get; private set; }
   38:         public HttpStatusCode StatusCode { get; private set; }
   39:         public string NotificationStatus { get; private set; }
   40:         public string DeviceConnectionStatus { get; private set; }
   41:         public string SubscriptionStatus { get; private set; }
   42:     }
   43: }

Next we are going to set up our object-relational mapping so that we can easily interact with data. There are a couple of ways to do this easily, but they both involve clear-text passwords. LINQ to SQL is by far the easiest method, but in the interest of security we’ll use LINQ to Entity Framework instead. By default, the database password for your SQL Azure database is stored in the App.config file, but LINQ to EF gives you the option to set the password programmatically later. We’ll pass on that extra step for now, though.

First, make sure you open the Azure portal at https://windows.azure.com. Navigate to the Database tab and look at the firewall rules. If you haven’t already configured your local PC to be within the range of acceptable connections, now would be a good time to do that, otherwise you won’t be able to connect to the database.
Earlier I mentioned that we’d put our ORM stuff in the Utilities project, so both the WCF web role and the worker role can access it. Right click the Utilities project and add a new folder. I called my folder ORM in case we would ever add other models to this project. Then, right click the ORM folder and add a new item. Go to the Data filter on the left and choose ADO.NET Entity Model. I named my file Users.edmx.
This will start the Entity Data Model wizard. First, choose Generate from Database, since we’ve already built the SQL Azure table.
On the “Choose your data connection,” choose the data connection you created if you already created one. Otherwise, click New Connection and fill in the data. For Server Name, use the server name as shown on your Windows Azure portal for the database server under “Fully Qualified DNS Name.” My connection properties look a bit like this. Make sure to click Test Connection before proceeding. If you get an error about the firewall, make sure you add your current IP address range to the firewall rules at windows.azure.com.

This puts you back at the “Choose your Data Connection” screen. It’s also important to select whether or not to include sensitive information in the connection string. For this simple example it’s okay to do that, as long as you don’t use that password anywhere else. Common sense, right? In the real world I would pull the password at runtime instead (using the “I will set it in my application code” setting).
Next, you can choose the tables you want to utilize in the model; in this case I just want the entire Users table. You can accept the other default settings and click Finish; you’ll have an easy-to-use interface to live SQL Azure data now. Make sure you rebuild now so the other projects have access to this new goodness.

Writing the Worker Role

Next, we will pump some code into the worker role. This code is responsible for polling the stocks and then sending push notifications “when appropriate” (in quotes because this is really subjective to what you are trying to accomplish in the app).

Add a new class to the StockMonitor project called Quote.cs. This is just a simple struct-esque data structure to hold the data we get from polling the quote service. The code for Quote.cs is below:
```
namespace StockMonitor
{
    public class Quote
    {
        public string Symbol;
        public float PercentChange;
        public float CurrentPrice;

        public Quote() { }
    }
}
```
In the StockMonitor project, add a project reference to the Utilities project as well as a reference to System.Data.Entity.
Open the StockMonitor project / WorkerRole.cs.

Add these three using directives at the top:

 using Utilities.ORM;
using Utilities.PushNotifications;
using System.IO;
using System.Text;

Add a new region at the class level called Helper Methods. This is where our new code will go.

In the Helper Methods region, add a new method called GetQuoteToPushForUser as shown below. This method does two things. First of all, it gets quotes for a given user based on what’s in the database for their account. They have an entry called AlertKeywords which contains a string like: MSFT,GE,F. This method splits on the comma and then figures out the “most interesting” stock (in this case it is the biggest percent change, but you can use more sexy LINQ goodness to change the criteria for which quote gets pushed if you want).

My not-necessarily-bug-free code for this method is below:

    1: private Quote GetQuoteToPushForUser(User user)
    2: {
    3:     // Generate the URL for the Yahoo service.
    4:     StringBuilder sb = new StringBuilder("https://quote.yahoo.com/d/quotes.csv?s=");
    5:     string[] symbols = user.AlertKeywords.Split(',');
    6:     for (int i = 0; i < symbols.Length; i++)
    7:     {
    8:         sb.Append(symbols[i]);
    9:         if (i < symbols.Length - 1)
   10:             sb.Append("+");
   11:     }
   12:  
   13:     // Append the format characters. We want ticker, current price and change in percent (s, l1, p2).
   14:     sb.Append("&f=sl1p2");
   15:  
   16:     string queryUrl = sb.ToString();
   17:  
   18:     // Query the service and parse the results.
   19:     WebClient client = new WebClient();
   20:     StreamReader reader = new StreamReader(client.OpenRead(queryUrl));
   21:     List<Quote> quotes = new List<Quote>();
   22:     while (reader.EndOfStream != true)
   23:     {
   24:         string line = reader.ReadLine();
   25:         line = line.Replace("\"", "");
   26:         line = line.Replace("%", "");
   27:         string[] info = line.Split(',');
   28:         quotes.Add(new Quote { 
   29:             Symbol = info[0].Replace("\"", ""), 
   30:             CurrentPrice = float.Parse(info[1]), 
   31:             PercentChange = float.Parse(info[2]) 
   32:         });
   33:     }
   34:  
   35:     // Return the quote with the highest percent change.
   36:     return quotes.First(q => q.PercentChange == (quotes.Max(p => p.PercentChange)));
   37: }

Next, we need another helper method called Push that is actually responsible for sending the push notification. Now, we will be writing to the trace so that we can see what is happening later when we monitor the Azure fabric console in the development environment. This is an easy way to debug services. We’ll use the Push Notification library to send a toast notification with quote data to a specified user. Here’s the code for this method:

    1: private void Push(Quote q, User u)
    2: {
    3:     PushNotificationSender sender = new PushNotificationSender();
    4:  
    5:     // This message will look like this on the phone:
    6:     // Stocks: MSFT: $27.39 (0.4%)
    7:     string message = q.Symbol + ": $" + 
    8:         Math.Round(q.CurrentPrice, 2) + 
    9:         " (" + Math.Round(q.PercentChange, 1) + "%)";
   10:  
   11:     // All of this is for debugging
   12:     StringBuilder trace = new StringBuilder();
   13:     trace.AppendLine("************************************");
   14:     trace.AppendLine("Sending push notification to user " + u.Username);
   15:     trace.AppendLine("Push notification URI: " + u.PushNotificationURI);
   16:     trace.AppendLine("Message: " + message);
   17:     Trace.Write(trace.ToString());
   18:     // End debug
   19:  
   20:     // Send the notification
   21:     sender.SendToastNotification(
   22:         new System.Uri(u.PushNotificationURI), "Stocks: ", message);
   23: }

Declare the following member variables at the class level. They are used in the waiting cycle and for data access.

 DateTime lastRun = DateTime.MinValue;
const ushort MINUTES_TO_WAIT = 2;
AzureAlertsEntities entities = new AzureAlertsEntities();

In the Run method, we will wait every 2 minutes (as defined by the constant above). When waiting time is over, we’ll loop through every user in the database who wants to use push notifications. Then, we’ll try to push them a quote.

Obviously in the real world you would not send an invasive toast-style stock notification every 2 minutes. That’s overkill. But we want to see this dang thing work, don’t we?

Here’s the Run method. Again, no guarantees about bug-freeness…

 public override void Run()
{
    // This is a sample worker implementation. Replace with your logic.
    Trace.WriteLine("StockMonitor entry point called", "Information");

    while (true)
    {
        // Give the processor a little break.
        Thread.Sleep(10000);

        // only do this every 2 minutes
        if (DateTime.Now.Subtract(lastRun).Minutes >= MINUTES_TO_WAIT ||
            lastRun == DateTime.MinValue)
        {
            Trace.WriteLine("Looking for users");                                     

            var users = entities.Users.Where(u => u.UsePushNotifications);
            foreach (User u in users)
            {
                Trace.WriteLine("Searching stocks for " + u.Username);

                try
                {
                    Quote q = GetQuoteToPushForUser(u);
                    Push(q, u);
                }
                catch
                {

                }
            }

            lastRun = DateTime.Now;
        }
    }
}

And We’re Done (For Now!)

That’s it for the Worker Role side of things. If you were to push F5, you’d see the Azure fabric spin up and you’d be able to see a little of what was happening if you opened the console. We’ll cover all of that later. For now, hang on tight to the next blog post where we expose the service for a phone to register for push notifications with our system, followed by the phone app which actually registers for the notifications.

Share via