- Presentations on Windows Azure and What's New in .NET 4.0
-
I've been asked few times for the slide decks of presentations I have done recently on overview of Windows Azure and on what's new in .NET 4.0. You can find them here:
Cloud
Computing: Practical Innovation (November 2009)
What's
New in .NET 4.0 (September 2009)
Please note, there are notes in the Notes section of each slide that help to understand the content when needed. Hope these help someone.
- UPDATE: Porting Silverlight RIA to Windows Azure (Data Access)
-
Given all latest changes to SDKs and tools, I think a significant update is needed for my old blog post on creating a data access layer for Azure Table Storage in Silverlight (Porting Silverlight RIA to Windows Azure - Part 1). It is now a part of whitepaper I'm working on, soon to be released on MSDN.
Note: Steps 1-3 represent one server-side model class, step 4 represents server-side data service class and steps 5-7 represent third, client-side view model class.
Step 1. Create POCO classes to represent your data model
Without EF-generated entities, one can create data model manually with help of plain old CLR objects (POCO) that look like this:
|
public class Report : TableServiceEntity
{
public Report()
{
}
public Report(string title, string description, string initiatorId, string solverId, string attachments)
: base(title, title)
{
Title = title;
Description = description;
InitiatorId = initiatorId;
SolverId = solverId;
Attachments = attachments;
DateCreated = DateTime.UtcNow;
ReportId = Guid.NewGuid();
}
public string Title { get; set; }
public string Description { get; set; }
public string Solution { get; set; }
public string InitiatorId { get; set; }
public string SolverId { get; set; }
public string Attachments { get; set; }
public DateTime DateCreated { get; set; }
public Guid ReportId { get; set; }
} |
Please note the use of TableServiceEntity as a base class – it is a class from built-in Windows Azure StorageClient library representing an entity in ATS. Among other things it adds ATS-specific attributes such as definitions of RowKey and PartitionKey to these entity classes.
Step 2. Create a data model class to be used with ADO.NET DS
Now, we need to wrap our entities into a model class that will serve as a representation of our entities visible to all clients. This not only allows us to separate the database representation of the entities from what we want to expose to the clients but also allows us to take advantage of StorageClient’s TableServiceContext class that will be used later by WCF DS.
|
public partial class AzureModel : TableServiceContext
{
private const string ReportsTableName = "Reports";
public AzureModel(string baseAddress, StorageCredentials credentials) : base(baseAddress, credentials)
{
}
public IQueryable<Report> Reports
{
get { return CreateQuery<Report>(ReportsTableName); }
}
... |
Step 3. Provide implementation for IUpdatable
The reason you need IUpdatable implementation is because ADO.NET uses it to update the storage. The ADO.NET Entity Framework's ObjectContext does not implement IUpdatable, but its ObjectContextServiceProvider does, so building a Data Service with an ObjectContext gives you an update support for free; using LINQ to SQL with a DataContext results in using the ReflectionServiceProvider that has no built in support for IUpdatable operations and relies on the data service context to implement it; same applies to any other DataServiceContext-based context classes such as StorageClient’s TableServiceContext used here. Therefore, we need to create our own IUpdatable implementation. After that is done, you need to do is to inherit your model class from it.
|
public partial class AzureModel : IUpdatable
{
public void AddReferenceToCollection(object targetResource, string propertyName, object resourceToBeAdded)
{
AddLink(targetResource, propertyName, resourceToBeAdded);
}
public void ClearChanges()
{
// clear out links
foreach (LinkDescriptor link in Links)
{
DetachLink(link.Source, link.SourceProperty, link.Target);
}
// clear out entities
foreach (EntityDescriptor entity in Entities)
{
Detach(entity.Entity);
}
}
public object CreateResource(string containerName, string fullTypeName)
{
object obj = Activator.CreateInstance(GetType().Assembly.GetType(fullTypeName));
AddObject(containerName, obj);
return obj;
}
public void DeleteResource(object targetResource)
{
DeleteObject(targetResource);
}
public object GetResource(IQueryable query, string fullTypeName)
{
object resource;
// fullTypeName can be null for deletes
if (fullTypeName == null)
{
resource = query.Cast<BaseEntity>().SingleOrDefault();
}
else
{
// Check for types that will be updated or deleted
if (fullTypeName == typeof(Report).FullName)
{
resource = query.Cast<Report>().SingleOrDefault();
}
…
UpdateObject(resource);
}
return resource;
}
public object GetValue(object targetResource, string propertyName)
{
Type type = targetResource.GetType();
PropertyInfo propInfo = type.GetProperty(propertyName);
if (propInfo == null)
{
throw new Exception(string.Format("Can't find given property '{0}'.", propertyName ?? "NULL"));
}
object val = propInfo.GetValue(targetResource, null);
return val;
}
public void RemoveReferenceFromCollection(object targetResource, string propertyName, object resourceToBeRemoved)
{
DeleteLink(targetResource, propertyName, resourceToBeRemoved);
}
public object ResetResource(object resource)
{
Detach(resource);
return resource;
}
public object ResolveResource(object resource)
{
// Already gave object in CreateResource
return resource;
}
void IUpdatable.SaveChanges()
{
try
{
SaveChanges();
}
catch (KeyNotFoundException e)
{
var exception = e;
}
}
public void SetReference(object targetResource, string propertyName, object propertyValue)
{
SetLink(targetResource, propertyName, propertyValue);
}
public void SetValue(object targetResource, string propertyName, object propertyValue)
{
PropertyInfo propInfo = targetResource.GetType().GetProperty(propertyName);
if (propInfo == null)
{
throw new Exception(string.Format("Can't find property '{0}'", propertyName ?? String.Empty));
}
propInfo.SetValue(targetResource, propertyValue, null);
}
} |
The overall flow of data is shown in Figure 1. Please note that we are calling DataServiceContext.UpdateObject() when we need to “synchronize” two contexts – client and server – as they don’t automatically update each other.
Figure 1. Accessing data in ATS via WCF DS
Please note that we are updating the object when it is being requested to accommodate all changes that occured on the client to this object as they don’t automatically propagate to the other client context we use to access data in ATS.
Step 4. Create Data Service in a web role
Add new ADO.NET Data Service item to your Azure web role project. You’ll have to reference your DAL project to have access to the model and then you can base your data service on it:
|
public class AzureDS : DataService<AzureModel>
{
public static void InitializeService(IDataServiceConfiguration config)
{
Logger.Info("InitializeService");
try
{
config.SetEntitySetAccessRule("*", EntitySetRights.All);
config.SetServiceOperationAccessRule("*", ServiceOperationRights.All);
}
catch (Exception e)
{
Logger.Error(e.Message);
}
}
protected override AzureModel CreateDataSource()
{
var cloudStorageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
string baseAddress = cloudStorageAccount.TableEndpoint.ToString();
StorageCredentials credentials = cloudStorageAccount.Credentials;
var azureModel = new AzureModel(baseAddress, credentials);
return azureModel;
}
} |
Step 5. Add a service reference
Now, add a service reference to the data service in your Silverlight project as you would do with any other web service:
Figure 2. Adding a service reference to WCF Data Service
Step 6. Create a view model
In view model class, you need to initialize data context with service URI. It’s a good idea to abstract the data service context with help of object property:
|
protected AzureModel Context
{
get
{
if (_model == null)
{
_model = new AzureModel(new Uri("AzureDS.svc", UriKind.Relative))
{
MergeOption = MergeOption. PreserveChanges
};
}
return _model;
}
} |
Please note - you might want to switch between NoTracking and PreserveChanges merge options here to achieve maximum performance of the queries as NoTracking is faster but can be used safely only when there are no changes being made in the client context. You can find more about various optimizations in the next chapter.
Step 7. Populate view model with service operations
Modify all your Silverlight code so that it uses LINQ to SQL to query objects and uses DataServiceContext’s BeginSaveChanges to make updates.
|
public void CreateReport(string liveUserId, string title, string description, string categoryId)
{
//Associate with current user
var query = (DataServiceQuery<User>)(from c in Context.Users
where c.RowKey == liveUserId
select c);
// Execute the query
query.BeginExecute(a =>
{
try
{
_matchingUsers = query.EndExecute(a).Select(user => (IUserElement)new UserElement(user));
IUserElement userElement = _matchingUsers.FirstOrDefault();
var report = new Report
{
Title = title,
Description = description,
InitiatorId = userElement.LiveUserId,
DateCreated = GetServerDateTime(),
CategoryId = categoryId,
Solution = String.Empty,
SolverId = String.Empty,
Attachments = String.Empty
};
report.RowKey = string.Format("{0:D19}", DateTime.MaxValue.Ticks - DateTime.UtcNow.Ticks);
report.PartitionKey = report.DateCreated.ToString("yyyy-MM");
report.ReportId = Guid.NewGuid();
Context.AddToReports(report);
Context.BeginSaveChanges(CreateReport_EndSaveChanges, report);
}
catch (Exception ex)
{
RaiseErrorHandler(new ErrorEventArgs(ex));
}
},
null);
} |
Step 8. Call EndSaveChanges to persist changes to ATS
|
private void CreateReport_EndSaveChanges(IAsyncResult result)
{
try
{
Context.EndSaveChanges(result);
CreateReportCompleted(this, new CreateReportEventArgs(((Report)result.AsyncState)));
}
catch (DataServiceRequestException ex)
{
HtmlPage.Window.Alert("Error occurred while creating report: " + ex.Response);
RaiseErrorHandler(new ErrorEventArgs(ex));
}
catch (Exception ex)
{
HtmlPage.Window.Alert("Error occurred while creating report: " + ex.Message);
RaiseErrorHandler(new ErrorEventArgs(ex));
}
} |
Now, all you have to do is to wire up your view model with your views and you can safely use ATS data in Silverlight and enjoy benefits of loosely coupled MVVM implementation.
- What is RCA (Rich Cloud Application)?
-
Rich Cloud Application (RCA) is a concept describing a new type of Rich Internet Application (RIA) served from the cloud. To the end user it would look exactly like RIA but with additional functions that take it to the next level - peer awareness, session awareness and duplex communication with a server. Client user doesn't have to know what is powering all these functions on the server, but presence of larger, richer context than before will be apparent - traditional client-server relationship will be effectively replaced with relationship with other users, some of which will be exactly at the same level of functionality and some will be at different level (power users, admins, etc.).
What this new rich context gives is the ability to create massive-scale applications that have not just scale in the back-end (traditional cloud) but also massive scale across their client base. Depending on how application is designed it might potentially multiply the amount of resources the application has at its disposal as clients effectively become part of the infrastructure application may depend on (with notion of loose coupling and random availability of course), but this is not the primary goal of RCA. The most important benefit is that applications can gain peer-to-peer and collaboration capabilities that scale massively in the cloud.
How does this all map to Microsoft technology stack? Microsoft cloud platform is Windows Azure, so all cloud-based services are going to be Azure applications with web or worker roles (depending on type of endpoints needed and whether some sort of background processing is needed on user's data). Rich Clients can be either managed applications or Silverlight applications. What Silverlight gives here is seemless deployment and much larger client base due to its cross-platform nature. For the purposes of RCA it doesn't lacks any of the functions compared to WPF. In next few posts I will show how various concepts of RCA can be implemented with Azure and Silverlight.
- AppFabric as a Foundation for Smart Grid
-
There's quite a lot of buzz around smart grid lately - many companies jumping on the green tech band wagon and investing resources in making all kinds of applications for smart grid - the next-generation energy infrastructure. Microsoft is no exception - it's developing Microsoft Hohm - a consumer-oriented service to help home residents to reduce their energy bills based on detailed usage report created from the data their utilities provide. My group is also helping our partner ISVs to develop smart-grid solutions based on Microsoft products and technologies. Most recently I helped Invensys to develop Smart Grid Pilot (SGP) application on top of Windows Azure, AppFabric (new name for .NET Services) and Silverlight. The idea behind Smart Grid Pilot is pretty simple - connect all smart grid participants (energy producers, utilities and consumers - businesses and homes) into one distributed network that can reach massive scale but would be easy to use.
So, why Azure, AppFabric and Silverlight? The main obstacle on the path to adoption of smart grid is not the outdated power infrastructure (it's actively being modernized), laws (many countries adopted very favorable laws for smart grid businesses) or lack of willingness of consumers to adopt (it means more savings for them). It is the lack of software infrastructure. Simply put, there's no such smart grid software solution right now that could scale to millions of homes, businesses, and most importantly devices. It's not just about connectivity (Internet is everywhere where energy might be these days) but mostly about applications and resources they use to serve the myriad of users, components and data streams in real time, changing constantly in both volume and distribution patterns.
Here comes AppFabric Service Bus. By design, Service Bus is supposed to serve exactly this kind of applications - super-scalable, distributed and involving many otherwise unconnected and sometimes incompatible clients and services. For example, to distribute energy prices in real time, you can use Service Bus to distribute this pricing data to all subscribed users (and subscribed here merely means "having well-known URL for the endpoint") in multicast pattern. As a service publishing these prices you simply have to know the endpoint to publish them to and use appropriate binding (e.g. NetEventRelayBinding).
Why Azure? Having messaging set up and configured for smart-grid services is not enough. You have to aggregate certain data streams and spit out only meaningful portions of information to connected clients. This aggregation requires some sort of scalable resource to serve as many clients as smart-grid solution will support (and that means millions potentially). Moreover, the load will probably not be well-balanced throughout time and geographies and hence the need for scale-up and scale-down capabilities of this resource with some geographical affinity. Windows Azure fits the bill perfectly.
Why Silverlight? Creating user-friendly applications that actually look like next-gen is a key for adoption of smart grid solutions on a wide scale. Dynamic nature of data streams being utilized also requires dynamic and rich execution environment. Last but not least multiple platforms have to be supported to extend the reach to as many consumers as possible. There are many functions that Smart Grid Pilot will finally support on the client side - it's not just reports and dynamic pricing charts - it's operations with electrical appliances that users will be able to perform remotely just by navigating to their account web page (hosted by utility in Windows Azure), billing and payment operations and customer support. Silverlight is probably best fit to satisfy these requirements at the moment.
Since many different roles with different responsibilities and permissions are supported (consumers getting the prices and publishing utilization information and issuing commands to their appliances through the service, energy producers publishing the wholesale prices, utility companies collecting all this data and producing retail price feeds, local authorities monitoring the utilization and generating all kinds of reports), a flexible security mechanism has to be adopted so entire system is secure yet easy to connect to by growing number of users. AppFabric ACS (Access Control Service) serves as a security glue here to provide claims-based authentication and authorization functions for applications and services connecting to both Windows Azure and Service Bus from either Silverlight or other runtimes.
We recently presented this project at PDC2009 and published a case study that should help other companies in adopting Windows Azure and AppFabric for their smart-grid solutions.
- Scalable Duplex Messaging with Silverlight 3 and Windows Azure
-
Silverlight 3 comes with very good PollingDuplex library allowing two-way communication between a server and a client but it lacks scalability features allowing it to work from behind a session-less load balancer such as one in Windows Azure. This limits options for implementing large-scale applications requiring notifications from server to be sent to clients.
The solution described below is based on modified version of server-side PollingDuplex library that Tomasz Janczuk of Silverlight WCF team recently put together. This modification eliminated in-memory client state registry that prevented System.ServiceModel.PollingDuplex library to scale. This opened the door for scalable technologies such as Windows Azure Storage, SQL Azure and codename "Velocity" to store duplex client session information. To work in real-world scalable environment this modified library needed scalable storage "connector" that would implement pub/sub pattern and connect asynchronous polling duplex service with asynchronous scalable storage. This is what my solution is all about. It uses Windows Azure Storage as a scalable storage for both client subsriptions (Azure Tables) and messages (Azure Queues). Please note: in contrast to previous solution for the same problem I described before this solution does not need any client modifications (they basically think they talk to the server over regular Silverlight 3 PollingDuplex) and does not have any significant latency in a single-message delivery.
Here's the basic workflow:
1. Client establishes communication with Polling Duplex Service (PDS), which is derived from PollingDuplex.Scalable by sending an asynchronous “connect” request with PollingDuplexBinding
2. Polling Duplex Service emulates server-side System.ServiceModel.PollingDuplex for the client but delegates all pub/sub tasks to PubSubStorage handler (PSS).
3. New client connects and subscribes to a topic; PSS registers client’s session in Subscriptions database and creates corresponding queue per client
4. Client publishes new message; PSS correlates the message’s topic with existing subscriptions and puts message copy to corresponding queues
5. Client sends a long poll request to server; PDS issues a dequeuing request to PSS and provides a callback delegate
6. PSS establishes an event sink for the queue corresponding to the client and call the callback method on PDS passing the message content with session information
7. PDS then propagates the message content to a specific client session in the body of asynchronous connect response
8. Inactive client sessions and all the resources allocated to support them are purged after specified timeout.
Note: the only limitation of the sample below is that it deliveres messages one-by-one and does not contain any batching capabilities.
You can find the source code for entire solution (including new version of polling duplex library) here.
You need to do following to get it running locally (no need to have Azure account):
1. Make sure all pre-requisites are installed:
a. VS2008SP1 + SQL Express / SQL Server
b. SL3 SDK + Tools
c. Azure SDK (July 2009 CTP)
2. Unzip StorageClient solution from samples.zip in Azure SDK and compile it
3. Unzip this solution and add a reference to StorageClient library (e.g. C:\Program Files\Windows Azure SDK\v1.0\samples\StorageClient\Lib\bin\Debug\StorageClient.dll) in WebRole project and set its "Copy Local" property to "true".
4. Compile the solution (but don’t run yet)
5. Run setupdevstorage.bat and make sure PubSubDB and corresponding table have been created in local dev storage
6. Open Development Storage UI and Select PubSubDB as a default table storage DB and start storage services
7. Launch the solution from within Visual Studio. PubSub Silverlight client should show up in the browser.
8. Open additional browser windows and point them to the same URL as in the first browser
9. Subscribe to the same topic in all browsers and start typing the messages that are received by all active subscribers.
Please provide your comments and suggestions. We really appreciate them!
Update: This sample only works with Azure SDK July 2009 CTP. Newer versions will need some minor code modifications due to breaking changes in Azure SDK.
- [Really] scalable Silverlight-Azure duplex implementation
-
Following up on my previous posts on this topic, this time I want to present really scalable solution to the problem of scaling client-server two-way communication in Silverlight 3 in the cloud. This solution is based on great implementation of HTTP polling duplex my colleague Tomasz Janczuk of Silverlight WCF team has just described on his blog.
One major difference between this and previous solutions is that server side of PollingDuplex channel (Microsoft.ServiceModel.PollingDuplex.dll) is now replaced with implementation that does not hold any client session state in memory of a single machine but rather provides "hooks" for scalable storages such as Windows Azure Storage and Velocity. Leveraging this new concept I've implemented Windows Azure Storage-based session tracking. The final solution does not need any SL3 clients previously using PollingDuplex to be modified - only server side has to replace a reference to Microsoft.ServiceModel.PollingDuplex.dll with a new library reference. This solution scales well in both local development and live Azure Storage and can be further extended to accomodate other scalable storage technologies. I will publish the source code and more detailed description soon. Stay tuned.
- Azure-friendly Polling Duplex in Silverlight 3
-
One of the most serious limitations of current implementation of PollingDuplex mechanism (one that allows server to talk to Silverlight applications in the browser) in Silverlight 3 is its lack of support for multiple backends. In other words, if your environment includes multiple machines or does not provide intelligent routing of messages from one client to only one server in the backend cluster, your otherwise perfect Silverlight applications utilizing PollingDuplex binding will simply not work - clients will poll servers that have no record of them and the duplex channel will simply fail.
This limitation, combined with Azure load balancer's lack of session "stickiness", renders the only backend to frontend communication mechanism in Silverlight 3 unusable there - as soon as you start taking advantage of Azure scalability and provision two or more web roles, your Silverlight applications based on PollingDuplex binding will start to fail.
The bad news is there is no easy solution to this - PollingDuplex server-side library in SL3 keeps client sessions in memory and doesn't allow these sessions to be shared among other instances or machines.
The good news is it's really not that hard to implement distibuted flavor of Long Polling pattern, which is utilized by SL3. Here's my solution:
1. Login / Silverlight XAP download
2. Client establishes communication w/ Duplex Service by sending a “connect” request over basicHttp
3. Duplex Service registers client by creating its GUID and corresponding queue for that GUID in AQS and returns this GUID to the client in HTTP response
4. Client remembers the GUID and considers itself connected
5. Client sends a long poll request to server with GUID in it, web role waits for requested timeout period and sends empty response. Connection with web role ends.
6. Web role puts messages from client into the outbound queue (one per app) in response to custom operation call from the client
7. Worker role’s timer picks up messages one by one from outbound queue and routes to external destinations based on config/payload
8. Worker role receives a message from external source, extracts client GUID from config/payload and routes message to specific inbound queue
9. Web role’s timer picks up messages and immediately passes it to accessor on Duplex Service
10. Duplex Service sends message in response on open connection with client. Connection with web role ends.
I have modified Silverlight Chat sample to follow this behavior and you can find it here.
The only thing you need for it is an active Azure account. Insert account name/key for storage (only queues are used) to web.config (used by web roles) and ServiceConfiguration.cscfg (used by worker roles), set the number of web roles you want and launch it along with two or more browsers pointing to SilverlightClientTestPage.html. Let me know if you have any feedback on it – it might be very useful for future Silverlight communication stack functionality planning.
- Porting Silverlight RIA to Windows Azure - Part 3: Implementing Service Bus Azure Proxy for Silverlight (SBAPS)
-
One major area where hosting RIA in Azure is significantly different from hosting in a regular web server is the use of the foundational piece of Azure platform itself - .NET Services Service Bus. Service Bus is an “intermediary in the cloud” allowing you to implement all kinds of patterns which were not possible before, magically connecting cloud applications, on-premise clients, web clients, devices and various combinations of all of the above, and doing it through variety of the protocols supported by WCF 3.5 SP1.
Unfortunately, this variety is unavailable to Silverlight whose Core CLR doesn’t support everything WCF 3.5 supports, for example custom TCP-based bindings (unavailable even in Silverlight 3). Now, since Silverlight application is served from a web server these mechanisms could be moved to the Azure’s web role where full .NET 3.5 may be available, right? Yes, but with one significant exception which pretty much renders most interesting communication architectures unusable – Azure is load-balanced and denies any control over the load balancer, which means you have no idea what web role instance requests from your Silverlight client will hit next and, more importantly,- you lose control over inbound communication with Silverlight clients. All in all – even simple multicasting (1:N communication) that is so effectively done in massive scale through Service Bus is unavailable for Silverlight clients hosted in Azure, let alone more advanced architectures.
Let’s see how we can work around this problem in the multicast example. Here’s the architecture we're going to implement:
Figure 1. Service Bus Azure Proxy for Silverlight (SBAPS)
Step 1. Configure Azure worker role for use with Service Bus.
Azure VMs don’t yet contain all binaries needed to work with other parts of Windows Azure Platform (WAP) such as Service Bus, therefore you might run into situation when the Azure solution works fine in your local development fabric since all the binaries are GAC’ed after installing .NET Services SDK but after deploying you will see roles not starting without any explanation – this is a sign that some of the references binaries are not only there but are not configured properly (remember, .NET Services SDK also puts its configuration into your local machine.config file). To encapsulate the deployment and the configuration of the Service Bus library in your worker role (web role is not suitable for hosting services), you need to set the referenced Microsoft.ServiceBus.dll assembly’s “Copy Local” property to “true” in Visual Studio and move all portions of your machine.config file related to the Service Bus to the App.config file in the worker role project.
Note: Due to a bug in Azure SDK July CTP, you need to set global environment variable AddAppConfigToBuildOutputs to “true” before starting Visual Studio in order for all the config file to be published along with the solution.
Also, note that we’ve added netEventRelayBinding definition that we need this configuration to support in the first place. This is the only binding we’re going to use to communicate with Service Bus in this sample.
We also need to add client and service configurations to the same serviceModel section:
|
<client>
<endpoint name="RelayEndpoint"
contract="psoWorkerRole.IMulticastContract"
binding="netEventRelayBinding"
bindingConfiguration="default"
address="http://AddressToBeReplacedInCode/" />
</client>
<services>
<service name="psoWorkerRole.MulticastService">
<endpoint name="RelayEndpoint"
contract="psoWorkerRole.IMulticastContract"
binding="netEventRelayBinding"
bindingConfiguration="default"
address="" />
</service>
</services> |
Note: Contract and service will be defined in the subsequent steps. Endpoint addresses will be inserted at runtime as we’re going to be establishing the channel with these endpoints explicitly in the code.
Step 2. Add Azure Storage Configuration.
In order to communicate between worker and web role, you’ll need to utilize reliable communication mechanism such as Azure Queue Storage. Add following section to both worker role’s App.config and web role’s Web.config:
|
<appSettings>
<add key="QueueStorageEndpoint" value="http://queue.core.windows.net/" />
<add key="AccountName" value="<INSERT ACCOUNT NAME>" />
<add key="AccountSharedKey" value="<INSERT SHARED KEY>" />
</appSettings> |
Step 3. Implement Service Bus client in worker role.
Add following code to the main worker role class derived from RoleEntryPoint:
|
// Create queue from config - exactly the same as in web role
StorageAccountInfo accountInfo = StorageAccountInfo.GetDefaultQueueStorageAccountFromConfiguration();
QueueStorage queueStorage = QueueStorage.Create(accountInfo);
MessageQueue outboundQueue = queueStorage.GetQueue("out");
outboundQueue.CreateQueue();
programInstance = new PSOHost(ConnectivityMode.AutoDetect);
programInstance.Run();
while (true)
{
// Get message from the queue
outboundMessage = outboundQueue.GetMessage();
if (outboundMessage != null)
{
programInstance.BroadcastText(outboundMessage.ContentAsString());
outboundQueue.DeleteMessage(outboundMessage);
}
else
{
Thread.Sleep(5000); // Adjust this sleep interval
}
} |
We are creating only outbound queue here which will be used by web roles to send messages to other recipients through Service Bus. Then we start polling this queue for messages and immediately broadcast them to the Service Bus:
|
// Broadcast the text through ServiceBus
public void BroadcastText(string text)
{
channel.Say(null, text);
} |
Here, Say is one of the operations on the multicast contract we associate with our Service Bus channel:
|
namespace psoWorkerRole
{
using System;
using System.ServiceModel;
[ServiceContract(Name = "IMulticastContract", Namespace = "http://pso")]
public interface IMulticastContract
{
[OperationContract(IsOneWay=true)]
void Enter(string nickName);
[OperationContract(IsOneWay = true)]
void Say(string nickName, string text);
[OperationContract(IsOneWay = true)]
void Exit(string nickName);
}
public interface IMulticastChannel : IMulticastContract, IClientChannel { }
} |
And finally, Say operation, which is called as soon as the Service Bus delivers the message – in this case to the same worker role instance, – invokes DumpMessage, which propagates the message to the inbound queues so they could be picked up by the web roles:
|
void DumpMessage(string text)
{
RoleManager.WriteToLog("Information", text);
Console.WriteLine(text);
foreach (MessageQueue inboundQueue in inboundQueues)
inboundQueue.PutMessage(new Message(text));
} |
For more information on how to implement multicasting pattern with Service Bus, please refer to MulticastSample sample in .NET Service SDK.
Step 4. Implement duplex communication in web ole and Silverligh client.
Silverlight 3 provides an out of the box mechanism for bi-directional communication with the server. In fact, the only server-side assembly in Silverlight SDK is System.ServiceModel.PollingDuplex.dll. We’re going to utilize it to talk to Silverlight clients. Reference this assembly in web role.
To close the loop and communicate with worker role we need to add operations with Azure queues:
|
Timer timer;
StorageAccountInfo accountInfo;
QueueStorage queueStorage;
MessageQueue outboundQueue;
MessageQueue inboundQueue;
public Communicator() : base()
{
// create queue from config
accountInfo = StorageAccountInfo.GetDefaultQueueStorageAccountFromConfiguration();
queueStorage = QueueStorage.Create(accountInfo);
outboundQueue = queueStorage.GetQueue("out");
inboundQueue = queueStorage.GetQueue("in" + HttpContext.Current.Application["roleId"]);
inboundQueue.CreateQueue();
outboundQueue.CreateQueue();
//Set up a message update every few seconds
this.timer = new Timer(new TimerCallback(PollInboundMessages),
null, 0, 10000);
} |
Note: We’re pulling the name of the inbound queue for each web role from the ASP.NET session. We create the inbound queue with unique name each time web role starts as we have to distinguish the queues somehow so the messages delivered from the worker role to them don’t get picked up by some other web role running in the application. Worker role doesn’t need to know the names – it just delivers the messages to all queues which name starts with "in".
|
void PollInboundMessages(object o)
{
// Get message from the queue
Microsoft.Samples.ServiceHosting.StorageClient.Message inboundMessage = inboundQueue.GetMessage();
if (inboundMessage != null)
{
TextChatMessageFromServer duplexMessage = new TextChatMessageFromServer();
duplexMessage.acceptsPublicMessages = true;
duplexMessage.text = inboundMessage.ContentAsString();
RoleManager.WriteToLog("Information", "Received from inbound queue: " + duplexMessage.text);
// Propagate message to connected clients
PushToAllClients(duplexMessage);
inboundQueue.DeleteMessage(inboundMessage);
}
} |
Here, PushToAllClients method delivers the message to the Silverlight clients connected to the web role through PollingDuplex protocol. The implementation of the duplex communicator pattern suitable for Azure can be taken from Silverlight Chat sample (http://code.msdn.microsoft.com/wcfazure/Wiki/View.aspx?title=SilverlightChat&referringTitle=Home).
- Porting Silverlight RIA to Windows Azure - Part 2: Efficient data access
-
Firstly, why is implementing efficient data access important for Azure RIAs in particular? The cloud databases in general are not the most performant databases in the world – they scale well but any given response by itself is probably going to be slower (e.g. much slower) than in traditional stand-alone or even client/server architectures. On the other hand, RIAs tend to be very interactive and therefore are expected to provide very responsive user experience. To avoid this conflict of architectures, one has to implement really efficient data delivery mechanism from cloud to the client and back. There are few steps to follow in this process.
Step 1. Implement efficient partitioning schema.
Azure utilizes partition key to “paginate” the data – that is entries with same partition key can be expected to be on the same virtual machine and entries with different partition keys can be spread across multiple virtual and physical machines in the cloud. Therefore, it’s best to group entries together using same partition key while creating them and filter them by the same partition key while querying.
The grouping should be primarily done based on business logic – of certain groups of objects represented by entries are more likely to be queried together than separately, than it’s best to assign them the same partition key value. However, if no grouping is possible, it’s best to assign each entry’s default PartitionKey property a unique value and query on it.
Another default property, RowKey can be used to further uniquely identify each entry. Using PartitionKey and RowKey in filters seems to be the most way to query data from Azure Table Storage in terms of speed of the request.
Step 2. Optimize data queries.
Based on partitioning implemented in Step 1 all queries that filter on certain entry properties should first include PartitionKey, then RowKey and only then other properties if needed. Following this method yields fastest results. Queries that ignore PartitionKey should not be too frequent and should artificially limit the amount of entries they request.
For example, a query from the last post could be modified like this:
|
var query = (from c in Context.Users
where c.PartitionKey == LiveUserId
select c) as DataServiceQuery<User>; |
Another efficient optimization of queries relies on more efficient use of LINQ where it’s possible to combine two or more queries together just by manipulating the syntax. Holding the results of common queries in memory (or Silverlight’s Isolated Storage) can further improve the performance by saving round-trips to the cloud and database (remember there’s always a layer between Silverlight and ATS which effectively doubles the number of round-trips required).
Step 3. Manipulate the merge option.
Each DataServiceContext-derived class (called AzureModel in previous part) has a property called MergeOption which significantly affects the performance, especially on large entry sets. It’s obvious, that MergeOption.NoTracking would be the best choice for sets that don’t need modification and MergeOption.PreserveChanges (used in previous part’s example) is the safest when sets need to be modified (for example, when user add an entry to the list of previously retrieved from the storage). Now, many developers resort to use one versus another or even creating multiple contexts with each option, when it’s totally feasible to just switch your DataServiceContext.MergeOption property based on current operation – NoTracking for reads and PreserveChanges for modification.
Step 4. Apply general ADO.NET DS performance optimizations.
A good set of ADO.NET DS-specific optimizations is described in this MSDN forum post. A word of caution here - all of these will need to be retested with every new ADO.NET DS and Azure SDK CTP release as they are subject to change.
- Porting Silverlight RIA to Windows Azure - Part 1
-
UPDATE: Please refer to an updated version of this post for up-to-date steps working with Azure SDK November CTP and Silverlight 3 and Silverlight 4 Beta.
Windows Azure is a great platform for taking traditional web applications and rich internet applications (RIAs) to the next level with its high availability and scalability. After all, it has a scalable web role which many understand as “the cloud web server”. However, it’s not exactly the same thing and your applications might not and likely will not work as usual after moving to Windows Azure. Any more or less complex web application, especially LOB application, might be affected by one or several of the following aspects:
· Azure web role is stateless, it doesn’t maintain any session state and your applications shouldn’t rely on continuing to do whatever they’re doing at any given moment in time
· There is no traditional database in the cloud therefore most database access technologies simply don’t work or work differently in Azure
· Azure load balancer is controlled by Azure fabric and you have no control over it, so your requests can be routed to any Azure instance breaking some "sticky" communication patterns
· Azure applications cannot rely on a specific web server IP address
· There are open port and component deployment limitations (what’s available is available and upgrading or adding might mean moving to different service level agreement (SLA), so this is not a technical problem you can solve).
In this series of posts I want to show how to overcome some of these limitations while porting traditional (if that’s the right word) Rich Internet Applications to Windows Azure. I’m going to be using Silverlight 3 Beta for this exercise. Now, you might wonder – why would Silverlight application be affected by changes on the server? After all it’s a client technology, right? Yes and no. Even though Silverlight has only client runtime, Silverlight application (even in SL3) is never a 100% client application (like mouse driver, for example) because it depends on the context which is provided by the server, therefore it must care about that server connection even if it’s running out of browser, especially if it’s serving as a UI for cloud application now.
I’m going to start with one of the most common scenarios – moving the data access layer to the cloud. This is something that almost every application will probably face considering that database in the cloud is different from traditional RDBMS in many ways and even though SQL Data Services is closing the gap very quickly, you’d still need to do things differently both on the server (Azure web/worker role) and on the client (Silverlight). Building a simple DAL for web-hosted Silverlight application has been pretty straightforward – point the ADO.NET EF Wizard to your database, select the data for a model, wrap that model with ADO.NET DS data service, expose it as a Silverlight-enabled WCF web service, add a service reference in your SL application and you’re ready to go with full set of CRUD capabilities on a nicely looking object model. This breaks at the very first step with Azure though – there’s no Entity Framework available for either Azure Storage or SDS yet. Below I describe one way to port to Azure Table Storage with minimal changes (I’ll cover SDS next month when it becomes available publicly with new RDBMS-like architecture and API).
Note: Steps 1-4
represent one server-side model class, step 5 represents another server-side data
service class and steps 7-9 represent third, client-side data access class. Dots (...) represent similar or auxiliary code.
Step 1. Create POCO classes to represent your data model.
POCO stands for plain old CLR objects and looks like this:
|
public class Report
{
public string PartitionKey { get; set; }
public string RowKey { get; set; }
public string Title { get; set; }
public string Description { get; set; }
public string Solution { get; set; }
public string InitiatorId { get; set; }
public string SolverId { get; set; }
public DateTime DateCreated { get; set; }
public Report()
{
}
public Report(string title, string description, string initiatorId, string solverId)
{
this.Title = title;
this.Description = description;
this.InitiatorId = initiatorId;
this.SolverId = solverId;
this.DateCreated = DateTime.UtcNow;
this.PartitionKey = title;
this.RowKey = title;
}
} |
Step 2. Decorate your POCO classes with DataServiceKey attribute:
|
[DataServiceKey("PartitionKey", "RowKey")]
public class Report
{
… |
Step 3. Create a data model class to be used with ADO.NET DS:
|
public partial class AzureModel : TableStorageDataServiceContext
{
public IQueryable<Report> Reports
{
get { return this.CreateQuery<Report>("Reports");}
}
…
} |
Step 4. Provide implementation for IUpdatable
Now, this is interesting part. The reason you need IUpdatable implementation is because ADO.NET uses it to update the storage. The ADO.NET Entity Framework's ObjectContext does not implement IUpdatable, but its ObjectContextServiceProvider does, so building a Data Service with an ObjectContext gives you an update support for free; using LINQ to SQL with a DataContext results in using the ReflectionServiceProvider that has no built in support for IUpdatable operations and relies on the data service context to implement it; same applies to any other DataServiceContext-based context classes such as StorageClient’s TableStorageDataServiceContext used here. There are several custom IUpdatable implementations out there, but this one by Tom Laird-McConnell is particularly simple and works in Azure without modifications. Only thing you need to do is to inherit your model class from it:
|
public partial class AzureModel : IUpdatable
{
#region IUpdatable Members
… |
Step 5. Create Data Service in a web role
Add new ADO.NET Data Service item to your Azure web role project. You’ll have to reference your DAL project to have access to the model and then you can base your data service on it:
|
namespace psoWebRole
{
public class AzureDS : DataService<AzureModel>
{
public static void InitializeService(IDataServiceConfiguration config)
{
config.SetEntitySetAccessRule("*", EntitySetRights.All);
config.SetServiceOperationAccessRule("*", ServiceOperationRights.All);
}
}
} |
Step 6. Add a service reference to your data service in your Silverlight project as you would do to any other web service.
Now your AzureModel class becomes a data context and is almost ready to be used in Silverlight.
Step 7. Initialize data context with service URI
|
// This is a context similar to entity context for ADO.NET EF -
// generated entities
AzureModel ctx = new AzureModel(new Uri("AzureDS.svc", UriKind.Relative));
// Safest merge option that allows not to lose any values that
// were changed on the client
ctx.MergeOption = MergeOption.PreserveChanges; |
Please note - you might want to switch between NoTracking and PresenerveChanges merge options here to achieve maximum performance of the queries, but more about this in the next part.
Step 8. Modify all your Silverlight code so that it uses LINQ to SQL to query objects and uses context’s BeginSaveChanges to make updates:
// Persist report
public void CreateReport(string LiveUserId, string title, string description)
{
//Associate with current user
var query = (from c in Context.Users
where c.LiveUserId == LiveUserId
select c) as DataServiceQuery<User>;
// Execute the query
if (query != null) query.BeginExecute(new AsyncCallback(a =>
{
try
{
matchingUsers = query.EndExecute(a);
currentUser = matchingUsers.FirstOrDefault();
Report report = new Report();
report.Title = title;
report.Description = description;
report.InitiatorId = currentUser.PartitionKey;
report.DateCreated = DateTime.UtcNow;
report.PartitionKey = report.InitiatorId;
report.RowKey = report.DateCreated.Ticks.ToString();
ctx.AddToReports(report);
ctx.BeginSaveChanges(OnCreateReportChangesCompleted, report);
}
catch (Exception ex)
{
if (CreateReportError != null)
CreateReportError(this, new MessagingErrorEventArgs(ex));
}
}), null);
} |
Step 9. Call EndSaveChanges to persist changes to Azure Table Storage:
|
void OnCreateReportChangesCompleted(IAsyncResult result)
{
try
{
ctx.EndSaveChanges(result);
CreateReportComplete(this, new CreateReportEventArgs((result.AsyncState as Report).Title));
}
catch (DataServiceRequestException ex)
{
HtmlPage.Window.Alert("Data service error occurred while creating report: " + ex.Response);
}
catch (Exception ex)
{
HtmlPage.Window.Alert("Error occurred while creating report: " + ex.Message);
}
} |
Of course, I omitted assembly reference and event handler steps for brevity but that’s how process looks like in a nutshell. I hope it saves you some time.
- New Slides on Future of UX and Azure Overview (RU)
-
I've done couple of presentations for our partners in Europe recently and want to share the scrubbed slides with everyone. First one summarizes the UX Continuum story on Microsoft platform as it relates to software architects and decision makers and touches on Windows 7, Surface, WPF, Silverlight and Windows Mobile.
Second one is in Russian and gives an overview of entire Azure Services Platform and gives few examples of real-world patterns that work well in Azure and are understood by traditional developers.
Enjoy!
- Product Support Online Architecture
-
Following up on my previous post - the reference application is now called PSO (Product Support Online). It will be developed in two phases to align features better with the changing nature of Azure and SDS APIs, but more importantly, to draw a clear line between on-premise applications and cloud-based applications so developers consuming the corresponding guidance would undestand how difficult (or how easy, I hope) it is to move this kind of application to Azure Services Platform.
High-level architecture diagrams for both phases are shown below. As you can see, Phase I is a pretty typical web application adding Silverlight UI and Live ID delegated authentication to the mix. The idea here is to isolate all operations with data to a separate data access layer (DAL) which can later be switched from SQL Server to SDS just by changing a connection string (this is promised by the SDS team and will be delivered sometime in the end of May 2009 in invitation-only SDK and in June-July 2009 in a public CTP).
Phase II is all about the cloud. It will add integration with Access Control Service so claims based federated security can be utilized by tenants of the application:
The most interesting part here is, of course, Task Engine, but it deserves a separate topic; the only thing I can disclose at this moment is that it's supposed to be flexible enough to cover many more scenarios than product support and feedback.
- "Product support in the cloud" reference implementation
-
Here’s new project I'm working on which is taking after my CPM pet project and various Azure and Silverlight projects I've done with partner ISVs.
It is an easy-to-use application for providing basic product support in the cloud, collecting feedback about products and exchanging opinions with other users. It's intended to be used by developers as a reference application utilizing both Windows Azure and Silverlight for rich yet scalable and highly available business solutions.
Here's how it works:
Internally, it will use Silverlight on the front-end and Azure, ADO.NET EF, ADO.NET DS and SQL Data Services on the back-end, but more about details later. The plan of record is to make it available on MSDN sometime in the summer 2009 with full source code and corresponding guidance.
- MIX09 Session comparing Flash to Silverlight
-
This week I attended a MIX09 session (well, the one that is supposed to be presented there in March anyway) by Rick Barraza, Senior Experience Architect at Cynergy. Rick is well known for his work on Flash, Flex, AIR, Silverlight and WPF projects and is a big proponent of Silverlight as the RIA platform. He shared his impressions of development experience with Silverlight and compared it to Flash and Flex in few ways that seemed very interesting to me, so I decided to share with you my understanding of the points expressed:
· Performance of Silverlight is sometimes few times higher than Flash, especially when it comes to animation of large number of objects on the same page – Flash usually doesn’t scale well beyond animating 10-15 primitive objects while Silverlight is capable of animating hundreds of them without significant slowdown on the same hardware
· “10 second Wow vs 10 minute Wow”: Flash applications are usually good to wow people for first few seconds but then users ask “now what?” when come across usability and productivity problems of traditional Flash applications (Flex solves this to certain extent but loses some of the “10 second Wow” capabilities of Flash)
· Conversion of Flash apps to Silverlight apps is done manually most of the time because of different ways they work with assets and different approach to transformations (timeline vs DispatcherTimer)
· Silverlight on the other hand has all functionality for producing 10 second Wow being followed by good usability and productivity of modern business applications needed for 10 minute Wow and beyond
· There are few things in Silverlight that are still hard to do (like PNG hacking, hard to do the same way due to lack of timeline, but used extremely often by designer community) but flexibility of the .NET tools and languages allows implementation of workarounds
· Traditional workflow-like approach to creating Silverlight applications isn’t always efficient for Flash-like applications that designers got used to develop for the Web and hence there’s big pushback against it. Instead they crave for state-machine approach, which is again possible in Silverlight, but somewhat different from the way .NET developers tend to think
· Traditional grid-based layout isn’t familiar to Flash developers either – they got used to creating free-form layout with (x,y) coordinates with a bunch of custom objects (vs predefined controls in Silverlight).
Finally, Rick suggested looking at Project “Rosetta Stone” which shows Flash developers how to create effective Silverlight applications using their existing skills and shares few interesting workarounds.
- What is CPM (Community Project Management)?
-
In response to the questions I received about CPM, I want to clarify what it is and who it is for.
What is it?
CPM is really a general concept that doesn't mandate any specific implementation or platform or even architecture. It has few central notions defined on top of and complementing the features of Social Network and Collaborative Innovation Network concepts:
· Composition of the collaboration rooms into projects based on stated goal
· Vertical hierarchical composition of projects into other higher-level projects based on stated criteria
· Dynamic regrouping of rooms and projects based on goal and criteria changes
· Dynamic progress and completion update
Who is it for?
Everyone. Literally. Consider a very typical situation when you have a question and you don't really know who to ask or don't have time to find an answer yourself. You go to a friend or a colleague or get online and ask it in a newsgroup or a forum. But what if this question isn't really that simple and has many parts to it which are not known to any single person or even a single community of people? You're either out of luck or will spend much more time and possibly money trying to get a solution. By using CPM system you do the same amount of work (describing your question and solution criteria) but you get much more - your problem becomes a living project that could be part of a larger problem or could be broken into pieces if it's too complex. Moreover, this set of projects can be made visible to different kinds of communities - part of it may be solved by the professionals, part of it can go to an academic community, part of it may get resolved by someone smart who happened to see it online or in his/her inbox. But most importantly, the solutions generated in the course of these projects, since they are classified and categorized from the beginning, can be "automagically" merged into a solution that you were originally looking for. If one of the parts is stuck without a solution, you have an ability to refactor it or redirect it to some other community or expert. The bottom line is - you get a solution eventually with much higher probability than without using CPM.
