So one of my favorite parts of working in the Windows Azure team is the camaraderie that grows when you take a bunch of really smart people and apply them to a big problem. I’m very fortunate to be part of three extremely interesting Windows Azure v-teams (teams assembled for a particular project or work stream); our TAP team (I’ll explain a little more about that in a later post), our billing team, and our deployments team.

One thing that I have noticed during the course of this year as we work towards PDC2009, is that each team has it’s own dynamic, it’s own challenges and issues, and most of all, it’s own in-jokes. Along the ship path, as the pressure increases, it’s easy to forget to have fun, so the other day, I jumped onto Zazzle, and captured some of the more interesting moments on our road to PDC09 so far.

Filing Bugs Is Free!

My first one comes from what we call a ship room, which essentially is a regular meeting (we hold ours twice a week for an hour) where everyone involved in shipping the next release (in our case, billing) attends to triage bugs from the previous release, and discuss blocking issues for the upcoming release. Everyone attends; developers, testers, program managers, release managers, everyone. As the billing team is relatively new, not everyone had worked together. Now, as every dev/test knows, filing bugs is not a trivial task, there is a tension between overfiling (carelessly filing bugs which adds work to the triage team and can cause wasted effort) and underfiling (not wanting to file bugs because it may make the developer look bad, however has the effect of giving an incorrect representation of how solid the codebase is). It’s a tension that exists in every project, and there is no right or wrong balance, you have to take a position that suits your objective. Entering billing ship room, there was definitely a tendency to underfile, but given the importance of billing, we decided that it was better to overfile, wear the cost of triage, to ensure we didn’t miss any bugs, than underfile to optimize for speed of delivery. Hence, the shirt below!

My Rack’s Too Fat for Singapore!

The second one comes from my experience in our infrastructure deployments team. This team is responsible for deploying all new Windows Azure capacity to support the cloud, which includes selecting hardware/network gear, designing the deployment topology/config, rolling out and setting up the physical assets, and wiring it all up to the Windows Azure grid, both from a wire and software point of view. As you can imagine, it’s a pretty fast paced environment, and I can only imagine, is akin to landing planes; you need to get lots of iron and copper on the ground, in different countries, very very quickly, in quick succession. The other aspect is that we don’t use your standard enterprise configurations, which means are racks tend to be tall, heavy, and many. So it some times happens that a datacenter location may not be able to support our deployments, which was the case with a location we were looking at in Singapore. I couldn’t resist capturing that little moment, in a very literal way in the shirt I’m wearing today!

So keep this in mind, next time your sitting with your project crew, toiling through the hurdles and issues, and that little moment pops up that makes everyone have a laugh and relax, capture it in some way, shape or form. It’s a great way to remind everyone that we do this stuff because we love it, and it also helps keep those moments alive down the path, where trust me, you’ll need them! ;)

Enjoy! :)

So I’ve been working on a proof of concept this past week, and one of the biggest issues I’ve had has been trying to see what has been going on with my “container” when I deploy my app to Windows Azure. It’s not that the logs aren’t good enough, but sometimes I need to try interactive commands to see the state of the environment or test something out.

First, let’s recap a little Windows Azure 101; when you deploy your application, we spin up a VM and deploy your app into that VM. That VM has some policies set, which restrict what you’re able to do compared to your local machine or dev server.

So how do you see what your VM is doing? Well, since we don’t really provide any option to TS (remote admin) into your VM instances, I had to come up with a slightly different approach. The first step was to flick on a little known setting, but very powerful one:

Enabling Native Code Execution turns off the partial trust policy in Windows Azure, allowing you to run a bunch of functions that you were previously able to run in the old Nov CTP 2008 days.

My next task was to code up a little page that would allow me to run commands in the VM environment, kind of like a remote console, so I could see what was happening when I was trying to run my app.

The main bit of code is quite simple, and uses the System.Diagnostics.Process to create a new command window to run my commands, and capture the output back into my ASP.NET page. Here is the code to create the command process and get the return data:

So I have a simple page now, where I can enter commands, and see what is returned when run within the VM:

Oh the things one finds out about their VM! :) For example, I can run commands like:

SET

NETSTAT

TASKLIST

This type of insight is really important, for example, in my task, I was trying to start a new process (Apache Tomcat), but couldn’t get a response from the default address that Tomcat binds to (http://127.0.0.1:8080), and couldn’t work out why it worked on my local dev fabric and not in the cloud. I then ran NETSTAT and saw that the 8080 port was already bound to a virtual address, hence why it wouldn’t work for me.

Anyway, there is a wealth of information that can help you as a developer, when trying to debug/troubleshoot an issue, that can only be gleaned from the live environment, so tuck in!

Enjoy! :)

So anytime you’re doing something more than once, there is an opportunity to do it better every consecutive time. Over my career, I’ve worked in a number of different project teams; software teams, hardware teams, software/hardware teams, even a marketing team!

One thing that has been consistent about my experience in teams though, is that if you are trying to get many people to work efficiently on one project, you need a process. Now, I know there are those who don’t favor process, and I agree with them that a bad process is just as destructive as no process, but a good process, that’s a different story. And also, sometimes, there isn’t any process in place whatsoever, and I don’t mean there isn’t any processes in existence, it just that the team and project you’re working on hasn’t adopted one.

So where do you start? Well, I have some principles I’ve always used to guide me thinking.

First, start with a blank sheet of paper. I have a philosophy that by even having a blank sheet of paper, that represents your process, you’re already off to a good start, because you now have a process to improve on. But if you don’t even have a blank piece of paper, you haven’t got anything to improve on, so you’re doomed to keep starting from scratch and feeling your way through every project, painful ad-hoc step by painful ad-hoc step.

Second, think logically about the stages/phases of your process, and the functional groups involved. For example, let’s say your trying to build a new product, you can start simply by saying, what are the big steps we will go through as a team, and what are the functional roles that the team will play. This is important if you want to avoid having everyone involved in every part of the project. This kind of high contact leads to burnout and failure, as everyone is always on, even when they don’t need to be, and your project ends up looking like a rave party than a slow waltz. So an example process sheet would look like (I like to use Visio Cross Functional Flowcharts):

Next, you put a simple box in each intersection where a process activity needs to occur. Now, I never put more than a single box in an intersecting square. The purpose of the chart above is to highlight a work interaction between a functional team and a phase, you can drill down into the actual deliverables and steps later. So my process might look like this.

So for example, in the Requirements phase, the business team are responsible for building the customer requirements. To start with, you may not have any steps, so at the project kickoff, you would start by defining the steps for this project to satisfy that part of the process. Next time around though, you could look back at the last project and say, “Wow, out of the 3 things we did, the focus groups worked really well, so let’s add that to the best practices list for Customer Requirements tasks”. Again, it’s not about forcing people to follow anything more than the high level process, but you also want to capture the prior learning, and express it in a way where someone can look over the task list for a process stage and say, “This was used to get this outcome, and I think I need that outcome for my project, so I’ll use that task”, but also enable people to add new tasks to the catalog.

The next step is you have to have a basic communications plan. In too many projects, emails fly around between project members and other connected people, with everything from FYI’s to work items and change requests. It’s dangerous because the wrong people can affect the outcome of your project, and critical items get lost in the email explosion. Set up some basic email aliases for your project, at a basic level, have 3:

1. A basic project comms alias for general FYI comms

2. A change request alias where requests can be managed

3. An oversight/stakeholder alias where critical decisions that need stakeholder approval/review can be sent

You should also attach a contract to each alias, for example, for the oversight alias, you might say that all the people on that alias commit to responding to any email sent to that list within 24 hours. This way, the alias doesn’t just become an vacuum or black hole, and people don’t lose faith in the alias and start emailing people directly.

And finally, you should have a clear accountability matrix. This ensures that the right people are involved in the right parts of the process, and more importantly, the wrong people are not involved. You matrix should map to your functional groups from your process chart, and should follow a model such as OARPi (Owner, Approver, Reviewer, Participant, Informed). This way, everyone knows when they are involved, and in what capacity. And also, they know who is not involved, so they can filter those people from the process. It really improves the signal:noise ratio for a project.

And that’s pretty much it. Each time you runt he process for a project, take the time at the end of the project to reflect, then update the body of knowledge for the process, highlighting tasks that worked really well in a phase for a functional group, and also tasks that didn't’, and include enough information for the next person so they know what you were trying to achieve, and what the outcome was.

Process is good, process is your friend! :)

So one of the key responsibilities of a Program Manager at Microsoft is to drive features. Features are the same as functions in some software nomenclature, and essentially, they map to something the software performs to satisfy a customer need.

Now, over the course of my career as a software engineer, I’ve seen two major disconnects in product teams. The first is where product teams never seem to be able to pick the right features to build. They have great talent, amazingly smart people, lots of resources, but just no direction, so they end up with awesome software that no one wants. I’ve also seen the opposite, teams with great feature lists, but a lack of diligence and professionalism required to ship quality features, so they build a product everyone wants but no one can use or will pay money for. This has led me to distill a little saying which has always held me in good stead: build the right feature, build the feature right. What does that even mean?

Well, knowing what features to build is an art in itself. When building a global product, it’s not easy to simply ask your customers what features you should build. Instead you have to use different techniques to ascertain what features are valuable to them, and which are not. You need to take into account multiple inputs, such as focus groups, customer visits, user feedback, market movements and pressures, and insights into areas of possible innovation. By doing this, you make sure that all the hard work the developers will do to produce your feature, is worthwhile not only to the customer, but to the business too.

Now once you’ve worked out the right features, you need to work out how to build those features right. See, this is where I’ve seen many a software group come unstuck. Picking a good feature is only half the battle, delivering it with quality and style is another. For example, in order to ship a great feature, many factors need to be considered, such as how the customer intends to use the feature, how the customer would like to use the feature, the best way to design the feature so the customer can be efficient and comfortable in using it, and can leverage existing skills and experience to be productive. It’s not good enough to simply cut some code, there needs to be a quality process from design through to deployment to ensure the feature is rock solid and appealing to the customer.

Shipping great software is about embracing both of these aspects, not just one of the other.

I came across this video today and really found it simple yet powerful.

For me, the best part of this video is it puts a real context around the whole S+S story, which to date, has been pretty hard to decipher.

The main takeaway as I see it, is that you can’t achieve everything you need from just desktop/server software or the cloud, you need a little bit of both. Why not have your cake and eat it too I say! ;)

When designing any piece of software, there are always going to be design tenants that you practice based on the type of software being developed.

For service software that is designed for a global, business critical audience, there are three key tenets in my opinion. In fact, there not so much tenets as they are assumptions.

So when you start designing your service(s), you need to design with the assumption that your service is going to do the following, and do it on a regular basis:

• Fail
  • This does not only mean your service will fail during runtime execution, but also during upgrades, deployment of new functionality, bug fixes, configuration changes to the environment
  • Your operating environment will fail. Power will go out, network appliances will die, hard disks will crash, CPU and Memory will fault, assume it will all happen, and probably all at once
  • In a standard data center deployment, you need to work out what your minimum unit of failure is. Is it a rack? Is it a switch? The data center? Knowing this helps you work out how to distribute your service to counter failures.
• Grow
• Your application is going to need to serve lots and lots of requests, so think about the scale of your application. Not only scale out (as in, the ability to serve lots of incoming requests simultaneously) but also scale up (make sure your application can dole out capacity)
• Bandwidth and Ingress/Egress is also a concern, pipes in and out of your service are constrained by physical limitations, so you’re going to have to think about how to deal with heavy load on your network and how your application will deal with this during peak and off-peak periods
• At some point, you service is going to exceed the place its located, as in, there just won’t be any more machines to deploy your service to, and there won’t be any more space to deploy more machines, so you’re going to have to design your service with the goal that you can deploy it across many servers across many data centers
• Distribute
  • Your customers invariably will want your service closest to them, and they will invariably never be located closest to your primary deployment data center, so design your service in a way that it can serve your customers at the point that closest to them. It reduces latency and improves service quality and experience

Also, designing a service involves thinking through the whole application lifecycle. Traditionally, we think about issues and bugs at the runtime level, but with services, because they are always on, living, breathing systems, you have to design them in a way that they can be changed (upgraded, configured differently, etc) at any time without interrupting its availability (the ability for customers to connect and use your service) and its consistency (the state maintained by the system must always be consistent and correct).

Now, this is by no means the magna carta of service design, but as you start thinking about designing your always available, highly scalable, geo-distributable service, the tenets above should be good food for thought.

Enjoy :)

OK folks, so all the announcements have been made for Mix, so it’s time to get cranking!

We have a new release of tools and SDK bits.

First, you’ll need to install some pre-reqs, but only if you are going to exploit some of the new capabilities:

If you don’t plan to use native code, FastCGI or URL rewriting, then don’t worry about the pre-reqs above.

Then, the tools and/or SDK. One big thing to note about the March release is we now have two install options:

• If you don’t use Visual Studio, then you can download the standalone Windows Azure SDK here, or
• If you use Visual Studio, then you can now get an all up VS Tools for Windows Azure and Windows Azure SDK installer here.

Enjoy!

My colleague and fellow Windows Azure PM Matt Kerner put me on to this site:

http://courtoons.wordpress.com/

Very fun cartoons related to Law/Legal subjects. My favorite (so far):

He he :)

So one of the things I’ve noticed since moving to the U.S is the complete obsession American’s (now, I’m being facetious, this observation is based purely on advertising) have with clothing you can wear while on the sofa/couch!

First I saw the “Snuggie”:

Essentially, it’s a blanket with sleeves, and has raised the ire of other perplexed consumers!

But then Yil sent me this today!

The Pantalaine “Couch Dress”! WTF!?

To be honest, if I’m going to get out and about with a piece of clothing that seconds as a coverlet, I’m going with the Pantalaine, I mean, just look how hot the model looks! Smoking!

Enjoy :)

So I’m back from Oz and have been tinkering with a small worker role app that grabs my Twitter feed and aggregates how many posts my friends have been doing as a scoreboard (I’ll post that tomorrow).

One of the design decisions I’ve had to make has been where to store the configuration settings.

See, in Windows Azure, you can either:

• Store your configuration settings using the App.config file for your project, then use the standard libraries from .NET to read those, or
• Use the ConfigurationSettings element within your Service Configuration and Service Definition files, and use the RoleManager.GetConfigurationSetting() method to read the setting

What’s the diff?

Well, when you store your settings in your Service Configuration and Definition files, you are able to change those settings without having to redeploy your application. If you use the App.config file, any change you make won’t be picked up until you redeploy your app.

Allow me to demonstrate.

So, first you need to define your setting in your ServiceDefinition file:

And add the ConfigurationSetting section into it:

<?xml version="1.0" encoding="utf-8"?>
<ServiceDefinition name="SettingsExample" 
                   xmlns="http://schemas.microsoft.com/_
                   ServiceHosting/2008/10/ServiceDefinition">
  <WebRole name="WebRole">
    <InputEndpoints>
      <!-- Must use port 80 for http and port 443 
      for https when running in the cloud -->
      <InputEndpoint name="HttpIn" protocol="http" port="80" />
    </InputEndpoints>
    <ConfigurationSettings>
      <Setting name="DefaultWelcome"/>
    </ConfigurationSettings>
  </WebRole>
</ServiceDefinition>

Next, define some values in the ServiceConfiguration file:

And set the values:

<?xml version="1.0"?>
<ServiceConfiguration serviceName="SettingsExample" 
                      xmlns="http://schemas.microsoft.com/_
                      ServiceHosting/2008/10/ServiceConfiguration">
  <Role name="WebRole">
    <Instances count="1"/>
    <ConfigurationSettings>
      <Setting name="DefaultWelcome" value="Manishkta"/>
    </ConfigurationSettings>
  </Role>
</ServiceConfiguration>

Next, you need to access your setting in your app, so you use the RoleManager.GetConfigurationSetting() to read it, like so:

using Microsoft.ServiceHosting.ServiceRuntime;

namespace SettingsExample_WebRole
{
    public partial class _Default : System.Web.UI.Page
    {
        protected void Button1_Click(object sender, EventArgs e)
        {
            lblGreeting.Text = 
            RoleManager.GetConfigurationSetting("DefaultWelcome") + 
            " " + txtName.Text;
        }
    }
}

Then deploy your app:

Now, when I run my app, it pulls the setting from my Windows Azure app deployment:

Next, let’s go change the setting live without messing with our deployment, to start, click the configure button:

Then change and save the setting:

Windows Azure will reconfig your app:

And it will now use the new setting:

This saves you from having to completely package up and redeploy your app every time you want to change a setting (due to changes in business rules or between production and staging environments for example).

Also, if you want to know how to paste code into your posts, check out this Live Writer Add-in! It rocks!

Chau :)

So I've blogged previously about a known issue regarding port translation/mapping in Windows Azure, and today (well, today and a few weeks ago) I came across an interesting side effect of this issue when accessing the Request.Url property within a Windows Azure web role.

Now, anytime you access the HttpContext.Request.Url property (or any derivation of that), you will be unknowingly getting some extra info you may not want.

For example (and as it happens, the piece of code at the center of today's investigation), if you are building a new URL in your Windows Azure application by calling:

string RootUri = HttpContext.Current.Request.Url.GetComponents(UriComponents.SchemeAndServer, UriFormat.Unescaped);

You're going to get something along the lines of:

Scheme: Http

Server: x.cloudapp.net

Port: 20000 <-- Aha! (I added the <-- Aha! for dramatic effect)

This is going to cause your app issues if you are then trying to use that in a dynamic link or web request.

The solution is simple, anytime you call something that returns a Request.Url object, make sure you ensure any calls you make on that Url object do not explicitly return the port (this gets tricky when doing local testing where you need the port, so you'll need to manage that). Construct your new URL with only host or path details, otherwise you'll make mischief for yourself!

Enjoy :)

So with the new release of our SDK and VS Tools and the recent release of Windows 7 Beta, some folks may have tried to run one on the other...

Well, in case you have and are wondering why some parts don't work, or in case you haven't and were thinking of trying to, or haven't and were thinking you might look good in mauve stretchy velour pants (like I did that one time at last years MSFT Christmas party), I have some info for you.

First, there is an issue with running the SDK (not the tools) on Windows 7 Beta. Essentially, you can install the SDK, but when you try to run your project on the local developer fabric, it will fall over.

Second, we're actively working on the problem, both us and the Windows 7 team, so as soon as we work out a solution we'll get it out to folks.

So what do you do in the meantime? My personal approach was to download the VSTS 2008 SP1/TFS SP1 free trial VPC (it expires on the 31st of December, 2009, which should give us enough time to sort the issue out ;)), install the SDK on that, then run that inside Virtual PC 2007 SP1.

Update: So as usual, I rushed through my post, made a bunch of bad assumptions, and ended up costing some folks a lot of wasted time... I'm sooo sorry! What happened? Well, to get past the Win7/Windows Azure SDK issue, I run a VPC environment on my laptop for demos. I actually got this VPC from an internal location, and thought it was VSTS 2008 running on Windows Server 2003. I installed the SDK, it runs great, so I then posted the above, pointing folks to a publicly available VPC that I thought/assumed was the same as mine. :( It wasn't. Turns out the one I have is actually Windows Server 2008, not Windows Server 2003. And as I'm not able to post the one I have, I have a work around.

Step 1. Grab a base VHD from here, either the Vista or Windows Server 2008 one.

Step 2. Download and install one of the VS 2008 trials.

Step 3. Install the Windows Azure SDK and VS Tools for Windows Azure on that.

Again, so sorry for the muck-up.

Enjoy :)

Today we released refreshes to both the Windows Azure SDK and the VS Tools for Windows Azure!

• Windows Azure SDK (this has the dev fabric and local storage stuff)
• VS Tools for Windows Azure (this has the templates and integration functionality to connect to the local and live environments for developing Windows Azure applications using Visual Studio)

What's hot in the new refresh?

Well, first we fixed a pesky little problem with VS hanging when you hit F5. For some customers, this ended up locking up the environment and you had to kill VS to get it all sorted, this is a huge fix.

We also fixed the limitation of using special characters in entities, which affected some update or delete operations.

There have also been a bunch of performance fixes, so the overall user experience feels a lot nicer compared to the PDC release.

There's quite a bit more in this release, however I only mentioned the issues that affected me most, I'm selfish like that (I'm sorry, please don't judge me).

So get cranking and download the new version, you won't be disappointed (especially me, since tonight is pizza night too!!).

Enjoy :)

A question popped up today about sending emails from within the Windows Azure environment, specifically using SmtpClient.

So, to cover off some basics:

1. The Windows Azure environment itself does not currently provide a SMTP relay or mail relay service
2. .NET applications run under a variation of the standard ASP.NET medium trust policy called Windows Azure Trust policy
3. Port 25 is open (well, all ports are open actually) to outbound traffic for Windows Azure applications

As we don't provide a host for sending or relaying SMTP mail messages, you'll need to find one that will. Your ISP or free mail provider will do providing they are setup to enable SMTP relaying.

Next, there is an issue with using the SmtpClient class with its Port property set to something other than 25 and its trust election, see this forum thread:

• http://social.msdn.microsoft.com/Forums/en-US/windowsazure/thread/409f1c6b-ba7a-4efa-aecf-51900d7f8c64/

So to demo this, I whipped up a quick sample app that sends STMP emails through a relay host as a web role:

The code is here:

• http://davidlem.svn.beanstalkapp.com/windowsazure/trunk/Windows%20Azure%20Hacking/AzureMail/

And the app is deployed here:

• http://196ac97b-025f-46e8-9216-c573a28590f5.cloudapp.net/Default.aspx

Enjoy :)

So I've had a couple of questions about hosting web services (SOAP style services in the form of *.aspx pages in a Web Role or *.svc files in the form of WCF services) in Windows Azure, specifically, how to get them to work?

Now, as I've mentioned before, Windows Azure uses a design where the executing environment that your hosted service is running in is actually just a pre-configured VM running Windows Server 2008 x64 with .NET 3.5 Fx installed amongst other things. So technically, running an ASP.NET web service or a WCF service should be nothing more than adding the required components (pages) to your web role. But before you get to your VM, there is some infrastructure that you need to step across, and this is what is currently providing some challenges to running hosted web services on Windows Azure.

So right now, if you create a new web role project, add an asmx page to it, publish it to the cloud, and run the page, it will work a treat. If you then add that web reference to a client project, you will get an error. The error happens because of the way we currently configure the internal Windows Azure environment to handle internal and external comms pathways (how your external URI maps to your internal machine URI).

Now this is a known issue but until we release a fix, the workaround is:

1. Create your web role and add your service definition (asmx or svc component)

2. Run the project using the local development fabric

3. While the service is running, add the local reference to your client project which will generate a compliant wrapper. This is important, because if you try to add a reference to your published service, you will run into the issue above. By adding the reference against the local service, you get the service definition added to your project.

4. Now publish the service, and record the URI (should be something like http://guidforstaging_or_accountnameforlive.cloudapp.net/service.asmx_or_svc)

5. Go into your client project, and update the web service reference URI from the local address to your live address

Your service will now be able to connect to your live service and have all the information it needs to communicate without having to do the discovery process against the live environment.

Here is a quick screen cast that demonstrates the work around process.

Also, check out the following links for more info:

• http://social.msdn.microsoft.com/Forums/en-US/windowsazure/thread/0be800f9-f7b3-4b27-bd85-10e81c5ef669/
• http://blogs.msdn.com/jnak/archive/2008/11/10/asp-net-mvc-on-windows-azure-with-providers.aspx

Enjoy :)