So anytime you’re doing something more than once, there is an opportunity to do it better every consecutive time. Over my career, I’ve worked in a number of different project teams; software teams, hardware teams, software/hardware teams, even a marketing team!

One thing that has been consistent about my experience in teams though, is that if you are trying to get many people to work efficiently on one project, you need a process. Now, I know there are those who don’t favor process, and I agree with them that a bad process is just as destructive as no process, but a good process, that’s a different story. And also, sometimes, there isn’t any process in place whatsoever, and I don’t mean there isn’t any processes in existence, it just that the team and project you’re working on hasn’t adopted one.

So where do you start? Well, I have some principles I’ve always used to guide me thinking.

First, start with a blank sheet of paper. I have a philosophy that by even having a blank sheet of paper, that represents your process, you’re already off to a good start, because you now have a process to improve on. But if you don’t even have a blank piece of paper, you haven’t got anything to improve on, so you’re doomed to keep starting from scratch and feeling your way through every project, painful ad-hoc step by painful ad-hoc step.

Second, think logically about the stages/phases of your process, and the functional groups involved. For example, let’s say your trying to build a new product, you can start simply by saying, what are the big steps we will go through as a team, and what are the functional roles that the team will play. This is important if you want to avoid having everyone involved in every part of the project. This kind of high contact leads to burnout and failure, as everyone is always on, even when they don’t need to be, and your project ends up looking like a rave party than a slow waltz. So an example process sheet would look like (I like to use Visio Cross Functional Flowcharts):

Next, you put a simple box in each intersection where a process activity needs to occur. Now, I never put more than a single box in an intersecting square. The purpose of the chart above is to highlight a work interaction between a functional team and a phase, you can drill down into the actual deliverables and steps later. So my process might look like this.

So for example, in the Requirements phase, the business team are responsible for building the customer requirements. To start with, you may not have any steps, so at the project kickoff, you would start by defining the steps for this project to satisfy that part of the process. Next time around though, you could look back at the last project and say, “Wow, out of the 3 things we did, the focus groups worked really well, so let’s add that to the best practices list for Customer Requirements tasks”. Again, it’s not about forcing people to follow anything more than the high level process, but you also want to capture the prior learning, and express it in a way where someone can look over the task list for a process stage and say, “This was used to get this outcome, and I think I need that outcome for my project, so I’ll use that task”, but also enable people to add new tasks to the catalog.

The next step is you have to have a basic communications plan. In too many projects, emails fly around between project members and other connected people, with everything from FYI’s to work items and change requests. It’s dangerous because the wrong people can affect the outcome of your project, and critical items get lost in the email explosion. Set up some basic email aliases for your project, at a basic level, have 3:

1. A basic project comms alias for general FYI comms

2. A change request alias where requests can be managed

3. An oversight/stakeholder alias where critical decisions that need stakeholder approval/review can be sent

You should also attach a contract to each alias, for example, for the oversight alias, you might say that all the people on that alias commit to responding to any email sent to that list within 24 hours. This way, the alias doesn’t just become an vacuum or black hole, and people don’t lose faith in the alias and start emailing people directly.

And finally, you should have a clear accountability matrix. This ensures that the right people are involved in the right parts of the process, and more importantly, the wrong people are not involved. You matrix should map to your functional groups from your process chart, and should follow a model such as OARPi (Owner, Approver, Reviewer, Participant, Informed). This way, everyone knows when they are involved, and in what capacity. And also, they know who is not involved, so they can filter those people from the process. It really improves the signal:noise ratio for a project.

And that’s pretty much it. Each time you runt he process for a project, take the time at the end of the project to reflect, then update the body of knowledge for the process, highlighting tasks that worked really well in a phase for a functional group, and also tasks that didn't’, and include enough information for the next person so they know what you were trying to achieve, and what the outcome was.

Process is good, process is your friend! :)

So one of the key responsibilities of a Program Manager at Microsoft is to drive features. Features are the same as functions in some software nomenclature, and essentially, they map to something the software performs to satisfy a customer need.

Now, over the course of my career as a software engineer, I’ve seen two major disconnects in product teams. The first is where product teams never seem to be able to pick the right features to build. They have great talent, amazingly smart people, lots of resources, but just no direction, so they end up with awesome software that no one wants. I’ve also seen the opposite, teams with great feature lists, but a lack of diligence and professionalism required to ship quality features, so they build a product everyone wants but no one can use or will pay money for. This has led me to distill a little saying which has always held me in good stead: build the right feature, build the feature right. What does that even mean?

Well, knowing what features to build is an art in itself. When building a global product, it’s not easy to simply ask your customers what features you should build. Instead you have to use different techniques to ascertain what features are valuable to them, and which are not. You need to take into account multiple inputs, such as focus groups, customer visits, user feedback, market movements and pressures, and insights into areas of possible innovation. By doing this, you make sure that all the hard work the developers will do to produce your feature, is worthwhile not only to the customer, but to the business too.

Now once you’ve worked out the right features, you need to work out how to build those features right. See, this is where I’ve seen many a software group come unstuck. Picking a good feature is only half the battle, delivering it with quality and style is another. For example, in order to ship a great feature, many factors need to be considered, such as how the customer intends to use the feature, how the customer would like to use the feature, the best way to design the feature so the customer can be efficient and comfortable in using it, and can leverage existing skills and experience to be productive. It’s not good enough to simply cut some code, there needs to be a quality process from design through to deployment to ensure the feature is rock solid and appealing to the customer.

Shipping great software is about embracing both of these aspects, not just one of the other.

I came across this video today and really found it simple yet powerful.

For me, the best part of this video is it puts a real context around the whole S+S story, which to date, has been pretty hard to decipher.

The main takeaway as I see it, is that you can’t achieve everything you need from just desktop/server software or the cloud, you need a little bit of both. Why not have your cake and eat it too I say! ;)

When designing any piece of software, there are always going to be design tenants that you practice based on the type of software being developed.

For service software that is designed for a global, business critical audience, there are three key tenets in my opinion. In fact, there not so much tenets as they are assumptions.

So when you start designing your service(s), you need to design with the assumption that your service is going to do the following, and do it on a regular basis:

• Fail
  • This does not only mean your service will fail during runtime execution, but also during upgrades, deployment of new functionality, bug fixes, configuration changes to the environment
  • Your operating environment will fail. Power will go out, network appliances will die, hard disks will crash, CPU and Memory will fault, assume it will all happen, and probably all at once
  • In a standard data center deployment, you need to work out what your minimum unit of failure is. Is it a rack? Is it a switch? The data center? Knowing this helps you work out how to distribute your service to counter failures.
• Grow
• Your application is going to need to serve lots and lots of requests, so think about the scale of your application. Not only scale out (as in, the ability to serve lots of incoming requests simultaneously) but also scale up (make sure your application can dole out capacity)
• Bandwidth and Ingress/Egress is also a concern, pipes in and out of your service are constrained by physical limitations, so you’re going to have to think about how to deal with heavy load on your network and how your application will deal with this during peak and off-peak periods
• At some point, you service is going to exceed the place its located, as in, there just won’t be any more machines to deploy your service to, and there won’t be any more space to deploy more machines, so you’re going to have to design your service with the goal that you can deploy it across many servers across many data centers
• Distribute
  • Your customers invariably will want your service closest to them, and they will invariably never be located closest to your primary deployment data center, so design your service in a way that it can serve your customers at the point that closest to them. It reduces latency and improves service quality and experience

Also, designing a service involves thinking through the whole application lifecycle. Traditionally, we think about issues and bugs at the runtime level, but with services, because they are always on, living, breathing systems, you have to design them in a way that they can be changed (upgraded, configured differently, etc) at any time without interrupting its availability (the ability for customers to connect and use your service) and its consistency (the state maintained by the system must always be consistent and correct).

Now, this is by no means the magna carta of service design, but as you start thinking about designing your always available, highly scalable, geo-distributable service, the tenets above should be good food for thought.

Enjoy :)

OK folks, so all the announcements have been made for Mix, so it’s time to get cranking!

We have a new release of tools and SDK bits.

First, you’ll need to install some pre-reqs, but only if you are going to exploit some of the new capabilities:

If you don’t plan to use native code, FastCGI or URL rewriting, then don’t worry about the pre-reqs above.

Then, the tools and/or SDK. One big thing to note about the March release is we now have two install options:

• If you don’t use Visual Studio, then you can download the standalone Windows Azure SDK here, or
• If you use Visual Studio, then you can now get an all up VS Tools for Windows Azure and Windows Azure SDK installer here.

Enjoy!

My colleague and fellow Windows Azure PM Matt Kerner put me on to this site:

http://courtoons.wordpress.com/

Very fun cartoons related to Law/Legal subjects. My favorite (so far):

He he :)

So one of the things I’ve noticed since moving to the U.S is the complete obsession American’s (now, I’m being facetious, this observation is based purely on advertising) have with clothing you can wear while on the sofa/couch!

First I saw the “Snuggie”:

Essentially, it’s a blanket with sleeves, and has raised the ire of other perplexed consumers!

But then Yil sent me this today!

The Pantalaine “Couch Dress”! WTF!?

To be honest, if I’m going to get out and about with a piece of clothing that seconds as a coverlet, I’m going with the Pantalaine, I mean, just look how hot the model looks! Smoking!

Enjoy :)

So I’m back from Oz and have been tinkering with a small worker role app that grabs my Twitter feed and aggregates how many posts my friends have been doing as a scoreboard (I’ll post that tomorrow).

One of the design decisions I’ve had to make has been where to store the configuration settings.

See, in Windows Azure, you can either:

• Store your configuration settings using the App.config file for your project, then use the standard libraries from .NET to read those, or
• Use the ConfigurationSettings element within your Service Configuration and Service Definition files, and use the RoleManager.GetConfigurationSetting() method to read the setting

What’s the diff?

Well, when you store your settings in your Service Configuration and Definition files, you are able to change those settings without having to redeploy your application. If you use the App.config file, any change you make won’t be picked up until you redeploy your app.

Allow me to demonstrate.

So, first you need to define your setting in your ServiceDefinition file:

And add the ConfigurationSetting section into it:

<?xml version="1.0" encoding="utf-8"?>
<ServiceDefinition name="SettingsExample" 
                   xmlns="http://schemas.microsoft.com/_
                   ServiceHosting/2008/10/ServiceDefinition">
  <WebRole name="WebRole">
    <InputEndpoints>
      <!-- Must use port 80 for http and port 443 
      for https when running in the cloud -->
      <InputEndpoint name="HttpIn" protocol="http" port="80" />
    </InputEndpoints>
    <ConfigurationSettings>
      <Setting name="DefaultWelcome"/>
    </ConfigurationSettings>
  </WebRole>
</ServiceDefinition>

Next, define some values in the ServiceConfiguration file:

And set the values:

<?xml version="1.0"?>
<ServiceConfiguration serviceName="SettingsExample" 
                      xmlns="http://schemas.microsoft.com/_
                      ServiceHosting/2008/10/ServiceConfiguration">
  <Role name="WebRole">
    <Instances count="1"/>
    <ConfigurationSettings>
      <Setting name="DefaultWelcome" value="Manishkta"/>
    </ConfigurationSettings>
  </Role>
</ServiceConfiguration>

Next, you need to access your setting in your app, so you use the RoleManager.GetConfigurationSetting() to read it, like so:

using Microsoft.ServiceHosting.ServiceRuntime;

namespace SettingsExample_WebRole
{
    public partial class _Default : System.Web.UI.Page
    {
        protected void Button1_Click(object sender, EventArgs e)
        {
            lblGreeting.Text = 
            RoleManager.GetConfigurationSetting("DefaultWelcome") + 
            " " + txtName.Text;
        }
    }
}

Then deploy your app:

Now, when I run my app, it pulls the setting from my Windows Azure app deployment:

Next, let’s go change the setting live without messing with our deployment, to start, click the configure button:

Then change and save the setting:

Windows Azure will reconfig your app:

And it will now use the new setting:

This saves you from having to completely package up and redeploy your app every time you want to change a setting (due to changes in business rules or between production and staging environments for example).

Also, if you want to know how to paste code into your posts, check out this Live Writer Add-in! It rocks!

Chau :)

So I've blogged previously about a known issue regarding port translation/mapping in Windows Azure, and today (well, today and a few weeks ago) I came across an interesting side effect of this issue when accessing the Request.Url property within a Windows Azure web role.

Now, anytime you access the HttpContext.Request.Url property (or any derivation of that), you will be unknowingly getting some extra info you may not want.

For example (and as it happens, the piece of code at the center of today's investigation), if you are building a new URL in your Windows Azure application by calling:

string RootUri = HttpContext.Current.Request.Url.GetComponents(UriComponents.SchemeAndServer, UriFormat.Unescaped);

You're going to get something along the lines of:

Scheme: Http

Server: x.cloudapp.net

Port: 20000 <-- Aha! (I added the <-- Aha! for dramatic effect)

This is going to cause your app issues if you are then trying to use that in a dynamic link or web request.

The solution is simple, anytime you call something that returns a Request.Url object, make sure you ensure any calls you make on that Url object do not explicitly return the port (this gets tricky when doing local testing where you need the port, so you'll need to manage that). Construct your new URL with only host or path details, otherwise you'll make mischief for yourself!

Enjoy :)

So with the new release of our SDK and VS Tools and the recent release of Windows 7 Beta, some folks may have tried to run one on the other...

Well, in case you have and are wondering why some parts don't work, or in case you haven't and were thinking of trying to, or haven't and were thinking you might look good in mauve stretchy velour pants (like I did that one time at last years MSFT Christmas party), I have some info for you.

First, there is an issue with running the SDK (not the tools) on Windows 7 Beta. Essentially, you can install the SDK, but when you try to run your project on the local developer fabric, it will fall over.

Second, we're actively working on the problem, both us and the Windows 7 team, so as soon as we work out a solution we'll get it out to folks.

So what do you do in the meantime? My personal approach was to download the VSTS 2008 SP1/TFS SP1 free trial VPC (it expires on the 31st of December, 2009, which should give us enough time to sort the issue out ;)), install the SDK on that, then run that inside Virtual PC 2007 SP1.

Update: So as usual, I rushed through my post, made a bunch of bad assumptions, and ended up costing some folks a lot of wasted time... I'm sooo sorry! What happened? Well, to get past the Win7/Windows Azure SDK issue, I run a VPC environment on my laptop for demos. I actually got this VPC from an internal location, and thought it was VSTS 2008 running on Windows Server 2003. I installed the SDK, it runs great, so I then posted the above, pointing folks to a publicly available VPC that I thought/assumed was the same as mine. :( It wasn't. Turns out the one I have is actually Windows Server 2008, not Windows Server 2003. And as I'm not able to post the one I have, I have a work around.

Step 1. Grab a base VHD from here, either the Vista or Windows Server 2008 one.

Step 2. Download and install one of the VS 2008 trials.

Step 3. Install the Windows Azure SDK and VS Tools for Windows Azure on that.

Again, so sorry for the muck-up.

Enjoy :)

Today we released refreshes to both the Windows Azure SDK and the VS Tools for Windows Azure!

• Windows Azure SDK (this has the dev fabric and local storage stuff)
• VS Tools for Windows Azure (this has the templates and integration functionality to connect to the local and live environments for developing Windows Azure applications using Visual Studio)

What's hot in the new refresh?

Well, first we fixed a pesky little problem with VS hanging when you hit F5. For some customers, this ended up locking up the environment and you had to kill VS to get it all sorted, this is a huge fix.

We also fixed the limitation of using special characters in entities, which affected some update or delete operations.

There have also been a bunch of performance fixes, so the overall user experience feels a lot nicer compared to the PDC release.

There's quite a bit more in this release, however I only mentioned the issues that affected me most, I'm selfish like that (I'm sorry, please don't judge me).

So get cranking and download the new version, you won't be disappointed (especially me, since tonight is pizza night too!!).

Enjoy :)

A question popped up today about sending emails from within the Windows Azure environment, specifically using SmtpClient.

So, to cover off some basics:

1. The Windows Azure environment itself does not currently provide a SMTP relay or mail relay service
2. .NET applications run under a variation of the standard ASP.NET medium trust policy called Windows Azure Trust policy
3. Port 25 is open (well, all ports are open actually) to outbound traffic for Windows Azure applications

As we don't provide a host for sending or relaying SMTP mail messages, you'll need to find one that will. Your ISP or free mail provider will do providing they are setup to enable SMTP relaying.

Next, there is an issue with using the SmtpClient class with its Port property set to something other than 25 and its trust election, see this forum thread:

• http://social.msdn.microsoft.com/Forums/en-US/windowsazure/thread/409f1c6b-ba7a-4efa-aecf-51900d7f8c64/

So to demo this, I whipped up a quick sample app that sends STMP emails through a relay host as a web role:

The code is here:

• http://davidlem.svn.beanstalkapp.com/windowsazure/trunk/Windows%20Azure%20Hacking/AzureMail/

And the app is deployed here:

• http://196ac97b-025f-46e8-9216-c573a28590f5.cloudapp.net/Default.aspx

Enjoy :)

So I've had a couple of questions about hosting web services (SOAP style services in the form of *.aspx pages in a Web Role or *.svc files in the form of WCF services) in Windows Azure, specifically, how to get them to work?

Now, as I've mentioned before, Windows Azure uses a design where the executing environment that your hosted service is running in is actually just a pre-configured VM running Windows Server 2008 x64 with .NET 3.5 Fx installed amongst other things. So technically, running an ASP.NET web service or a WCF service should be nothing more than adding the required components (pages) to your web role. But before you get to your VM, there is some infrastructure that you need to step across, and this is what is currently providing some challenges to running hosted web services on Windows Azure.

So right now, if you create a new web role project, add an asmx page to it, publish it to the cloud, and run the page, it will work a treat. If you then add that web reference to a client project, you will get an error. The error happens because of the way we currently configure the internal Windows Azure environment to handle internal and external comms pathways (how your external URI maps to your internal machine URI).

Now this is a known issue but until we release a fix, the workaround is:

1. Create your web role and add your service definition (asmx or svc component)

2. Run the project using the local development fabric

3. While the service is running, add the local reference to your client project which will generate a compliant wrapper. This is important, because if you try to add a reference to your published service, you will run into the issue above. By adding the reference against the local service, you get the service definition added to your project.

4. Now publish the service, and record the URI (should be something like http://guidforstaging_or_accountnameforlive.cloudapp.net/service.asmx_or_svc)

5. Go into your client project, and update the web service reference URI from the local address to your live address

Your service will now be able to connect to your live service and have all the information it needs to communicate without having to do the discovery process against the live environment.

Here is a quick screen cast that demonstrates the work around process.

Also, check out the following links for more info:

• http://social.msdn.microsoft.com/Forums/en-US/windowsazure/thread/0be800f9-f7b3-4b27-bd85-10e81c5ef669/
• http://blogs.msdn.com/jnak/archive/2008/11/10/asp-net-mvc-on-windows-azure-with-providers.aspx

Enjoy :)

Wow, so it's 5:38pm on Wednesday the 31st of December, 2008, I've just wrapped up some last minute posts and am getting ready to head home.

What a year 2008 has been, just thinking back over it, it's been one of the most amazing years of my life.

What were some of the highlights?

• My daughter Oli was born :)

• Yil, Oli, Monix, Nelson and I moved to Washington, U.S

• I helped launch the Azure Services Platform at PDC 2008

• I scored a dream job as a Senior Program Manager on the Windows Azure Engineering team 

• My niece Madeline was born :)

• Our first Christmas in the U.S turned out to be a wonderful white one!

What will 2009 hold? Who can say?

Hope everyone has a safe, amazing and most of all, fun 2009!

Cya 2008 :)

So I've been playing around with the logging capabilities in Windows Azure, and have some thoughts to get down.

Firstly, Windows Azure provide provides an object called the RoleManager as part of its Runtime API.

With the RoleManager, you can WriteToLog (or write to a log). You can write different levels of messages, such as Critical, Error, Warning, Information and Verbose.

Once you start writing to a log, you need to do a couple of things to get the logs.

First, you need to dump the logs to a storage project. To do this:

1. Click on the Configure button on one of your hosted services

2. Select a Storage Account where you want your logs dumped to, and either use the generated name or insert your own into the Container name field. Your log files will be dumped into blobs within this container. I did not use my application storage project for logs, I created a second storage project called Logs, and dumped my logs to that.

Now, you just need to get them! I wrote a simple WPF app to grab these using my storage lib:

Easy! If you need the app, you'll need the following projects:

1. LogBrowser

2. Windows.Azure.Storage

Enjoy!