Keep track of all the latest news and events on developer tools and technologies you care about
By Will Lovegrove and Ian Cox
Our company, Release Consulting Ltd, is a small software development agency based in London. In 2012 we won a UK Government Technology Strategy Board research and development grant to build a highly innovative cloud-hosted data-sharing platform that would make API technology accessible to businesses of any size. That data-sharing platform (called datownia) was built on Windows Azure and launched in October 2012. Today we’re using datownia to provide APIs-as-a-Service to our SME clients, enabling them to easily open up and integrate data to their business applications.
This article looks back at our technical and financial experiences of our first cloud hosted project. It highlights a key finding for us which is that cloud computing’s ‘pay for what you use’ financial model means that system architecture and IT project management decisions now have an immediate and instant financial impact on our business in a way which is very different from when we used traditional non-cloud hosted infrastructure services.
The decision to implement our platform on Windows Azure was more a question of “why not Azure” than “why Azure”. Our staff are experienced in building applications with Microsoft software development tools. Plus Windows Azure provides a 90 day free trial period. It was an obvious decision to leverage our internal skills and the free offer. So in retrospect, given the lack of due diligence we performed when selecting our cloud infrastructure provider, can we say we are happy and willing advocates of Windows Azure? The answer is yes. But it comes with a caveat: carefully evaluate upfront your system design and project management choices. The decisions you make in the early stages of your project will affect the rate at which your project consumes cloud resources, which in turn affects the rate at which you are charged for those resources. In the early months of your project keep a close eye on those charges. It could be you are paying for cloud resources you don’t need.
The clear and obvious advantage of Windows Azure has been the shallow learning curve associated with the tools and technology. It took hours, not days, to set-up a sample project in the development emulator because we were already familiar with Visual Studio, SQL Server and other MS technology solutions. Time is money. Implementing on Windows Azure removed all effort associated with hosting infrastructure procurement, shortened our overall project time frame and immediately reduced our software infrastructure expenditure.
“Windows Azure gave us well formed tools and services that were a natural evolution of technologies that we were already familiar with.”
There were other aspects of the Windows Azure service which also impressed us:
1 Comprehensive diagnostics. Windows Azure makes it easy to log diagnostics across multiple instances. That gave us good data to analyse to work out what was going on with our roles and whether they were running within capacity. Which in turn helped us to optimise the performance of our application.
2 The speed of deployment of web and worker roles is faster than performing the same tasks in a custom virtual machine. That saved us time and made our team more efficient. For example, it shortened the issue-fix-redeploy iterations that our team followed in their agile methodology.
3 The ability to upload custom virtual machines using the VM role (and now also Azure Virtual Machines) was helpful. It enabled us to do things that were not so simple with a pure web or worker role, such as running third-party windows services that have previously proved awkward to get running within a worker role.
Windows Azure gave us well formed tools and services that were a natural evolution of technologies that we were already familiar with. That familiarity de-risked the technical jump into cloud. But it simultaneously introduced a new financial risk that was not obvious to us at the time: you pay for what you use. Whilst this statement is touted as an advantage of cloud computing it also means that unless you do diligent project planning and system architecture design up front then its very difficult to accurately predict your resulting cloud computing charges. This is especially true on your first Windows Azure project.
“We get good data to analyse to work out what is going on with our roles and whether they are running within capacity.”
For small companies who advocate ‘an agile approach’, or prefer a ‘just do it’ mentality, a lack of upfront planning and system architecture review processes could result in cloud computing charges which are higher than expected. For many SMEs reviewing the financial bottom line is a daily reality and our company is no exception. Our first monthly Windows Azure charges were simultaneously intriguing and a concern. We discovered that ‘pay-for-what-you-use’ brought system architecture decisions directly into the financial spotlight in a way which we hadn’t anticipated. Here are three examples which illustrate that point:
1 Log files are helpful but they consume resources. Yes, Windows Azure diagnostics are comprehensive but you pay for the storage of your log files. That could mean Windows Azure charges are higher than you expect while you run those logs. Keep an eye on your log files and tweak your diagnostic settings to get the right balance between storage taken up and data. Diagnostics can be configured in several places, so make sure it is doing what you want.
2 Worker roles vs Web roles. Although architecturally a worker role is cleaner and more scalable, it is more expensive to host a separate worker role when a web role can fire off a background task that can itself monitor a message queue. You have to get the balance right between cost, scalability and performance.
3 Being charged for deployments that are not actually running. Currently a non-running deployment costs as much as one that is running. To accurately forecast your Windows Azure costs you need to plan the deployments you need before your project commences, not during it.
The key learning from our first Windows Azure project is that in the early stages of the project it was very difficult to forecast Windows Azure charges. That created a sense of cloud computing being a complex financial model, even though the level of transparency and detail of the historical financial billings and usage logs are very good. The situation is best summarised by saying that cloud computing is good at telling you what you have spent, but its difficult to predict what you will spend in the future. By comparison, physical data center hosting services made it easy to financially predict hosting charges but came with the realisation that you were paying for infrastructure resources which you may not have utilized.
Our project (datownia) was not for a client. It was an internal project co-sponsored by the company’s CEO and CTO. That helped defuse any tension about unforeseen charges and the causes of those charges. However, the bigger issue for all cloud computing infrastructure service providers and their software developer customers is how to manage the financial expectations of external project sponsors (e.g. clients) in the early months of a project and especially when its the first cloud project for the client. The free 90 day trial of Windows Azure helps this issue by giving everyone concerned 90 days to build, observe and then trend the consumption of resources which affect the financial billing. But not every project can be implemented from the ground up in 3 months meaning that it may not be possible to reliably predict your expenditure before the end of the 90 day trial. The 90 days free trial also creates a ‘free ride’ for the project. Its not until the free 90 day trial expires, and the charges come through, that the serious questions are asked about how the charges relate to the application design. By that time it may be too late or too expensive to revisit core system architecture decisions.
Cloud computing is built on the principle of you pay for what you use. But until you build a project and operate it under production conditions you cannot be exactly sure what your cloud charges will be because you cannot be exactly sure what your requirements are in terms of logs, number of roles and number of instances of those roles. This puts technical project managers and system architects under more pressure to firstly predict their infrastructure costs in advance and then secondly to justify the design decisions that resulted in the corresponding cloud resource consumption profile.
As a small business it has become very clear to us that cloud-computing exposes technical project decisions to more scrutiny and exposes our company to a different type of financial risk than traditional procurement of infrastructure hosting services. This new risk can be mitigated by best practise system architecture and project management methodologies. Yet these methodologies introduce a new layer of project management overhead which counterbalances the speed and flexibility of creating new virtual machines and cloud resources.
On balance it’s been our experience that the long term benefit of ‘pay for what you use’ easily outweighs the initial lack of clarity about what exactly you are paying for. The more we learn about Windows Azure from a technical and financial perspective, and the more we learn about our own application, the better we are able to fine tune the application to optimise both the application performance and our Windows Azure charges. The ability to quickly trim the fat from cloud hosted projects and instantly optimise our costs is highly valuable to us.
In summary, the advice we have for other companies considering building applications in the cloud comes in two parts. Firstly, carefully manage your financial stakeholders expectations concerning the charges for the period in which your application is in development and also for the initial period in which your application operates in a ‘production’ or ‘general availability’ capacity. You will need to observe and trend the key resource consumption indicators for a significant period of time before you can build your own financial model that will predict cloud-computing charges. Your cloud project may very well be cheaper to run than its equivalent implementation on traditional infrastructure services. But it will take time to accurately model the actual savings. Keep a close eye on the cloud charges and the performance of your application in the interim period.
Secondly, understand that the culture of IT project management is inextricably linked to the financial management of a project. Cloud computing changes the cost model of IT projects. Expect a similar level of change in the culture of your IT project management the first time you build on cloud technology. That cultural change may begin as soon as the first cloud computing charges are invoiced and your company’s financial managers start to ask for accurate infrastructure cost forecasts from your software development team.
About the authors:
Will Lovegrove is CEO of Release Consulting Ltd, which trades as Release Mobile Ltd: a multi-platform mobile business solutions agency.
Visit Will on LinkedIn: uk.linkedin.com/in/willlovegrove/
Twitter: @willlovegrove @datownia
Ian Cox is CTO of Release Consulting Ltd. He is the architect and designer of all the companies software solutions which encompass .NET enterprise applications, iOS mobile applications and Android Mobile Applications.
Visit Ian on LinkedIn: uk.linkedin.com/pub/ian-cox/2/43/71a
‘datownia’ is a data-sharing platform owned by Release Consulting Ltd and implemented on Windows Azure. datownia is a simple but powerful data-sharing tool that helps its customers build secure IT system data interfaces. It flips Spreadsheets (stored in Dropbox or Box) into secure REStful web-service APIs and presents them within a Developer Portal branded with your company name. Customers administrate their Developer Portal meaning they control who can and cannot access their data via the APIs. datownia is free to try for 3 months and available at http://www.datownia.com