Eugenio Pace - Cloud Computing Guidance

First experiments with (new) SQL Data Services

eugeniop — Fri, 12 Jun 2009 19:03:29 GMT

Last week I got my new login to the new SQL Data Services. As a reminder for all readers:

SDS accelerates its plans to offer relational capabilities

May 11, 2009 - Based on customer feedback, SDS has accelerated its plans and will be offering true relational capabilities through SQL Server’s existing network protocol, Tabular Data Stream (TDS) and existing query language Transact-SQL (T-SQL). This will provide customers direct access to the familiar relational model, T-SQL programming language and the existing development and management tools, while continuing to deliver on our key value props of fault tolerance, high availability, friction free provisioning and pay as you grow scaling. For more information, see the SDS product site and the MSDN Library.

What I’ve done? After some initial “hello world-ish” tests, I wanted to try something more interesting so I decided to port IssueTracker into SDS.

As you know, IssueTracker was originally designed for SDS’ previous ACE model (Authority, Container, Entity), so my first task was to re-write the data access layer to use SQL Server.

One of my goals in this experiment was to test SDS “impedance match” with on-premises SQL Server. Also, I wanted to develop independently of the availability of SDS. Not that SDS is unreliable, but currently it is available only inside Microsoft’s corporate network. I didn’t want to VPN into corpnet for this when working from home.

So I chose to develop exclusively against my local SQL Express instance first and then make a switch to the real SDS.

Fortunately, the app was designed with a couple of layers that isolated the persistence details, so writing the new data tier was a fairly mechanical process.

This diagram roughly captures the architecture:

The repository classes implement a common interface the app uses, the Model is just a collection of rather simple C# objects with no knowledge of the database being used. The Mappers are responsible for the transformations between the application model and the entities that do have knowledge of the database.

In the diagram, classes marked with * are new, the numbers indicate variability points in the implementation, meaning that I can switch between one implementation and the other. Because I used LINQ to SQL, the types in the box labeled as “SQL Model” were generated
automatically by the LINQ to SQL designer.

When my unit tests compiled again, I switched the connection string to point from the “.\SQLEXPRESS” to the SDS instance in our network and…it worked! First attempt!

Overall, it was a rather painless and pleasant experience. Of course the data model in the app is simple and I’m not using any advanced queries or any sophisticated features in SQL yet.

Things missing and Possible next steps:

The original implementation had 2 requirements that leveraged features in SDS previous ACE model:

1- Multi-tenant isolation: achieved through containers. Each tenant got its own container.

2- Schema flexibility: tenants could customize the application, extending the schema of some core entities. Flexible entities made this very easy, because they are essentially property bags.

For #1, I considered two options:

1- Partitioning by tenant

2- Do not partition at all and have all tenants on the same database (single-instance, multi-tenant)

The first option is fairly straight forward. Each tenant gets its own database that is created at provisioning time. The “tenant id” is part of the calling context in the application, so I dynamically connect to each database as needed. Two advantages of this approach: there’s high isolation between tenants (no data from one can leak into another), and the application code is simpler, because from the data perspective, the application is “single-tenant”.

I haven’t implemented the extensibility feature yet, but I’m planning on reusing some techniques we did some research on in the past, probably through extension tables.

There’re other interesting areas for research such as:

1- Strategies for partitioning: in discussions with Ryan, he suggested I should consider more sophisticated ways of partitioning the information: by tenant, by tenant + project, etc. and I agree this would be interesting .

2- Unit of Work: currently I’m simply reusing the original ACE implicit UoW that comes with each interaction. This is, each time you called Create, Delete or Update on SDS, the operation was completed in the context of a unit of work. You could not logically group multiple operation (say, 2 creates and 1 delete). This is suboptimal with the SQL implementation, because the new SDS supports transactions and I would like to leverage that.

3- Performance and scalability issues: I haven’t spent any time looking at the application’s “chattiness” with the database that might lead to degraded performance, or any other data access optimizations. This is a whole area in itself, but not very different from “regular” application development. The only exception perhaps is that, in theory at least, the app and the database can be hosted in different datacenters (say the app in Amazon and the data in SDS). I’m not sure that would be a good idea anyway, probably not for this scenario. If the app was hosted in Windows Azure and used SDS, then they would be close in terms of network distance (low latency & high bandwidth).

Windows Azure 101 – Primitives and Application Patterns – Playing Mendeleyev

eugeniop — Wed, 18 Mar 2009 23:42:44 GMT

Windows Azure’s primitives are very simple, but as in many other things, the power comes from the combination of these simpler primitives to create more complex things.

Look around and see how many things can be assembled from a little more than 100 “simple” elements.

In Windows Azure, there are essentially 2 types of building blocks: code hosts blocks and persistent bocks

The code hosts run (your) code, the persistence blocks store data.

There are 2 types of Code Hosts:

Interactive: ASP.NET & WCF
Non-interactive: Worker

The interactive building blocks, whether it is a human initiated interaction (ASP.NET) or a programmatic interaction (WCF), is what is referred in Windows Azure terminology as a “Web Role”. The web role is specialized in “request – response” types of interactions. A user or a program submits a requests, the request is received, analyzed and processed, then a response is sent back. The goal is to process a lot of these concurrent requests and to keep the time between a request and a response as small as possible.

The non-interactive building block is known in Windows Azure as “Worker Role”, and it is the classic background processor.

There are 3 persistence building blocks. All of them store information, but have specialized functions:

Table: stores records with properties
Blobs: stores “things” with associated metadata
Queue: stores strings with FIFO semantics for retrieval

That’s it.

So let’s explore what you could do with this.

A relatively simple web site, like a simple blog engine would be this:

The front end web role is the app itself: pages, views, controllers, (whatever you use for the logic of the app). All operations (reads/writes) against the store where posts, comments and images would be stored are synchronous.

Adding one block will give you an RSS feed (e.g. using Syndication APIs in WCF):

And now you can independently manage (e.g. scale) your web viewers from those using an aggregator.

Now let’s imagine you’d like to create a heat map similar to the one you see in my blog, showing where are your readers are located. One possible way of solving this calling a components in the RSS or Web nodes providing as input the IP address of the requestor. The component would then lookup somewhere the country or region associated with the IP address and add one to the counter of that specific country/region. This computation will take penalize the request/response for something that the reader is not necessarily interested in. Besides the lookup IP/country might depend on an external call to another service, with even further penalties.

A better solution would be to offload these to another (background) process that con compute the information with minimal cost to the original request:

Now the front end nodes will only pay the cost of writing to a queue. The lookup/conversion/heat map generation is done in the background by the worker. You can imagine dynamically creating new instances of the worker if the queue gets too long. Anything that can be postponed for a while, can be pushed to an asynchronous worker for processing (e.g. reporting, analysis, etc)

These are just 6 elements in Microsoft’s larger table of elements for cloud development (.NET Services, SQL Data Services, etc).

These patterns are of course well known (and old :-)), but are proven. Windows Azure gives us a nice way of implementing them plus a way of managing them once they are deployed.

Technorati Tags: Azure

Azure IssueTracker Enterprise - Simple Demos

eugeniop — Mon, 16 Mar 2009 20:09:35 GMT

Provisioning IssueTracker Enterprise:

Highlights:

There’s no direct interaction with Access Control Service. IssueTracker uses ACS API to create the scopes, rules and the issuer (Contoso).
The provisioning form captures all the required information to setup the trust relationship between Access Controls Service and the tenant (certificate, etc)

Tenant (Contoso_Enterprise) uses IssueTracker Enterprise from a Smart Client (Active Profile):

Highlights:

Tenant STS is configured:
- Tenant name that must be the same as the name used in the provisioning form.
- Signing certificate thumbprint: this is used internally to retrieve the certificate form the store. Thumbprint can be obtained from the certificate properties.

Tenant Manages IssueTracker Enterprise from PowerShell scripts:

Highlights:

PowerShell CmdLets are registered
Management User disables the application (passing a parameter to define reason)
Business User attempts to use the system, gets an error message (with the above reason)
Management user enables application back

Tenant changes STS configuration issuing different Claims:

Highlights:

Tenant changes one of the output claims to “Program Manager”. In the real implementation this could be a user moving from group in Active Directory to another
System rejects access as the claim is not recognized as input to any rule in ACS

Architecting Cloud Application for the Enterprise – Part V – Management

eugeniop — Thu, 12 Mar 2009 22:41:26 GMT

Having solved the identity issue, the SuperCloudySoftware team focuses then on the management requirements raised in the meeting with VeryBigCorp CIO.

To recap, they want to:

Be able to to monitor IssueTracker from their existing infrastructure, that happens to be System Center Operations Manager (SCOM) and eventually author new rules to correlate IssueTracker specific events with other events that are generated inside their boundaries.
Be able to logically disable and enable IssueTracker for users from SCOM and
Be able to change claim mappings for authorization purposes from within standard tools (e.g. they don’t want to use a new web portal but would be happy with an MMC Snap-In)
As with their other (business) users, they want seamless integration for their IT staff: no login, no pop-up, etc. If an authenticated user of their network is authorized to do monitoring, he /she should be able to do it without any additional checks.

John and his team realize that the Web dashboard for monitoring is not enough for these requirements. It is still a good thing for their smaller Customers that don’t care too much about it anyway except when there are problems, but it will not work for VeryBigCorp.

The team realizes that they need to do a couple of things:

Instrument the application to gather information and to provide some level of control (e.g. enable/disable)
Create an external API for management
Create common clients for that API (e.g. PowerShell CmdLets, SCOM Management pack, etc)

Of course securing the API is relatively easy now that they have a claims based architecture as described in the previous post and can federate with consumers. They can leverage the same rules for granting access to this API: for example, VeryBigCorp would just need to define what claims will be mapped to the “Enable Application” claim that IssueTracker will expect.

The high level solution is illustrated below:

To simplify things even further, SuperCloudySoftware pre-configures the client library and the clients. So, just after provisioning, a client library will be automatically generated for the Customer with all the necessary information: more specifically, the information to setup WS-TRUST between IssueTracker, Access Control Service and the company (STS address, certificate, etc):

VeryBigCorp IT staff can then download the (pre-configured) tools from IssueTracker web site and they will “just work” as long as the logged in user provides the right credentials:

Notice there’s no need to specify who the tenant is, where the IssueTracker Management endpoint is, etc. All of that is built into the client library that the CmdLet uses.

SuperCloudySoftware strategy is to supply 3 experiences: MMC, PowerShell CmdLets and a SCOM Management Pack, but if there’s a customer with special needs (e.g. they are using Tivoli or HP OpenView), they also provide an API (secured with WS-TRUST).

Next article we will dive deeper into some implementation details of both the security and management aspects. As mentioned before, all of these is available in the download here.

Architecting Cloud Applications for the Enterprise – Part IV - SuperCloudySoftware sketches IssueTracker Enterprise Edition

eugeniop — Tue, 10 Mar 2009 03:15:00 GMT

In the previous article, we explored the challenges of building services for an enterprise, illustrating those through a dialogue between VeryBigCorp CIO and a team from SuperCloudySoftware.

VBC requirements can be summarized as:

Identity integration (including Single Sign On and management of access policies)
Management integration (ability to monitor and act on IssueTracker from within VBC’s management environment)
Application integration (ability to call IssueTracker programmatically from other VBC’s applications)

Let’s start with the beginning and explore the Authentication and Authorization aspects of IssueTracker.

In its current release, IssueTracker’s approach for user AuthN and AuthZ is straightforward username + passwords, and a user profile associated with it. This works ok for very small organizations (or even bigger organizations with a few people using it) but breaks quickly for companies like VeryBigCorp for many reasons:

People forget passwords and VeryBigCorp’s help desks would add a new task of resetting credentials.
People need to be retrained into new procedures. (“For IssueTracker password reset, please press 23”)
People get fired from VeryBigCorp’s and can the log on to the system from their homes and “creatively edit” the company’s information.
VeryBigCorp’s wants a seamless experience for their users (no pop-up, no login, nothing). It should just work.

Besides, VeryBigCorp already has a user repository (maybe more than one?) and they already have problems provisioning new users on it (them) when they hire new employees, etc. Adding a new repository for just 1 app is not going to happen.

The team @ SuperCloudySoftware considers one option:

Synchronizing VBC existing user repository with IssueTracker’s. This certainly can be done but is too ad-hoc, complex, cumbersome, error prone, non-standard. it wouldn’t work with other Customers and SuperCloudySoftware would much rather build one solution for all, and not treat each case as an exception.

A better solution is to use claims and federated identity. This allows the service to rely on somebody else that it trusts to authenticate users (VeryBigCorp) and deal with an abstracted set of facts (claims) about the requestor that are used for authorization.

Fortunately, the team went to Microsoft PDC last year and learnt about some technologies that would be very appropriate for these scenario: Geneva Framework and .NET Services Access Control Service and do all the heavy lifting of setting up federation and dealing with claims:

In this architecture the web site (IssueTracker) defines a set of claims that are required to access its resources, for example perform certain operation such as "Create a Project”, “Delete an Issue”, etc. Whoever brings those claims will be allowed to access the resource (e.g. complete the operation). If you don’t provide the claims, then you would get “access denied”. Claims can be obtained from a trusted issuer (VeryBigCorp’s identity provider such as Active Directory).

Simple analogy: This is like going to a bar and providing the bartender with a valid ID that will attest to your age. The resource is the beer, the claim is your age, the issuer is the government department that gave you the ID, the rule is “beer is fine if you are > 21 years old” (in the US)

Now, each company will attest to different facts about its users (like their name, which organization they work for, who is their manager, where they are located, etc.). These facts are irrelevant for IssueTracker, so some sort of mapping the original claims to those understood by IssueTracker needs to happen.

The translator allows VeryBigCorp to define rules of access. For example, they might say:

“Any employee who is a Program Manager will be able to Create New Projects in IssueTracker”

This implies taking the “being a member of the Program Manager group” claim and transform it into a “Create New Project” claim. Other organizations might have completely different rules and IssueTracker wouldn’t care as long as what it receives is something it understands.

That’s what Access Control Service is used for: convert claims from different issuers into others.

Following the bar analogy: the resource is the beer, the claim is your age, but instead of the bartender having to verify all sorts of different issuers of claims (passports, driving licenses, birth certificates, identity cards, etc), you simply put a guy in the door that verifies that and then gives you a colored bracelet. The rule for the bartender becomes: “anyone with a colored bracelet can have beer”. How you obtain it is someone else’s responsibility (hint: bribe the guy in the door :-) ).

This solution is actually very scalable and has a nice bonus: Access Control Service is already federated with a very popular identity provider like LiveID. So, by implementing this, SuperCloudySoftware has a LiveID enabled website for its smaller Customers for “free”.

The other nice thing is that this works with both Web Sites and Web Services (the so called “passive” and “active” profiles). So If IssueTracker exposes an API instead of a human readable web page, the same principles (and almost the same implementation) applies.

Note: IssueTracker has an API because of requirement #2 of veryBigCorp –> being able to integrate with other systems.

This is exactly what SuperCloudySoftware implemented and if you are curious about the details, then download the code and check for yourself!

IssueTracker Standard Edition (with federation with LiveID)
IssueTracker Enterprise Edition (with federation with a custom identity provider)

I’ll expand on the implementation details in the next articles.

P.S.: if you want to learn more about claims, federation and identity “theory” in general (including the alcoholic analogy :-)) then you should bookmark and read from il Maestro Vittorio Bertocci.

Updates: fixed a few typos.

The need for “standards for application logic” in PaaS. Really?

eugeniop — Wed, 25 Feb 2009 00:37:41 GMT

In his latest post on Coghead’s demise, Phil argues that:

“What this highlights is the lack of any standard for transferring not just data but application logic between such platforms.”

My argument is that those standards already exist and are widely adopted:

“Standards for capturing application logic already exist: Java & .NET (and COBOL). Coghead "mistake" was to try to develop their own development platform from scratch, instead of leveraging what already existed and provide value on top of that.”

Phil replied that:

“Doesn't solve the problem

But you still can't *transfer* logic from one development platform to another, say from COBOL to Java, or from Java to .NET, without completely rewriting it. What I'm advocating would be helpful to people developing on established platforms too. My point is that it's essential in a PaaS context.”

My response was getting too long, so I decided to post here instead.

Sure, having an abstract model for your app logic and then deciding implementation details would be great. I buy the attractiveness of such an approach and I understand why people would like this. (I’m sure it will sound familiar to my friends @ ArTech), but there’re problems too (e.g. “minimum common denominator” syndrome, lack of finer grained control, not being able to take advantage of the latest features in a given implementation, etc).

However, I certainly don’t think it is *essential* for PaaS. Nice, desirable, yes. Essential, I don’t think so.

Phil says that “With PaaS, the lack of such mechanisms could become a huge barrier to adoption as customers become fearful of which platform might be next to switch off the lights.”

True to some extent, but there are ways of mitigating this *today* without waiting for the uber-cross-platform-cross-cloud-ocean-boiling model.

Coghead could have chosen to offer app hosting for .NET and/or Java based apps (or PHP or COBOL for that matter), and attract 10,000’s of ISVs that have already bet on those platforms. Instead, they created a *new* platform from scratch. They not only required everybody to learn their new abstractions, their new language, their new tools, etc. Those by themselves are strong adoption barriers, not impossible to overcome, but quite tough.

But they also asked everybody to bet their operational business on them (the “aaS in “PaaS”), because nobody had access to their runtime except them. The lethality to the business viability is in the combination of these two factors. Platforms are catalysts, and as a consequence, they usually don’t do anything useful by themselves. They need to be bootstrapped.

So, if Coghead had chosen say .NET (I’m biased of course :-)) as their underlying programming model, barriers of entry would have been much lower for many reasons. Among them:

ISV would have had less cost in creating a “Coghead” solution (they would have reused all their existing skills, tools, knowledge, etc).
The cost of re-targeting their app would have been lower in the case the hoster goes out of business. Some work would have been required anyway, but not as high as with the current model.

In this hypothetical scenario, instead of parsing the XML files, they would have a bunch of .NET (C# or VB.NET) assemblies.

Some PaaS offerings, such as Apprenda, have taken this path. In my opinion a much healthier and pragmatic path.

The other obvious way of addressing these risks is with a “reverse escrow” from PaaS providers to their ISVs: giving out the runtime to the ISVs if they go out of business. Worse case, ISVs would buy time to port the application into another runtime. (like .NET).

Architecting Cloud Applications for the Enterprise - Part III - SuperCloudySoftware meets VeryBigCorp CIO

eugeniop — Wed, 10 Dec 2008 17:50:00 GMT

Actors: the CIO at VeryBigCorp, John (SuperCloudySoftware's CEO) and a bunch of senior developers and architects.

CIO: Gentlemen, welcome. Let me start by saying that our internal Customer is absolutely fascinated with your application. Believe me, they are hard to please and I've never seen them so excited. Congratulations!

John (CEO): thank you! We are really committed to our customer satisfaction

CIO: great to hear that. Now... the reason I wanted to meet with you is because we want to use the system for the entire division...

John (thinking): yes! yes! yes! 10,000 seats...

CIO: ...that would be 10,000 users approximately. The thing is that the current authentication approach you guys have today, username and password, doesn't really work for us. I don't want yet another password to maintain, troubleshoot, etc. We have enough identities already as a result of our latest set of mergers and acquisitions...

John (Glancing at the other tech folks, nodding and still thinking about 10K seats): Sounds completely reasonable to me...

CIO: Besides the maintenance hassles of dealing with people forgetting passwords, provisioning new employees, etc. there is a serious liability in having you guys managing authentication this way. If any of these employees is fired, nothing prevents him to logon from his home and creatively edit the data in the system. That's too risky for us. So we want Single Sign On between our own identity infrastructure and yours...

John (carefully but confidently): right, makes total sense...

CIO: Great. The second thing I want to discuss with you is integration. Event though IssueTracker would actually replace a few systems that we host ourselves, there are still other applications that need to pull and push information to IssueTracker. In particular, our ISO9000 support system absolutely needs to be fully integrated in real time with IssueTracker. We cannot have people duplicating entries on these two different systems.

John (cautiously): mhm

CIO: By the way, the ISO9000 system runs on our AS400 minicomputer...

John (thinking): A...S...what?

John (suddenly excited): well, there are export/import features in IssueTracker. Currently you could download or upload an Excel spreadsheet, and ....

CIO (interrupting John): sure, that might help, but what we need is that every time someone creates a new Issue in IssueTracker, the same information is replicated in the AS400 almost real-time...makes sense? Some delay is admissible, but it must be as online as possible. Couple of minutes at most...

John (worried): mhm

CIO: great we understand each other! You guys are really easy to work with!

John: Sure. We are really committed to our customers satisfaction. I'd like to meet with my team and take a look at this requirements. I'm sure we can develop a proposal for you in the next week or so...

CIO (interrupting again): ...wait, there's one more thing...

John: oh! sorry... (thinking: what else could they possibly want...)

CIO: IssueTracker will become a critical system for us. If something goes wrong, and believe me, it will; we need to know. We need our help desk to be fully integrated with yours. Our users are trained to call help desks for problems with any homologue system they interact with. So when a trouble ticket is generated here, we need that to be escalated to you.

The Architect (who had remained silent until then): hey, you could use IssueTracker to track issues with IssueTracker! To iterate is human, to recurse is divine...

John (piercing the architect with his eyes): thanks for the "suggestion", now could you please expand on your needs?

CIO: yeah...anyway, we recently invested on a new set management tools (we just deployed Microsoft System Center Operations manager), so ideally, we would like all administrative tasks to be launched from there...ok? I want to be able to track response time, correlate potential problems with other conditions...

John (Recovering): ok... We need to look at this. Frankly, we don't have those things in place today, but we are very interested in making this work for you.

I'm very comfortable with the technical capabilities of my team so I'm sure we will find a solution.

I'll come back to you next week with a proposal. ok?

John and his team leaves VeryBigCorp offices. John is really wondering if this thing of expanding into the enterprise segment is really worth it. Why not just continue to sell to the SMB segment as they've been doing so far? Anyway, John wants to explore the options and make a decision on more than initial gut feeling.

At the same time, his instincts tell him that VBC requirements are not really unique. If they solve them for VBC, they would solve them for all companies with that profile and position SCS competitively.

Next chapter: SuperCloudySoftware whiteboards options to address VeryBigCorp challenges.

Architecting Cloud Applications for the Enterprise - Part II - VeryBigCorp buys IssueTracker

eugeniop — Mon, 08 Dec 2008 05:02:00 GMT

Now that we have introduced our characters, we will start this story describing the acquisition process of IssueTracker in VeryBigCorp.

IssueTracker was first used in VBC by a very limited number of people. Someone in some business unit somehow learnt about it and thought it would be a good tool to automate their work.

A senior manager in that business unit approved the expense, a subscription was bought and IssueTracker became a new software asset in VBC. IT was not involved in this process and the acquisition happened in the "black market" of IT.

And thus, IssueTracker became an "illegal immigrant" in VBC IT department: not everybody knows it is there, some people think it is great, other would like to get rid of it, most regulators (in IT) would simply ignore it, as long as it doesn't cause any trouble.

This limbo status could last for a long time. Ironically, IssueTracker is so useful that it gains notoriety. It becomes clear to the business unit managers that it this little service is actually the perfect solution for their needs. Moreover, users love it. It's simple to use, it's intuitive and it does its job efficiently. Users productivity soars. IssueTracker is the little service that could.

All this goodness eventually reaches the business unit senior manager and he decides it is something that must be used by everybody. Just imagine all the productivity gains for the entire division! Increasing the efficiency of 10,000 employees under his leadership will have a tremendous impact in his bottom line.

So the senior manager calls the CIO and IssueTracker existence is made transparent. Not only the CIO learns about this "illegal immigrant", he now has to legalize it :-).

The CIO calls SuperCloudySoftware sales department and requests a meeting with their technical folks to "sort out a few issues" of turning IssueTracker into a qualified citizen of VBC IT.

On the other side of the line, SuperCloudySoftware just can't believe what has just happened. VeryBigCorp wants to buy 10,000 seats of Issuetracker. 10,000 seats!

The best technical guys in SuperCloudySoftware are sent to the meeting then. Their mission: do whatever it takes to win this Customer! Failure is not an option.

Next chapter of this story is the meeting between the CIO and SuperCloudySoftware.

Architecting Cloud Applications for the Enterprise - Part I - Introducing the Actors

eugeniop — Fri, 05 Dec 2008 19:14:00 GMT

I will start this series by introducing the main characters of our scenario.

First, we have VeryBigCorp. VBC is a large corporation, with multiple branches and subsidiaries, thousands of employees, etc. VBC is the typical organization with a rather complex business environment: multiple business units, complex rules, regulations, etc.

VBC IT department is a reflection of this complexity: they have lots of legacy components, multiple networking stacks and a rich myriad of technologies coexist in its data centers. VBC develops custom applications for some of their business units, but they also buy packages from specialized vendors.

VBC IT has multiple processes in place to deal with all these challenges: there are architecture and development guidelines that everyone is supposed to follow, there are software development lifecycle processes, standards, naming conventions, etc. All these are there for good reasons, but sometimes creates a perception of lack of agility and excessive bureaucracy.

Most technology acquisitions in VBC are handled by the IT department following strict steps.

The second character in our story is SuperCloudySoftware, a service provider (a "cloud ISV" if you want)

SCS has embraced the web since its foundation. SCS innovates very quickly, pushes updates on its service regularly based on customer feedback, focuses on user experience, etc. They are the ultimate "agilists".

SCS focused initially on smaller businesses, even some consumers. Their flagship service is IssueTracker a task tracking service.

IssueTracker is only available as a service. That means that you can't buy a license of it and deploy it in your own data center.

From the beginning SCS made the strategic decision of making IssueTracker available through "multiple heads":

There is a Web Client that only requires a browser
There's a Smart Client that provides a richer UX and enhanced connectivity options (e.g. working offline) and
There's also a Web Services API for all functions, that allows anybody to create their own clients or want to integrate with other client environments such as Microsoft Office.

IssueTracker itself relies on cloud building blocks. For example, the persistence of the application is based on SQL Data Services. This of course is completely opaque to their customers.

Next chapter will cover IssueTracker acquisition process in VeryBigCorp.

Architecting Cloud Services for the Big Enterprise

eugeniop — Thu, 04 Dec 2008 12:08:00 GMT

For the last couple of months, we’ve been working on scenarios that involve consuming cloud services and applications from "Big Enterprise". The focus being the technical obstacles for adopting cloud services.

Big Enterprise is the moniker we use to describe an organization with rather heavy investments on on-premises software.

Not surprisingly then, the aspects we have explored are integration in these three areas: security, management and application integration.

No important application in Big Enterprise lives in isolation so integration of cloud service on these three levels is key.

The good news is that there's new exciting tools, frameworks and services ISV can leverage today to address these requirements.

My session at PDC was the first time we presented this work. In general there’s agreement that these are recurring, common and important challenges; which is good validation.

Unfortunately not all demos @ PDC worked as I had expected (and endlessly practiced before), for various reasons. In retrospective, I’m glad we went through that pain, because there were lots of lessons learnt. Failure is a great teacher.

Since PDC, we’ve been working on improving the reference implementations our demos were based on, by incorporating all this experience: obscure pitfalls, wrong assumptions, unexpected limitations, lack of complete symmetry between production and pre-production, pure randomness, etc.

I'm certain many of you our there in the real world will face many of these same challenges, so I hope our pain will help you be more successful with the Azure Platform.

Since PDC, we presented the same (updated) content on various events: Teched EMEA, TechDays in Milano a few days ago and RAF in Madrid. In general we've got great feedback and good validation of our assumptions.

My plan is to drill down on each of these aspects in the next (several) couple of blog posts, so stay tuned!

Last but not least, most of the sample code demonstrated will be available for you to download and explore in the upcoming weeks.

Blog activity suspended until after PDC

eugeniop — Fri, 03 Oct 2008 17:10:07 GMT

I've got like 5 posts in my queue. All of them are drafts, all need work, all need nice graphics to illustrate the concepts and all are related to my session at PDC.

I'm 100% focused on actually having all my demos finished, deck's polished, etc. and I've frequently found myself back and forth editing and updating the articles. I will defer posting here anything until I'm done, which will probably be around the time PDC starts in 4 weeks.

See you there (and here)!

Tagged again

eugeniop — Thu, 25 Sep 2008 05:34:03 GMT

About a year and a half ago, some proto-ethnologist started a tagging blogo-game: you where asked to write 5 facts about you and then tag 5 other people you knew. Not sure if someone eventually did any analysis of the spread, lifetime and reach of the game. But regardless of the spreading speed, coverage, and eventual death, I found it to be a great way of learning something more personal about the people I usually read.

The game has started again, being tagged by my friend Gianpaolo. (I'm flattered he considers me intellectually sharp). This time the request is to name at least 5 books that influenced you.

Interestingly, the last fact I wrote about in the tag-blog post was about my library. Books have been a companion throughout all my life. They were the windows and doors to the whole world, to other people's ideas, to remote places. I traveled, I fought, I raised to glory, I cried with defeat and betrayal, I laughed, I loved, I cursed, I doubted and I learnt. My parents always told me: "toys, clothes and candy are optional. We might say 'no', but we will never say no to a book".

In the 1960 film The Time Machine, (adapted from the classic H.G. Wells classic book) the friends of the time traveler notice he left again and took 3 books with him. Then one asks: "Which three books would you have taken?", presumably to rebuild the exploited Eloi civilization in the future. That is brutal stack ranking! How could I just pick up 3! But wait...the request is to name at least 5 this time. Great!

As I mentioned in my previous post, I was lucky to inherit two libraries: my grandfather's and my godfather's. Besides my parents never said 'no' to books, so early on I had a fairly large collection. Each inherited collection was very different though: my grandfather's was an eclectic collection mostly about poetry (not Vogon), history, politics and philosophy. He had all the classics: Voltaire, Rousseau, Montesquieu, Kant, Francisco de Quevedo, Cervantes, Borges, etc. I read many, but most when I grew older and could actually digest what I read. Some at school of course. (and some, I confess I never finished). He mostly bought (very) cheap editions, so I had to be very careful not to break them, yet I can still recall the smell of those pages.

My godfather's books, on the other hand, where all about adventures: Emilio Salgari, Julio Verne, Daniel Defoe, Alexandre Dumas, etc. so before high school, I focused mainly on these. Many of them, I read many many times.

Memorable titles from that age: all of Salgari's (the "Sandokan" series and many others from this prolific writer), the "Voyages" from Verne, Quevedo's "Satiric Poems" which were awesome considering they were written in the XIV century and contained a lot of cursing in ancient spanish. This guy was like a "fake Steve Jobs" of the Spanish Golden Era.

One book that impacted me a lot was "Shakleton's Incredible Voyage", by Alfred Lansing. This was about a real adventure. Ernest Shakleton's was my hero. I admired his leadership and will, even in the extreme situation he and his team were in. From him I learnt: "never give up". I did a lot of research on Antarctic expeditions after that. (Many years later, while serving in the Army, I was assigned to a unit that supported the antarctic bases).

High school was all about fantasy and science fiction: J.L. Borges, Ray Bradbury, Theodore Sturgeon, Aldous Huxley, Stapledon, George Orwell, Fredric Brown, William Gibson, Philip Dick, Isaac Asimov, Ursula Le Guin, Tolkien, Lovecraft, E. A. Poe, Brian Aldiss. I loved all these stories.

Highlighted ones: Sturgeon's "More than Human", Asimov's robot stories, all of Borges' (never read Borges? try "The Library of Babel"), Aldiss' trilogy "Helliconia" (fascinating). I also did theater during high school, so I studied of course many plays, especially those I acted on :-). Very different, but very enjoyable too: "The Importance of being Ernest", "My Fair lady", among others.

One book I remember that also triggered a lot of further study was Eco's "The Name of the Rose". It was much later I linked Jorge de Burgos, the blind librarian of the story with Jorge Luis Borges the author. Himself a librarian and also blind. Excited about this first encounter, I tried other books from Eco, but they never quite made it as "The name...".

At that time I also became fascinated by great military campaigns & military technology: roman army and battles, WWII were my favorites. I knew everything about roman formations, shields and helmets, lorica segmentata and Adrian's wall; T-34 vs Tiger vs Sherman tanks, differences between BF-109 and Spitfire fighters, etc.

During late high school and University I read mainly technical stuff, but I also discovered Rudy Rucker who mixed history, mathematics & science fiction. "Infinity and the Mind", "The Hacker and the Ants", "Mindtools", the "* ware" series all great titles.

Later years until now, I've been reading some great books on many other topics. Besides all the classics of my profession (COM+, Cardspace, WCF Unleashed, etc) I've read (and read) a lot about social & economic sciences: Taleb's "Black Swan", Jared's "Collapse" & "Guns, Steel & Germs", Druon's "The accursed Kings", Pollan's "Omnivore's Dilemma" (great book), Jim Collins' "Good to Great" and "Build to Last". I also enjoy historical novels: Folletts' "The Pillars of the Earth", Seynor's "Roma" are great books and some of Arturo Perez Reverte books (El Pintor de Batallas, La Sombar del Aguila, La Carta Esferica).

Now my turn to tag: Ed, Glenn, Blaine, Ricardo and Matias

Northwind Hosting exists, it's better than what you saw and it's called SaaSGrid

eugeniop — Fri, 12 Sep 2008 03:32:09 GMT

Sinclair Schuller, CEO of Apprenda was kind enough to demo SaaSGrid to me earlier this week, patient enough to answer all my questions and I have to say, that I'm really impressed.

What is SaaSGrid? In their own words:

If you're building an on-demand business application, using SaaSGrid as your foundation will help you build quickly and inexpensively, ensuring that your customers can access your application's value sooner rather than later. Furthermore, you’ll then be able to deploy your SaaSGrid application to SaaSGrid itself, which provides a safe and robust virtual application container and hosting environment for your delivery needs. SaaSGrid's comprehensiveness ensures that your business and revenue approach will be able to flex and grow with changing market demands.

Sinclair walked me through the whole experience of building an app, on-boarding it, deploying it in different environments and then consuming it.

SaaSGrid has some very nice properties for ISVs. Besides the whole value proposition of hosting the app, SaaSGrid offers higher value services such as application lifecycle management, billing and metering, tenant management, etc. All of this without requiring you to learn a new programming language, paradigm and/or platform: everything is based on standard .NET so if you are familiar, skilled and trained on the standard Microsoft platform, leveraging SaaSGrid should really be very easy. Furthermore, you retain the freedom to deploy your app in other ways: on-premises, on-demand, etc.

This "non-intrusiveness" of SaaSGrid is a property of PaaS offerings we have studied in the past. I personally believe that all offerings requiring an ISV to re-write an app, or re-learn a whole new development paradigm (custom language, non-mainstream storage, etc) will be at a disadvantage compared to PaaS offerings that will make the most of your existing investments and strengths, and therefore adoption will be hurt (probability +80% :-)).

Last year we developed and wrote "Northwind Hosting concept demo" with the intent of elevating concepts we believed were important in PaaS offerings. We also wrote a couple of white papers describing this topic (e.g. ISVs are from Mars, Hosters are from Venus; Efficient Software Delivery Through Service Delivery Platforms).

Many concepts discussed there are now implemented for real in SaaSGrid, but I also saw lots of new innovative features which I found extremely useful.

Last, having an idea is great, but cheap :-). Writing about an idea takes some non-trivial effort (it is non-trivial for me at least). Turning an idea into a real offering is very hard. So kudos to Sinclair and his team for their hard work.

Concurrency in SSDS

eugeniop — Fri, 01 Aug 2008 23:12:31 GMT

A common concern with SSDS, and a common question I get in many presentations I've given is how to handle concurrency and entity versioning.

Suppose you have the following sequence of events:

By default, SSDS will just accept the last Update and overwrite any changes made in between. If you want SSDS to be strict about versioning, then you need to express this intent in the scope of the call.

If you are using SOAP, you just need to create a new instance of the VersionMatch class and specify your requirement:

public void Update(T entity)
{
     try
     {
         Scope scope = CreateScope();
         scope.VersionMatch = new VersionMatch() 
         { 
            MatchType = VersionMatchType.Match, 
            Version = entity.Version 
         };

         scope.EntityId = entity.Id.ToString();
         Entity flexibleEntity = entityMapper.FromType(entity);
         proxy.Update(scope, flexibleEntity);
     }
     catch (FaultException<Error> ex)
     {
        throw new UpdateException(ex);
     }
}

A side note: in writing a new test to verify this behavior, I dumped ExceptionExpected attribute in the test method altogether for this one, that gives me much more control on the exact place I expect the exception to occur:

Assert.IsTrue(ThrowsException<UpdateException>(() => rb.Update(b2)));

ThrowsException<E> is:

     private static bool ThrowsException<E>(Action f) where E : Exception
     {
            try
            {
                f();
            }
            catch (E)
            {
                return true;
            }
            catch
            {
            }

            return false;
     }

Claim based security made easy

eugeniop — Thu, 10 Jul 2008 17:22:35 GMT

When we implemented claim based authorization in LitwareHR, we had to write a lot of code and play with non-trivial configurations (LitwarehR includes 2 STS and all the supporting infrastructure for securing the web services and the callers to them).

Not being a security expert myself, I found the “theory” behind this amazingly simple and powerful, but the “practice” quite complex.

The good news is that all this just got much easier with the release of “Zermatt”:

“Zermatt” is a .NET developer framework and SDK that helps developers build claims-aware applications to address today’s application security requirements using a simplified model that is open and extensible, can improve security, and boosts productivity for developers. Developers can build externalized authentication capabilities for “relying party” applications and build custom “identity providers”, often referred to as Security Token Services (STS). With these components, developers can build applications that meet a variety of business needs more quickly.

Quoting my good friend Peter Provost: “I love deleting code!”. “Zermatt” will allow us to get rid of a ton of "plumbing" code in LitwareHR.

Update: if you look at LitwareHR code, you will see that the approach used is very similar to Zermatt's, so it is great to see that we were on the right direction. Obviously, Zermatt's scope is larger.

Resources:

Link to the beta: http://go.microsoft.com/fwlink/?LinkId=122266

Download Keith Brown's Whitepaper: http://go.microsoft.com/fwlink/?LinkId=122266

More info on MSDN: http://msdn.microsoft.com/en-us/security/aa570351.aspx

Maestro Bertocci's blog: http://blogs.msdn.com/vbertocci

Kim Cameron blog: http://www.identityblog.com

Keith Brown blog & article: http://www.pluralsight.com/community/blogs/keith/archive/2008/07/09/introducing-microsoft-code-name-zermatt.aspx

Requirements:

“Zermatt” requires .Net 3.5 to be installed. It has been verified on Windows 2K3 SP2 with IIS 6.0 and Windows Vista SP1 and Windows Server 2008 with IIS 7.0.