Eugenio Pace : SaaS

Exploring the Service Provider track – Fabrikam Shipping Part II (Solution)

eugeniop — Fri, 04 Sep 2009 07:45:10 GMT

Now that we presented the scenario & the requirements, let’s take a look at the solution.

What is conceptual solution we propose?

Fabrikam Shipping in the pre-Claims era:

This diagram shows Fabrikam Shipping today if used by Adatum (no claims, no federation):

You will see the usual suspects for a typical .NET web application. Furthermore, Fabrikam is using standard providers for authentication, authorization and profile. In this configuration, everyone in Adatum must use, of course, user name & passwords. The username is the handle associated with a role in the roles database, which drives application behavior (what you can do).

In the example, John from sales, can only Order New Shipments, but Peter from Customer Service, can Manage them.

Making Fabrikam Shipping Claims-Aware

What we want now, is Fabrikam to be claims aware and trust claims issued by Adatum. Claims issued by Adatum will be used for authentication and authorization. We also want to map Adatum internal roles to Fabrikam’s for authorization purposes: who will be a “Shipment Creator”? Who will be an “Administrator”?

Let’s see how this would work:

When John attempts to use FS for the first time (e.g. https://adatum.fabrikamshipping.com), because there’s no session established yet (John is un-authenticated from FS point of view) he will be redirected to Fabrikam’s Issuer (e.g. https://login.fabrikam.com). Fabrikam’s Issuer is trusted by the application.
Again, John will be redirected to Adatum’s Issuer, because that is what Fabrikam’s Issuer trusts.
If John uses a domain joined desktop, he’d already be authenticated in his network and will have a valid Kerberos token. This token is used by the Adatum’s Issuer to create Adatum’s claims: employee name, employee address, cost center, and department John works for.

The process unwinds then:

Adatum’s claims are sent back to Fabrikam’s Issuer, where they are transformed:

- Name, address and cost center are simply copied (no transformation)

- Other rules are applied that will result in a “role” claims to be issued (any of the valid roles FS understands)

More examples of mappings:

exists([issuer == "Adatum"]) => issue(type = "Role", value = "Shipment Creator");

Which can be interpreted as:

“Any employee from Adatum can create shipment orders”

c:[type == “http://schemas.xmlsoap.org/claims/Group”, value == "Shipments"] => issue(type = “Role”, value = “Shipment Manager”);

that would implement the rule:

“Any employee from Adatum in “Shipments” (indicated by group membership) department can manage shipment orders”

2. After these transformation happens, John is finally directed back to the application with the transformed claims.

Adatum could issue Fabrikam’s specific claims, but we don’t want to pollute Adatum’s Issuer with Fabrikam specific concepts (like Fabrikam roles). Fabrikam will allow Adatum to issue any claims they want or can, and then will allow Adatum to configure the system to map these Adatum claims into Fabrikam claims.

Fabrikam will do this for every new Customer using Fabrikam Shipping. Yet, their application will always understand the same set of claims: “Shipment Creator”, etc. FS stays decoupled.

Note 1:
This scenario is almost identical to IssueTracker. If you feel deja-vu, don’t be surprised. Only in IssueTracker, we used .NET Services ACS as the Service Provider (Fabrikam) Issuer.

Note 2:
This scenario is also similar (but not quite the same) to Adatum’s a-Order. Some key differences: Fabrikam is a multi-tenant system, probably with a provisioning experience, that a-Order lacked. This is because in our fictitious (but hopefully realistic) world, the Customer churn in Fabrikam Shipping is much higher than in a-Order. That is, we assume the frequency customers join and leave Fabrikam is higher. Thus, Fabrikam needs to automate this as much as possible.

Note 3:
Yes, there will be another post with Adatum’s side of the story. But I’m sure by now you’ll guess what’s in there.

I’ll cover provisioning in the next post, as it has some interesting discussion points. But you can see some hints here.

Feedback very much welcome.

Post-post announcement:

We hope to have a some running code and much much polished chapters soon. We’ll probably upload those to a CodePlex site. Stay tuned!

Exploring the Service Provider track – First station: Fabrikam Shipping – Part I (the scenario & challenges)

eugeniop — Wed, 02 Sep 2009 01:57:10 GMT

Once again, thanks everybody that wrote us with reviews, feedback and suggestions! Please keep it coming! Also: we hope to have soon a CodePlex site where we can start sharing more. We are still working out some details.

As usual, the Disclaimer: this post and the next ones are early drafts to share with you the direction we are taking. They might (and I hope they will) change quite a bit in the actual Guide! We might end up not covering one of these scenarios in the book.

An additional disclaimer for this post: I wrote the whole scenario following the same template of the previous posts and it resulted in a very loooong article. So I divided it into two parts. This is Part I –> the scenario, the challenges and the requirements. Part II will be the solution.

An Ode to Simplification: there’s been quite some debate internally to this project as how to name things, especially “STS” vs. “Issuer” vs. “I-STS” vs. “R-STS” vs. “FP”, etc. Keith has started this on his blog some time ago. We definitely want to keep things simple. As simple as possible, but not simpler. For now we have settled on the term “Issuer”, independently of the logical role it takes part in. In simpler words: what we used to call “Identity Provider” is now an “Issuer”. What we called a “Federation provider” is also an “Issuer”.

Keith is writing a whole section of our book on “Jargon” and meaning of the different terms.

Credits: this scenario is largely inspired on Vittorio’s PDC demo. See here.

The themes for the first “Service Provider” scenario are:

Identity in a SaaS application
Federation with multiple Customers

There’s 1 variations in this scenario:

Automating the on-boarding process

The Introduction

Fabrikam is a company that provides shipping services. As part of their offering, they have an application (Fabrikam Shipping – FS) that allows its customers to create new shipping orders, track them, etc. Fabrikam Shipping is delivered as a service and runs in Fabrikam’s datacenter. Fabrikam Customers use a browser to access it.

FS is a fairly standard .NET web application: the web site is based on ASP.NET 3.5, the backend is SQL Server, etc. In the current version, users are required to authenticate using (guess what): username and password!!

Fabrikam uses ASP.NET standard providers for authentication (Membership), authorization (Roles provider) and personalization (Profile).

Fabrikam Shipping is also a multi-tenant application: the same instance of the app is used by many customers.

One sunny day in Seattle, they sign a great deal with a marquee Customer: Adatum Corp. And Adatum doesn’t like the username and password, because they are working hard to get rid of identity silos. They have 3 concerns:

Usability for their employees. Lack of SSO, forgetting passwords, using sticky notes to remember them, etc.
Maintenance costs:

What happens if an employee forgets his or her corporate password? He will probably call Adatum’s IT help desk. What happens if they use FS and they forget its password? who should they call? Consider this:

If they instruct employees to call Fabrikam’s help desk, there would be a special procedure for IT guys, would probably require training, etc.
If they instruct employees to call Fabrikam directly, they would impact #1

When a new employee is hired, he is already provisioned in Adatum’s systems. They don’t want special processes for FS.

Liability:
1. Adatum has authentication policies that are there for a reason. They also want to retail control on who has access to what (regardless of where that is deployed) and FS is no exception.
2. If an employee leaves the company, he should not have access to FS anymore, effective immediately. If they used username / passwords, they could potentially access FS from other places, even if they are not an Adatum employee anymore.

Back to FS:

Access Control to FS is based on Roles. There are 3 roles:

“Shipment Creators”. Anyone in this role can create new orders.
“Shipment Managers”. Can create and modify existing shipment orders.
“Administrators”. Can configure the system (e.g. look and feel, shipping preferences, billing, etc).

FS also keeps profile information for users, to avoid repeatedly entering common information and preferences. More concretely, FS allows its users to store:

Package Sender information (sender address)
Cost Center information for billing

Fabrikam can open the bills to its Customers by Cost Center. With this, 2 employees from Adatum belonging to 2 different departments would get 2 different bills.

Key Requirements.

Adatum wants SSO for its employees.

Fabrikam wants to avoid storing configuration information about the shipment that can become stale later on (e.g. the package sender information).

Fabrikam wants to bill customers by Cost Center if they supply one.

Some assumptions:

Adatum has an Issuer (see Scenario #1)
Fabrikam can change anything in their application

We’ll look at the solution space in the next post.

Announcing new project – patterns & practices - Claims based Authentication & Authorization Guide

eugeniop — Tue, 11 Aug 2009 23:02:22 GMT

For the next couple of months I’ll be working on a new project here at patterns & practices, developing a new guide for claims based authentication and authorization.

I’m personally very happy to be working on this project, for many reasons. I believe frameworks like “Geneva” (previously known as “Zermatt”, now Windows Identity Foundation), products like “Geneva Server” (now ADFS) are great platform additions to enable a new set of scenarios.

I realize that SSO, Federated Identity and Claims are not new. It’s just that we have much better tools and higher abstractions to implement these scenarios much more easily than ever.

I also feel privileged to work with such a great team. I’ll be sitting on giants shoulders: Dominick Baier, Vittorio Bertocci, Keith Brown, David Hill and Matias Woloski. Many others are joining as advisors and reviewers.

As it is customary now in the patterns & practices team, we will be publishing our content often and very early. I’ll post details here soon.

We also want to try a few new things in this project. In this guide we want to be very focused on the practices rather than on the “theory”, the “principles” or “philosophy” of claims based security.

We want to have very concrete scenarios, with a high fidelity of what happens out there in the real world. Almost a “case study” approach in which we weave a story across the book that takes the reader into more ambitious requirements as he proceeds.

With each chapter, we will introduce more complex solutions to address increasingly more ambitious requirements.

The current backlog for the scenarios we want to cover is illustrated below. Each “station” is a core scenario. Some will have small variations (like Azure hosting in the first one).

The two lines (yellow and light blue) refer to the two perspectives we plan to include: that of someone consuming software (the blue), and that of some building software (the yellow).

Stay tuned!

Eugenio

Update: fixed image size.

First experiments with (new) SQL Data Services

eugeniop — Fri, 12 Jun 2009 22:03:29 GMT

Last week I got my new login to the new SQL Data Services. As a reminder for all readers:

SDS accelerates its plans to offer relational capabilities

May 11, 2009 - Based on customer feedback, SDS has accelerated its plans and will be offering true relational capabilities through SQL Server’s existing network protocol, Tabular Data Stream (TDS) and existing query language Transact-SQL (T-SQL). This will provide customers direct access to the familiar relational model, T-SQL programming language and the existing development and management tools, while continuing to deliver on our key value props of fault tolerance, high availability, friction free provisioning and pay as you grow scaling. For more information, see the SDS product site and the MSDN Library.

What I’ve done? After some initial “hello world-ish” tests, I wanted to try something more interesting so I decided to port IssueTracker into SDS.

As you know, IssueTracker was originally designed for SDS’ previous ACE model (Authority, Container, Entity), so my first task was to re-write the data access layer to use SQL Server.

One of my goals in this experiment was to test SDS “impedance match” with on-premises SQL Server. Also, I wanted to develop independently of the availability of SDS. Not that SDS is unreliable, but currently it is available only inside Microsoft’s corporate network. I didn’t want to VPN into corpnet for this when working from home.

So I chose to develop exclusively against my local SQL Express instance first and then make a switch to the real SDS.

Fortunately, the app was designed with a couple of layers that isolated the persistence details, so writing the new data tier was a fairly mechanical process.

This diagram roughly captures the architecture:

The repository classes implement a common interface the app uses, the Model is just a collection of rather simple C# objects with no knowledge of the database being used. The Mappers are responsible for the transformations between the application model and the entities that do have knowledge of the database.

In the diagram, classes marked with * are new, the numbers indicate variability points in the implementation, meaning that I can switch between one implementation and the other. Because I used LINQ to SQL, the types in the box labeled as “SQL Model” were generated
automatically by the LINQ to SQL designer.

When my unit tests compiled again, I switched the connection string to point from the “.\SQLEXPRESS” to the SDS instance in our network and…it worked! First attempt!

Overall, it was a rather painless and pleasant experience. Of course the data model in the app is simple and I’m not using any advanced queries or any sophisticated features in SQL yet.

Things missing and Possible next steps:

The original implementation had 2 requirements that leveraged features in SDS previous ACE model:

1- Multi-tenant isolation: achieved through containers. Each tenant got its own container.

2- Schema flexibility: tenants could customize the application, extending the schema of some core entities. Flexible entities made this very easy, because they are essentially property bags.

For #1, I considered two options:

1- Partitioning by tenant

2- Do not partition at all and have all tenants on the same database (single-instance, multi-tenant)

The first option is fairly straight forward. Each tenant gets its own database that is created at provisioning time. The “tenant id” is part of the calling context in the application, so I dynamically connect to each database as needed. Two advantages of this approach: there’s high isolation between tenants (no data from one can leak into another), and the application code is simpler, because from the data perspective, the application is “single-tenant”.

I haven’t implemented the extensibility feature yet, but I’m planning on reusing some techniques we did some research on in the past, probably through extension tables.

There’re other interesting areas for research such as:

1- Strategies for partitioning: in discussions with Ryan, he suggested I should consider more sophisticated ways of partitioning the information: by tenant, by tenant + project, etc. and I agree this would be interesting .

2- Unit of Work: currently I’m simply reusing the original ACE implicit UoW that comes with each interaction. This is, each time you called Create, Delete or Update on SDS, the operation was completed in the context of a unit of work. You could not logically group multiple operation (say, 2 creates and 1 delete). This is suboptimal with the SQL implementation, because the new SDS supports transactions and I would like to leverage that.

3- Performance and scalability issues: I haven’t spent any time looking at the application’s “chattiness” with the database that might lead to degraded performance, or any other data access optimizations. This is a whole area in itself, but not very different from “regular” application development. The only exception perhaps is that, in theory at least, the app and the database can be hosted in different datacenters (say the app in Amazon and the data in SDS). I’m not sure that would be a good idea anyway, probably not for this scenario. If the app was hosted in Windows Azure and used SDS, then they would be close in terms of network distance (low latency & high bandwidth).

Windows Azure 101 – Primitives and Application Patterns – Playing Mendeleyev

eugeniop — Thu, 19 Mar 2009 02:42:44 GMT

Windows Azure’s primitives are very simple, but as in many other things, the power comes from the combination of these simpler primitives to create more complex things.

Look around and see how many things can be assembled from a little more than 100 “simple” elements.

In Windows Azure, there are essentially 2 types of building blocks: code hosts blocks and persistent bocks

The code hosts run (your) code, the persistence blocks store data.

There are 2 types of Code Hosts:

Interactive: ASP.NET & WCF
Non-interactive: Worker

The interactive building blocks, whether it is a human initiated interaction (ASP.NET) or a programmatic interaction (WCF), is what is referred in Windows Azure terminology as a “Web Role”. The web role is specialized in “request – response” types of interactions. A user or a program submits a requests, the request is received, analyzed and processed, then a response is sent back. The goal is to process a lot of these concurrent requests and to keep the time between a request and a response as small as possible.

The non-interactive building block is known in Windows Azure as “Worker Role”, and it is the classic background processor.

There are 3 persistence building blocks. All of them store information, but have specialized functions:

Table: stores records with properties
Blobs: stores “things” with associated metadata
Queue: stores strings with FIFO semantics for retrieval

That’s it.

So let’s explore what you could do with this.

A relatively simple web site, like a simple blog engine would be this:

The front end web role is the app itself: pages, views, controllers, (whatever you use for the logic of the app). All operations (reads/writes) against the store where posts, comments and images would be stored are synchronous.

Adding one block will give you an RSS feed (e.g. using Syndication APIs in WCF):

And now you can independently manage (e.g. scale) your web viewers from those using an aggregator.

Now let’s imagine you’d like to create a heat map similar to the one you see in my blog, showing where are your readers are located. One possible way of solving this calling a components in the RSS or Web nodes providing as input the IP address of the requestor. The component would then lookup somewhere the country or region associated with the IP address and add one to the counter of that specific country/region. This computation will take penalize the request/response for something that the reader is not necessarily interested in. Besides the lookup IP/country might depend on an external call to another service, with even further penalties.

A better solution would be to offload these to another (background) process that con compute the information with minimal cost to the original request:

Now the front end nodes will only pay the cost of writing to a queue. The lookup/conversion/heat map generation is done in the background by the worker. You can imagine dynamically creating new instances of the worker if the queue gets too long. Anything that can be postponed for a while, can be pushed to an asynchronous worker for processing (e.g. reporting, analysis, etc)

These are just 6 elements in Microsoft’s larger table of elements for cloud development (.NET Services, SQL Data Services, etc).

These patterns are of course well known (and old :-)), but are proven. Windows Azure gives us a nice way of implementing them plus a way of managing them once they are deployed.

Technorati Tags: Azure

Azure IssueTracker Enterprise - Simple Demos

eugeniop — Mon, 16 Mar 2009 23:09:35 GMT

Provisioning IssueTracker Enterprise:

Highlights:

There’s no direct interaction with Access Control Service. IssueTracker uses ACS API to create the scopes, rules and the issuer (Contoso).
The provisioning form captures all the required information to setup the trust relationship between Access Controls Service and the tenant (certificate, etc)

Tenant (Contoso_Enterprise) uses IssueTracker Enterprise from a Smart Client (Active Profile):

Highlights:

Tenant STS is configured:
- Tenant name that must be the same as the name used in the provisioning form.
- Signing certificate thumbprint: this is used internally to retrieve the certificate form the store. Thumbprint can be obtained from the certificate properties.

Tenant Manages IssueTracker Enterprise from PowerShell scripts:

Highlights:

PowerShell CmdLets are registered
Management User disables the application (passing a parameter to define reason)
Business User attempts to use the system, gets an error message (with the above reason)
Management user enables application back

Tenant changes STS configuration issuing different Claims:

Highlights:

Tenant changes one of the output claims to “Program Manager”. In the real implementation this could be a user moving from group in Active Directory to another
System rejects access as the claim is not recognized as input to any rule in ACS

Architecting Cloud Application for the Enterprise – Part V – Management

eugeniop — Fri, 13 Mar 2009 01:41:26 GMT

Having solved the identity issue, the SuperCloudySoftware team focuses then on the management requirements raised in the meeting with VeryBigCorp CIO.

To recap, they want to:

Be able to to monitor IssueTracker from their existing infrastructure, that happens to be System Center Operations Manager (SCOM) and eventually author new rules to correlate IssueTracker specific events with other events that are generated inside their boundaries.
Be able to logically disable and enable IssueTracker for users from SCOM and
Be able to change claim mappings for authorization purposes from within standard tools (e.g. they don’t want to use a new web portal but would be happy with an MMC Snap-In)
As with their other (business) users, they want seamless integration for their IT staff: no login, no pop-up, etc. If an authenticated user of their network is authorized to do monitoring, he /she should be able to do it without any additional checks.

John and his team realize that the Web dashboard for monitoring is not enough for these requirements. It is still a good thing for their smaller Customers that don’t care too much about it anyway except when there are problems, but it will not work for VeryBigCorp.

The team realizes that they need to do a couple of things:

Instrument the application to gather information and to provide some level of control (e.g. enable/disable)
Create an external API for management
Create common clients for that API (e.g. PowerShell CmdLets, SCOM Management pack, etc)

Of course securing the API is relatively easy now that they have a claims based architecture as described in the previous post and can federate with consumers. They can leverage the same rules for granting access to this API: for example, VeryBigCorp would just need to define what claims will be mapped to the “Enable Application” claim that IssueTracker will expect.

The high level solution is illustrated below:

To simplify things even further, SuperCloudySoftware pre-configures the client library and the clients. So, just after provisioning, a client library will be automatically generated for the Customer with all the necessary information: more specifically, the information to setup WS-TRUST between IssueTracker, Access Control Service and the company (STS address, certificate, etc):

VeryBigCorp IT staff can then download the (pre-configured) tools from IssueTracker web site and they will “just work” as long as the logged in user provides the right credentials:

Notice there’s no need to specify who the tenant is, where the IssueTracker Management endpoint is, etc. All of that is built into the client library that the CmdLet uses.

SuperCloudySoftware strategy is to supply 3 experiences: MMC, PowerShell CmdLets and a SCOM Management Pack, but if there’s a customer with special needs (e.g. they are using Tivoli or HP OpenView), they also provide an API (secured with WS-TRUST).

Next article we will dive deeper into some implementation details of both the security and management aspects. As mentioned before, all of these is available in the download here.

Architecting Cloud Applications for the Enterprise – Part IV - SuperCloudySoftware sketches IssueTracker Enterprise Edition

eugeniop — Tue, 10 Mar 2009 06:15:00 GMT

In the previous article, we explored the challenges of building services for an enterprise, illustrating those through a dialogue between VeryBigCorp CIO and a team from SuperCloudySoftware.

VBC requirements can be summarized as:

Identity integration (including Single Sign On and management of access policies)
Management integration (ability to monitor and act on IssueTracker from within VBC’s management environment)
Application integration (ability to call IssueTracker programmatically from other VBC’s applications)

Let’s start with the beginning and explore the Authentication and Authorization aspects of IssueTracker.

In its current release, IssueTracker’s approach for user AuthN and AuthZ is straightforward username + passwords, and a user profile associated with it. This works ok for very small organizations (or even bigger organizations with a few people using it) but breaks quickly for companies like VeryBigCorp for many reasons:

People forget passwords and VeryBigCorp’s help desks would add a new task of resetting credentials.
People need to be retrained into new procedures. (“For IssueTracker password reset, please press 23”)
People get fired from VeryBigCorp’s and can the log on to the system from their homes and “creatively edit” the company’s information.
VeryBigCorp’s wants a seamless experience for their users (no pop-up, no login, nothing). It should just work.

Besides, VeryBigCorp already has a user repository (maybe more than one?) and they already have problems provisioning new users on it (them) when they hire new employees, etc. Adding a new repository for just 1 app is not going to happen.

The team @ SuperCloudySoftware considers one option:

Synchronizing VBC existing user repository with IssueTracker’s. This certainly can be done but is too ad-hoc, complex, cumbersome, error prone, non-standard. it wouldn’t work with other Customers and SuperCloudySoftware would much rather build one solution for all, and not treat each case as an exception.

A better solution is to use claims and federated identity. This allows the service to rely on somebody else that it trusts to authenticate users (VeryBigCorp) and deal with an abstracted set of facts (claims) about the requestor that are used for authorization.

Fortunately, the team went to Microsoft PDC last year and learnt about some technologies that would be very appropriate for these scenario: Geneva Framework and .NET Services Access Control Service and do all the heavy lifting of setting up federation and dealing with claims:

In this architecture the web site (IssueTracker) defines a set of claims that are required to access its resources, for example perform certain operation such as "Create a Project”, “Delete an Issue”, etc. Whoever brings those claims will be allowed to access the resource (e.g. complete the operation). If you don’t provide the claims, then you would get “access denied”. Claims can be obtained from a trusted issuer (VeryBigCorp’s identity provider such as Active Directory).

Simple analogy: This is like going to a bar and providing the bartender with a valid ID that will attest to your age. The resource is the beer, the claim is your age, the issuer is the government department that gave you the ID, the rule is “beer is fine if you are > 21 years old” (in the US)

Now, each company will attest to different facts about its users (like their name, which organization they work for, who is their manager, where they are located, etc.). These facts are irrelevant for IssueTracker, so some sort of mapping the original claims to those understood by IssueTracker needs to happen.

The translator allows VeryBigCorp to define rules of access. For example, they might say:

“Any employee who is a Program Manager will be able to Create New Projects in IssueTracker”

This implies taking the “being a member of the Program Manager group” claim and transform it into a “Create New Project” claim. Other organizations might have completely different rules and IssueTracker wouldn’t care as long as what it receives is something it understands.

That’s what Access Control Service is used for: convert claims from different issuers into others.

Following the bar analogy: the resource is the beer, the claim is your age, but instead of the bartender having to verify all sorts of different issuers of claims (passports, driving licenses, birth certificates, identity cards, etc), you simply put a guy in the door that verifies that and then gives you a colored bracelet. The rule for the bartender becomes: “anyone with a colored bracelet can have beer”. How you obtain it is someone else’s responsibility (hint: bribe the guy in the door :-) ).

This solution is actually very scalable and has a nice bonus: Access Control Service is already federated with a very popular identity provider like LiveID. So, by implementing this, SuperCloudySoftware has a LiveID enabled website for its smaller Customers for “free”.

The other nice thing is that this works with both Web Sites and Web Services (the so called “passive” and “active” profiles). So If IssueTracker exposes an API instead of a human readable web page, the same principles (and almost the same implementation) applies.

Note: IssueTracker has an API because of requirement #2 of veryBigCorp –> being able to integrate with other systems.

This is exactly what SuperCloudySoftware implemented and if you are curious about the details, then download the code and check for yourself!

IssueTracker Standard Edition (with federation with LiveID)
IssueTracker Enterprise Edition (with federation with a custom identity provider)

I’ll expand on the implementation details in the next articles.

P.S.: if you want to learn more about claims, federation and identity “theory” in general (including the alcoholic analogy :-)) then you should bookmark and read from il Maestro Vittorio Bertocci.

Updates: fixed a few typos.

The need for “standards for application logic” in PaaS. Really?

eugeniop — Wed, 25 Feb 2009 03:37:41 GMT

In his latest post on Coghead’s demise, Phil argues that:

“What this highlights is the lack of any standard for transferring not just data but application logic between such platforms.”

My argument is that those standards already exist and are widely adopted:

“Standards for capturing application logic already exist: Java & .NET (and COBOL). Coghead "mistake" was to try to develop their own development platform from scratch, instead of leveraging what already existed and provide value on top of that.”

Phil replied that:

“Doesn't solve the problem

But you still can't *transfer* logic from one development platform to another, say from COBOL to Java, or from Java to .NET, without completely rewriting it. What I'm advocating would be helpful to people developing on established platforms too. My point is that it's essential in a PaaS context.”

My response was getting too long, so I decided to post here instead.

Sure, having an abstract model for your app logic and then deciding implementation details would be great. I buy the attractiveness of such an approach and I understand why people would like this. (I’m sure it will sound familiar to my friends @ ArTech), but there’re problems too (e.g. “minimum common denominator” syndrome, lack of finer grained control, not being able to take advantage of the latest features in a given implementation, etc).

However, I certainly don’t think it is *essential* for PaaS. Nice, desirable, yes. Essential, I don’t think so.

Phil says that “With PaaS, the lack of such mechanisms could become a huge barrier to adoption as customers become fearful of which platform might be next to switch off the lights.”

True to some extent, but there are ways of mitigating this *today* without waiting for the uber-cross-platform-cross-cloud-ocean-boiling model.

Coghead could have chosen to offer app hosting for .NET and/or Java based apps (or PHP or COBOL for that matter), and attract 10,000’s of ISVs that have already bet on those platforms. Instead, they created a *new* platform from scratch. They not only required everybody to learn their new abstractions, their new language, their new tools, etc. Those by themselves are strong adoption barriers, not impossible to overcome, but quite tough.

But they also asked everybody to bet their operational business on them (the “aaS in “PaaS”), because nobody had access to their runtime except them. The lethality to the business viability is in the combination of these two factors. Platforms are catalysts, and as a consequence, they usually don’t do anything useful by themselves. They need to be bootstrapped.

So, if Coghead had chosen say .NET (I’m biased of course :-)) as their underlying programming model, barriers of entry would have been much lower for many reasons. Among them:

ISV would have had less cost in creating a “Coghead” solution (they would have reused all their existing skills, tools, knowledge, etc).
The cost of re-targeting their app would have been lower in the case the hoster goes out of business. Some work would have been required anyway, but not as high as with the current model.

In this hypothetical scenario, instead of parsing the XML files, they would have a bunch of .NET (C# or VB.NET) assemblies.

Some PaaS offerings, such as Apprenda, have taken this path. In my opinion a much healthier and pragmatic path.

The other obvious way of addressing these risks is with a “reverse escrow” from PaaS providers to their ISVs: giving out the runtime to the ISVs if they go out of business. Worse case, ISVs would buy time to port the application into another runtime. (like .NET).

Architecting Cloud Applications for the Enterprise - Part III - SuperCloudySoftware meets VeryBigCorp CIO

eugeniop — Wed, 10 Dec 2008 20:50:00 GMT

Actors: the CIO at VeryBigCorp, John (SuperCloudySoftware's CEO) and a bunch of senior developers and architects.

CIO: Gentlemen, welcome. Let me start by saying that our internal Customer is absolutely fascinated with your application. Believe me, they are hard to please and I've never seen them so excited. Congratulations!

John (CEO): thank you! We are really committed to our customer satisfaction

CIO: great to hear that. Now... the reason I wanted to meet with you is because we want to use the system for the entire division...

John (thinking): yes! yes! yes! 10,000 seats...

CIO: ...that would be 10,000 users approximately. The thing is that the current authentication approach you guys have today, username and password, doesn't really work for us. I don't want yet another password to maintain, troubleshoot, etc. We have enough identities already as a result of our latest set of mergers and acquisitions...

John (Glancing at the other tech folks, nodding and still thinking about 10K seats): Sounds completely reasonable to me...

CIO: Besides the maintenance hassles of dealing with people forgetting passwords, provisioning new employees, etc. there is a serious liability in having you guys managing authentication this way. If any of these employees is fired, nothing prevents him to logon from his home and creatively edit the data in the system. That's too risky for us. So we want Single Sign On between our own identity infrastructure and yours...

John (carefully but confidently): right, makes total sense...

CIO: Great. The second thing I want to discuss with you is integration. Event though IssueTracker would actually replace a few systems that we host ourselves, there are still other applications that need to pull and push information to IssueTracker. In particular, our ISO9000 support system absolutely needs to be fully integrated in real time with IssueTracker. We cannot have people duplicating entries on these two different systems.

John (cautiously): mhm

CIO: By the way, the ISO9000 system runs on our AS400 minicomputer...

John (thinking): A...S...what?

John (suddenly excited): well, there are export/import features in IssueTracker. Currently you could download or upload an Excel spreadsheet, and ....

CIO (interrupting John): sure, that might help, but what we need is that every time someone creates a new Issue in IssueTracker, the same information is replicated in the AS400 almost real-time...makes sense? Some delay is admissible, but it must be as online as possible. Couple of minutes at most...

John (worried): mhm

CIO: great we understand each other! You guys are really easy to work with!

John: Sure. We are really committed to our customers satisfaction. I'd like to meet with my team and take a look at this requirements. I'm sure we can develop a proposal for you in the next week or so...

CIO (interrupting again): ...wait, there's one more thing...

John: oh! sorry... (thinking: what else could they possibly want...)

CIO: IssueTracker will become a critical system for us. If something goes wrong, and believe me, it will; we need to know. We need our help desk to be fully integrated with yours. Our users are trained to call help desks for problems with any homologue system they interact with. So when a trouble ticket is generated here, we need that to be escalated to you.

The Architect (who had remained silent until then): hey, you could use IssueTracker to track issues with IssueTracker! To iterate is human, to recurse is divine...

John (piercing the architect with his eyes): thanks for the "suggestion", now could you please expand on your needs?

CIO: yeah...anyway, we recently invested on a new set management tools (we just deployed Microsoft System Center Operations manager), so ideally, we would like all administrative tasks to be launched from there...ok? I want to be able to track response time, correlate potential problems with other conditions...

John (Recovering): ok... We need to look at this. Frankly, we don't have those things in place today, but we are very interested in making this work for you.

I'm very comfortable with the technical capabilities of my team so I'm sure we will find a solution.

I'll come back to you next week with a proposal. ok?

John and his team leaves VeryBigCorp offices. John is really wondering if this thing of expanding into the enterprise segment is really worth it. Why not just continue to sell to the SMB segment as they've been doing so far? Anyway, John wants to explore the options and make a decision on more than initial gut feeling.

At the same time, his instincts tell him that VBC requirements are not really unique. If they solve them for VBC, they would solve them for all companies with that profile and position SCS competitively.

Next chapter: SuperCloudySoftware whiteboards options to address VeryBigCorp challenges.

Architecting Cloud Applications for the Enterprise - Part II - VeryBigCorp buys IssueTracker

eugeniop — Mon, 08 Dec 2008 08:02:00 GMT

Now that we have introduced our characters, we will start this story describing the acquisition process of IssueTracker in VeryBigCorp.

IssueTracker was first used in VBC by a very limited number of people. Someone in some business unit somehow learnt about it and thought it would be a good tool to automate their work.

A senior manager in that business unit approved the expense, a subscription was bought and IssueTracker became a new software asset in VBC. IT was not involved in this process and the acquisition happened in the "black market" of IT.

And thus, IssueTracker became an "illegal immigrant" in VBC IT department: not everybody knows it is there, some people think it is great, other would like to get rid of it, most regulators (in IT) would simply ignore it, as long as it doesn't cause any trouble.

This limbo status could last for a long time. Ironically, IssueTracker is so useful that it gains notoriety. It becomes clear to the business unit managers that it this little service is actually the perfect solution for their needs. Moreover, users love it. It's simple to use, it's intuitive and it does its job efficiently. Users productivity soars. IssueTracker is the little service that could.

All this goodness eventually reaches the business unit senior manager and he decides it is something that must be used by everybody. Just imagine all the productivity gains for the entire division! Increasing the efficiency of 10,000 employees under his leadership will have a tremendous impact in his bottom line.

So the senior manager calls the CIO and IssueTracker existence is made transparent. Not only the CIO learns about this "illegal immigrant", he now has to legalize it :-).

The CIO calls SuperCloudySoftware sales department and requests a meeting with their technical folks to "sort out a few issues" of turning IssueTracker into a qualified citizen of VBC IT.

On the other side of the line, SuperCloudySoftware just can't believe what has just happened. VeryBigCorp wants to buy 10,000 seats of Issuetracker. 10,000 seats!

The best technical guys in SuperCloudySoftware are sent to the meeting then. Their mission: do whatever it takes to win this Customer! Failure is not an option.

Next chapter of this story is the meeting between the CIO and SuperCloudySoftware.

Architecting Cloud Applications for the Enterprise - Part I - Introducing the Actors

eugeniop — Fri, 05 Dec 2008 22:14:00 GMT

I will start this series by introducing the main characters of our scenario.

First, we have VeryBigCorp. VBC is a large corporation, with multiple branches and subsidiaries, thousands of employees, etc. VBC is the typical organization with a rather complex business environment: multiple business units, complex rules, regulations, etc.

VBC IT department is a reflection of this complexity: they have lots of legacy components, multiple networking stacks and a rich myriad of technologies coexist in its data centers. VBC develops custom applications for some of their business units, but they also buy packages from specialized vendors.

VBC IT has multiple processes in place to deal with all these challenges: there are architecture and development guidelines that everyone is supposed to follow, there are software development lifecycle processes, standards, naming conventions, etc. All these are there for good reasons, but sometimes creates a perception of lack of agility and excessive bureaucracy.

Most technology acquisitions in VBC are handled by the IT department following strict steps.

The second character in our story is SuperCloudySoftware, a service provider (a "cloud ISV" if you want)

SCS has embraced the web since its foundation. SCS innovates very quickly, pushes updates on its service regularly based on customer feedback, focuses on user experience, etc. They are the ultimate "agilists".

SCS focused initially on smaller businesses, even some consumers. Their flagship service is IssueTracker a task tracking service.

IssueTracker is only available as a service. That means that you can't buy a license of it and deploy it in your own data center.

From the beginning SCS made the strategic decision of making IssueTracker available through "multiple heads":

There is a Web Client that only requires a browser
There's a Smart Client that provides a richer UX and enhanced connectivity options (e.g. working offline) and
There's also a Web Services API for all functions, that allows anybody to create their own clients or want to integrate with other client environments such as Microsoft Office.

IssueTracker itself relies on cloud building blocks. For example, the persistence of the application is based on SQL Data Services. This of course is completely opaque to their customers.

Next chapter will cover IssueTracker acquisition process in VeryBigCorp.

Architecting Cloud Services for the Big Enterprise

eugeniop — Thu, 04 Dec 2008 15:08:00 GMT

For the last couple of months, we’ve been working on scenarios that involve consuming cloud services and applications from "Big Enterprise". The focus being the technical obstacles for adopting cloud services.

Big Enterprise is the moniker we use to describe an organization with rather heavy investments on on-premises software.

Not surprisingly then, the aspects we have explored are integration in these three areas: security, management and application integration.

No important application in Big Enterprise lives in isolation so integration of cloud service on these three levels is key.

The good news is that there's new exciting tools, frameworks and services ISV can leverage today to address these requirements.

My session at PDC was the first time we presented this work. In general there’s agreement that these are recurring, common and important challenges; which is good validation.

Unfortunately not all demos @ PDC worked as I had expected (and endlessly practiced before), for various reasons. In retrospective, I’m glad we went through that pain, because there were lots of lessons learnt. Failure is a great teacher.

Since PDC, we’ve been working on improving the reference implementations our demos were based on, by incorporating all this experience: obscure pitfalls, wrong assumptions, unexpected limitations, lack of complete symmetry between production and pre-production, pure randomness, etc.

I'm certain many of you our there in the real world will face many of these same challenges, so I hope our pain will help you be more successful with the Azure Platform.

Since PDC, we presented the same (updated) content on various events: Teched EMEA, TechDays in Milano a few days ago and RAF in Madrid. In general we've got great feedback and good validation of our assumptions.

My plan is to drill down on each of these aspects in the next (several) couple of blog posts, so stay tuned!

Last but not least, most of the sample code demonstrated will be available for you to download and explore in the upcoming weeks.

Northwind Hosting exists, it's better than what you saw and it's called SaaSGrid

eugeniop — Fri, 12 Sep 2008 06:32:09 GMT

Sinclair Schuller, CEO of Apprenda was kind enough to demo SaaSGrid to me earlier this week, patient enough to answer all my questions and I have to say, that I'm really impressed.

What is SaaSGrid? In their own words:

If you're building an on-demand business application, using SaaSGrid as your foundation will help you build quickly and inexpensively, ensuring that your customers can access your application's value sooner rather than later. Furthermore, you’ll then be able to deploy your SaaSGrid application to SaaSGrid itself, which provides a safe and robust virtual application container and hosting environment for your delivery needs. SaaSGrid's comprehensiveness ensures that your business and revenue approach will be able to flex and grow with changing market demands.

Sinclair walked me through the whole experience of building an app, on-boarding it, deploying it in different environments and then consuming it.

SaaSGrid has some very nice properties for ISVs. Besides the whole value proposition of hosting the app, SaaSGrid offers higher value services such as application lifecycle management, billing and metering, tenant management, etc. All of this without requiring you to learn a new programming language, paradigm and/or platform: everything is based on standard .NET so if you are familiar, skilled and trained on the standard Microsoft platform, leveraging SaaSGrid should really be very easy. Furthermore, you retain the freedom to deploy your app in other ways: on-premises, on-demand, etc.

This "non-intrusiveness" of SaaSGrid is a property of PaaS offerings we have studied in the past. I personally believe that all offerings requiring an ISV to re-write an app, or re-learn a whole new development paradigm (custom language, non-mainstream storage, etc) will be at a disadvantage compared to PaaS offerings that will make the most of your existing investments and strengths, and therefore adoption will be hurt (probability +80% :-)).

Last year we developed and wrote "Northwind Hosting concept demo" with the intent of elevating concepts we believed were important in PaaS offerings. We also wrote a couple of white papers describing this topic (e.g. ISVs are from Mars, Hosters are from Venus; Efficient Software Delivery Through Service Delivery Platforms).

Many concepts discussed there are now implemented for real in SaaSGrid, but I also saw lots of new innovative features which I found extremely useful.

Last, having an idea is great, but cheap :-). Writing about an idea takes some non-trivial effort (it is non-trivial for me at least). Turning an idea into a real offering is very hard. So kudos to Sinclair and his team for their hard work.

End to end demo of LitwareHR on SSDS

eugeniop — Sun, 27 Apr 2008 23:51:00 GMT

Here's an end-to-end demo of LitwareHR using SSDS. The total demo lasts about 11 min and you will see:

Tenant Provisioning and customization (takes the first 5 min approximately). I show the initial tenant provisioning (creation of a new tenant in LitwareHR, initial configuration, etc) and then a basic customization (e.g. look & feel, position entity shape, etc). You'll notice that I switch back and forth between www.litware.com and SSDS to show how entities look like in the store.
Using the new created instance (this takes the last 6 min). I show a hypothetical recruiter logging-in, opening new positions, then an applicant browsing the open positions and submitting a Resume and finally the recruiter browsing the posted applications. Again, you'll see how do these entities look like in SSDS.

If you want to see the video on a different window, click here.