Michaeljon Miller : Web 2.0

Sitting in deployment – Microsoft Hohm goes live

mikemill — Mon, 06 Jul 2009 21:20:33 GMT

Other than a few minor DNS hiccups, Microsoft Hohm went live this morning. The team’s been sitting in a conference room on campus since about 5:30 this morning watching things spin up. We’re triaging other issues, but so far there have been no show-stopper bugs and no reason for any of the dev team to stay late tonight.

Way to go Hohm team. Considering this is the first major web application that most of the team has shipped, things went extremely well. More later.

What does SaaS mean (to me)

mikemill — Wed, 19 Sep 2007 22:03:00 GMT

I've been thinking about this software-as-a-service thing for many, many years now (prior to joining MSFT I wrote one of the first internet chat applications with my buddy Scott, and later on I worked on the leading internet-delivered skills assessment system). Clearly there are different camps and that's a good thing. It means we understand that there are different classes of needs and desires.

The recent push toward social networking (hmm…) has its roots in the individual or casual group. That is, social applications are about connecting people and groups based less on their work affiliations and more on those intangibles: likes, wants, and desire to belong. This aspect of the new internet has bled nicely into other application spaces, like the one that I work in.

My work for the last few years has been around creating great experiences in line of business applications. These applications are usually known more by their acronyms: CRM, ERP, ERM, HRM, etc. It's also interesting to note that these applications traditionally are not productivity applications such as Office, but are more about helping to automate certain business operations.

It's this concept of user though that made me start thinking hard about how social networking applications are changing the face of LOB applications. For the last year my team has been doing research into SaaS LOB applications to understand why customers would consider purchasing them instead of traditional on-premise software. There are a few easily guessable and recognizable patterns in the purchase decision such as security, cost-cutting, and availability. These were aspects that we assumed customers were looking for and we weren't overly surprised to find them.

What was surprising, at least to a number of people on the team (I'm not saying I knew this, but I suspected it) was that customers were beginning to purchase SaaS because they expected to collaborate seamlessly with their trusted partners. This was the basis for the work that my previous team was doing last summer and fall. What's interesting about it is that it goes against the core design of (nearly) every LOB application ever built.

I think of SaaS in terms of some additional intangibles: collaboration, community, connectedness, completeness, and changeability (the 5 Cs for now). For me it's not about multi-tenant architectures, or feature sets, or any of the other things that we as architects typically worry about. It's about how to create great experiences for businesses. I'll talk more about the 5 Cs over the next week or so and try to explain how this new web is having an impact on the status quo.

What do I mean by hostable LOB application?

mikemill — Fri, 17 Nov 2006 02:05:00 GMT

At the risk of upsetting a few people, I'm going to take a shot at defining what I think a hostable line-of-business application platform might look like. Those of you familiar with the CRM platform will recognize quite a few of the concepts.

First, by hostable I mean an application and platform running as a web-based client, rendered using HTML / CSS and supporting any number of modern web browsers. Note that I didn't say an application running inside of an ActiveX control or over something like Terminal Server. Yes, these are "hosted" applications, but only in a very limited sense of the word.

Line-of-business should be pretty clear. This isn't an email application, it's not an online auction site, nor is it a search engine (all of which are perfectly viable consumer-oriented applications). I'm also not talking about hosted productivity applications like word processors, spreadsheets, or presentation tools. I'm talking about those "big" applications that businesses bet their company on which automate "big" processes such as sales, financials, support, marketing, and commerce.

Clearly there are a few different classes of users of these systems. In particular there are members of the business licensing the software, there are customers of the business, and there are those folks on the periphery such as partners, vendors, and suppliers. I'm assuming that all of these "users" need some level of access to the underlying business process and data encompassed by the software. I'm also assuming that there are very different limitations on how the different users interact with the software (let's call this a role that the user plays with respect to the software). That is, the software experience is tailored to the individual using the software at a given time to perform a given task.

Let's look at an example of where software morphs itself - a sales order "process". First, a customer comes to the business' web site and browses their online catalog. They add a silver toaster to a shopping cart, specify some shipping and billing details, and place their order. Inside the software a sales order document is probably created, but the customer doesn't know this and certainly doesn't care how it's done as long as their toaster arrives in time for breakfast. Now, the system, if constructed to actually help the business, is busy fulfilling this order. To do this it breaks the order into a series of work items (let's assume for just one minute that this business builds toasters just in time) and distributes those work items to actors who can perform the work. Each of the actors (I don't really want to call them users or people because we might have a robot on the manufacturing floor that makes the darkness dial and that robot technically isn't a "person") interacts with the sales order indirectly through a role-specific view, but they're probably still using the same software system.

Anyway, I digress, as usual. You get the point that I'm talking about big software the helps the business do what the business does. Back to the discussion at hand. Each business is different and prides itself on being different. That means that they need software which is also different and tailored to how they best do their business. That is, the software is customized to their needs. I'm also assuming that the company hosting the software for the business (let's call them the provider) wants to make money at this providing business in which case they want to run as many of their customers as possible on as few pieces of hardware and software as possible. The provider wants to capitalize on the economies of scale that are possible with natively multi-tenant software. Ideally, if a provider could purchase a single machine with a single "license" and run all of their customers from it, while simultaneously supporting the customers' varied customization needs, then they would do that. That's what I mean by multi-tenant.

I'm not going to go into all the gory details about how to build such a system, but I will state the requirements here. First, the user experience must be tailored to the user's specific role with respect to the company. Next, the user experience must be standards-based (i.e. it must run in a browser and not use any of the weird extensions that many browsers have). Third, the overall business experience must include a level of customization that lets the business completely tailor the software to meet the business' specific needs. Next, the entire interaction model from purchase to provision to customization to use must also be browser-based (can't have any weird requirements to download some fancy development environment to make this work). Finally, the whole thing needs to be cost-effective from the provider's viewpoint (otherwise they can't offer it to their customers in a cost-effective fashion).

Quick update: one more way to think about the hostable line-of-business platform is that someone can come along and build a completely different application on it. It's a platform and it's application-agnostic.

Watch for part two…

Any uptake on the RSS connector for CRM?

mikemill — Sat, 20 May 2006 00:17:00 GMT

Is anyone actively using the RSS connector CRM? If so, how are you using it and did you customize it any way? If not, what are the reasons for not using it?

This is really for my information. The CRM team might watch this space and have some desire to use the information, but that's not why I'm asking. My new team is looking at some related technologies and we're / I'm wondering how this one fit.

Easing the release tax burden

mikemill — Wed, 10 May 2006 22:20:00 GMT

I've started thinking again about how to release software quickly. This isn't anything new to me, I've spent a long time with several start-ups working out the details of getting a release into customers' hands, getting their feedback (sometimes immediately), and getting the next release moving. This doesn't seem to be rocket science, there isn't a lot of hard work involved, and there shouldn't be any heavyweight process.

Why am I bringing this up? I have this sneaking suspicion that it's not really possible to release software quickly if you're releasing it under the Microsoft brand. The folks in MSN, Windows Live, and Office Live will likely take issue with that suspicion. In fact, they're out to show the world that Microsoft can release relevant software quickly. I live in a different part of the company where we feel the brunt of long-cycle upgrades. That is, our customers typically buy a product and sit on it for somewhere in the seven-year range.

Now, don't get me wrong, I'm all for buying a piece of software and realizing that investment over something longer than 12 months. And, I sincerely believe that software manufacturers can make that model work very well - they can make money, the customers get new features, and everybody is happy.

But, I've been working on a "new" software delivery model recently and it's all about speed. We are trying to define the product, build enough of it, and get it into customers' hands on very short development cycles. What hit me was the amount of non-essential work necessary, within the confines of Microsoft, to actually release that product. We have so much process around the edges that we can easily overwhelm the actual development time.

Sure, this is a tax that we pay for different reasons: legal issues, security, trademark, copyright, sustained engineering, globalization concerns, and so on. This is the price we pay (and there are more taxes, these are the ones that jumped to mind) to get a piece of software out the door. So, if we're talking about a one- to three-month development cycle, then we're looking at somewhere around six months to release whatever it was that we built (Charles calls this "turning the crank" and claims it's a fixed cost regardless of the project size). With that tax, is it even possible to get on a fast-release train?

Maybe I'm wrong, maybe this is just the view that I've had releasing six products over my Microsoft career. If I am wrong, then let me know what I can do to make the process lighter. If I'm not wrong, then let's work together to see what we can do to ease the release tax.

Announcing the RSS connector for MS-CRM 3.0

mikemill — Sat, 25 Mar 2006 00:01:00 GMT

We've finally released the RSS connector for MS-CRM. I've mentioned this tool a few times. This has been a long release mired in a few documentation, legal, and technical issues. But, that's not your concern, you probably just want to download this thing, install it, and make things happen. Well, here's the backgrounder on what's making the connector tick. The MSDN article which comes with the connector download covers some basic information. Look for a longer whitepaper from our UE team in the next few weeks.

We want to hear how you've used, modified, or extended the connector. If you use it, let me know. This is the last bit of code that I built for the CRM team (well, it's the last bit that I released, I was still working on the add-entity framework and address book right up to RTM) and I'd like to see what happens with it (and no, this isn't typical of the code quality that I usually write, this was a prototype first and a public release second; if we were going to release this is would be very different). We're releasing this under a different model (see the EULA) and we're very interested in its life once it leaves here.

Basics of the connector

The RSS connector for CRM is built on top of the advanced find and web service Fetch functionality. For the most part it directly executes the requested query and returns the results as RSS-formatted XML. However, there are a few changes that are made to the base query (if you're wondering, all queries are stored as serialized <fetch> requests, which means the connector gets to mess with XML).

First thing that happens is that the connector loads the actual <fetch> definition for the requested user or system query. Next, it creates an array of the columns specified in the query's grid. These are used for specifying the simple list extension attributes for IE7 sorting and grouping. The one change from the grid columns is that the connector adds the modifiedon attribute if it's part of the underlying entity's definition.

Once the connector has cached away the display attributes it modifies the in-memory copy of the query so that all attributes are available. The query definition has all of its selection criteria removed as well so that the feed data is as broad as possible given the caller's security attributes.

User queries vs. system queries

Under the covers system queries and user queries are structurally the same. They are stored in different tables in the database (don't ask, it was a decision that couldn't be undone by the time it was noticed), they have the same columns, and they have the same semantics. The primary difference is that the security model changes: system queries are effectively public and user queries are effectively private. A secondary difference is that they had different APIs during the TAP and alpha releases because they just happen to be written in two different languages (again, don't ask).

This does have the nice side-effect that the RSS connector simply reads the query definition through the proper entity. It's really just a <fetch> that changes entity names and some minor decorations.

When the RSS connector displays the nice HTML-based list of available feeds it does so in two sections. The top contains user-specific feeds and the bottom contains all accessible system feeds. The connector also generates HTML <link> elements for each user query. This tells RSS-aware browsers and readers that there are feeds published on the page. The connector only does this for user queries otherwise the list would be unreasonable long.

How the connector selects attributes for display

Because the connector modifies the query definition to force all attribute selection (this isn't just done for display, it's also done to support instance delivery, but more on that in a minute) it's able to present a rich view of the instance data to the RSS aggregator. In WriteItemData the connector loops over the entity's attribute list in a semi-intelligent order. It writes the primary field, any audit attributes, any state or status attributes, ownership data, any "description" attributes, and then all the rest. Nearly all attributes retrieved from the platform are displayed: there are a few attributes that have no public view and no reasonable display label, so those are skipped.

Riding on coattails - using the list extensions

The connector uses the query's grid column to select the set of attributes used in the list extensions elements. Technically, this is a de-selection, because the connector rewrites the query to remove the attribute list and adds an <all-attributes> clause to the <fetch>. In the foreach loop in WriteCrmAttributes there's a check to see if the "current" attribute is in the cell list and if it's not it's skipped. The data is written in a manner that makes list extension display useful to the user and executable to the extension processor. That is, all "codes" and other internal details are tossed away and nice display values are used instead.

The lightweight metadata cache and the service proxy

Two things that held up earlier release of the connector were technologies that I used to make the connector happen but which aren't supported outside of the CRM team. These are use of the 1.2 COM proxy (which is finally gone in the upcoming CRM release - I hated that thing because I had to code it over my wife's birthday a few years back and that got me in a lot of trouble) and the internal metadata cache assemblies.

I didn't want to freak out our development team so I had to use the same metadata interfaces that everyone else uses. The problem is that the MD web service delivers too much data too slowly for me (speaking of slow, one optimization I'd like to see in the connector is to read the queries in one batch instead of on a per-entity type basis). To get around the metadata problem I rolled a very lightweight and purpose-built cache that uses the web service to read the metadata and keep it around in a static. There are a ton of problems with this approach: there's another copy of the cache floating around and CRM is already memory-hungry, and this cache isn't aware of customization changes (i.e. Publish) so it can get out of sync. I didn't consider either of these show-stoppers for this add-on, but the PM in charge of programmability does and he's doing something about it for V.next.

One thing that I did that might be a little surprising was that I asked for the WSDL and then hand-edited down to its absolute basic bits for this solution. I didn't want a 650Kb proxy loaded into the connector and I didn't want the connector to pay the late compilation and reflection hit when W3WP loaded the proxy. The connector only uses the Fetch method and the SOAP header. That means I was able to strip all the types and method definitions out (sorry Kevin and Arash). And no, I'm not using this as an apology for the web service shape in V3, it's a good thing. I didn't have to do the same thing with the metadata proxy because it's fairly small and the connector needs a lot of the definitions from it.

If you've gotten this far make sure you read my entry on using the offline client hosting process otherwise known as Cassini. I used the connector to verify that I could make the offline client web services work.

Optional non-IE7 "list extension" behavior

When the RSS feeds are displayed to the user there are two RSS icons and a text-based hyperlink. The two icons represent the "simple" RSS feed and the RSS feed with the complete instance data. The text link will show a down-level IE representation of the IE7 RSS viewer. This bit of code is a very early prototype put together by the IE and RSS team to show what the IE7 experience might look like. I lifted the code from those teams for the PDC demo and just never got around to removing it. Someone better versed in cross-browser AJAX stuff might be able to make this work better in other browsers. For now, this link can be ignored (and I would recommend replacing the link with the "real" RSS link and let the browser figure it out).

Delivering a complete CRM instance in the <item> data

The RSS connector has the capability to deliver, as part of the item data, the XML serialized representation of a complete entity instance. It does this to enable a set of scenarios supported by really simple sharing and by some hub and spoke delivery models that we're looking at. When this option is enabled the <channel> element contains the underlying entity's XSD (this is a different XSD generation process than the WSDL uses). When a smart RSS aggregator loads a feed with the CRM namespace it knows that the entity definition and entire entity instances are available to it. This means you can tunnel select CRM instance data over RSS without exposing the CRM web services. RSS provides the pipe through which this data moves. We've come up with dozens of applications for this delivery mechanism and will start building some software based on this model over the summer. (This is the project that I've left the CRM team to work on and I couldn't be more excited about it.)

Wrapping things up

The rest of the code is just infrastructure used to make CRM data into RSS. It's missing support for HTTP 304 and ETags. I'm hoping that someone will add that and drop me an update so I can reverse-integrate it into the code. I'm assuming that the connector will fall under the "unsupported sample code" umbrella which means that there isn't a formal support infrastructure in place for it. However, if you post a comment to this entry I'll see that they get to someone in the CRM team.

Building and installing the connector is easy. I'm assuming that the MSDN document talks about this, but if it doesn't here's the short and sweet. With the connector code is a small CMD script that if executed from a VS2003 command window will compile and copy the assembly to the bin directory. My demo installation uses the ISV extensions to add a "Web feeds" item to the menu which points at the RSS feed display and a convenient OPML page. There's a 16x16 PNG file that fits nicely in the menu and just happens to match the IE7 and Firefox RSS icons.

Three faces of SaaS

mikemill — Fri, 24 Feb 2006 01:59:00 GMT

I was talking to some folks yesterday about this whole Software as a Service thing. While we were talking I started trying to define to myself what SaaS is. Now, I know there are many smarter more experienced people doing the same and that there is no right answer here. Instead I decided to break it down into three different business models that are supported by internet-based software.

Software supporting a business -- the Software as a Storefront model

Think of this as the Amazon model. Amazon is the product resale business. Their business model is about selling things and their storefront is an application on the internet. Amazon doesn't make money by providing their application any more than a dry cleaner with a web presence makes money through that presence. It's an enabling technology. (Let's ignore some of Amazon's new service offerings for the moment.)

Business selling interactive software- the Software as an Application model

This is the SalesForce model. SalesForce sells their online application and their business model is about making money selling that application. Another model comes to mind here that's like the SF model, but slightly different. SF sells their software directly to their customers. One (unnamed) company that I'm thinking of sells their software as a service to other companies who turn around and resell it. Maybe this is the service side of the Software as an Application model. In this case the company is an application provider.

Business selling service software - Software as a Service

In my opinion this is the model that makes Web 2.0 spin. It's about exposing interesting functionality so that other consumers can combine and compose new applications on top. A handful of "name brand" (without naming the brands) services jump to mind: map services, auction services, product inventory, and an ISV community around existing Software as an Application vendors.

This model doesn't imply anything about the user or developer experience. The service is the interesting bit to the provider. I can see service vendors also selling application software or other services (see one of the above models). This is the area that I'm most interested in right now.

What's interesting here is the business model. That is, if the service is effectively headless, and the end user doesn't ever see the 'service' then how does a company make money with this model? That's a question we'll all need to think about in the long run.

RSS and CRM - a little history

mikemill — Wed, 21 Sep 2005 00:55:00 GMT

Wow, just over a week since MS-CRM hit the big time and showed up on stage during a PDC keynote. That’s pretty cool. So, how did it get there? Well, the product is really cool and might have made it up there on its own merit. But then again, this is a PDC, not TechEd (and some of us are still laughing about ANY business application on stage).

It got up there because we had a lot of spirited discussion in the weeks and months leading up to PDC and the Business Summit about what would change in the world if we could democratize data. That is, could we do something that would unlock a company’s business data in a controlled way that could also make the entire supply chain work better.

What initially came out of those discussions was that CRM had a pile of cool technologies available that at least enable us to start exposing the data. Everything is exposed using web services (and yes, the V3 web services address all of the V1.x problems), it’s secure, and we’ve got a reasonably decent query definition and execution environment (CRMQuery and <fetch>).

We needed to take the next step which was to think through scenarios where access to subscription data was useful and to think about what a subscription was (in terms of business data). What we decided to do was leverage the query definition and execution services to generate a list of “interesting data”. Since queries themselves are secured objects in CRM we were able to ride on that security model to only allow access to specific queries (both access and the ability to execute a query is constrained).

Once we had that the next step was to look at what we could do to expose the underlying query results. Sure, we could have just given the caller the XML. Or we could have generated HTML. Or, just slightly better, we could have provided the XML with an XSL stylesheet. All those ideas were good, but they weren’t good enough. Instead, we converted the query results into a reasonably standard, well-supported, and well-understood protocol – RSS.

We took stock RSS, added in support for list extensions, and sprinkled in a few other goodies (watch for these in upcoming entries and in the final release) to generate a deployment-specific, partner-specific feed. Each feed honors the deployment’s configuration metadata (custom entities, entity and attribute names, display values) and the calling user’s privileges and rights.

Sorry, but I don’t really have a witty wrap-up for this. There’s a lot more to the feed generator than I’m letting on about. You’ll have to wait to see the other cool stuff we’re doing with it.

Web services aren’t distributed objects

mikemill — Wed, 01 Dec 2004 23:15:00 GMT

Why do we continue to try forcing a distributed object model into the world of web services? Nowhere in the name is there any hint that web service architecture has anything to do with distributed objects. Nowhere is there an implication that there are even objects involved in web service architecture. Sure, underneath the covers, in the private service implementation there are certain to be some kind of objects, if the service was written any time in the last decade or so.

But objects on the outside? Where did that notion ever come into play? Werner Vogel did a great piece [1] back in December 2003 that tried to dispel some of the myths that keep creeping in. I keep reading that article, and talking to developers around MBS, to see why web services keep getting lumped in with distributed object architectures. The only reason I can come up with is that the development community has had OO on the collective brain for quite a while and anything that looks like it might fit into the CORBA, DCOM, or RMI mold is dismissed as another distributed OO attempt. Web services will never take off until we start thinking in terms of services.

It’s really hard to do that though. Objects have all those nice characteristics that are pounded into our heads in school: data abstraction, information hiding, loose-coupling, separation of code from data… Wait a minute here. Aren’t those all things that web services provide too? Maybe that’s the reason we keep getting wrapped around the axle when we try to build new systems in a web services world. We want all that goodness, and the thing we’re taught to use is OO, so it must be the case that OO and web services are equivalent architectures. What can we do about it though?

Well, one of the key tenets of OO is that we should only ever deal with an object via its public interface. That means we shouldn’t worry about what’s inside the object, and we probably never should be shown what’s inside the object. That’s all good. What if we just change all occurrences of “object” in the preceding sentence with “service”? Close, but no cigar. Why? Because there’s a programming model difference between objects and services. When I create a solution using OO techniques I get to assume that I can create an object instance and hang on to it for as long as I want. I might even decide to pass it to another object instance for consumption. As the developer I probably get to hold on to precious resources, even though I don’t know that I’m doing it, because I get to control the object lifetime.

Hmm. That doesn’t sound right. What if one of those precious resources is a row in a table somewhere? (I know, it’s pretty hard to hold on to a row, but it’s not too hard to create a database transaction that holds a row.) Maybe the issue is that there are two levels of processing involved. Maybe the Smalltalk guys had the terminology right. Maybe what we send to an object is a message. That sounds right. Now, if we replace “object” with “service” we get to send messages to services. We don’t get to hold on to the “object” because it’s not there any more, it’s been replaced by a service.

Ok, so this is a lot of rambling. Is there a point? Sort of. The point is that we need to change the way we think about the programming model when we talk about building distributed systems using web services. We don’t get to control object lifetimes, we don’t get to hold on to precious resources, we don’t get to control transactions across multiple services. Instead, we get to ask a service to go do something for us and we let the infrastructure handle the how.

What’s the tie-in to CRM? Only that there are no distributed objects in the product. It’s all web services, and the “objects” are simply data structures that barely pass for documents. Why all that XML in the first two releases? Well, that was the technology available to us when we started and that technology solved the problem of getting programmers away from thinking that they needed to create objects and call methods.

[1] http://weblogs.cs.cornell.edu/AllThingsDistributed/archives/000343.html