Architecture + Strategy : Platform

Cloud Computing and the Microsoft Platform

dachou — Tue, 13 Jan 2009 14:09:14 GMT

It has been a couple of months since I wrote about cloud computing and Microsoft’s plans and strategies. Now that Azure Services Platform has been unveiled at PDC2008, and after having the opportunities to discuss it with a community of architects from major enterprises and startups via the Architect Council series of events, I can talk about cloud computing from the perspective of the Microsoft platform, and the architectural considerations that influenced its design and direction.

Okay – cloud computing today is a really overloaded term, much more than SOA (service-oriented architecture) when it was the hottest ticket in IT. There are a lot of different perspectives on cloud computing, adding to the confusion and the hype. And unsurprisingly, there are a lot of confusion around Microsoft’s cloud platform too. So here is one way of looking at it.

Microsoft’s cloud includes SaaS (Software-as-a-Service) offerings as shown in the top row of the above diagram, such as Windows Live and the Business Productivity Online Suite; and the PaaS (Platform-as-a-Service) offering currently branded as the Azure Services Platform. For the rest of this article we will focus on the Azure Services Platform, as it represents a platform on top of which additional capabilities can be developed, deployed, and managed.

Comprehensive Software + Services Platform

At Microsoft, we believe that the advent of cloud computing does not necessitate that existing (or legacy) IT assets be moved into the cloud, as it makes more sense to extend to the cloud as opposed to migrate to the cloud. We think that eventually, a hybrid world of on-premise software and cloud-based services will be the majority norm, although the balancing point between the two extremes may vary greatly among organizations of all types and sizes. As a platform company, Microsoft’s intention is to provide a platform that can support the wide range of scenarios in that hybrid world, spanning the spectrum of choices between on-premises software and cloud-based services.

Thus Microsoft’s cloud platform, from this perspective, is not intended to replace the existing on-premises software products such as our suite of Windows Server products, but rather, completes the spectrum of choices and the capabilities required for a Software + Services model.

Cloud Platform as a Next-Generation Internet-Scaled Application Environment

So what is a cloud platform? It should provide an elastic compute environment that offers auto-scalability (small to massive), and ~100% availability. However, while some think that the compute environment means a server VM (virtual machine) allocation/provisioning facility that provides servers (i.e., Windows Servers, Linux Servers, Unix Servers, etc.) for administrators to deploy applications into, Microsoft’s approach with the Azure Services Platform is remarkably different.

Azure Services Platform is intended to be a platform to support a “new class of applications” – cloud applications.

On the other hand, the Azure Services Platform is not a different location to host our existing database-driven applications such as traditional ASP.NET web apps or third-party packaged applications deployed on Windows Server. Cloud applications are a different breed of applications. Now, the long-term roadmap does include capabilities to support Windows-Server-whichever-way-we-want-it, but I think the most interesting/innovative part is allowing us to architect and build cloud applications.

To clarify, let us take a quick look at the range of options from an infrastructure perspective.

The diagram above provides a simplified/generalized view of choices we have from a hosting perspective:

On-premises: represents the traditional model of purchasing/licensing and acquiring software, install them, and manage them in our own data centers
Hosted: represents the co-location or managed outsourced hosting services. For example, GoGrid, Amazon EC2, etc.
Cloud: represents cloud fabric that provides higher-level application containers and services. For example, Google App Engine, Amazon S3/SimpleDB/SQS, etc.

From this perspective, “Hosted” represents services that provide servers-at-my-will, but we will interact with the server instances directly, and manage them at the server level so we can configure them to meet our requirements, and install/deploy applications and software just as we have done with existing on-premises software assets. These service providers manage the underlying infrastructure so we only have to worry about our servers, but not the engineering and management efforts required to achieve auto-scale and constant availability.

“Cloud” moves the concerns even higher up the stack, where application teams only need to focus on managing the applications and specifying to the environment their security and management policies, and the cloud infrastructure will take care of everything else. These service providers manage the application runtimes, so we can focus on deploying and managing business capabilities, as well as higher-level and differentiating aspects such as user experience, information architecture, social communities, branding, etc.

However, this does not mean that any one of these application deployment/hosting models is inherently better than the other. Yep, while most people look at “hosted” and “cloud” models as described here, both as cloud platforms, they are not necessarily more relevant than the on-premises model for all scenarios. These options all present varying trade-offs that we as architects need to understand, in order to make prudent choices when evaluating how to adopt or adapt to the cloud.

Trade-Offs in the Cloud

Let us take a closer look at the trade-offs between the on-premises model and the cloud (as differences between “hosted” and “cloud” models are comparatively less).

At the highest level, we are looking at trade-offs between data consistency and scalability/availability. This is a fundamental difference between on-premises and cloud-based architectures, as “traditional” on-premises system architectures are optimized to provide near-real-time data consistency (sometimes at the cost of scalability and availability), whereas cloud-based architectures are optimized to provide scalability and availability (by compromising data consistency).

One way to look at this, for example, is how we used to design and build systems using on-premises technologies. We used pessimistic locking, optimistic locking, two-phase commit, etc., methods to ensure proper handling of updates to a database via multiple threads. And this focus on ensuring the accuracy and integrity of the data was deemed one of the most important aspects in modern IT architectures. However, data consistency is achieved by compromising concurrency. For example, in DBMS design, the lowest transaction isolation level “serializable” means all transactions occur in a serial manner (in a way, single-threaded) which promises safe updates from multiple clients. But that adversely impacts performance and scalability in highly concurrent systems. Raising the isolation level helps to improve concurrency, but the database loses some control over data integrity.

Furthermore, as we look at many of the Internet-scale applications, such as Amazon S3/SimpleDB, Google BigTable, and the open source Hadoop; their designs and approaches are very different from traditional on-premises RDBMS software. Their primary goal is to provide scalable and performant databases for extremely large data sets (lots of nodes and petabytes of data), which resulted in trading off some aspects of data integrity and required users to accommodate data that is “eventually consistent”.

Amazon Web Services CTO, Werner Vogels, has recently updated his thoughts on “eventual consistency” in highly distributed and massively scaled architectures. An excellent read for more details behind the fundamental principles that contribute to this trade-off between the two models.

Thus, on-premises and cloud-based architectures are optimized for different things. And that means on-premises platform are still relevant, for specific purposes, just as cloud-based architectures. We just need to understand the trade-offs so each can be used effectively for the right reasons.

For example, an online retailer’s product catalog and storefront applications, which are published/shareable data that need absolute availability, are prime candidates to be built as cloud applications. However, once a shopping cart goes into checkout, then that process can be brought back into the on-premise architecture integrated with systems that handle order processing and fulfillment, billing, inventory control, account management, etc., which demand data accuracy and integrity.

The Microsoft Platform

I hope it’s kind of clear why Microsoft took this direction in building out the Azure Services Platform. For example, the underlying technologies used to implement Azure include Windows Server 2008, but Microsoft decided to call the compute capability Windows Azure, because it represents application containers that operate at a higher level in the stack, instead of Windows Server VM instances for us to use directly. In fact, it actually required more engineering effort this way, but the end result is a platform that provides extreme scalability and availability, the transparency of highly distributed and replicated processes and data, while hiding the complexities of the systems automation and management operations on top of a network of globally distributed data centers. This should help clarify, at a high level, as to how Azure can be used to extend existing/legacy on-premise assets, instead of being just another outsourced managed hosting location.

Of course, this is only what this initial version of the platform looks like. From a long-term perspective, Microsoft does plan to increase parity between the on-premise and cloud-based platform components, especially from a development and programming model perspective, so that the applications can be more portable across the S+S spectrum. But the fundamental differences will still exist, which will help to articulate the distinct values provided by different parts of the platform.

Thus the Azure Services Platform is intended for a “new class of applications”. Different from the traditional on-premise database-driven applications, the new class of “cloud applications” are increasingly more “services-driven”, as applications operate in a service-oriented environment, where data can be managed and provisioned as services by cloud-based database service providers such as Amazon S3/SimpleDB, Google MapReduce/BigTable, Azure SQL Services, Windows Azure Storage Services, etc., and capabilities integrated from other services running in the Web, provisioned by various private and public clouds. This type of applications inherently operate on an Internet scale, and are designed with a different set of fundamentals such as eventual consistency, idempotent processes, federated identity, services-based functional partitioning and composition (loose-coupling), isolation, parallel and replicated data and process architecture, etc.

This post is part of a series of articles on cloud computing and related concepts.

Architect Council - Mountain View 2008.11.18

dachou — Wed, 19 Nov 2008 22:19:00 GMT

Here are the content for the Architect Council event at Microsoft's Mountain View campus on Tuesday, November 18, 2008. We would like to thank everyone who made the time to attend, and sharing your feedback. We appreciate the kind comments, as well as areas we need to improve upon. If you have any further questions and/or comments, please feel free to reach out to us (via blogs listed on this site, our email addresses, or other information contained in the slide decks).

Looking forward to another set of events next quarter!

Microsoft And Cloud Computing

View SlideShare presentation or Upload your own. (tags: microsoft cloud)

Azure Services Platform

View SlideShare presentation or Upload your own. (tags: microsoft azure)

A Lap Around The Mesh Services Woodyp

View SlideShare presentation or Upload your own. (tags: services mesh)

Announcements (PPTX):

Microsoft and Cloud Computing (PPTX):

A Lap Around the Mesh Services (PPTX):

Azure Services Platform (PPTX):

And a couple of upcoming events:

XAMLFest - Dec. 16-18 – Mountain View, CA

http://www.isvconcierge.com/Lists/Articles/DispForm.aspx?ID=335

Architect Council - Dec. 18 - San Francisco, CA

http://msevents.microsoft.com/CUI/InviteOnly.aspx?EventID=5D-BC-15-14-81-B7-05-DA-9F-C0-E7-E4-23-88-67-BE&Culture=en-US

SOA & Business Process Conference 2009 - Jan. 27-30 – Redmond, WA

http://www.regonline.com/Checkin.asp?EventId=628664

MSDN Developers Conference 2009 - Feb. 19 – San Francisco, CA

http://www.msdndevcon.com/Pages/SanFrancisco.aspx

MIX 2009 - Mar. 18-20 – Las Vegas, NV

http://2009.visitmix.com/

Tech•Ed 2009 - May 11-15 – Los Angeles, CA

http://www.microsoft.com/events/teched2009/

Professional Developers Conference 2009 - Nov. 17-20 – Los Angeles, CA

http://www.microsoftpdc.com/

PDC2008 - A Futures Look at the Microsoft Platform

dachou — Fri, 11 Jul 2008 19:27:56 GMT

Microsoft's Professional Developer Conference (October 27-30, 2008) is coming to the Los Angeles Convention Center again!

2005 was the last time PDC was held, also in LA. That event unveiled the pieces that make up of what we know today as .NET Framework 3.0, as well as Windows Vista, Windows Server 2008, etc. Things such as Windows Communications Foundation (WCF; then-called "Indigo"), Windows Presentation Foundation (WPF; then-called Avalon), and Windows Workflow Foundation all represented significant advances in the .NET Framework, and how they impact application development efforts on the Microsoft platform today.

Unlike the TechEd events held annually at Orlando, Florida, PDC is focused on the leading-edge technologies and platform components and has usually been held once every two years. So Microsoft spent 3 years in hibernation this time (arguably for a number of reasons), and it certainly is a much different world today.

Today we live in an environment where cloud computing has obtained mainstream status, and developers & organizations have a larger number of platforms to consider, and new & viable vendors to partner with. However, cloud computing also brings along a set of new concerns, models, and architectures.

As a platform company, Microsoft has also been spending time to shift towards cloud computing. Our approach is to provide a platform that spans the cloud, enterprises, desktops, and devices; a full spectrum of choices that, we think, are relevant for the foreseeable future. And the Microsoft platform is designed to bring all of those previously silo'ed areas together, in a seamless and consistent manner, that fully addresses the wide range of concerns, models, and architectures in this new environment. Thus at PDC we can expect to get an inside look at how Microsoft's platform for the future has evolved, and how we can leverage existing skillsets to build applications for the future.

PDC2008 features more than 160 sessions covering a wide range of topics for professional developers and architects. These sessions provide an in-depth technical understanding of Microsoft’s future platform and offer practical guidance to help plan the evolution of your own products.

The topics include:

Cloud services - SQL Server Data Services, messaging and identity services, Live platform services, etc.
Live Mesh - Mesh services, FeedSync, device P2P, Mesh Operating Environment, etc.
Silverlight - mobile, deep dives, business apps, etc.
Cloud synchronization - Sync Framework, ADO.NET Data Services (Astoria), SQL Server project Velocity, etc.
.NET Framework - F#, C# futures, VB futures, dynamic languages, COM interop advances, WPF futures, WF futures, Workflow Services, etc.
Windows 7 - touch computing, native Web services
Windows Mobile - location-based services, Web development

And many, many more as more details emerge.

Visit the Microsoft PDC website for up-to-date information.

For registration details, see http://microsoftpdc.com/Registration/. $200 early bird discount before August 15.

Hope to see you there! Please say hi if you happen to run into me. :)

Microsoft Implementing Software Plus Services

dachou — Wed, 16 Apr 2008 01:09:00 GMT

Microsoft has been talking about "Software + Services" (S+S) as its vision of the future for a while now (see related posts on S+S: Microsoft Platform Overview & Talking about Software Plus Services). People like Bill Gates and Ray Ozzie often talk about the applicable patterns and trends that exemplify this concept, even though they don't always mention the moniker.

And Microsoft's execution on this direction is quite visible too. From continued investments on the desktop and enterprise software, to the latest and still growing cloud platform that brings many of the traditional capabilities into the Web.

For example, many of the enterprise servers - Exchange, SharePoint, Office Communications, and eventually Biztalk and SQL Server as well, are all being implemented as services in the cloud that users can use directly, without investing in their own physical infrastructures to host and manage them. There are also a lot of progress being made in the consumer space in the form of Windows Live services.

However, a major value proposition in S+S is the ability to integrate traditional software with distributed services, and bring the best of both worlds together. What has Microsoft done so far to implement that S+S vision?

Basically, many efforts are happening across the board. Some of the more visible ones include:

Exchange - it supports multiple delivery means (hosted on-premise, outsourced hosting/management by a partner, and cloud-based service from Microsoft), it supports many clients (Outlook, OWA, Outlook Mobile, Outlook Voice Access), multiple licensing models - traditional perpetual and subscription; plus itself can be a consumer of attached services such as Forefront spam/filtering services

Office System - Office clients combined with SharePoint server represents a business productivity platform (client-server interaction and leveraging the many valuable enterprise services in SharePoint such as enterprise search, content management, business data catalog, business intelligence, etc.). Excel spreadsheets can be published into SharePoint and then provisioned as web services, InfoPath forms, stored as part of SharePoint’s InfoPath services, can be rendered on InfoPath clients but can also be rendered directly from SharePoint as forms services. Office clients themselves can also be extended with .NET to connect to back-end systems whether directly or via SharePoint or Biztalk. For example, Office Live Workspaces which is a cloud-based SharePoint service for consumers, SharePoint Online for businesses, etc.

SharePoint - SharePoint Server itself can be deployed on-premise, outsourced hosting, or accessed as a subscription service from Microsoft (SharePoint Online). It also has many other flavors such as Office Live, Office Live Workspaces that live in the cloud as services for consumers to use

Windows Live - known as a set of cloud-based services, but Microsoft has also delivered a set of client-side software (Mail, Messenger, PhotoGallery, Toolbar, Writer) to improve the user experience, in addition to the browser-based interfaces. Also many of the services offer API’s for people to build applications with.

Office Communications Server - similar to Exchange, it now also has a cloud-based service for people to use (Office Communications Online), plus API's for developers to build specific branding and user experiences

Duet - a product that integrates Microsoft Office with SAP. Basically users can use the Office clients as the UI to SAP services

Xbox - Xbox Live is one of the first examples of S+S

Dynamics - similar model to Exchange - multiple deployment/delivery models, licensing models, and client access channels

Windows - Windows Update is a componentized client and cloud-based service interaction model; similar is OneCare

These examples all demonstrate the fundamental principles of S+S:

One recent offering that is particularly interesting, is Windows Live Workspaces (http://workspace.officelive.com). This service offering, in a way, is Microsoft's response to Google Apps. Instead of converting the Office client software suite (Outlook, Word, Excel, PowerPoint, Groove, OneNote, Visio, InfoPath, Access, etc.) into browser-based solutions to compete head-on with Google Apps, Windows Live Workspaces was delivered to offer the sharing and collaborating capabilities that have been cited as the biggest shortcoming when using the Office clients.

Now Microsoft actually has been delivering SharePoint services for a number of years now to provide that file sharing and collaboration scenarios for workgroups and enterprises. But there was a gap for consumers and inter-organizational scenarios that traditional SharePoint deployments (inside the firewalls) don't address very well.

Thus Windows Live Workspaces is still built on SharePoint, but has been designed specifically to support consumer and end-user collaboration. It provides capabilities for fine-grained document-level access control, ubiquitous access, cloud-based storage, and client-side add-on's that integrate directly into the Office clients. So users can create/open/save documents into Windows Live Workspace directly from Word or Excel, for example. And of course, user always have the option to save documents locally until they're ready to share with other people.

This approach illustrates the S+S approach by leveraging best of both worlds. Rich client-side software (criticized as bloatware sometimes but it can also be perceived as having the capabilities ready-to-use regardless of where a user is; having internet access or not) that fully leverages the power of the client device platform to maximize individual productivity, while leveraging cloud-based platforms for sharing and collaborating with others to maximize group productivity.

Microsoft Platform Overview

dachou — Tue, 15 Apr 2008 23:57:09 GMT

I also had the privilege of speaking at the South Bay .NET User Group, at their April monthly meeting, held at the Honda Motors U.S. headquarters campus in Torrance, CA.

The topic of this presentation was an overview of the neat and new things on the broad Microsoft platform, to help distill an understanding of how Microsoft is evolving the platform in response to major trends in IT environment today.

| View | Upload your own

We took a quick glance over many interesting platform components from Microsoft:

Software + Services (S+S): http://msdn2.microsoft.com/en-us/architecture/aa699384.aspx
S+S Blueprints: http://www.ssblueprints.net/
SOA - Oslo: http://www.microsoft.com/soa/products/oslo.aspx
BizTalk Services: http://labs.biztalk.net/
Windows Live Platform: http://dev.live.com/
Office Business Applications: https://www.obacentral.com/
Office Live Workspaces: http://workspace.officelive.com/
Silverlight: http://www.silverlight.net/
Silverlight Blueprint for SharePoint: http://msdn2.microsoft.com/en-us/sharepoint/cc303301.aspx
Silverlight Streaming: http://silverlight.live.com
Popfly: http://www.popfly.com/
Robotics Studio: http://www.microsoft.com/robotics/
XNA Game Studio: http://www.microsoft.com/xna/
Surface: http://www.microsoft.com/surface/
Photosynth: http://labs.live.com/photosynth/
HD View: http://research.microsoft.com/ivm/hdview/hdgigapixel.htm

The intention is to show that, in addition to building .NET applications on the core .NET platform (ASP.NET, Atlas/AJAX, WinForms, WPF, WCF, WF, etc.), there are many rich frameworks for building different kinds of applications, and often available at a higher abstraction level or specialized in specific scenarios. Having an awareness of these components means additional options for .NET developers to address specific problems or implement specific capabilities. The skills and knowledge on the .NET platform, such as programming in C# and familiarity with the Visual Studio development environment, can easily be extended to create solutions using these rich frameworks and platform components.

All this is being brought together under the context of Microsoft's perception of the the future of technology, influenced by major trends today including SOA, Web 2.0, Software-as-a-Service, etc. Microsoft uses the term "Software + Services" to describe this vision, where rich and targeted software components (client-side and installed on-premise) connect to and leverage distributed services (server-side and cloud-based).

The big question is, is this "Software + Services" view of the future relevant? Arguably Microsoft seems to be the only one advocating this view of the world where both client software and distributed services combine to deliver compelling user experiences, when mainstream mindshare today seems to be focusing on browser-based applications. And while it is worth noting that most of the major services players, such as SalesForce, Google, Adobe, Yahoo, Mozilla, etc., all are delivering desktop components that live outside of the browser (or at least work in off-line modes), their approach seems to be client-side software as an augmentation to cloud-based services (i.e., Google Desktop, Adobe AIR, etc.).

It is still difficult to say whether "Software + Services" will be more relevant, or browser platforms will become more dominant than they already are. As we can expect to see that the browser platform will become more sophisticated, whether via continued improvements in HTML and JavaScript or shift to RIA platforms such as Adobe Flex and Microsoft Silverlight (and Java FX, Open Laszlo, etc.); and that smart client applications will become easier to distribute and manage (like how FireFox manages its own updates). But I do think probability is higher that we can expect that not everything will be delivered through browsers.

In particular, we should expect that organizations will continue to invest in additional channels beyond the browser to reach customers. Desktop gadgets, desktop applications, plug-ins or add-ons to existing desktop application platforms (such as Office clients, Windows Live Mesenger, Vista Sidebar, SideShow, etc. on the Microsoft side), multiple device platforms (such as Windows Mobile, XBox, Zune, Media Center, Windows Embedded, etc.; again on the Microsoft side), and various services platforms (such as Windows Live, Popfly, SharePoint Online, etc.; on the Microsoft side), are all potential channels to add value to browser-based user experiences, and in many cases, very viable options to differentiate from others.

Now Microsoft may be the most vocal about the value of client-side software combined with server-side services, and building a platform that provides a spectrum of choices (which may be criticized as adding complexity as opposed to simplifying and unifying into a "good enough" approach). Similar approaches can also be identified from other leaders in the industry. Google for one is delivering more and more platform components - Google Apps, Apps Engine, Android, Desktop, GrandCentral, iGoogle, Search/Analytics/Ads, Youtube, and many more in the pipeline such as audio and video advertising, etc. From a high-level the visible trend is that Google is aggressively diversifying its platform and providing value by allowing customers to leverage the capabilities in those platforms.

Thus, we can expect to see that the technology "platform" is evolving into a much more diversified set of capabilities, and increasingly, those capabilities can be leveraged via a multitude of means beyond tradition API-based or Web services-based integration; beyond writing code.

2007.12.04 MSDN PowerSeries Event in Irvine, CA

dachou — Thu, 06 Dec 2007 11:06:09 GMT

I had the opportunity to present at the MSDN PowerSeries event in Irvine, on Software + Services, Windows Live Platform, and the Office Platform. It was a tough act to follow after Lynn Langit's morning session, but the audience was great and allowed me to talk about these topics at an architecture level and not providing implementation-level details.

For those who attended the event - thank you again for your time at the event and the feedback you've provided; both positive and otherwise. Here are the presentations:

Announcements

| View | Upload your own

Software + Services Overview

| View | Upload your own

Windows Live Platform Overview

| View | Upload your own

Office Platform Overview

| View | Upload your own

The presentation decks have also been uploaded to my Windows Live Skydrive in PowerPoint 2007 (PPTX) format. If you don't have Office 2007, you can download the free PowerPoint Viewer 2007. Here are the direct links to the presentation slide decks in PowerPoint 2007 (PPTX) format:

Announcements
Software + Services
Windows Live Platform
Office as a Platform

Share this post :

Describing Web Platform Stack

dachou — Sat, 01 Dec 2007 22:30:11 GMT

Context

In an earlier blog post I talked about "Web as a Platform" (in Web 2.0's context) and briefly described a layered and componentized perspective in looking at the Web platform in general. And I thought it would be more clarifying to illustrate what a Web platform stack might look like, so this post is intended to describe (not define) a stack view of the Web platform.

Plus, the evolution of this stack is the result of collective innovation contributed by many brilliant minds, and not driven by any single entity. Just as Eric Schmidt had said, "don't fight the Internet", I also don't think we need to model the Web after a specific prescribed framework. Rather, just allow the collective consciousness continue to innovate organically.

Thus this is a description of the Web platform (not a definition), as it is merely an attempt at categorizing the observed patterns and trends, and their relationships and dependencies, in the Web 2.0 phenomenon, into a structured context. There are many ways to describe and categorize these patterns, so this view is not necessarily exact and accurate, but hopefully it can provide some clarifications into the way Web is evolving.

Architecture of the Web

Below is a high-level rendering of the layered components architecture view of the Web platform stack:

The choice of words used is questionable, but the intention here is to highlight the trends and patterns (and their relationships) and hoping to effectively convey the concepts, without spending the time to make sure they are semantically accurate.

Just as I mentioned in my earlier blog post, layers towards the bottom of the stack are progressively closer to raw data and IT architectures, and layers towards the top are closer to people. In general, this layered stack view is used as I think lower layers serve as platforms that encapsulate the underlying complexities and provide abstraction and support to the upper layers. Even though this also kind of describes the evolutionary path (or a maturity model) of the Web in the past few years, I think this stack view is relevant as innovation is still occurring across this entire view.

Some examples to help clarify (nowhere near a comprehensive list; just intended to illustrate the categorizations):

Infrastructure:

Standards - XML, HTML, CSS, SOAP, REST, Atom, RSS, BitTorrent, HTTP, SMTP, FTP, SMS, VoIP, etc.
Tools - LAMP, WISA, JavaScript, .NET, Java, Visual Studio, Eclipse, etc.
Media - video streaming, podcasts, vcasts, electronic gaming, interactive TV, Microsoft IP TV, Microsoft Media Center
Runtimes - hosting environment, servers, desktops, browsers, clients, mobile devices, Microsoft Xbox, Sony Playstation, Nintendo Wii, Adobe AIR, Microsoft Silverlight, etc.
Networks - Internet, Wi-Fi, VPN, WAN, cellular, wireless LAN, DSL, FiOS, etc.

Foundation:

Utilities - Amazon EC2, programmableweb, etc.
Data - Amazon S3, Google Base, Microsoft Astoria, etc.
Storage - Google GDrive, Windows Live Skydrive, XDrive, DriveHQ, Box.net, Elephant Drive, etc.
Messaging - Amazon SQS, Microsoft BizTalk Services, etc.
Identity - Windows Live ID, Google Accounts, Yahoo! Accounts, OpenID, etc.

Framework:

Personalization - My Yahoo!, iGoogle, Netvibes, Windows Live, bookmarks, favorites, etc.
Transformation - Microsoft BizTalk Services (part of Don Ferguson's description of the "Internet Service Bus")
Composition - Yahoo! Pipes, Google Mashup Editor, Microsoft BizTalk Services, etc.
Orchestration - Microsoft BizTalk Services (part of Don Ferguson's description of the "Internet Service Bus")
Privacy - TBD; in general, interoperable services to give users control over what parts of their online presences to share and what not to share

Applications:

Information - Google Analytics, Google Trends, MSN, Yahoo! News, Yahoo! Finance, Upcoming, etc.
Visualization - Google Maps, Virtual Earth, Yahoo! Maps, Google Gadgets, Windows Live Gadgets, Vista Sidebar Gadgets, mobile clients, etc.
Commerce - Amazon, eBay, Paypal, Google Checkout, MSN Shopping, Microsoft Points, etc.
Monetization - Google AdSense, Google AdWords, Microsoft AdCenter, pay-per-click, cost-per-action, impressions, etc.
Accessibility - TellMe, Google Translate, Live Search Translator, services for the visually impaired like Google Accessible Search, plusmo, ZapText, etc.

Integration:

Search - Google Search, Yahoo! Search, Ask, Windows Live Search, etc.
Distribution - Facebook Platform, Microsoft Popfly, etc.
Aggregation - Newsgator, Bloglines, Rojo, NetNewsWire, My Yahoo!, Windows Live, iGoogle, PageFlakes, etc.
Syndication - Twitter, Jaiku, Pownce, Facebook Newsfeed, Feedburner, Technorati, etc.
Portability - Gadgets, Widgets, Google OpenSocial, etc.

Participation:

User Content - blogs, wikis, reviews, photo sharing, Blogger, WordPress, LiveJournals, Wikipedia, CrowdRules, Flickr, Youtube. Epinions, Urban Dictionary, Trip Advisor, eHarmony, Match, etc.
Communities - MySpace, Facebook, Orkut, hi5, Bebo, Windows Live Spaces, Friendster, LinkedIn, World of Warcraft, Xbox Live, Second Life, etc.
Folksonomies - del.icio.us, Digg, reddit, Simpy, Furl, Netvouz, etc.
Collaborative Filtering - Amazon, half.ebay.com, NetFlix, TiVO, Last.fm, StumbleUpon, etc.
Mashups - Microsoft Popfly, JackBe, etc.

Interaction (emerging):

Social Graphs - capability to enable social network analysis and moving towards mapping physical relationships
Collective Intelligence - capability to comprehend and extract meaning from composite/aggregate communities
Microformats - creation of various metadata formatting approaches to add contextual relationships
Semantic Relationships - adding meaning and contextual mapping to various forms of content available on the Web
Implicit Networks - frameworks for capturing and analyzing dynamic aspects of user activities on the Web

Interpretation (futures):

Derived Intelligence - a form of artificial intelligence derived from collective intelligence to aid predictive analysis
User Intent - discernment of user intentions based on historical user activities, responses, and collective trends and patterns
Dynamic Relationships - ability to map dynamic aspects between user activities throughout the Web

In general, each component in each layer is worthy of a separate detailed analysis, and only a minor fraction of things that are examples of a particular category have been listed. Though the intention is to show that each site or service individually is not representative of the layer component; it is the network effects created by the collection of sites and services in that category. Similarly for the layers in the platform stack, it is the aggregation of individual components that really exemplify the characteristics of that layer.

As a result, we can see that each layer in this stack view has dependencies on the services offered in the underlying layers. And each layer itself provides a level of abstraction and support to the layers above. Thus architecting solutions using the Web as a platform is quickly becoming a process of choosing a target layer (where the solution will reside), and choosing the appropriate combination of support services from the underlying layers.

This view can also be helpful in visualizing general trends of innovative development on the Web, and identify potential spots where opportunistic developments can occur, and areas where they have been turned over to systematic developments. For example, the current state (as of this writing) is that mainstream efforts can be categorized as focused in the "Participation" layer, and is where many of the opportunistic developments are being turned into systematic ones (basically, gaining maturity). The "Interaction" and "Interpretation" layers are considered to be the subject of the next Web (or Web 3.0), and is where much of the research & development efforts are focused in.

One fundamental aspect is that, the Web platform is created and maintained by the collective wisdom contributing to it. It is too big and too diverse for any one entity to own, even though many organizations have investments in multiple areas, and some more than the others. But it is interesting to see how this view of the Web platform is taking shape, based on the inter-dependencies and (almost "symbiotic") relationships established between the clusters of sites and organizations operating on the Web.

Hypothesis

The general concept here is that, Web 2.0 applications are taking on a new form. They are composite applications in nature, and increasingly can be created and hosted completely in the Web (cloud), without any dedicated on-premise infrastructure. And they are increasingly being implemented at higher levels of abstraction (moving up the stack).

For established enterprises, this marks a shift in Web application models. From approaches to open up enterprise data silos and providing value-added services to customers ("Applications" layer aspects), to migrate to a model where various higher-level components of the Web ("Integration" and "Participation" layer aspects) can be integrated and leveraged to connect to the communities. For emerging businesses, it is now possible to quickly establish an initial online presence by completely building on the cloud-based Web platform, while looking to add differentiating values with a variety of options (such as dedicated on-premise solutions).

From a user participation perspective, lower layers are progressively closer to people with higher technical expertise, but are populated by smaller communities. On the other hand, upper layers are progressively closer to larger communities as barriers to entry, from a technology perspective, are increasingly lower. This aspect demonstrates the power of network effects in enabling the participation age, and fueling the explosive pace of innovation towards creating a Web that connects/involves more people and is more relevant and intelligent.

Certainly, areas where boundaries are being pushed may still sound like science fiction, and it's fun to imagine that new breakthroughs will bring about sea changes that will overthrow all conventional wisdom. The blogosphere already has tons of speculations in that respect. Though I believe "could" does not equate to "should", such that change for change sake will not add value; only changes that lead to better outcomes will gain adoption. Thus my assessment is that, significant changes are surely imminent, but conventional wisdom will also not cease to complete irrelevance. Eventually, when the pendulum settles, we usually see a hybrid world, with some changes more dominant, and some changes less. The Web is a place where rapid changes are occurring, and as architects and strategists, using a pragmatic viewpoint when facing these changes may help us better plan the migration path between current and future states.

Share this post :

This post is part of a series:

Popfly as a Web Platform

dachou — Mon, 26 Nov 2007 14:12:43 GMT

Primer

Microsoft Popfly (www.popfly.com), currently in beta since October 2007, is a web site and tool to help people create and share web sites, mashups, and other kinds of experiences.

This service, in my opinion, is a really interesting and innovative product Microsoft has delivered this year. From an architect's perspective, Popfly can be considered as a Web platform, along with the many other interesting ones created this year, such as the Facebook Platform.

Many people also saw Popfly's potential as a Web platform. For example, Mary Jo Foley correlated it to Yahoo! Pipes, Tom Foremski described how easy it is to build a Facebook app with Popfly, John Mullinax provided a business perspective on how to leverage Popfly, and Denny Boynton with some architectural thoughts.

A Web Platform

In an earlier blog post I talked about "Web as a Platform" (in Web 2.0's context) and briefly described a layered and componentized perspective in looking at the Web platform in general. Popfly fits in that perspective very well, and can be categorized into a composition tools layer that doesn't seem to have received a lot of attention from the general Web 2.0 community. Specifically, in the programmable Web aspect of Web 2.0, the focus has been on creating the APIs, frameworks, runtime environments, standards, etc. to facilitate the various kinds of applications and social interactions. But the tasks of developing these applications still rely on traditional code-based environments. Popfly represents a major innovation on the composition tool side, and does it in an elegant way that transformed the bootstrapping requirements of various kinds of services and APIs available in the cloud, into, literally, building blocks that people without any technical background can piece together (like LEGO!) and create all kinds of composite applications (or mashups). It also offers a provisioning and syndication system so these applications can be deployed (or embedded into web pages) anywhere on the Web (and coined the term "mashout").

Popfly has been compared to Yahoo! Pipes, which provides a very elegant composition tool for aggregating and manipulating syndicated content (and a wickedly cool implementation of JavaScript in its development environment). It is a very powerful platform in terms of programmability in the context of mashing up data. Another is Google Mashup Editor, which is also a very powerful tool that helps people quickly create mashup applications. Without turning this into a comparison of the three tools, in general I think each provides a distinct value and meet different needs. For example, Yahoo! Pipes provides a graphical drag-and-drop development model in using syndicated data, and Google Mashup Editor provides a code environment particularly targeted for utilizing Google services and products; though the target audience for both of them tend to be developers.

Popfly differs in its approach to democratize development by raising the level of abstraction and narrowing down options in block configurations. This greatly simplifies the process of piecing together building blocks, and it is this simplicity that offers Popfly's greatest advantage at making development social, and potentially more appealing to a wider audience.

The public beta provides many kinds of building blocks - display, fun & games, images & video (media), local information, maps, news & RSS, shopping, social networks, tools (programming utilities such as RegExp), and others. These building blocks represent configurable components that map to many different kinds of cloud-based service APIs, such as Flickr, Facebook, Live Search, AOL Video Search, Yahoo! Videos, Virtual Earth, Yahoo! Traffic, Digg, Yahoo! News, Twitter, Technorati, etc.; the list goes on. The rich collection (and growing) of building blocks allows not just the mashup of functions and data, but also adding an interchangeable visualization and interaction layer to the applications.

Popfly boostrapped these cloud-based service APIs, and exposed their methods, input parameters, and results as configurable elements in each building block. In addition, Popfly also pre-defines and maintains compatible relationships between these APIs so in many cases, default configurations are sufficient for creating a mashup without requiring the user to perform any configuration changes. Simply drag and drop, and connect the dots will do.

Popfly itself is implemented using a combination of traditional Web application technologies (ASP.NET, AJAX, JavaScript, HTML, etc.) hosted in a highly available server infrastructure, and a Silverlight implementation of the in-browser development environment.

The challenge for Popfly is reaching critical mass in adoption. Just like the Facebook Platform, which is really a software distribution platform, harnesses its power from the lively communities in Facebook. Popfly can achieve similar goals if its adoption can be turned into a self-propelling virtuous cycle, when a healthy growth in adoption can be facilitated.

Thus Popfly really is a platform in the Web 2.0 world. It provides an environment where people without a significant technical background can build stuff in, and hides the complexities in the underlying infrastructure. It also articulates many of the Web 2.0 principles, such as enabling participation and harnessing collective intelligence, leveraging the long tail, lightweight development models, rich user experiences, etc. For businesses and organizations looking to open up their data and services, or to interact with the user communities, participating in the Popfly ecosystem could be a simple way to enable viral adoption in the distribution channel (and for some, utilizing the monetization methods).

A 1-Minute Mashup Application

To illustrate Popfly's simplistic elegance, I created a mashup between a Flickr picture set and a visualization block that uses Silverlight. A snapshot of the application in edit mode is shown below.

Without going into a detailed step-by-step replay, all I did was drag/drop the Flickr block, configure it with the Flickr set ID that contains the pictures I want to use, drag/drop the Carousel block, then drag/drop a connector from the Flickr block to the Carousel block. Hooking up the output from Flickr with input parameters in Carousel was done automatically and seamless. That's it! And the application is now ready to be deployed across the Web.

The resulting mashup application is embedded below. I picked a presentation block that uses Silverlight, but there are blocks that are pure HTML and JavaScript too.

Share this post :

This post is part of a series:

Web 2.0 - A Platform Perspective

dachou — Mon, 26 Nov 2007 07:39:28 GMT

Background & Primer

"Web as a Platform" has been a much discussed topic since Tim O'Reilly used it as a tagline in the first Web 2.0 conference back in October of 2004, then described in more detail in a 2005 article, and the subsequent "Mind Map" graphic:

Since then many interpretations of the "Web platform" have existed, ranging from technical perspectives that focused on tools such as AJAX, RSS, REST, SOAP, mashups, composite applications; user-generated content and collective intelligence such as Wikipedia, Youtube; social bookmarking/syndication such as del.icio.us, Digg; to social networks such as Facebook, Myspace, etc. Just to list a few, but the list of sites and categories of sites that exemplify Web 2.0 principles has undergone an explosive growth in the past few years.

Collectively, the rich cluster of "Web 2.0" sites on the internet form a services foundation from which applications and functionalities can be built upon, without needing any additional dedicated infrastructure. This marks a significantly different approach from "Web 1.0" site implementations where each organization has to procure dedicated hardware, software, hosting environment, etc. in order to provision a new application on the internet. As a result, the collection of cloud-based services form a new kind of "platform" to create a new breed of applications.

Understanding Web as a Platform

Without making this yet another attempt at trying to define the specifics of Web 2.0 (or even Web 3.0 for that matter) and the internet platform, delegating it to those who focus on semantics, I think we can look at "Web as a Platform" in its broadest terms. That is, a platform that provides some sort of framework which allows people to build stuff upon, while encapsulating (or hiding) some of the underlying complexities.

But this doesn't point directly to technical solutions; it really encompasses many categories of "stuff" (such as media, social interactions, implicit relationships, semantic connections, monetization methods, etc.) that can be leveraged and implemented on the Web today. I liked how Fred Wilson said it:

I believe the web is a platform. And that everything we need for an open ad market, or an open data architecture, or frankly most anything else, is available on the "web platform" today.

So what can we do with the Web platform? There are many perspectives on this as well. Such as Marc Andreesen's "layered" perspective:

Level 1 - API access - Flickr, Delicious, Twitter, etc.
Level 2 - API plug-in - Facebook
Level 3 - Runtime environment - Ning, Salesforce.com, etc.

And Alex Iskold's "building blocks" perspective:

Storage Services - Amazon S3, GDrive, Windows Live Skydrive, etc.
Messaging Services - Amazon Simple Queue Service, BizTalk Services, etc.
Compute Services - Sun Grid
Information Services - Amazon E-Commerce, Yahoo! Answers, Virtual Earth, etc.
Search Services - Google Search API, Alexa Search Platform, Live Search, etc.
Web 2.0 Services - del.icio.us, Flickr, Basecamp, etc.

Again, without questioning the validity of these categorizations used (as there are lots of discussion about that as well), I think from a general sense, both perspectives are valid. I think that building blocks do exist, but at the same time, there are multiple layers of building blocks (or categories) in the Web platform.

What this means, is that building blocks in each layer can be utilized in various combinations/permutations to create the next layer up. These layers span between two extremes - information and people. The layers closer to information consist of Web application platforms as we know today, such as ASP.NET, Silverlight, LAMP, Java, Ruby on Rails, etc.; that require more expert knowledge in development and technology but smaller parts of the overall population. The layers closer to people are still being formed as we speak, but in general they rely on higher forms of abstraction that provide services closer to our lives, while enabling the broad reach of larger pools of audiences (consumerization and democratization of technology comes to mind). And today we are seeing higher and higher layers of platforms being created that allow people to connect, to organize, to find and use resources, to be social, and to basically "live" on the Web.

Of course, the word "platform" is being used very loosely today, and new "platforms" and layers of platforms are being created almost on a daily basis. Marshall Kirkpatrick took a real brief look at some of the most hyped new platforms today. For example, the most recent and significant incarnations of higher-level Web platforms are probably Facebook Platform and Google OpenSocial.

From a platform layer perspective, the Facebook Platform and Google OpenSocial, even though aimed at doing different things (lots of debate on this too), are built on top of other existing layers. Applications built on top of the Facebook Platform use a combination of traditional Web app technologies like HTML, CSS, JavaScript, XML, etc., but their benefits are derived from building blocks available on the Facebook Platform, in the form of mashups of external services building blocks, explicit foundation blocks (such as News Feeds, Status, Events, FBML, FQL, configuration and provisioning systems, etc.), and implicit foundation blocks (social graphs, software distribution/dissemination channel, monetization, 50+ million and still growing user base, etc.). A major characteristic of this platform is that it is very easy to develop against, which democratizes development and allows more and more people to participate in the social experience. In essence this platform furtherly narrows the gap between technology and people (thus categorized as a higher-layer platform). This resulted in a wildly viral and vital platform that has accounted more than 5,000 applications deployed today and growing exponentially.

From a higher level, it seems that a "Web OS" of some sort is starting to take shape, as we can draw many parallels to the layered, subsystems and componentized approaches in modern computer operating system and software architectures. But I am not yet sure that it would be of value to try to apply traditional thinking in defining a "standard" Web platform stack, by needlessly preempting more knowledgeable people, and risk further defragmenting the evolution.

In general though, by today we can definitely see the Web maturing as a very viable platform. News such as Amazon S3 exceeds 99.99% uptime should remove most doubts about the reliability of cloud-based services. But I think it is a platform with a spectrum of choices (layers and building blocks) where people with different skillsets can look to leverage and add value. The choices available in the full spectrum are all relevant, despite some idealists' claim that newer and higher-level models (such as higher layers of the platform used in the context of this post) will completely commoditize and subsume older and lower-level models. I tend to think that, while it is true that more and more attention will be focused on newer and higher-level models, we will continue to see lots of innovation on the lower-level layered platforms. We will just see that more and more people will be involved in the overall ecosystem, with a large infusion of participants with non-technical skillsets increasingly more involved at the higher levels. This I think is the true goal of Web 2.0, connecting people and democratizing/bridging the technology chasm.

What's Next?

It's always interesting to try to take a peek at what may be possible in the future.

Democratization in software development - Recent advances in the Web platform (raising layers of abstraction), model-driven architectures, etc., will increasingly simplify software development efforts for the higher level platforms. Two very notable examples are Yahoo! Pipes and Microsoft Popfly.

The Implicit Web - Increasing specialization in making sense of the dynamic aspects of user behaviors and activities in the online world. For example, search engines to finally grasp user intent (via click streams, combinational media consumption habits, etc.). This is also an area where the Facebook Platform may be able to glean from the reactions its applications can elicit from the members, based on the static social graphs.

Privacy Controls - With so much attention on enabling the "read-write" Web, and increasing openness, a need for better privacy control will inevitably arise. Web idealists argue that traditional data silos (or intellectual property as we know today) will need to be opened up and interoperate in the new world. Again, I believe a hybrid model somewhere between the two extremes (of fully open and completely closed architectures) usually work out better to the benefit of its users. From this perspective, yes the highly protected enterprise data silos today will need to open up, but should be just enough to add value for the users. To do that, some kind of interoperable privacy controls is required.

Ubiquitous Access w/ Rich User Experiences - A consistent and seamless experience for people accessing their information, applications, and services, across a full spectrum of connected devices and systems. At the same time, highly targeted user experiences implemented for the appropriate form factors are available to take advantage of the latest hardware and device innovations.

There are many more, such as the data/semantic Web, evolutionary intelligence, changes in social trends, etc. It'll be interesting to see how things pan out in this space.

Share this post :

This post is part of a series: