All postings/content on this blog are provided "AS IS" with no warranties, and confer no rights. All entries in this blog are my opinion and don't necessarily reflect the opinion of my employer.
Primer
Microsoft Popfly (www.popfly.com), currently in beta since October 2007, is a web site and tool to help people create and share web sites, mashups, and other kinds of experiences.
This service, in my opinion, is a really interesting and innovative product Microsoft has delivered this year. From an architect's perspective, Popfly can be considered as a Web platform, along with the many other interesting ones created this year, such as the Facebook Platform.
Many people also saw Popfly's potential as a Web platform. For example, Mary Jo Foley correlated it to Yahoo! Pipes, Tom Foremski described how easy it is to build a Facebook app with Popfly, John Mullinax provided a business perspective on how to leverage Popfly, and Denny Boynton with some architectural thoughts.
A Web Platform
In an earlier blog post I talked about "Web as a Platform" (in Web 2.0's context) and briefly described a layered and componentized perspective in looking at the Web platform in general. Popfly fits in that perspective very well, and can be categorized into a composition tools layer that doesn't seem to have received a lot of attention from the general Web 2.0 community. Specifically, in the programmable Web aspect of Web 2.0, the focus has been on creating the APIs, frameworks, runtime environments, standards, etc. to facilitate the various kinds of applications and social interactions. But the tasks of developing these applications still rely on traditional code-based environments. Popfly represents a major innovation on the composition tool side, and does it in an elegant way that transformed the bootstrapping requirements of various kinds of services and APIs available in the cloud, into, literally, building blocks that people without any technical background can piece together (like LEGO!) and create all kinds of composite applications (or mashups). It also offers a provisioning and syndication system so these applications can be deployed (or embedded into web pages) anywhere on the Web (and coined the term "mashout").
Popfly has been compared to Yahoo! Pipes, which provides a very elegant composition tool for aggregating and manipulating syndicated content (and a wickedly cool implementation of JavaScript in its development environment). It is a very powerful platform in terms of programmability in the context of mashing up data. Another is Google Mashup Editor, which is also a very powerful tool that helps people quickly create mashup applications. Without turning this into a comparison of the three tools, in general I think each provides a distinct value and meet different needs. For example, Yahoo! Pipes provides a graphical drag-and-drop development model in using syndicated data, and Google Mashup Editor provides a code environment particularly targeted for utilizing Google services and products; though the target audience for both of them tend to be developers.
Popfly differs in its approach to democratize development by raising the level of abstraction and narrowing down options in block configurations. This greatly simplifies the process of piecing together building blocks, and it is this simplicity that offers Popfly's greatest advantage at making development social, and potentially more appealing to a wider audience.
The public beta provides many kinds of building blocks - display, fun & games, images & video (media), local information, maps, news & RSS, shopping, social networks, tools (programming utilities such as RegExp), and others. These building blocks represent configurable components that map to many different kinds of cloud-based service APIs, such as Flickr, Facebook, Live Search, AOL Video Search, Yahoo! Videos, Virtual Earth, Yahoo! Traffic, Digg, Yahoo! News, Twitter, Technorati, etc.; the list goes on. The rich collection (and growing) of building blocks allows not just the mashup of functions and data, but also adding an interchangeable visualization and interaction layer to the applications.
Popfly boostrapped these cloud-based service APIs, and exposed their methods, input parameters, and results as configurable elements in each building block. In addition, Popfly also pre-defines and maintains compatible relationships between these APIs so in many cases, default configurations are sufficient for creating a mashup without requiring the user to perform any configuration changes. Simply drag and drop, and connect the dots will do.
Popfly itself is implemented using a combination of traditional Web application technologies (ASP.NET, AJAX, JavaScript, HTML, etc.) hosted in a highly available server infrastructure, and a Silverlight implementation of the in-browser development environment.
The challenge for Popfly is reaching critical mass in adoption. Just like the Facebook Platform, which is really a software distribution platform, harnesses its power from the lively communities in Facebook. Popfly can achieve similar goals if its adoption can be turned into a self-propelling virtuous cycle, when a healthy growth in adoption can be facilitated.
Thus Popfly really is a platform in the Web 2.0 world. It provides an environment where people without a significant technical background can build stuff in, and hides the complexities in the underlying infrastructure. It also articulates many of the Web 2.0 principles, such as enabling participation and harnessing collective intelligence, leveraging the long tail, lightweight development models, rich user experiences, etc. For businesses and organizations looking to open up their data and services, or to interact with the user communities, participating in the Popfly ecosystem could be a simple way to enable viral adoption in the distribution channel (and for some, utilizing the monetization methods).
A 1-Minute Mashup Application
To illustrate Popfly's simplistic elegance, I created a mashup between a Flickr picture set and a visualization block that uses Silverlight. A snapshot of the application in edit mode is shown below.
Without going into a detailed step-by-step replay, all I did was drag/drop the Flickr block, configure it with the Flickr set ID that contains the pictures I want to use, drag/drop the Carousel block, then drag/drop a connector from the Flickr block to the Carousel block. Hooking up the output from Flickr with input parameters in Carousel was done automatically and seamless. That's it! And the application is now ready to be deployed across the Web.
The resulting mashup application is embedded below. I picked a presentation block that uses Silverlight, but there are blocks that are pure HTML and JavaScript too.
This post is part of a series:
Background & Primer
"Web as a Platform" has been a much discussed topic since Tim O'Reilly used it as a tagline in the first Web 2.0 conference back in October of 2004, then described in more detail in a 2005 article, and the subsequent "Mind Map" graphic:
Since then many interpretations of the "Web platform" have existed, ranging from technical perspectives that focused on tools such as AJAX, RSS, REST, SOAP, mashups, composite applications; user-generated content and collective intelligence such as Wikipedia, Youtube; social bookmarking/syndication such as del.icio.us, Digg; to social networks such as Facebook, Myspace, etc. Just to list a few, but the list of sites and categories of sites that exemplify Web 2.0 principles has undergone an explosive growth in the past few years.
Collectively, the rich cluster of "Web 2.0" sites on the internet form a services foundation from which applications and functionalities can be built upon, without needing any additional dedicated infrastructure. This marks a significantly different approach from "Web 1.0" site implementations where each organization has to procure dedicated hardware, software, hosting environment, etc. in order to provision a new application on the internet. As a result, the collection of cloud-based services form a new kind of "platform" to create a new breed of applications.
Understanding Web as a Platform
Without making this yet another attempt at trying to define the specifics of Web 2.0 (or even Web 3.0 for that matter) and the internet platform, delegating it to those who focus on semantics, I think we can look at "Web as a Platform" in its broadest terms. That is, a platform that provides some sort of framework which allows people to build stuff upon, while encapsulating (or hiding) some of the underlying complexities.
But this doesn't point directly to technical solutions; it really encompasses many categories of "stuff" (such as media, social interactions, implicit relationships, semantic connections, monetization methods, etc.) that can be leveraged and implemented on the Web today. I liked how Fred Wilson said it:
I believe the web is a platform. And that everything we need for an open ad market, or an open data architecture, or frankly most anything else, is available on the "web platform" today.
So what can we do with the Web platform? There are many perspectives on this as well. Such as Marc Andreesen's "layered" perspective:
Level 1 - API access - Flickr, Delicious, Twitter, etc. Level 2 - API plug-in - Facebook Level 3 - Runtime environment - Ning, Salesforce.com, etc.
And Alex Iskold's "building blocks" perspective:
Storage Services - Amazon S3, GDrive, Windows Live Skydrive, etc. Messaging Services - Amazon Simple Queue Service, BizTalk Services, etc. Compute Services - Sun Grid Information Services - Amazon E-Commerce, Yahoo! Answers, Virtual Earth, etc. Search Services - Google Search API, Alexa Search Platform, Live Search, etc. Web 2.0 Services - del.icio.us, Flickr, Basecamp, etc.
Again, without questioning the validity of these categorizations used (as there are lots of discussion about that as well), I think from a general sense, both perspectives are valid. I think that building blocks do exist, but at the same time, there are multiple layers of building blocks (or categories) in the Web platform.
What this means, is that building blocks in each layer can be utilized in various combinations/permutations to create the next layer up. These layers span between two extremes - information and people. The layers closer to information consist of Web application platforms as we know today, such as ASP.NET, Silverlight, LAMP, Java, Ruby on Rails, etc.; that require more expert knowledge in development and technology but smaller parts of the overall population. The layers closer to people are still being formed as we speak, but in general they rely on higher forms of abstraction that provide services closer to our lives, while enabling the broad reach of larger pools of audiences (consumerization and democratization of technology comes to mind). And today we are seeing higher and higher layers of platforms being created that allow people to connect, to organize, to find and use resources, to be social, and to basically "live" on the Web.
Of course, the word "platform" is being used very loosely today, and new "platforms" and layers of platforms are being created almost on a daily basis. Marshall Kirkpatrick took a real brief look at some of the most hyped new platforms today. For example, the most recent and significant incarnations of higher-level Web platforms are probably Facebook Platform and Google OpenSocial.
From a platform layer perspective, the Facebook Platform and Google OpenSocial, even though aimed at doing different things (lots of debate on this too), are built on top of other existing layers. Applications built on top of the Facebook Platform use a combination of traditional Web app technologies like HTML, CSS, JavaScript, XML, etc., but their benefits are derived from building blocks available on the Facebook Platform, in the form of mashups of external services building blocks, explicit foundation blocks (such as News Feeds, Status, Events, FBML, FQL, configuration and provisioning systems, etc.), and implicit foundation blocks (social graphs, software distribution/dissemination channel, monetization, 50+ million and still growing user base, etc.). A major characteristic of this platform is that it is very easy to develop against, which democratizes development and allows more and more people to participate in the social experience. In essence this platform furtherly narrows the gap between technology and people (thus categorized as a higher-layer platform). This resulted in a wildly viral and vital platform that has accounted more than 5,000 applications deployed today and growing exponentially.
From a higher level, it seems that a "Web OS" of some sort is starting to take shape, as we can draw many parallels to the layered, subsystems and componentized approaches in modern computer operating system and software architectures. But I am not yet sure that it would be of value to try to apply traditional thinking in defining a "standard" Web platform stack, by needlessly preempting more knowledgeable people, and risk further defragmenting the evolution.
In general though, by today we can definitely see the Web maturing as a very viable platform. News such as Amazon S3 exceeds 99.99% uptime should remove most doubts about the reliability of cloud-based services. But I think it is a platform with a spectrum of choices (layers and building blocks) where people with different skillsets can look to leverage and add value. The choices available in the full spectrum are all relevant, despite some idealists' claim that newer and higher-level models (such as higher layers of the platform used in the context of this post) will completely commoditize and subsume older and lower-level models. I tend to think that, while it is true that more and more attention will be focused on newer and higher-level models, we will continue to see lots of innovation on the lower-level layered platforms. We will just see that more and more people will be involved in the overall ecosystem, with a large infusion of participants with non-technical skillsets increasingly more involved at the higher levels. This I think is the true goal of Web 2.0, connecting people and democratizing/bridging the technology chasm.
What's Next?
It's always interesting to try to take a peek at what may be possible in the future.
Democratization in software development - Recent advances in the Web platform (raising layers of abstraction), model-driven architectures, etc., will increasingly simplify software development efforts for the higher level platforms. Two very notable examples are Yahoo! Pipes and Microsoft Popfly.
The Implicit Web - Increasing specialization in making sense of the dynamic aspects of user behaviors and activities in the online world. For example, search engines to finally grasp user intent (via click streams, combinational media consumption habits, etc.). This is also an area where the Facebook Platform may be able to glean from the reactions its applications can elicit from the members, based on the static social graphs.
Privacy Controls - With so much attention on enabling the "read-write" Web, and increasing openness, a need for better privacy control will inevitably arise. Web idealists argue that traditional data silos (or intellectual property as we know today) will need to be opened up and interoperate in the new world. Again, I believe a hybrid model somewhere between the two extremes (of fully open and completely closed architectures) usually work out better to the benefit of its users. From this perspective, yes the highly protected enterprise data silos today will need to open up, but should be just enough to add value for the users. To do that, some kind of interoperable privacy controls is required.
Ubiquitous Access w/ Rich User Experiences - A consistent and seamless experience for people accessing their information, applications, and services, across a full spectrum of connected devices and systems. At the same time, highly targeted user experiences implemented for the appropriate form factors are available to take advantage of the latest hardware and device innovations.
There are many more, such as the data/semantic Web, evolutionary intelligence, changes in social trends, etc. It'll be interesting to see how things pan out in this space.
December 4, 2007 - Irvine, California
The emerging technology macro-trend today is the convergence of the Web and physical worlds, often referred to as the disappearing boundaries between the on-line and off-line worlds. This trend underscores a shift from using web browsers on personal computers as the only way to connect on-line, to a model of personal computing where seamless, consistent, yet focused experiences are delivered via a variety of connected devices. Join us for three ArcReady sessions designed to demystify how the Software Plus Services (S+S) vision effectively supports this view of the future and how Microsoft intends to implement this vision, how the Windows Live Platform enriches online experiences, and how Microsoft Office can serve as a composite application platform bringing together online experiences. Microsoft architects and key partners delivering these sessions will give you the knowledge you need to make informed decisions. ArcReady – reinforcing the foundation of solution architecture.
We are giving away an Xbox 360 at the event. :)
ArcReady: "Software Plus Services" (S+S) Distilled
Service-Oriented Architecture (SOA), Web 2.0, Software-as-a-Service (SaaS), and User Experience (UX) are among of the most prominent trends driving a new era in technology. Today's environment is marked by the commoditization and consumerization of technology, ubiquitous Internet connectivity, new and highly scalable monetization methods, collective edge intelligence, design-driven product development and more. In this session, you'll learn about Microsoft's unifying response to this new environment, called the "Software plus Services" (S+S) strategy.
We'll cover:
ArcReady: Windows Live Platform Overview
As the Web has grown and evolved, so have user expectations. Today's Internet users still go online to get information, but also to manage productivity, consume media and entertainment, and interact with others. Users expect great Web experiences that let them bring their personal context (relationships, interactions, memories, locations) with them wherever they go. In this session, we'll look at how services from the Windows Live Platform enable developers to give users the most compelling personal experiences on the Web.
You'll get:
ArcReady: Office as a Platform
Organizations are adopting service orientation as a design principle to drive enterprise agility and business innovation. With service-oriented architecture (SOA), they can efficiently respond to business changes and rapidly evolve their IT systems. This session will explore how 2007 Microsoft Office System clients, servers, and tools deliver a development platform that can be used to create collaborative, role-based, easy-to-use solutions and composite applications that extend the traditional business applications and enterprise systems.
To register, go to the event website.
I had the opportunity to attend the WebbyConnect Summit on October 3-5 at Laguna Beach. It was a series of non-technical panel discussions on the various growing trends on the Web, and their social and cultural impacts, especially with respect to the media and internet industries. Topics discussed include:
It was a very insightful event, as the various very notable speakers presented their thoughts on how media and social trends are impacting the evolution of the Web, and vice versa.
Some highlights:
My thoughts:
It was very refreshing to hear from the thought leaders in the social Web and media and advertising agency spaces. Particularly many campaigns that worked wildly successfully, by leveraging many social elements of Web 2.0. For example, “Trevor” as world's intern where anyone can schedule Trevor's time and ask him to do things (like sending a heavy metal birthday gram), effectively used a combination of user-generated content, viral advertising, live web feeds, instant messaging, video streaming, community establishment in Facebook and Myspace, etc., that drew an extremely popular response on the Web. The campaigns discussed were just about the most effective use of Web 2.0 platforms I've seen.
To me it boils down to brand management, and the emergence of using Web-oriented techniques to drive a new breed of brand campaigns on the Web. However, a few principles remain the same (lessons for a technologist such as myself, but obvious to people working in these areas):
On the general trend of convergence:
This was my biggest takeaway from the summit, as it was clear that the pendulum has swung back away from the extremes, in many different areas. As noted earlier, online and off-line, traditional media and online media, collective intelligence and expert editorial, etc. Basically people are no longer claiming the absolute dominance of one extreme over the other, but are seeing trends that hybrid and combinatorial models work much more effectively. In the technology world, the focus is now shifting towards bringing information and access to the users in a seamless and consistent manner, but with targeted user experiences for different platforms.
Coming back closer to home, from a technology architecture perspective, I found all these trends of convergence are very analogous to the core of Microsoft's "Software Plus Services" (S+S) strategy (and moniker). I akin S+S to Microsoft's vision of the future, in response to all the recent innovation and mindshare on SOA, Web 2.0, SaaS, etc. I will leave the details of my thoughts on S+S to another post, but just briefly here - it's an architecture of "AND", where both local software and cloud-based services work together to provide the most targeted user experiences. Vinton Cerf noted a similar thought in his keynote, that the internet is still largely a transport that is agnostic of the context and data that traverses it. We still need software to interpret and visualize the information in meaningful ways. And the fact is, the dominant players in the online space, such as Google, Yahoo, eBay, Saleforce.com, etc., are all moving towards this middle by providing client-side software. It seems their current approach can be categorized as "Services Plus Software" where cloud-based services are augmented by client-side software, compared to Microsoft's approach of "Software Plus Services" where client-side software is augmented by "Web as features". It is arguable which approach (or direction) is more relevant, but I think in general the macro-trend that is occurring is that of convergence between software and services.
This week I had the opportunity to speak at the IT Architect Regional Conference in San Diego, on the subject of architecting enterprise SOA security. It is an interesting event, with speakers from Microsoft, IBM, Oracle, TIBCO, Fair Issac, and many other organizations. We even gave away a brand new XBox 360 and a Zune!
In a nutshell, my presentation was intended to point out the security aspects of planning an enterprise SOA, and a few topics that don't seem to be covered very often, and with an emphasis towards the future and navigating the organizational and cultural issues.
A brief overview -
Basically, some of the fundamental changes in SOA, such as:
Then of course, these changes also bring along many questions. Particularly many that represent conflicting approaches and each organization may come up with different solutions based on varying trade-offs.
For example,
In my opinion, trust-based architectures are much more flexible and scalable, and implementable by today's technology standards. And we couldn't completely eliminate trust in an impersonation/delegation model anyway. For example, a connected node/system has to "trust" service wrappers, agents, and/or local system components to verify user credentials against a centralized repository (such as Active Directory, LDAP, etc.) anyway.
On the other hand, having end-to-end security contexts is indeed conceptually more secure, as it can help better address the man-in-the-middle attacks, but in an SOA with a number of intermediaries between consumers and producers, there is still not an effective solution in managing public keys to support end-to-end message-level data encryption.
Finally, some overall talking points. One important and interesting point that was kind of new to many people is that security in SOA has to be planned and designed just like another process layer. If we overlook security and not plan it carefully, we may end up creating tightly coupled elements in the overall architecture, and impacting the agility we intended to create.
The most visible example of this is trying to implement message-level encryption for the sake of data integrity (message digests) and confidentiality. In order to establish an end-to-end security context (so that intermediaries, including the ESB, should not be able to decrypt sensitive data on transit to the destination), both the intended consumer and producer have to know exactly how to encrypt and decrypt data. And that depends on a previous exchange of public keys, which in this case had to occur directly between the consumer and producer endpoints. That in a way is tight coupling, as the consumer and producer endpoints have to know about each other, and are required to establish a one-to-one, peer-to-peer relationship in terms of public keys exchange used for encryption/decryption. To alleviate the situation, a centralized public key infrastructure can be implemented in an enterprise so that the management and decisions on public key usage can be externalized from endpoints and centralized. However, enterprise solutions in this area are still evolving, and we haven't yet seen effective solutions for doing similar things beyond the enterprise and on the Web.
Lastly, the most important point is that, just like SOA governance, security is also a huge factor of the organization and corporate culture. We have to take a process-first approach to the problem (instead of technology-first), then weave in the technology delivery part of it.
For those interested, the entire slide deck I used can be downloaded from my Windows Live SkyDrive. If you don't have Office 2007, you can download the free PowerPoint Viewer 2007.