Steven Lees

FeedSync and Live Mesh

2008-04-23T09:03:27Z

Congratulations to the Live Mesh team, who announced their Live Mesh Technology Preview release earlier this evening! Amit Mital gives a detailed overview in this post on http://dev.live.com.

You can read all about it in the usual places...so why do I mention it here? FeedSync is one of the core parts of the Live Mesh platform. One of the key values of Live Mesh is that your data flows to all of your devices. And rather than being hidden away in a single service, any properly authenticated user has full bidirectional sync capability. As I discussed in the Introduction to FeedSync, this really makes "your stuff yours".

To be clear, Live Mesh is a lot more than just FeedSync. Check out Mike Zintel's post on the Live Mesh blog, and the Channel 9 interview with Abolade Gbadegesin for a lot more technical background. And don't miss Ray Ozzie's discussion with Jon Udell.

There's also more to FeedSync than Live Mesh. It's great to see Live Mesh syncing with FeedSync; things will get even better when you can connect multiple services together through FeedSync. We'll have more information and code samples coming soon.

FeedSync now covered by the Open Specification Promise

2007-12-19T23:00:13Z

I've been talking to more people about FeedSync since I posted the v1 specification earlier this month. One common question people ask is about licensing terms for the spec, and what our thoughts are about FeedSync implementations.

We've always made it very clear that our goal is to have lots of independent implementations of FeedSync, and that we want to enable royalty-free use of the specification. The spec itself is released under a Creative Commons license; today we're also making the spec available under the Microsoft Open Specification Promise. The full details are in this post on the FeedSync news blog. The short version is that the OSP makes life very easy for implementers. By placing the FeedSync spec under the OSP we're saying even more clearly that developers have royalty free use of the spec, now and in the future.

If you're new to the Open Specification Promise, they have a great page with an FAQ that explains the program in detail: http://www.microsoft.com/interop/osp/default.aspx. The Feedback from Representatives of the Community link shows the response from several people in the open source community; it's been very positive. You can also take a look at our FeedSync FAQ for some specific questions and answers about FeedSync and OSP.

It's great to have an existing program like OSP that is such a good fit for FeedSync, and I've really appreciated the support from the folks in the Microsoft Interop group in bringing the FeedSync specification under the Open Specification Promise.

FeedSync Design Principles

2007-12-08T02:32:57Z

There's been a good discussion for the past couple of days on the atom-syntax list about sync, mostly focused on tombstones.

It's useful to know that our primary design point for FeedSync, as Joe Cheng points out, is around "mesh sync". That is, if you have a collection of resources that you can represent as a feed, FeedSync is designed to reliably synchronize that resource collection across any number of endpoints in any conceivable network topology. As the spec says,

The extensions described in the FeedSync specification enable feed readers and publishers to generate and process incoming item changes in a manner that enables consistency to be achieved.

The "achieving consistency" part means that you can be sure that if you implement the spec correctly, all nodes in the mesh will converge on the same data.

Another main design point is that FeedSync is designed for synchronizing human-scale data. That is, things like contact information, calendar items, task lists, and so on. A couple of people have asked me whether you could implement something like SubEthaEdit with FeedSync. You could...but there are sure to be sync methods that are better for that kind of task. On the other hand, if the result of your SubEthaEdit-ing session is a collection of documents that you want to synchronize across multiple machines, then FeedSync is a great way to accomplish that.

One of the cool things about FeedSync is that even though it is designed for two-way, multi-endpoint mesh sync, it is still a pretty Simple set of Extensions on top of Atom and RSS. That means that FeedSync is simple enough to be useful in much simpler scenarios, such as one-way publish-subscribe, or two-way client-server scenarios.

There were some comments (from James Holderness and James Snell, for example) that FeedSync is unnecessarily complex for a task like deleting blog spam from client aggregators. That might be the case; and I'm certainly an advocate of using the simplest solution that accomplishes the task. As I said above, FeedSync's primary design point is around broader scenarios than just blogs. But if you think about your blog as a collection of resources, and you want that resource synchronized across multiple endpoints, then FeedSync is worth a look.

FeedSync

2007-12-05T08:51:00Z

Today we published the final v1 spec for Simple Sharing Extensions, under a new name, FeedSync. The new name is a little simpler than the old one (kind of ironic!) and it captures the intent pretty well.

We also kicked off a new site today, http://feedsync.org/, which explains what FeedSync is about. There is a tutorial walkthrough, and pointers to some sample code in JavaScript, VB, and C# for those who are interested.

Sam Ruby was kind enough to update the Feed Validator for the changes. Thanks Sam!

The FeedSync site has pretty much everything you need to get going, but if you're looking for more, let us know what's missing.

Happy syncing!

About a "by"

2007-12-03T03:53:00Z

I was reviewing the spec comments that we received over the past few months on the SSE spec, and because there was a fair amount of feedback about the “by” attribute on sx:history, I’d like to explain where the spec ended up.

Just to refresh your memory: every time an item or entry in a feed is changed, the endpoint making the change needs to add an sx:history element to call out the changes. The element has three attributes: a sequence attribute (the change number), a when attribute (date/time for the change), and a by attribute (the endpoint that made the change).

For purposes of sync, the by attribute has to uniquely identify the participating “endpoint”. You can think of by as the “endpoint ID.” The real world entity that the endpoint stands for could be one of several different things depending on your implementation: it could represent a user; or a user on a device; or a process for a user on a device; or something even more specific. One important consideration in picking which one of those things is an endpoint is that the by attribute determines the granularity of conflict detection. Any time an update is made by the same endpoint (the same “by”), that update subsumes all previous updates from that endpoint.

For a practical example, consider the case where I’m syncing data from my laptop to my phone. I change entry A on my laptop; then later (but before I’ve had a chance to sync) I change entry A on my phone. When I sync, one of two things happens. If the by attribute on both my laptop and my phone is identical, then one of those changes is discarded, because as far as the algorithm is concerned, they’re from the exact same endpoint. If the by attribute is different on the two devices, then both changes are preserved. At some point I’ll get to view both changes and choose which one I want to keep; no data is lost.

There were some suggestions that the by attribute should be required or recommended to be a URI. The reason that might be interesting is to somehow identify the (human) author of the change, or at least provide a pointer to the author. That’s definitely a useful thing to do. In the end, though, we felt that such a recommendation would make it harder for implementers to craft useful and unique endpoint IDs for their application.

Our recommendation is to choose the by attribute based on your granularity of conflict detection. Use a different element such as atom:author in order to indicate the source of a change.

Thanks for the feedback (past and future), it’s much appreciated!

Simple Sharing Extensions in action!

2007-07-28T02:18:00Z

Jon Udell has a recent post about the Microsoft Humanitarian Systems team and their use of SSE:

...one of the latter involved the use of Simple Sharing Extensions (SSE), an extension of RSS, to synchronize location data between Google Earth and Microsoft Virtual Earth.

I wondered what broader role SSE might play, given that it extends a Groove-like data synchronization capability to a diverse set of applications. It turns out that Ted will be testing a prototype SSE adapter for Microsoft Access on a trip to Kabul next week:

From my perspective as a relief and development person for 20 years, you can’t overestimate the value of simple tools like good old Access. What if Access could relay messages and synchronize via SSE, so that you’ve got persistent statefulness and failover on highly intermittent and jittery networks? Suddenly Access becomes a much more lively player in the edge-based mesh. So now in Afghanistan we’ll actually be using this wonderful everyman’s tool, Access, enlivened with SSE adapters, to help out an NGO partner who’s told us that would really help them share data with the other stakeholders in the reconstruction project they’re working on.

Ted and Jon are referring to the prototype SSE adapter for Access that Ed Jezierski and his team have built. We're working on adapters for to connect to other endpoints as well, such as Groove and SharePoint. As Ted points out, it will be great to enable all of these established tools to sync to any SSE endpoint.

Simple Sharing Extensions spec updates

2007-07-28T01:50:00Z

I've received some great feedback on the Simple Sharing Extensions spec that I posted on MSDN last month. I'm working on updating the spec based on all of the comments, so I'm posting the proposed spec changes to the FEED-TECH list. I'll be rolling them out in a handful of emails; here is a link to the first one, and here is a link to the full list archive.

Comments and feedback are definitely appreciated! The best way to respond is to join the list and post your comments there.

SSE implementation on CodePlex

2007-07-04T03:37:46Z

Daniel Cazzulino has posted an SSE implementation on CodePlex: the Simple Sharing Extensions for .NET project. Cool! I'm looking forward to checking it out.

When to sync?

2007-06-13T02:25:06Z

Now that we've posted the updated SSE spec to MSDN, one of the next things on my list is to discuss how SSE works. Before jumping in to examples, though, I wanted to address a more fundamental question: when does it make sense to use SSE?

In my last post, I paraphrased the overview section of the spec which describes the scope of SSE. Here's what the spec says:

The scope of Simple Sharing Extensions (SSE) is to define the minimum extensions necessary to enable loosely-cooperating applications to use XML-based container formats such as Atom and RSS as the basis for item sharing – that is, the bi-directional, asynchronous synchronization of new and changed items amongst two or more cross-subscribed feeds.

When you are thinking about practical uses for SSE, one of the most important parts of the description above is the word bi-directional. SSE is going to be most interesting scenarios where you have multiple writers to the same underlying data store.

Let's work with a real world example. The New York Times has a desktop reader for their content called Times Reader. The reader caches the last seven days of news content on your computer so that you can access it even when you're offline. Should they change their application so that they are using SSE to sync that news content between your device and their server?

Probably not. The NYT is the authoritative publisher for their news content; even if you wanted to update it, I'm guessing they wouldn't want you to. You might think, though, that it would still be useful to produce an SSE-enabled feed to help the Times Reader cache the news content on your local machine, even if it's a read-only cache. It certainly wouldn't hurt anything to produce an SSE feed in this case, but all you really need to reliably cache the content is a unique ID for each content item, and SSE requires a bit more metadata that the reader application probably wouldn't end up using.

There is an interesting use case for SSE in the Reader application, though. The Times Reader includes the crossword puzzle. The "data store" in this case is the one where your crossword puzzle solution is stored. Wouldn't it be sweet if you could start working on the puzzle in the morning from home, then pick it up again at work and add a few more answers, then work on it during the afternoon commute from your smartphone (I'll assume you're not driving...), and have your answers automagically sync to all of those devices? You can if they're all syncing the data through SSE. You could do this through a browser-based service, too, but that typically requires you to be connected to the server whenever you want to work on the crossword puzzle.

Unlike the news content publishing case, there are multiple writers in the crossword puzzle scenario...but so far they all happen to be your own devices (home computer, work computer, and phone). Where SSE is most useful is when there are multiple people as well as multiple devices writing to the same data store. Imagine that your niece on the East coast happens to really like crosswords, and wants to work on the same puzzle with you. SSE tells the reader application what to do in order to merge the two sets of puzzle answers (aka data items) together.

That's just one example; SSE is certainly useful for other kinds of information beyond crossword puzzle answers. In Ray Ozzie's original blog post introducing SSE, he envisioned using SSE for calendar and contact information. SSE is also interesting in other configurations, such as server-to-server, as well as the peer-to-peer configuration described in this post.

Windows Live Writer: fun with spellchecking

2007-06-13T02:24:00Z

I've been writing up a posting on when to use SSE, and I discovered that Windows Live Writer checks your posts for spelling as you're writing. You get those familiar red squiggly lines that you're used to from Microsoft Word.

I guess I haven't been using computers long enough to get over the amusement when the spell checker doesn't recognize some bit of jargon that I'm using. In this case, it wasn't happy that I wrote "automagically", and suggested quite a few alternatives:

Still a great feature--I'll take the occasional entertainment value as a bonus.

Spec update for Simple Sharing Extensions

2007-06-12T02:04:00Z

It's been a while in the works, but I'm happy to announce that we've update the specification for Simple Sharing Extensions. There are some pretty significant changes to this edition, including:

We added an SSE binding to the Atom feed format.
We made several changes to the sync algorithm to improve accuracy and efficiency
We reorganized the spec to improve readability
We removed the OPML binding from the v1 edition of the spec.

I've also updated the FAQ doc to go with the current spec. There are HTML/Javascript based tutorials for both Atom and RSS as well.

For those who are new to the spec, the scope of Simple Sharing Extensions (SSE) is to define the minimum extensions necessary for loosely-coupled applications to use XML-based container formats like Atom and RSS feeds as a way to synchronize a set of items across a distributed mesh of endpoints. In slightly more plain English, that means that you can have a set of applications and services that are all sharing the same underlying data in a loosely coupled way.

Silverlight Streaming

2007-05-09T04:05:00Z

One of the big announcements at MIX this year was Silverlight Streaming, which you can find at http://silverlight.live.com/. I haven't tried it yet, but it looks pretty cool. The terms of use are listed at the bottom of the page--basically, while the service is in pre-release mode you can host up to 4Gb of content for no charge.

If you want to try it out, you'll want to check out the developer page on http://dev.live.com/silverlight/. Since the service hosts Silverlight applications, you'll need to build at least a simple one of those. There are some good quickstarts to get you up and running here: http://silverlight.net/quickstarts/.