Steven Lees : FeedSync

FeedSync Design Principles

StevenLees — Sat, 08 Dec 2007 02:32:57 GMT

There's been a good discussion for the past couple of days on the atom-syntax list about sync, mostly focused on tombstones.

It's useful to know that our primary design point for FeedSync, as Joe Cheng points out, is around "mesh sync". That is, if you have a collection of resources that you can represent as a feed, FeedSync is designed to reliably synchronize that resource collection across any number of endpoints in any conceivable network topology. As the spec says,

The extensions described in the FeedSync specification enable feed readers and publishers to generate and process incoming item changes in a manner that enables consistency to be achieved.

The "achieving consistency" part means that you can be sure that if you implement the spec correctly, all nodes in the mesh will converge on the same data.

Another main design point is that FeedSync is designed for synchronizing human-scale data. That is, things like contact information, calendar items, task lists, and so on. A couple of people have asked me whether you could implement something like SubEthaEdit with FeedSync. You could...but there are sure to be sync methods that are better for that kind of task. On the other hand, if the result of your SubEthaEdit-ing session is a collection of documents that you want to synchronize across multiple machines, then FeedSync is a great way to accomplish that.

One of the cool things about FeedSync is that even though it is designed for two-way, multi-endpoint mesh sync, it is still a pretty Simple set of Extensions on top of Atom and RSS. That means that FeedSync is simple enough to be useful in much simpler scenarios, such as one-way publish-subscribe, or two-way client-server scenarios.

There were some comments (from James Holderness and James Snell, for example) that FeedSync is unnecessarily complex for a task like deleting blog spam from client aggregators. That might be the case; and I'm certainly an advocate of using the simplest solution that accomplishes the task. As I said above, FeedSync's primary design point is around broader scenarios than just blogs. But if you think about your blog as a collection of resources, and you want that resource synchronized across multiple endpoints, then FeedSync is worth a look.

FeedSync

StevenLees — Wed, 05 Dec 2007 08:51:00 GMT

Today we published the final v1 spec for Simple Sharing Extensions, under a new name, FeedSync. The new name is a little simpler than the old one (kind of ironic!) and it captures the intent pretty well.

We also kicked off a new site today, http://feedsync.org/, which explains what FeedSync is about. There is a tutorial walkthrough, and pointers to some sample code in JavaScript, VB, and C# for those who are interested.

Sam Ruby was kind enough to update the Feed Validator for the changes. Thanks Sam!

The FeedSync site has pretty much everything you need to get going, but if you're looking for more, let us know what's missing.

Happy syncing!

About a "by"

StevenLees — Mon, 03 Dec 2007 03:53:00 GMT

I was reviewing the spec comments that we received over the past few months on the SSE spec, and because there was a fair amount of feedback about the “by” attribute on sx:history, I’d like to explain where the spec ended up.

Just to refresh your memory: every time an item or entry in a feed is changed, the endpoint making the change needs to add an sx:history element to call out the changes. The element has three attributes: a sequence attribute (the change number), a when attribute (date/time for the change), and a by attribute (the endpoint that made the change).

For purposes of sync, the by attribute has to uniquely identify the participating “endpoint”. You can think of by as the “endpoint ID.” The real world entity that the endpoint stands for could be one of several different things depending on your implementation: it could represent a user; or a user on a device; or a process for a user on a device; or something even more specific. One important consideration in picking which one of those things is an endpoint is that the by attribute determines the granularity of conflict detection. Any time an update is made by the same endpoint (the same “by”), that update subsumes all previous updates from that endpoint.

For a practical example, consider the case where I’m syncing data from my laptop to my phone. I change entry A on my laptop; then later (but before I’ve had a chance to sync) I change entry A on my phone. When I sync, one of two things happens. If the by attribute on both my laptop and my phone is identical, then one of those changes is discarded, because as far as the algorithm is concerned, they’re from the exact same endpoint. If the by attribute is different on the two devices, then both changes are preserved. At some point I’ll get to view both changes and choose which one I want to keep; no data is lost.

There were some suggestions that the by attribute should be required or recommended to be a URI. The reason that might be interesting is to somehow identify the (human) author of the change, or at least provide a pointer to the author. That’s definitely a useful thing to do. In the end, though, we felt that such a recommendation would make it harder for implementers to craft useful and unique endpoint IDs for their application.

Our recommendation is to choose the by attribute based on your granularity of conflict detection. Use a different element such as atom:author in order to indicate the source of a change.

Thanks for the feedback (past and future), it’s much appreciated!