Microsoft RSS Blog : Lists

Understanding Lists and SLE

rss — Wed, 29 Mar 2006 19:44:00 GMT

Hey! This is Arvind. I am a Software Design Engineer in Test in the RSS Team working on among other things, the implementation of the Simple List Extensions in IE7 and the Windows RSS Platform.

There have been a few posts on RSS in IE7 regarding the user experience, the RSS Platform, and the Simple List Extensions. As Sean mentioned in the previous post, there are two parts to the Simple List Extensions: the ability to treat a feed as list and the ability to define customized sorting and filtering functions.

I want to go into details about the benefits of using SLE to define a list and how it impacts the user experience and the platform. This is a fairly long post, so I’ve split it into 4 parts: the ‘why’, the ‘how’, ‘what’ and ‘when’ about lists!

I - Why do we need Lists

Feeds are used in various applications today. There are news feeds that keep users updated on the latest headlines; blogs that give an insight into experiences; media feeds of photos, audio, and video; travel feeds that provide discounted fares; and commerce feeds on product availability. By subscribing, users can continually get new information and updates to existing items across all of these feeds. However, there are cases when outdated items need to be treated differently.

Let us take a closer look by comparing two different feeds.

a) News Feeds

Every time a news site publishes new items on its feed, the RSS Platform downloads and adds these new items to the existing store of items (which posted earlier by the publisher). The user has the capability to view the recent additions as well as the items published a few days earlier since the RSS platform stores all items for this feed. News feeds and blogs, by their nature, are particularly well-suited to archiving.

b) Lowest Fares Feed from a Travel Agency

Now consider a feed from a travel agency of the latest & lowest fares available for purchase. If you view a travel website, it displays the fares that are relevant for the moment by removing fares that have been sold out or expired and introducing new promotional fares. The website’s feed also reflects the relevant fares and remove the outdated ones. When a user is subscribed to the feed, new and updated items in the feed are added to the aggregator’s local store.

The problem is that the local store reflects a mixture of new, changed, and old items that may be outdated or unavailable. There is no easy way for the user to distinguish the available fares from the unavailable ones. Clicking on an outdated fare may lead to an error that specifies that the 'Page does not exist.’ This can confuse the user and can lead to a unusable experience.

Detailed example

Let’s say that a user is subscribed to a travel agent feed on March 18th. The live feed is the feed published by the web server on the left, and on the right is the user’s subscribed feed that is consumed in a RSS aggregator.

By March 21st, there are new fares available on the live feed. Also, some of the fares have expired – like the trip to Chicago. A traditional aggregator adds the new fares but doesn’t delete the expired fares.

This means that the user now has to distinguish what is expired from the relevant items. The problem only gets worse over time. The experience after 3 months of subscribing to the feed results in more outdated items.

For these kinds of feeds, a better experience would to see the same data in the aggregator as the feed on the web server (live feed). The aggregator should also indicate which items are new since the last time the user viewed the feed. In order for a user to benefit from these kinds of feeds, there needs to be a way that a feed publisher can indicate to an aggregator to keep only the relevant information.

In the diagram below, we've gone back to the first update. On the right is a list feed that mirrors the feed on the web server. By subscribing to the list feed, outdated items are removed and new items are marked as ‘new.’

By having publisher express that their feed is really a list feed, users can gain value of subscribing to these feeds to automatically get new items and interact with only useful data.

II - How to create a List

The Simple List Extension’s “treatAs” feature was created for exactly this scenario – to allow feed publishers to indicate to an aggregator that only the current set is relevant to the user.

So, how does one mark a feed as a list? Really simple! Add the <cf:treatAs>list<cf:treatAs> tag as a child element of the top-level channel (or feed) element. SLE provides a useful feature with minimal developer work – and best of all, because its defined in a namespace, if an aggregator doesn’t understand the new element, it will ignore it and continue to handle the feed as it normally does (the results will be less-than-ideal, but at least the user will see the data).

RSS 2.0 Feed as a List

<rss version="2.0" xmlns:cf="http://www.microsoft.com/schemas/rss/core/2005">
<channel>
    <cf:treatAs>list</cf:treatAs>
    <title> Lowest Available Fares today </title>

    ..........
    ..........

</channel>
</rss>

Atom 1.0 Feed as a List

   <cf:treatAs>list</cf:treatAs>
   <title>Lowest Available Fares today</title>
   ..........
   ..........
   ..........

</feed>

III. How does the RSS Platform and IE feed view behave when viewing a list?

Let us look at how the RSS Platform and the IE7 Feed Reading View deal with list feeds.

The RSS Platform stores all of the items in a feed and on a scheduled basis, it checks the live feed (on the website) for updates. When it detects a list feed (via the presence of the treatAs element), it compares the existing items stored for the feed with the live feed items.

If an item in the local feed store is no longer in the live feed, the item in the feed store is removed.
If an item in the live feed is new, then it is added to the feed store and marked as ‘unread’ (i.e. it appears new to the user).
If an item in the live feed is already in the local feed store, but the content is different than the one stored locally, then the item in the feed store is updated.

The RSS Platform also preserves the order of the items exactly as it appears on the live feed. In some list feeds like “Top 10 Movies” the order in which the items appear bear relevance.

The user can read feeds in IE7 using the IE Feed Viewer. The reading experience for lists is virtually the same as feeds:

Items that are updated or new since the last published time are marked as ‘new’ using a blue color and bolded text.
Items that are NOT new are marked as ‘read’ using a grey color.
When a user views the list feed, all items are automatically marked as ‘viewed.’

The differences the user sees in a list feed that is different from a regular feed are:

By default, the items are sorted in order as they appear on the live feed (for ordinary feeds, the default is to sort by the date the item was published/modified). The user can reverse the sort order, or sort by any other relevant sorting.
By default, all items are shown (for ordinary feeds, the default is to show only unread items).

IV When to use lists

Here are some scenarios where list feeds are more appropriate than regular feeds. These applications almost always provide a relevant set of items and don’t want the user to see old/archived data.

1. Feeds that list the ‘Top N’

Examples:

The Top 10 Movies of the weekend
The ‘Top 10 Expensive Real Estates for the month
The 100 most downloaded songs of the week

Here are some examples of live Top N feeds that are using SLE today.

Yahoo! Music Top Songs
Yahoo! Music Top Videos
Yahoo! Music Top Albums

2. Feeds that reflect a user’s list of things (wish lists, bridal registries, todo lists, etc.).

Let’s take wishlists as an example. Imagine an item the user removed from the wish list because he already received it. If this was a regular feed and was archived, a friend who views this wish list could end up gifting the item the user already has!

This is an example of a Amazon wish list using SLE: Jeff Bezo's Wish List on Amazon.com.

You can make your own and share with your friends and family, too. Watch the video here.

3. Feeds of available products

For example, an auction site. The feed can contain sellers’ items with a particular product match.

Here's an example of an eBay search for Xbox 360.

4. Lists of open issues

The RSS Team is also using list feeds internally to improve the process of tracking active bugs in IE7 during the development process. New bugs get opened and old bugs are resolved every day, and all bugs are stored in a database. When a developer wants to know what bugs to work on, she queries the database. These queries can get slower over time as more people are doing queries.

To alleviate this issue (and to try out SLE, of course), we created a list feed that lists all the active bugs for each IE7 feature. Every day as the list of bugs changes, the feed updates itself to reflect this new list of bugs.

The traffic to the database server is substantially reduced because the database has to be contacted ONLY once to create the Bugs List as opposed to creating a connection with the database every time a user wants to find out the active list of bugs.

When developers on the team subscribe to this list feed and view it in IE7 (or any other application using the RSS Platform), they see ONLY the relevant active bugs opened against their feature. Using SLE, we have made the process of tracking active issues very simple, fast and scalable.

Thanks for reading. I’m excited to see feeds develop and take form as list feeds. If you have any feedback, please leave a comment.

- Arvind

Simple List Extensions in action

rss — Tue, 28 Mar 2006 18:25:00 GMT

Last summer, at Gnomedex 5.0, we announced the Simple List Extensions (SLE), which allow RSS and Atom feeds to be marked up with new tags that allow sorting/filtering and what we call "list semantics." There’s been a lot of feedback from the community and the specification has been updated and posted to a permanent location on MSDN.

At Mix06 last week, Dean Hachamovitch gave the keynote address on Next Generating Browsing, during which he showed off great live examples of feeds that implement the Simple List Extensions, built by Amazon, eBay, Yahoo, and MSN Spaces.

Amazon
The Amazon Web Services team did some great work in setting up RSS feeds for wishlists. Just go to amazon.com, find anyone’s wishlist and click on the fancy orange icon at the top of the page.

Here’s a live example, and a video from Mix06 showing these feeds in action. Jeff Barr posts more details over on the Amazon Web Services blog.

eBay
In addition to the feeds from eBay stores that they've had for a few months, eBay is now providing RSS feeds on search results (Arturo Zacarias posted the details on the eBay announcements list).

The search results and the existing store feeds are SLE-enhanced to allow very granular filtering on different categories and filters. This is one of the best examples to date of using the sorting and filtering to their fullest.

Pick an item you've been trying to find and just type it into the search box on ebay.com and subscribe to the feed using the button at the bottom of the page.

Yahoo! Music
Yahoo! Music has been providing RSS feeds for their Top Songs, Albums and Videos for some time. Now these feeds are enhanced with SLE to provide sorting and filtering controls, as well as to indicating that these feeds are really lists. SLE-supporting aggregators will treat these kinds of feeds in the way that the publisher intended – as a single entity. We’ll post later today about how exactly this "list semantic" works.

Here’s a direct link to the Yahoo! Music Top 10 Songs list. Ian Rogers posts more on the Yahoo! Music blog.

MSN Spaces
The folks over at MSN Spaces quietly added SLE support to all of their feeds earlier this month. Mike Torres blogged about what they've done. Here’s an example using his feed. This really shows how SLE can be used to enhance even blog feeds.

Needless to say, I’m very excited to see RSS used in these cool ways on sites as diverse as these, and I'm even more excited to see SLE being used to make these feeds even more useful to users.

On a related note, SLE has been getting some other mentions around the blogosphere. Charlie Wood predicts: “As more enterprise applications become RSS-enabled, I predict that an RSS reader that doesn't support Simple List Extensions will become as retrograde as a web browser that doesn't support tables.” I have to say that I agree completely :).

There’s no better way to find out how these Simple List Extensions work than to install IE 7 Beta 2 Preview (updated March 20 – check out the cleaner feed view!), and just click on the examples above. They all show off the sorting and filtering features of SLE amazingly well.

Thanks to everyone involved in getting these feeds online!
- Sean

Simple List Extensions - Spec update

rss — Fri, 26 Aug 2005 14:31:00 GMT

As promised, I've posted a local (draft) version of the updated Simple List Extensions 1.0 (which we're mirroring on this site while we get the MSDN site updated) as well as a companion document (SLX Explained) with samples, examples, and more detailed explanations.

The idea is that the companion document will evolve much more often as people ask questions or need clarification and will serve as a permanent record of useful questions and clarifications. Whenever we post a major change to either document, we'll post to the blog.

Thanks again!
Sean

Our welcome from (and to) the community

rss — Sat, 25 Jun 2005 19:31:00 GMT

Well, this is has been quite a fun 30 hours. Yesterday, at Gnomedex 5.0 Dean, GM of the Browsing and RSS Technologies team (and frequent IEBlog poster), announced a bunch of RSS-related features of Longhorn. Details of all that are on the MSDN site.

The other main thing that Dean talked about is our interest in helping to bring the concept of "lists" to RSS, since we see an amazing amount of user value in doing so. To help move that along, we posted a proposal on how to do this that contains the main features we think are important about lists.

(a) A way for the publisher to indicate to a client that the feed is a list. Consumers should then process the list in a slightly different way from normal feeds.

In the specification, we used a feed-level element called "cf:treatAs", with the value of "list" ("cf" represents the namespace we're using for these extensions -- it's defined in the specification).

(b) A way for the publusher to indicate to a client which of the elements on the items in the feed are interesting for the client to expose to the user for sorting and pivoting.
      In the spec, we used a set of elements at the feed level which are structured like so:

<cf:listinfo>
    <cf:sort>
        <element data-type="date|text|number">
            User-readable name for the element
         </element>
   </cf:sort>
   <cf:group>
      <element>
         User-readable name for the group.
      </element>
   </cf:group>
</cf:listinfo>

Well, we've been hearing a lot of great feedback about these lists, including Phil Ringnalda's comments which point out some concerns that he has with the way the spec re-uses element names which are technically defined in someone else's namespace.

In Phil's comments, Matthew Gertner, and Tim Bray suggest a different structure, which I'll reproduce here:

<cf:listinfo>
   <cf:sort name="element" ns="namespace" data-type="date|text|number">
      User-readable name for the element
   </cf:sort>
   <cf:group name="element" ns="namespace">
      User-readable name for the group
   </cf:group>
</cf:listinfo>

I think this is better. It still keeps the basic idea of helping the consuming app to present useful information to its clients, and its easier for people to parse.

So... what do people think? This is an easy change to make, and if the community is in favour, I will make change the spec to this format.

- Sean