Relevant to You? : Search

Small Update to the OPML Generator

andyed — Mon, 30 Oct 2006 06:28:00 GMT

The OPML generator revealed an interesting use case -- tracking the blogosphere's opinion on specific topics or products through comment feeds. However, this is typically not what folks want to add to their normal subscription list. I've refined control of the return of comment feeds and added the ability to include/exclude category/tag feeds.

I've also added the ability to nest exported OPML rows in a top level category. This facilitates grouping in some news readers. Enjoy your OPML.

Search Macros Recap: DiggRank and the Power of Trusted Networks

andyed — Tue, 17 Oct 2006 09:00:00 GMT

In case you couldn't tell, in microsoftie speak, I'm super excited about (advanced) search macros. I've created about 50 macros, the best of which I selected for the macro directory on this site. Almost all of these macros feature LinkFromDomain heavily.

Significant portions of my work experience since '96 have involved doing SEO, and I'm jazzed that the search engine community is appreciating the Live Search syntax for their assessment purposes. However, that's mising the really interesting and important point here.

Search Macros vs Rollyo/Sites Search
Macros are often compared to Rollyo or other site bundling search offerings, but I hope the blog post describing LinkFromDomain, LinkDomain, and featuring other operators, sets the record straight. Defining a set of sites to search is cool, and an idea well due to be commonly available. To be fair, Rollyo's UI and integration is slick, but Micah Alpern hacked up a search of his blog, the blogs he linked to, or the web at large with a Google API hack back in 2003!

Using the link domain operators, you can go well beyond a simple set of sites. You can:

keep a living list of the sites that link to you and search them
keep a living list of the sites you link to and search them
do the same for a set of trusted sites

Access to other advanced syntax differentiates further from simple site search amalgamations. Heck, Scoble pontificated about a search engine that excluded blogs that participate in pay per post. While I didn't figure out a way to focus this on only those PPP bloggers who don't disclose their interest, I think it's impressive that the basics can be done at all. It's called macro:andyed.realBloggers, and uses -inbody:counttrackula.com to exclude sites that use the PPP tracking script (I think!) and hasfeed: to restrict to blogs (or other pages with syndication).

Super Hubs: DiggRank
The promise of personal networks of trust in information retrieval is not fully realized by the macros offering, but it's an important step in the right direction. For super-hubs, like Digg or Delicious, linkFromDomain captures some really interesting human attentional residue.

Let me introduce macro:andyed.DiggRank. Try it for:

If you'd prefer to diversify from Digg, try macro:andyed.popularRank which maps from delicious, digg, stumbleupon, reddit, wink, and rojo:

This isn't just site search :)

One more example: there's a classic set of works by John Brunner that neatly fit into the cyberpunk genre, but predate the 80s rise of writers like Gibson, Sterling, and Stephenson. My macro:andyed.cyberpunk nails the simple query John Brunner and provides the perfect background for my previous sentence. While wikipedia dominates page 1, note the diversity of domains returned on page 2 and beyond. Only 7 domains were programmed into the macro, but they're used as jumping points to create a search set of 10,000 pages.

Live Search OPML Generator: Bootstrap Your Blogosphere Engagement

andyed — Thu, 12 Oct 2006 07:39:00 GMT

What does it do?: This app runs your query against the Live Search API with the feed: syntax, returning RSS feeds, and assembles an OPML file. You'll get the chance to choose from the results before generating the subscription list.

What is OPML?: An XML format commonly used for managing subscriptions to RSS feeds in readers known as aggregators. Typically, you'll find an import command in your aggregator where you can browse to a .opml.

What can I use it for?

Startup a Blog reading list: Subscribing to blogs is not as easy as it could be and can be time consuming. Bootstrap your blog subscription with this tool.
Subscribe to a community: By crafting a careful query, you can get an instant RSS synch with a whole community of bloggers.
Watch bloggers reactions: Enable comments to get specific blog post comment feeds to track what bloggers are saying about your topic of interest.

Try it out at http://surfmind.com/lab/msn/opml/

Wait until I tell you about combining this with macros! We've just posted the official blog post describing macros V2, following up on a core developer's account from a few weeks ago. There's still more to come.

There's another interesting extension to this technique -- creating a reading list from a search. Reading lists are groups of RSS feeds you can subscribe to, allowing the creator of the reading list to edit the list and you to get the updated feeds automagically. While support is limited across readers right now, it's a very cool idea. I'm not quite ready to invite a world of aggregators to ping my server regularly to get updates; perhaps someone out there is? The Live Search API is available!

Layered UI: Another Example in the Mac "Time Machine"

andyed — Tue, 08 Aug 2006 21:39:00 GMT

Nate put together a nice collage of some layered UI work we did back in the late 90s next to a recent ship from Apple:

The webgraphics UI allowed a user to see all 10 titles and hover in to see details. The time explorer uses a cleaner mapping of y-axis to time, but lacks the the visibility advantages of the wbg design.

Note: I'm pretty proud of the search stack UI, notably as it was a hidden frame AJAXy implementation for IE, fresh after the introduction of innerHTML.

News.com on MSR (re)Search. SIGIR in Seattle.

andyed — Tue, 08 Aug 2006 21:09:00 GMT

Wow, academic work covered by News.com; well, it is research by sone of MSR's most talented information retrieval folks and the search competitive space seems to be a media favorite. One of the works discussed by News.com explores analysis of user behavior given a strong statistical model of normal patterns.

SIGIR is this week in Seattle, so in addition to the staff presenting our dozen papers, we've got a stellar attendance from the search product team. I'll be on the panel at the Evaluating Exploratory Search workshop later this week where I'll be talking more about modeling user behavior for system diagnosticity.

Searching for personal homepages only at Live.com

andyed — Thu, 03 Aug 2006 16:42:24 GMT

Google Blogoscoped covers a recent Matt Cutt's video session where the question of searching only personal homepages is addressed. We also recognized this need and crafted a macro for this purpose at Live Search. Here's what it looks like:
prefer:intitle:""'s"" (homepage | intitle:""home"" | intitle:""'s page"" | ""my favorite"" | ""my interests"" | inurl:""~"")"

The prefer operator is a unique one to MSN/Live Search. It affects the ranking, but is not a requirement for a page to match. Note also the extensive use of the inurl and intitle operators. These are quite handy; at ACM CHI this year I demoed macros and was asked if I could create a search engine for canadian college history departments. Thirty seconds later I had an amazed audience with "inurl:.ca/history loc:CA prefer:(department | university | professor | research)"

Macros allow you to bundle up syntax like this and add them to your scope bar on live.com. Learn more about the Live Search advanced syntax on our help page or the blog post announcing our last round of syntax additions. Macros are only available on our beta live.com at the moment, but this will be addressed very soon.

Live Labs Funded Research Projects: Take #1

andyed — Sun, 09 Jul 2006 23:28:00 GMT

I'm excited about the work coming from External Research & Programs: Microsoft Live Labs: Accelerating Search in Academic Research 2006 RFP Awards.

One of the projects, from Suresh Bhanani and Lada Adamic at UMich, is going to take on deep clickstream analysis in a fixed content domain. In their words: "We will extract the most common health-related queries and use both human experts and natural language processing to identify key facts located on the Web pages returned by the search engine. We will then correlate the search engine ranking with the completeness of information on the page."

They offer the following sample visualization of how a search engine might distribute users through a set of linked content across related domains.

I'll visit some of the other Live Labs grant projects in the future.

The nature of browsing is changing - report from WWW'06

andyed — Wed, 21 Jun 2006 16:54:00 GMT

Some good folks at the University of Hamburg published an update to some long standing historical numbers in web usage research that reveal the way users are browsing the web has changed dramatically over the last few years. Here's the punchline:

Table 1: Comparison chart of three long-term studies

	Catledge & Pitkow³	Tauscher & Greenberg³	This Study
Time of study	1994	1995-1996	2004-2005
No. of users	107	23	25
Length (days)	21	35-42	52-195, ø=105
No. of visits	31,134	84,841	137,272
Recurrence rate	61%	58%	45.6%
Link	45.7%	43.4%	43.5%
Back	35.7%	31.7%	14.3%
Submit	-	4.4%	15.3%
New window	0.2%	0.8%	10.5%
Direct access	12.6%	13.2%	9.4%
Reload	4.3%	3.3%	1.7%
Forward	1.5%	0.8%	0.6%
Other	-	2.3%	4.8%

In particular, note the dramatic increase in new window usage (including tabs). Usage of the back button is decreased, probably directly related to new window/tabs as an alternate page view management strategy. This work also produced the sexiest image of the conference, a positional click map aggregated across all the users studied and all the pages they viewed.

The dense region at the lower right is the search engine next button. WWW has a long tradition of publishing the proceedings online. This year the papers are available in XHTML, here's the link for this work.