Relevant to You? : Research

Innovation in Feedback Collection with Windows Vista

andyed — Sat, 30 Sep 2006 23:15:00 GMT

I've written about the challenges of capitalizing on at-scale user feedback both academically and on the search blog (busted post alas, edit when fixed). One way to listen to users is to open up your bug tracking system to the world, as done by Mozilla.org. While this results in a great deal of information about user needs and pain points, it results in hundreds (if not thousands) of duplicates scattered around the bug database and associated component hierarchy.

We do a lot of structured feedback data collection at Microsoft, in search and across the organization, within Microsoft and in our customer's environments. As I've worked in this space for search, some good folks over in the operating system team have taught me a few tricks. Their approach is now available to the world of Vista users at the Windows Vista Scenario Voting site.

Collecting feedback in this way has numerous advantages over plain textual feedback or structured bug reports:

A user doesn't have to be a programmer to put their feedback into the right place, maximizing it's chance of impact
Communicating key scenarios focuses users on goals the software producer is committed to supporting
Feedback is encouraged to target key parts of the process, speaking the user language and using recognition not recall. That is, rather than requiring the user to remember the arbitrary "Step 4 of the network setup wizard", a plain language set of steps is provided

That last point may need a bit of elaboration. Here's what the scenarios page lists for View a summary of the latest content from RSS feeds.

Visit http://channel9.msdn.com/
View the RSS feed available from the page
Subscribe to it.
After an RSS feed has been subscribed to, view a summary of the latest content using the mini Explorer (bar on the side of IE, found by clicking the Favorites icon.)

Feedback can then easily refer to specific steps that are causing the user's problems. I expect gathering user reactions to and wishes for software products to be a key strategy for product success in the coming years. Already, some hypothesize that the ease of development and feedback loops are a key determinant of the web's success.

News.com on MSR (re)Search. SIGIR in Seattle.

andyed — Tue, 08 Aug 2006 21:09:00 GMT

Wow, academic work covered by News.com; well, it is research by sone of MSR's most talented information retrieval folks and the search competitive space seems to be a media favorite. One of the works discussed by News.com explores analysis of user behavior given a strong statistical model of normal patterns.

SIGIR is this week in Seattle, so in addition to the staff presenting our dozen papers, we've got a stellar attendance from the search product team. I'll be on the panel at the Evaluating Exploratory Search workshop later this week where I'll be talking more about modeling user behavior for system diagnosticity.

The nature of browsing is changing - report from WWW'06

andyed — Wed, 21 Jun 2006 16:54:00 GMT

Some good folks at the University of Hamburg published an update to some long standing historical numbers in web usage research that reveal the way users are browsing the web has changed dramatically over the last few years. Here's the punchline:

Table 1: Comparison chart of three long-term studies

	Catledge & Pitkow³	Tauscher & Greenberg³	This Study
Time of study	1994	1995-1996	2004-2005
No. of users	107	23	25
Length (days)	21	35-42	52-195, ø=105
No. of visits	31,134	84,841	137,272
Recurrence rate	61%	58%	45.6%
Link	45.7%	43.4%	43.5%
Back	35.7%	31.7%	14.3%
Submit	-	4.4%	15.3%
New window	0.2%	0.8%	10.5%
Direct access	12.6%	13.2%	9.4%
Reload	4.3%	3.3%	1.7%
Forward	1.5%	0.8%	0.6%
Other	-	2.3%	4.8%

In particular, note the dramatic increase in new window usage (including tabs). Usage of the back button is decreased, probably directly related to new window/tabs as an alternate page view management strategy. This work also produced the sexiest image of the conference, a positional click map aggregated across all the users studied and all the pages they viewed.

The dense region at the lower right is the search engine next button. WWW has a long tradition of publishing the proceedings online. This year the papers are available in XHTML, here's the link for this work.