While waiting for a flight back from the CSCW conference back in November, the topic of conversation turned to what people wanted to see on MSDN.  The most unique idea was to surface inbound links to a page as a way to measure the buzz of a given topic.  That spurred a few of us to see if we could build a quick and buzz control.

Key to solving this problem was determining where to get the data from.  Traditionally this type of information is processed out of IIS Log files simply by looking at the relationship between referrer and URI Stem.  However, there are many challenges with taking this route.  Log parsing systems require a dedicated infrastructure brining additional development and operational cost.  These cost only increase when you add log aggregation across multiple farms in multiple data centers.  Additionally, such systems run in some sort of batch mode, meaning that your data may or may not always be available to front end applications (granted that is only a problem for poorly designed systems).  However, the most telling weakness of a log parser based solution is simply getting your hands on the files in the first place.  In an effort to increase performance, many corporations use third party vendors to track traffic and/or turn off their logging altogether.

Since speed of development was key, as well as the fact that the log files were 404, we opted for a different data source.  Live Search had everything needed to pull it off.  In no time, we wrote a data source wrapper for the Live search services, as well as a simple control for display and user interaction.