Linked data, an exploding meme

Over the last week I went from not knowing about a new technological movement to seeing it everywhere and becoming a believer. Here's the story.

I'm something of a data wonk. On, my desk you'll find a copy of the economist book of facts and figures. I read it for fun. I often download the latest spreadsheets from OMB on the latest economic figures. I'm also the guy that spends time digging through the Customer Experience Improvement Program data looking for gems on how our customers use our products. And yes, I mine Twitter too.

But what's always frustrated me about this data is that it's hard to analyze and link together. The OMB data comes in a pretty nice spreadsheet but I always have to tweak it to get what I want. I can't link it to other data sets without significant work. This seriously limits the insights I might be able to achieve. I have to be pretty confident about what I'm going to find before I look for anything beyond the obvious. I've got so many data streams available and I just want to have the freedom to dump into an OLAP cube and play. Dealing with the limitations is an eternal source of pain for me. I live this stuff so I was primed.

And then there was the past week.

First I hear about Wolfram Alpha. It looks like a search engine but it calculates answers from data gleaned from the web rather than matching strings. Cool, I thought. But nothing started to flash in my head. Then I heard about a project from Amazon where they are publishing government data in an accessible format based on AWS. Also cool. I made a mental note to check it out the next time I had a hankering to go to OMB. But it still didn't click.

But then I was whiling away the time with my Zune at the kids swimming lessons and I thought I'd listen to a TED talk from Tim Berners-Lee. It's an idea of his for something called linked data. He's advocating for everyone to provide raw data over http for tools and services to build on. This talk is when everything came together. When you see three news items in the same area in a short period of time, you know something is up.

In some ways these projects seem unrelated, but that makes the meme even more powerful. Different people are coming to the same problem from different angles. Wolfram Alpha needs to data to work on. Amazon may be just looking for something to show off their platform with, but then again that's all Microsoft had in mind when it initially built Terraserver. Now Google Maps and Virtual Earth are big deals. Let's just I'm stoked.