I figure it won't be long before folks are asking me how to do this. Stephen Toub put up a great article in the MSDN that shows how:
He even has a followup post on handling associations for viewers:
For another example of how cool these viewers can be, Tim Heuer put together a viewer for Foxit:
Steve, it's probably worth noting that the article covers only previewers for Vista. As I understand it, it doesn't cover creating a previewer that will work in Outlook 2007 running on Windows XP. I hear that might be in the works.
It took a while but they finally started asking :)