Andrew Whitechapel posted a great article explaining the multiple versions of the VSTO runtime.
One anecdote that I find entertaining--the original VSTO runtime was code named White Rabbit. All the interface names were prefaced with "IWR" for White Rabbit as well.
In the Office source code, the developer that wrote the integration of the White Rabbit runtime into Office has a comment next to the line of code that releases and unloads the White Rabbit runtime that looks something like this:
// Bye bye bunny pWhiteRabbitRuntime->Release();