One of the great things about TFS is that you can place the "crown jewels" of your development activities in a single safe location. The consequence for me is that I'm somewhat paranoid about using a production TFS installation for TFS R&D activities. For example, should I install a process template provided by a third party on my production server simply to have a look at it? IMO, absolutely not!
It's not to say you can't use your production TFS installation for some R&D work. Where I get sensitive is where this work requires changes that can't be confined to a single team project. At my current site, planning for replacing a CruiseControl/NANT build solution with Team Build is being undertaken using a team project on the production TFS installation. It's nicely contained, and TFS is being used as intended... to support development. However, having looked at one or two commercial process template offerings which require me to run MSIs on the TFS servers, there is no way I would put these anywhere near production (Why do software houses still get MSIs wrong?).
Where does that leave us? In short, you will probably need somewhere to try new TFS "things" independently of production. Virtual machines are the obvious solution, however if you plan migrate a lot of source from VSS into TFS, I would recommend that you get yourself some physical hardware. This allows you to iron out migration issues in a semi-realistic fashion before you involve the production installation (see some of my previous posts on the problems of repeating a migration over and over on the same TFS installation). Regardless of the choice, you will need to consider how you're going the license the development TFS installation (development is production for TFS after all). For low usage (which should normally be the case), the Workgroup Edition of TFS may fit the bill. I'm actually a fan of both approaches. I use VMs to "play" and a physical development installation to implement the next round of TFS changes. If Humpty-Dumpty has a big fall, at least he won't take out all my (or my customer's) source and developer productivity with him.