Obviously I'm going to show you only one of the things that can hurt your performance. I've been doing Team System installs in a 2 server sceanrio. I've seen quiet bad performance results: 15 min. to create a new Team Project. I've pounded my head over the wall for a few hours to figure out why.
The installation process went without problems and everything was by the book. I had previously installed Team System in a 1 server scenarion on a lousy hardware and it worked faster.
Then I ran netstat whoch shows you the network connections your machine has and I noticed that I have an open connection to the proxy. Running 'netstat -b' showed me which process opened which port. I then saw that devenv.exe is going thru the proxy. Then it hit me, every hit to the Team Foundation server is basically a web service call and it is probably taking the proxy settings from IE.
Setting IE to not use a proxy dropped the time to create a new Team Project to 90 seconds.