When I first joined Microsoft in 1996 I was lucky enough to work with Compaq to create a SQL Server scalability lab for ISVs. I learnt a lot about both SQL Server and how to run scalability exercises that year. I also learnt that in such exercises you spend a lot of time waiting for long jobs to finish and they always finish at the most awkward times - 2am seemed to be a particular favourite. In 1996 the only answer we had was to make use of a nearby hotel and spend a lot of hours sitting in the lab with relatively little to do.
Things are so much better now.
I kicked off some work yesterday in our lab with an ISV to look at how their application will scale with a very large database (we are aiming for 8TB). We spent yesterday kicking boxes, getting the EMC SAN configured, loading their application etc. By end of the day we had everything in place including ...drum roll... remote access to all the machines via the plain old Internet. This is so much more convenient. I have been able to travel today and whenever I find a wireless connection, log in to see how it is progressing.
At the moment we are just loading up the 8TB of data using a simple load generator. As of 3:59pm, all is well with no memory leak and plenty of disk IO.
Fingers X'd it stays like that :-)