I've seen a few questions come up about this.  Two questions, actually:

Question: How do I delete a Team Project Collection on the Team Foundation Service?

Answer: You can't.  For now we automatically create one collection per account (called DefaultCollection) and you can't delete it, rename it, add another one or anything else.

Question: How do I delete a Team Project on the Team Foundation Service?

Answer: You can but it's ugly.  This is one of those unfinished scenarios that helps explain why this is a "preview" and not a finished service.  Buck wrote a post on how to do this.

 

Brian