Here are a few things about branching source code in VSTS via the IDE that I've learned the hard way:

First, don't try to branch the top most level node of the Team Project.  Branch subfolders.  At least in the beta, the root itself of the Team Project cannot be branched, this is the equivalent of telling TFS to give itself a nasty ingrown hair and results in what I perceive to be a fairly user surly error message.  I'm working to get you a cleaner user experience in RTM bits, but beta is what it is.

To fully graft in your branch, there are two steps.  (1) Create the branch, and then (2) check it in.  (1) To create the branch, pick your favorite subfolder in the version control explorer, right click, and then in the context menu there will be a simple option to create a branch.  After the thrill of seeing all the progress status indicators go away you might think you're done.  You're not.  It needs to be checked in.  Until you check it in, others on the team cannot see the branch and basic functionality like creating a team build type against it using the wizard will not be possible. (2) To check it in, go to the Pending Changes window by going to View->Pending Checkins and then select Check In.  Now you've done it.