To do this, you can't call query labels with a wildcard character, you need to do the following:
 
- Get all items blow $/Dir1 using GetItems
- Loop through them calling QueryLabels on each one.
 
Here's a code snippet:

VersionControlServer sc = tfs.GetService<VersionControlServer>();
List<VersionControlLabel> labels = new List<VersionControlLabel>(); 

foreach (Item i in sc.GetItems("$/Proj", RecursionType.Full).Items)
{
   labels.AddRange(sc.QueryLabels(null, null, null, false, i.ServerItem, null));
}