Unfortunately, there's not a fast, efficient way to see the list of labels in the system with the full comment without also seeing a list of all of the files included in a label.  You also can't efficiently answer the question, "What labels involve foo.cs?"  While this won't be changed for v1, you can certainly do it using code.  I mentioned on the TFS forum that I'd try to put together a piece of code to do this.  The result is the code below.

The code is really simple to do this, but I ended up adding more to it than I originally intended.  All that's really necessary here is to call QueryLabels() to get the information we need.

Let's look at the QueryLabels() call in a little detail, since it is the heart of the app.  Here is the method declaration from the VersionControlServer class.

public VersionControlLabel[] QueryLabels(String labelName,
                                         
String labelScope,
                                         String owner, 
                                         bool includeItems,
                                         String filterItem,
                                         VersionSpec versionFilterItem)

By convention, methods that begin with "Query" in the source control API allow you to pass null to mean "give me everything."  In the code below, I don't want to filter by labelName or owner, so I set those to null to include everything.

If the user specified a server path for the scope (scope is always a server path and not a local path), we'll use it, and otherwise we'll use the root ($/).  The scope of a label is, effectively, the part of the tree where it has ownership of that label name.  In other words by specifying the label scope, $/A and $/B can have separate labels named Foo, but no new label Foo can be used under $/A or $/B.  For this program, setting the scope simply narrows the part of the tree it will include in the output.  For example, running this with a scope of $/A would show only one label called Foo, but running it with $/ as the scope (or omitting the scope) would result in two Foo labels being printed.

D:\ws1>tree
Folder PATH listing for volume Dev
Volume serial number is 0006EE50 185C:793F
D:.
├───A
└───B

D:\ws1>tf label Foo@$/testproj/A A
Created label
Foo@$/testproj/A

D:\ws1>tf label Foo@$/testproj/B B
Created label
Foo@$/testproj/B

D:\ws1>d:\LabelHistory\LabelHistory\bin\Debug\LabelHistory.exe
Foo (10/25/2005 4:00 PM)
Foo (10/25/2005 4:00 PM)

The most important parameter here is actually includeItems.  By setting this parameter to false, we'll get the label metadata without getting the list of files and folders that are in the label.  This saves both a ton of bandwidth as well as load on the server for any query involving real-world labels that include many thousands of files.

The remaining parameters are filterItem and versionFilterItem.  The filterItem parameter allows you to specify a server or local path whereby the query results will only include labels involving that file or folder.  It allows you to answer the question, "What labels have been applied to file foo.cs?"  The versionFilterItem is used to specify what version of the item had the specified path.  It's an unfortunate complexity that's due to the fact that we support rename (e.g., A was called Z at changeset 12, F at changeset 45, and A at changeset 100 and beyond).  Before your eyes glaze over (they haven't already, right?), I just set that parameter to latest.

Here's an example of using the program with the tree mentioned earlier.  I modified the Foo label on A to have a comment, so it has a later modification time.

D:\ws1>tf label Foo@$/testproj/A /comment:"This is the first label I created."
Updated label Foo@$/testproj/A

D:\ws1>d:\LabelHistory\LabelHistory\bin\Debug\LabelHistory.exe
Foo (10/25/2005 4:05 PM)
   Comment: This is the first label I created.
Foo (10/25/2005 4:00 PM)

Then I added a file under A, called a.txt, and modified the label to include it.  Running the app on A\a.txt, we see that it is only involved in one of the two labels in the system.

D:\ws1>tf label Foo@$/testproj/A A\a.txt
Updated label
Foo@$/testproj/A

D:\ws1>d:\LabelHistory\LabelHistory\bin\Debug\LabelHistory.exe A\a.txt
Foo (10/25/2005 4:56 PM)
   Comment: This is the first label I created.

To build it, you can create a Windows console app in Visual Studio, drop this code into it, and add the following references to the VS project.

Microsoft.TeamFoundation.Client
Microsoft.TeamFoundation.Common
Microsoft.TeamFoundation.VersionControl.Client
Microsoft.TeamFoundation.VersionControl.Common

using System;
using System.IO;
using Microsoft.TeamFoundation;
using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.VersionControl.Client;
using Microsoft.TeamFoundation.VersionControl.Common;

namespace LabelHistory
{
    class Program
    {
        static void Main(string[] args)
        {
            // Check and get the arguments.
            String path, scope;
            VersionControlServer sourceControl;
            GetPathAndScope(args, out path, out scope, out sourceControl);

            // Retrieve and print the label history for the file.
            VersionControlLabel[] labels = null;
            try
            {
                // The first three arguments here are null because we do not
                // want to filter by label name, scope, or owner.
                // Since we don't need the server to send back the items in
                // the label, we get much better performance by ommitting
                // those through setting the fourth parameter to false.
                labels = sourceControl.QueryLabels(null, scope, null, false, 
                                                   path, VersionSpec.Latest);
            }
            catch (TeamFoundationServerException e)
            {
                // We couldn't contact the server, the item wasn't found,
                // or there was some other problem reported by the server,
                // so we stop here.
                Console.Error.WriteLine(e.Message);
                Environment.Exit(1);
            }

            if (labels.Length == 0)
            {
                Console.WriteLine("There are no labels for " + path);
            }
            else
            {
                foreach (VersionControlLabel label in labels)
                {
                    // Display the label's name and when it was last modified.
                    Console.WriteLine("{0} ({1})", label.Name,
                                      label.LastModifiedDate.ToString("g"));

                    // For labels that actually have comments, display it.
                    if (label.Comment.Length > 0)
                    {
                        Console.WriteLine("   Comment: " + label.Comment);
                    }
                }
            }
        }

        private static void GetPathAndScope(String[] args,
                                            out String path, out String scope,
                                            out VersionControlServer sourceControl)
        {
            // This little app takes either no args or a file path and optionally a scope.
            if (args.Length > 2 || 
                args.Length == 1 && args[0] == "/?")
            {
                Console.WriteLine("Usage: labelhist");
                Console.WriteLine("       labelhist path [label scope]");
                Console.WriteLine();
                Console.WriteLine("With no arguments, all label names and comments are displayed.");
                Console.WriteLine("If a path is specified, only the labels containing that path");
                Console.WriteLine("are displayed.");
                Console.WriteLine("If a scope is supplied, only labels at or below that scope will");
                Console.WriteLine("will be displayed.");
                Console.WriteLine();
                Console.WriteLine("Examples: labelhist c:\\projects\\secret\\notes.txt");
                Console.WriteLine("          labelhist $/secret/notes.txt");
                Console.WriteLine("          labelhist c:\\projects\\secret\\notes.txt $/secret");
                Environment.Exit(1);
            }

            // Figure out the server based on either the argument or the
            // current directory.
            WorkspaceInfo wsInfo = null;
            if (args.Length < 1)
            {
                path = null;
            }
            else
            {
                path = args[0];
                try
                {
                    if (!VersionControlPath.IsServerItem(path))
                    {
                        wsInfo = Workstation.Current.GetLocalWorkspaceInfo(path);
                    }
                }
                catch (Exception e)
                {
                    // The user provided a bad path argument.
                    Console.Error.WriteLine(e.Message);
                    Environment.Exit(1);
                }
            }

            if (wsInfo == null)
            {
                wsInfo = Workstation.Current.GetLocalWorkspaceInfo(Environment.CurrentDirectory);
            }

            // Stop if we couldn't figure out the server.
            if (wsInfo == null)
            {
                Console.Error.WriteLine("Unable to determine the server.");
                Environment.Exit(1);
            }

            TeamFoundationServer tfs =
                TeamFoundationServerFactory.GetServer(wsInfo.ServerName);
                                                      // RTM: wsInfo.ServerUri.AbsoluteUri);
            sourceControl = (VersionControlServer)tfs.GetService(typeof(VersionControlServer));

            // Pick up the label scope, if supplied.
            scope = VersionControlPath.RootFolder;
            if (args.Length == 2)
            {
                // The scope must be a server path, so we convert it here if
                // the user specified a local path.
                if (!VersionControlPath.IsServerItem(args[1]))
                {
                    Workspace workspace = wsInfo.GetWorkspace(tfs);
                    scope = workspace.GetServerItemForLocalItem(args[1]);
                }
                else
                {
                    scope = args[1];
                }
            }
        }
    }
}

[Update 10/26] I added Microsoft.TeamFoundation.Common to the list of assemblies to reference.

[Update 7/12/06]  Jeff Atwood posted a VS solution containing this code and a binary.  You can find it at the end of http://blogs.vertigosoftware.com/teamsystem/archive/2006/07/07/Listing_all_Labels_attached_to_a_file_or_folder.aspx.