Hi. Thanks for all the feedback on my last Blog entry. The C# team did a lot of work trying to understand our users, and it’s great to see where you all do and don’t agree with our conclusions. I thought it might be interesting for you all to get a peek into a different part of our product development process, when we usability test new features. In this entry I’ll briefly cover the C# specific usability study that we did for Generics, including a little on how we tested it, and what we found. It was interesting for me to study a language feature as apposed to a set of UI, and I think we learned a lot.

 

We're going way back here, to October 2002. I was working with Anson Horton, who was at the time the PM for the C# compiler. We wanted to get some understanding of how users would react to generics and our documentation for them. We wanted to know if users would be able to master the syntax with a minimum of frustration, and would they get how to use them to write clean code. We recruited 5 Visual C# developers to come to the Microsoft Usability labs, and asked them to try some programming with generics.

 

The Set Up

Anson wrote a simple application that used a regular old hashtable of objects from the Visual C# 2003 (Everett). We asked the documentation writer to get us the best version of the Generics documentation that they could before the study. We put a copy of the docs in the waiting room so the participants could read as much or as little of the documentation as they wanted before they started coding. We also taped a copy of the documents down to the table next to their computer where we could keep a remote camera on the docs and get an idea about what the users were reading. There were 56 pages of documentation altogether that covered the concepts of Generics, but mostly focused on the syntax of all the different parts of consuming and creating Generic classes.

 

We wrote out some things that we wanted the users to do with the Anson’s program. Here are two of the tasks we gave them:

 

  1. Pretend that Microsoft has just released a new version of Visual C# and the .NET Frameworks that includes some new strongly typed classes that use Generics. Your task is to modify your company’s contact management Windows Forms Application and change it to use the new strongly typed classes where appropriate.  The application is called “address” and is in the folder on your desktop called “address.

(Note: This task was used to get a sense of how well users understood the Generics concept. The code used a hashtable, and we wanted to see if they could recognize that the hashtable was a good candidate for Generics. Then we wanted to see their first experience using Generics was good and productive.)

 

  1. Pretend that part of your job is to manage shared classes for your company. Please take the linked list class, called “LinkedList” and make it strongly typed using the Generics feature. You will find the code in the “link” folder on your desktop.

(Note: This task was used to get a sense for how easy or hard it would be for users to create a class that was generic. We started them with a non-generic version so they could focus on just the generic code in the study.)

 

We also created an interview, and Anson interviewed each participant about their impressions of Generics after they tried it.

 

The Results

Here are some of the key results from the study.

 

In general, users will be able to consume generic collection classes. However, they will find the syntax of specifying the type parameters in the instantiation clause to be unintuitive.

Basically, what we saw was that when the developers should have written something like this:

List<string> myListOfStrings = new List<string>();

 

They would look at the docs and write:

 

List<string> myListOfStrings;

But they struggled to get: myListOfStrings = new List<string>();

 

There were a few of factors that made this hard it hard to write the full line of code (note that we were using very early bits for this, so all the namespaces and class names have since changed):

 

  1. It seemed weird to have to declare the type parameter twice, it was just not intuitive to them at first.
  2. It seemed weird to put the type parameter and the parens in the constructor call. Again, it was just not intuitive them the developers at first. It just sort of looks odd until you get used to it.
  3. The error messages from the compiler were not helpful. Here’s a sample of the compiler errors they got:

Included type parameters with declaration clause, but not with instantiation clause: “error CS0691: ‘ArrayList’ type not found.  ‘ArrayList’ has the wrong number of type parameters

Included type parameters with instantiation clause, but not with declaration clause: “error CS0029: Cannot implicitly convert type ‘Experimental.Collections.Generic.ArrayList<int>’ to ‘System.Collections.ArrayList’

I think you can see why the error messages didn’t help much for developers trying to learn the syntax. I think the current error messages are slightly better.

 

  1. Intellisense did not help. When we tested this the auto-complete features for Generics were not in the editor yet, so users had to read documentation. Now when you type “List<string> myListOfString = “ the clause “new List<string>()” pops up in the Intellisense completion list and is selected. I think this will go a long way to helping C# devs learn the syntax faster.

Users want to use code snippets to learn the syntax

Not much to say here. Basically, all the users except 1 skipped everything in the docs, and just skimmed for code until they saw a code fragment that did what they wanted. We see this effect all over the place, not just learning syntax, but it was especially noticeable here. I guess it figures that people would want to learn syntax by reading examples, but one of the developers did use the “grammar” section that describes the usage of the syntax in a general way, not with examples.

 

Users want specific code snippets for the class they are trying to consume

This observation followed the last one, but we saw that the developers skimmed the documentation for examples using the particular class the wanted to use, not just any example. For instance, if they were trying to instantiate a HashTable<K,V>, but the first example was for a List<T> they kept skimming. After they found out that the only example was for a List<T> they went back and used that. During the interview they said they would have been happier with examples for every generic class they could use.

 

After consuming a Generic class, users will be able to create their own Generic classes

We saw that by following the examples, the developers were able to convert Anson’s LinkedList to be strongly typed. However, a few of the developers felt that they wouldn’t really need to do so, because they don’t need much more then the .NET Framework was providing. Good news, it sounds like we have the right set of classes in the Framework.

 

C# coding will be better with Generics

After trying them, all 5 developers said that Generics was definitely a great addition to the language, and would make their code cleaner to read and run better because they would never get runtime exceptions related to casting. This was congruous from the feedback that we got from our other customer channels as well. The difference was that this feedback was based on actual experience of Generics in code, so we felt that our predictions about the customer reaction to Generics were probably true.

 

Templates don’t kill developers, developers kill developers

This harkens back a little to the “Battle Scars from C++” comment in my last blog entry. During the interview, the developers recounted stories of working on C++ projects where templates were hideously misused, and caused them endless pain trying to use other developers template code. They thought that this was a risk with Generics too. I think we need to provide strong guidelines on when to use, and especially when not to use, generic classes.

 

Feedback?

  • Have you tried Generics, and what do you think of the syntax for declaring, instanting, and creating them? Was it harder or easier to learn then you expected?
  • Do you think C# developers will have trouble learning the Generics syntax? What would be most useful, more IDE support, better error messages, or more and more examples?
  • Are Generics a purely positive addition to the C# language, or are there drawbacks that we haven’t considered?
  • Are any of you worried that Generics will be misused and that you’ll end up working on projects where misuse makes you less productive? Should we be doing anything to address possible misuse of Generics?