I spent the week at the ACM CHI conference in Montreal. I love attending CHI and seeing all the great stuff happening in research, academia, and product development. This year I presented an Experience report with my close compriate Donna Wallace. We presented our "Experience Engineering Framework" that we designed with Monty Hammontree. This is a tool that we created to help us tailor a user-centered design plan for every project we tackle.

I promised to post a slightly long, but still much condensed, article on the topic. Please check out the experience engineering framework, and let us know what you think.

Cheers, Rick

This was the title of the talk that I did with Brad Abrams yesterday at Mix06. This was one of the most fun days I have had at a conference. I met a lot of really neat people with different perspectives and ideas. Plus, there were several parties with open bars! Free drinks and unlimitted geek talk, yeehaaa!

One thing that made Mix06 a different conference for me, is that it's a"conversation". This definately held true for our talk.I think they will actually publish a link to the presentation, so I'll link to that when I get it. I've had a lot of really great conversations with the attendees here that are passionate about design. Here's the content of my part of the talk and some of the enusing questions so that we can keep the converstaion going.

Web 2.0 is a Designer Driven Revolution
Of course I am biased, and some of the developers here think this is a hilariously out of touch comment, but I really believe it. Look, the technology that enables AJAX and Web 2.0 apps has been around for years. For the past couple of years designers have been wielding the tools to create and push new experiences on the web. Libraries and other platform improvements have responded to this designer pressure to make it easier to deliver these experiences. The technology is changing to accomedate the experience we want to deliver instead of visa-versa.

Best Practices are Forming in Real Time in a Web 2.0 Way
I don't think we are necesssarily going to see a collection of definitive books on topics such as AJAX usability, AJAX design, etc... I think what we are seeing is that best practices are being proposed, linked, and strengthened by converging multiple perspectives in tight feedback loops. I refered to 2 web site in my talk:

• AJAX Patterns: discusses some desing principles and has a lot of details on how to implement them with basic AJAX code.
• AJAX Usability: discusses how AJAX can make your sites easier or harder to use.

I won't provide a detailed overview of these sites. But I think it is very cool how they refer to each, and other sites, including the AJAX Pratterns book as it is being writtern. However, here are three essential things that I take as a given for designers based on the community best practices:

1. There is a difference between a "web site" and a "web app." If you have a web site (colleciton of linked documents) then tryng to make it a web app, breaking years of web site conventions, etc.. will bust your user experience.
2. Obviously you should update only small parts of page or screen when it makes sense.
3. Good design and conventions still matter. Your experience as a designer is still relevant.

Now I talk about six "lessons from the trenches" that we have accrued as we have been creating user experiences with Windows Live Local. Of course there are a ton more details and perspectives that I could have gone into, but I think this was the right level for this talk. I’ll post a link to the talk here when (if) it’s on line. But quickly, here are the six that I added to the mix:

1.      Provide instant Feedback to the user.  When users click on something on your site, you can’t wait for the server side response to provide feedback that they did something, you have to respond immediately on the client. If you don’t your site could seem slower then it is, or it can even be downright confusing for longer server side tasks. AJAX Patterns discusses this idea as part of “Local Event Handling.”

2.      Predict user actions. By predicting what the user will do next, you can bring data, images, lists, etc… down to the client, and provide a snappier experience, help users discover options and features, and help user accuracy. AJAX Patterns discusses when they discuss “Predictive Download”, “Local cache”, and “Local Activity.”

3.      Preserve user state. When you have a rich set of interactions you invite users to invest time and energy into your site. You should honor that investment by saving and recreating user state if users close the browser, leave your site, etc… AJAX Usability mentions that as part of “Instant Save.”

4.      Allow users to Share state. Similar to preserving state, let users leverage their investment by sharing the fruits of their labors. On AJAX sites, this is often done with a “Permalink.”

5.      Create Controls and document APIs. Let web developers mix it up! Scripters should be able to extend the experiences you create and create new experiences. If you have nifty UI, let them host it in their site. If you have nifty data, let them access your web services.

6.      Strictly Separate presentation logic and implementation logic. ‘nuff said.

Hopefully you’ll be able to see some of the ensuing discussion in the video. They are after the part where we discuss how to apply these principles using ATLAS. There were a few threads to the conversation, including:

1.      Designers need web browsers to know the difference between a web app and a web site so that users know what set of conventions to use, and allow the browser to manage the saving and sharing of state, as apposed to the application.

2.      Part of #1 was “we hate permalinks!”

3.      At what point does an app get too rich to consider building it in a browser and moving it instead to a rich client app.

4.      Since web apps break conventions of web sites, how to we move the base of existing users over to a new emerging set of conventions?

Feedback, discussion, mix it up!

 

I just ran a usability study with 9 participants where we asked them to build two forms, one a WinForm and one a Web Form, that had a drop down list that drove the contents of a grid below it. The participants had to figure out how to fill the drop down from all the cities in the database, and then how to fill in all the rows that matched that city in the drop down (from a different table). We gave them a little more then an hour for each form.

Whidbey has a ton of new data binding features, and there were a couple of interesting observations out of this study. First, users got much farther along then they did in similar studies with the Everette feature set. All the participants got to see data in the form at least a little bit. This seemed to make the whole experience much more pleasing for them. This was a great thing to see. I think getting that positive reinformcement early in the process will improve the first impressions of .NET users and will contibute significantly to driving adoption during the evaluation phase.

The second interesting thing harkened back to something that I have observed many times over the years. Users like to layout the controls onto a form, and then they like to make those controls work. The only caveat to this is that some experience .NET users started the task by dragging data components out of the toolbox, because they knew from experience this was the most efficient way to accomplish the task.

In any case, there was definately a very strong “Toolbox First” approach to writing these forms. A lot of the new data features in both win and web scenarios are available and easily discoverable when using this approach, and users seemed to be both successful and pleased by features that worked with the toolbox first approach. The data features that did not work “toolbox first” were either not discovered, or users struggled to make them work and abandoned them.

So from now I think that when we think about feautres that support any UI programming, we should always think that the user is going to approach it by laying out the form first, and then making the design of the form work by writing code, running wizards, using other features, etc... If users have to do work somewhere else in Visual Studio before they can start laying controls onto the form, they probablyt won't discover this dependency, or won't want to use that feature.

Hi all,

Keep the feedback on the debugger usability problems coming. I've seen a few that have Whidbey solutions, and a few that don't. I'll capture all this feedback and make sure we use it.

Cheers, Rick

Yesterday I posted about how we think the typical C# user users the debugger. This posting talks about some of the usability problems that we observed, and also discusses how some of the new Whideby debugger features address those problems.

Users Work for the Debugger, the Debugger Doesn’t Work for Them

We noticed that as users were trying to inspect the values in their code, they spent most of their time interacting with the debugger UI, as apposed to thinking about their debugging problems. Some of the work that they had to do for the debugger included:

1. Mousing around to hover over values
2. Bringing up windows
3. Resizing and moving windows
4. Resizing columns
5. Copying and pasting from columns
6. Expanding tiny like “+”
7. Rewriting code

Since we observed over and over again that users chose to use datatips (the tooltips that you get when you hover over a variable) as their tool of choice for inspecting values, since datatips don’t require much “management” in terms of sizing, placing, etc… The debugger team, wisely in my opinion, invested heavily in enhancing how datatips worked. (For those of you who haven't seen this feature yet, data tips now let you drill down into a value so you can see what's inside a variable without going to a debugger window.)

An interesting aside here, our user research suggests that even hard core Unix developers who never use the mouse when writing code, do use the mouse when debugging code. Once of them told us that debugging is a visual activity, and as such, the mouse is efficient and natural while doing that task.

As cool and efficient as datatips are, I think we need to take it even farther. Why should users have even have to hover with the mouse to see the values? I think we should move toward a model where the watch window is basically embedded in the editor. Maybe with some gesture like clicking the mouse button

Usability Breakdown: Inspecting Complex Types

Often, when a user is debugging a framework object, for instance a Dataset, the actual data that the user wanted to find was essentially inaccessible because the debugger simply showed the internal data structure. So if you wanted to see what data was in a Dataset that you filled, you had to dig way down into the watch window, and hope you could find the information you needed.

The debugger team’s solution to this problem was Visualizers. I think Visualizers will save developers countless hours of debugging. (Again, if you haven't seen these yet, Visualizers provide a way to display information about an object on a Win Form, so developers can focus on the interesting part of the data. For example, if you are debugging a program that uses a dataset, you can view the data in the dataset in a grid, since this is usually what you want to know when debugging. Visualizers are an additional debugging method, you still have the ability to look at the dataset in a debugger window as before.)

Usability Breakdown: Inspecting Long Values

We saw a lot of users trying to inspect values that didn’t fit into the value column of a debugger window very well. For instance when a user wants to see the structure of some XML that they are reading in they have to write it to the output window to really see the structure of it and to be able to scroll through it effectively. Unfortunately this requires writing extra lines of debugging code. Then there are other data types, like bitmaps, which you simply can’t view in the debugger at all.

Visualizers to the rescue again! Now you can view strings, XML, etc… from the debugger in a reasonably readable format.

Usability Breakdown: Inspecting the Return Value of Method Calls

Users frequently want to know the return value of a function call they are making. Most users end up rewriting their code to set the return value to a local variable so they can see that value in a debugger window. This can make code hard to read and unnecessarily complex. Plus, it’s just extra work that users have to do for the debugger.

It turns out that some users discovered that they can add a method call to the watch window (not many C# users seem to use the immediate window to try out function calls). Unfortunately, for the users who did add the values to the watch window, they sometimes got foiled by side effects of methods. For example, we saw one user trying to debug a program that was reading in XML. He tried to see what the he was reading in by putting a “read()” method in the watch window. In this case, the read method read in the next node of XML, but also moved the cursor in the XML file to the node after. As a result, the debugger stole half of the XML and made the program act oddly.

This is a tough problem, and probably users will still have to rewrite code at least some of the time to debug effectively. However, there are a couple of features that the debugger team worked on in this version that should help address this problem. One thing they tried to do was to make it clear that in the watch window the method was executed, it didn’t just store the last return value.

Usability Breakdown: Setting a Breakpoint in a Large Project

We saw some users writing code for big projects, so they had a lot of code. Sometimes they would see a bug, but it was very hard for them to figure out where to set the first breakpoint to start stepping and inspecting, especially when the bug only shows up after some moderate user interaction with their program. While there are a couple of new features that will assist with this is a tough problem because a key user task that is not well supported by the debugger is...

Understanding Program Execution

Often developers have to figure out how a program works in a general way to start to get an idea about what could be causing a bug, or even where to set a breakpoint and start looking at it more closely. The thing that the debugger does so well is to provide a very detailed snapshot of the program, but it doesn’t do as good a job helping a developer understand how the program executes, it doesn’t give an overview. It’s like trying to understand the plot of a movie by looking at a few frames from the film.

From the point of view of the psychology of programming, I think this is a very interesting area to understand and to try to support developers. I know there are various profiling tools, etc… (also, see the Trace Points feature in Whidbey) that can help developers look at stack traces and event logs, etc… but I haven’t seen  any tools that seem to be built on a really solid foundation of how developers think about this. I think designing a debugger that helps a developer with this cognitive task would be an incredible benefit.

When developers build a mental model of how a program works, what do those models look like? How do they use tools to help build those models? How do they use those models to enhance or debug a program?

Hi. Thanks for all the feedback on my last Blog entry. The C# team did a lot of work trying to understand our users, and it’s great to see where you all do and don’t agree with our conclusions. I thought it might be interesting for you all to get a peek into a different part of our product development process, when we usability test new features. In this entry I’ll briefly cover the C# specific usability study that we did for Generics, including a little on how we tested it, and what we found. It was interesting for me to study a language feature as apposed to a set of UI, and I think we learned a lot.

 

We're going way back here, to October 2002. I was working with Anson Horton, who was at the time the PM for the C# compiler. We wanted to get some understanding of how users would react to generics and our documentation for them. We wanted to know if users would be able to master the syntax with a minimum of frustration, and would they get how to use them to write clean code. We recruited 5 Visual C# developers to come to the Microsoft Usability labs, and asked them to try some programming with generics.

 

The Set Up

Anson wrote a simple application that used a regular old hashtable of objects from the Visual C# 2003 (Everett). We asked the documentation writer to get us the best version of the Generics documentation that they could before the study. We put a copy of the docs in the waiting room so the participants could read as much or as little of the documentation as they wanted before they started coding. We also taped a copy of the documents down to the table next to their computer where we could keep a remote camera on the docs and get an idea about what the users were reading. There were 56 pages of documentation altogether that covered the concepts of Generics, but mostly focused on the syntax of all the different parts of consuming and creating Generic classes.

 

We wrote out some things that we wanted the users to do with the Anson’s program. Here are two of the tasks we gave them:

 

1. Pretend that Microsoft has just released a new version of Visual C# and the .NET Frameworks that includes some new strongly typed classes that use Generics. Your task is to modify your company’s contact management Windows Forms Application and change it to use the new strongly typed classes where appropriate.  The application is called “address” and is in the folder on your desktop called “address.

(Note: This task was used to get a sense of how well users understood the Generics concept. The code used a hashtable, and we wanted to see if they could recognize that the hashtable was a good candidate for Generics. Then we wanted to see their first experience using Generics was good and productive.)

 

2. Pretend that part of your job is to manage shared classes for your company. Please take the linked list class, called “LinkedList” and make it strongly typed using the Generics feature. You will find the code in the “link” folder on your desktop.

(Note: This task was used to get a sense for how easy or hard it would be for users to create a class that was generic. We started them with a non-generic version so they could focus on just the generic code in the study.)

 

We also created an interview, and Anson interviewed each participant about their impressions of Generics after they tried it.

 

The Results

Here are some of the key results from the study.

 

In general, users will be able to consume generic collection classes. However, they will find the syntax of specifying the type parameters in the instantiation clause to be unintuitive.

Basically, what we saw was that when the developers should have written something like this:

List<string> myListOfStrings = new List<string>();

 

They would look at the docs and write:

 

List<string> myListOfStrings;

But they struggled to get: myListOfStrings = new List<string>();

 

There were a few of factors that made this hard it hard to write the full line of code (note that we were using very early bits for this, so all the namespaces and class names have since changed):

 

1. It seemed weird to have to declare the type parameter twice, it was just not intuitive to them at first.
2. It seemed weird to put the type parameter and the parens in the constructor call. Again, it was just not intuitive them the developers at first. It just sort of looks odd until you get used to it.
3. The error messages from the compiler were not helpful. Here’s a sample of the compiler errors they got:

Included type parameters with declaration clause, but not with instantiation clause: “error CS0691: ‘ArrayList’ type not found.  ‘ArrayList’ has the wrong number of type parameters”

Included type parameters with instantiation clause, but not with declaration clause: “error CS0029: Cannot implicitly convert type ‘Experimental.Collections.Generic.ArrayList<int>’ to ‘System.Collections.ArrayList’”

I think you can see why the error messages didn’t help much for developers trying to learn the syntax. I think the current error messages are slightly better.

 

4. Intellisense did not help. When we tested this the auto-complete features for Generics were not in the editor yet, so users had to read documentation. Now when you type “List<string> myListOfString = “ the clause “new List<string>()” pops up in the Intellisense completion list and is selected. I think this will go a long way to helping C# devs learn the syntax faster.

Users want to use code snippets to learn the syntax

Not much to say here. Basically, all the users except 1 skipped everything in the docs, and just skimmed for code until they saw a code fragment that did what they wanted. We see this effect all over the place, not just learning syntax, but it was especially noticeable here. I guess it figures that people would want to learn syntax by reading examples, but one of the developers did use the “grammar” section that describes the usage of the syntax in a general way, not with examples.

 

Users want specific code snippets for the class they are trying to consume

This observation followed the last one, but we saw that the developers skimmed the documentation for examples using the particular class the wanted to use, not just any example. For instance, if they were trying to instantiate a HashTable<K,V>, but the first example was for a List<T> they kept skimming. After they found out that the only example was for a List<T> they went back and used that. During the interview they said they would have been happier with examples for every generic class they could use.

 

After consuming a Generic class, users will be able to create their own Generic classes

We saw that by following the examples, the developers were able to convert Anson’s LinkedList to be strongly typed. However, a few of the developers felt that they wouldn’t really need to do so, because they don’t need much more then the .NET Framework was providing. Good news, it sounds like we have the right set of classes in the Framework.

 

C# coding will be better with Generics

After trying them, all 5 developers said that Generics was definitely a great addition to the language, and would make their code cleaner to read and run better because they would never get runtime exceptions related to casting. This was congruous from the feedback that we got from our other customer channels as well. The difference was that this feedback was based on actual experience of Generics in code, so we felt that our predictions about the customer reaction to Generics were probably true.

 

Templates don’t kill developers, developers kill developers

This harkens back a little to the “Battle Scars from C++” comment in my last blog entry. During the interview, the developers recounted stories of working on C++ projects where templates were hideously misused, and caused them endless pain trying to use other developers template code. They thought that this was a risk with Generics too. I think we need to provide strong guidelines on when to use, and especially when not to use, generic classes.

 

Feedback?

• Have you tried Generics, and what do you think of the syntax for declaring, instanting, and creating them? Was it harder or easier to learn then you expected?
• Do you think C# developers will have trouble learning the Generics syntax? What would be most useful, more IDE support, better error messages, or more and more examples?
• Are Generics a purely positive addition to the C# language, or are there drawbacks that we haven’t considered?
• Are any of you worried that Generics will be misused and that you’ll end up working on projects where misuse makes you less productive? Should we be doing anything to address possible misuse of Generics?