I've been running another usability study on an API that makes heavy use of attributes and have made similar observations to a previous study (see http://weblogs.asp.net/stevencl/archive/2004/05/12/130826.aspx#FeedBack)

This time we've been looking at the Indigo APIs. Indigo makes heavy use of attributes throughout. For example, attributes are used to identify an interface as representing a service contract or a method inside that interface as a service operation.

In this study, we've observed similar issues to those observed before. One of the issues that has resurfaced is the apparent reluctance of participants to consider the attributes sprinkled throughout their code as potential causes of unexpected application behavior. As in the previous study, we've seen developers consistently neglecting to consider modifying any of the attributes that they have applied to their code when attempting to modify the behavior of their code. In many cases, the desired effect (changing the lifetime of a service for example) is done by simply changing a property of a particular attribute. But instead, many participants spend time investigating their own code and even changing their own code to try to achieve the change in behavior they are looking for. It seems too that this behavior isn't necessarily isolated to the usability labs. One of the Indigo team members told me of a recent example where some folks from the product team were attempting to debug some customer code. They spent a lot of time checking the IIS configuration, the customer code etc and overlooked the attributes in the code. It turned out that the wrong attribute had been used.

I think one factor in this is the low visibility of attributes. For example, one participant in the study this week was stepping through his code in the debugger when he noticed some unexpected behavior at some point during the excecution of his code. He was focused on a particular block of code and concentrated his efforts on understanding how that block of code might have caused the behavior he had just observed. The cause for that behavior was due to an attribute that had been applied to the class that defined the method the participant was stepping through. Thus when he was reading his code, the attribute was well out of his focus of attention.

However, such issues are typically an issue for the first few times only - once you learn the effect of an attribute, it's more likely that you'll remember that attribute and will revisit it if the same issue occurs again. And indeed, this is what we observed during the study.

Another factor involved is when there are relationships between two or more attributes. The problem gets worse when the change in application behavior necessitates modifying more than one attribute at a time. The problem is that the relationship between multiple attributes is often unclear. One of the tasks in the study I am running this week requires that participants apply an attribute to a method inside an interface declaration and another attribute to the class that implements that interface. Participants who have worked on this task have been unaware of this requirement and have not been able to complete the task without significant help. There is nothing about the design of the attributes that makes this relationship clear.

Once participants have been told about the multiple attributes and the requirement to modify them all in conjunction their reaction is always the same - "that's kludgy!". Having seen this in multiple studies on different APIs, I'm pretty much at the point where I would strongly recommend API teams avoid designing APIs that require developers to modify multiple attributes to achieve specific functionality. Instead, I'd recommend examining ways that the same functionality could be achieved through the use of one attribute or at the very least, consider naming the multiple attributes in such a way that the relationship between the attributes is much clearer.

It would be interesting to hear anyone's feedback on this issue. Is this something you have experienced yourself when using an attribute heavy API? How big of an issue do you think it is? What would be your preferred solution?