Motley: Checklist? We don't need no stinking checklist! Checklists are for anal people with short memories.
Maven: Checklists are very useful for activities like code reviews, where it is difficult to remember everything we need to look for. Good checklists are short, consistent, easy to understand, accessible and provide support information. Personal and team checklists are recommended.
[Context: Maven spent the morning looking over the code reviews in his Inbox and noticed a few odd things about one of Motley's code reviews from earlier in the day]
Maven: I was just looking over the code you sent out for review earlier in the day. Nice job on finding those errors you found. There are a few more things I might point out, however.
Motley: Did I miss a few? Hard to believe as I am the king of code reviews. I must have been having a bad morning. What did I miss?
Maven: Let me ask you this first: when you look for issues in a code review, what kinds of things do you look for?
Motley: The usual stuff - memory leaks, invalid usage of pointers, failure to check input parameters - that kind of thing. That covers the major ones.
Maven: What about issues like commenting, proper use of exceptions, freeing up resources, ensuring strings are localizable, assertions are used appropriately, unit tests are created, and checking for buffer overruns? I could go on and on.
Motley: I usually remember to check for those kinds of problems. Depends on my mood.
Maven: Do you ever use a checklist?
Motley: Checklist? We don't need no stinking checklist! Checklists are for anal people with short memories. Checklists will just slow us down. Plus, they get out of date quickly in my experience. Better to just get into the habit of knowing what to look for and then just do it!
Maven: Let me ask you this: do you remember everything you need to buy at the grocery store?
Motley: The common stuff I buy from week to week? Yes.
Maven: What about the one-off groceries? What if you want to bake a cake and you don't have any flour, which is not something you buy from week to week?
Motley: First of all, think about what you just said. Do you think I would ever be caught baking a cake? Wise up, Mave.
Maven: Good point. Sorry. The point is that for things you are in the habit of, your brain just automatically remembers them. I always know I need fruit when I go to the grocery store. However, for things that I don't buy regularly, like chocolate chips, I need a list.
Motley: Do what you need to do. I still try and get into the habit of looking for certain items in a code review, though, so I can avoid a checklist.
Maven: Getting into the habit is great, but with all the things a developer has to keep in his or her head, it's hard to remember everything when coding.
If we sat down and brainstormed every characteristic of production-quality code, we would probably fill a couple of whiteboards. Unfortunately, the human brain only retains about 7 +/-2 concepts in short term memory at a time. That's why phone numbers are seven digits (forget the area code, which is pretty static). For concepts outside of that core 9 - to be optimistic - we really need some tools that help us with reminders.
Motley: There are certain things that I "just remember", like checking for null pointers. I don't need a checklist to help me with that.
Maven: I agree.
Motley: You agree?!? Then what's the point?
Maven: For concepts that really are habit and don't require any extra effort to think about, consider not putting those on a checklist. The checklist is a helpful tool to ensure you remember those concepts that have not become habit. Additionally, if the team, for example, always remembers to pull strings out of resource files to help make the code localizable, then get that off the checklist and replace it with a reminder to check for a mistake that the team often makes. The highest priority items for the business or the release are also good candidates.
Motley: So now you are talking about maintaining checklists, which I indicated is cumbersome.
Maven: Keep checklists in a simple format that everyone can easily edit, like a wiki. Sure, you have to spend time maintaining them, but if it saves even one bug from shipping, it has just paid for itself 50 times over. Even better, you might want to have your personal checklist in a tool that you use every day and all day. For example, I noticed you check your e-mail a lot - you could have your checklist inside Microsoft Outlook as a task list. There it is easy to maintain and always available. The wiki works just as well though, particularly for team checklists.
Motley: Now you are talking about more than one checklist? It's getting worse.
Maven: A good practice is to have a team checklist and a personal checklist. The team checklist contains items that you want everyone to look for in a code review. It typically contains the most frequent errors made by the team, or major areas to focus on. Your personal checklist contains items specific to you, such as things you often forget to do or look for. For me, I often forget to make liberal use of assertions, so my checklist helps remind me to code defensively. A best practice is to use your person checklist to review your code prior to sending it out for review.
Motley: So I guess I start with the stuff I missed from this mornings code review. I am just going to keep it in a simple text file on my desktop where it is easy to find and edit.
Maven: Format is not important - whatever works for you. Here are a few more characteristics of good checklists:
Mixing styles makes the checklist difficult to read.
Motley: As always, you have lots of rules for everything. Don't you get tired of being so thorough?
Maven: Well, no. Ha ha ha. These "rules" are basically some intuitive guidelines (softer than a "rule") to help create a good checklist. For your personal checklist, you can do what you want as long as they help with the main goal of reminding you of important items.
Motley: Fine, fine, fine. Based on my poor code review performance this morning, I'll give it a shot. I'll create a quick checklist for code reviews.
Maven: Great! Checklists are also great for other activities, too, like design reviews, unit tests, code check-ins, and many other development activities.
Motley: Let's not get carried away. I said I'll create one for code reviews.
Maven: Ideally, you also think about building tools that automatically validate your checklist items without human interaction. That's what a tool like FxCop does - it takes a big checklist like the .NET Design Guidelines and automatically ensures your code satisfies those guidelines. Automated checklists are the best kind!
Motley: I'm always up for writing code to remove more manual labor. Let me give that some thought.
Maven's Pointer: One interesting format for checklists is in XML. With XML you can apply any type of XSLT transform to the data to generate different views. You can have one view that is a summary of the checklist item, and another view that contains all of the details. Each view is based on the same central XML file. Developing checklists using a tool like Microsoft InfoPath abstracts away the XML data format and still allows multiple views. Store the XML on a Microsoft SharePoint server to make them centrally accessible, and you have a great checklist solution.
I agree -- checklist are key for sharing expertise and improving results.
Sadly, I've come across very few useful checklists among the sea of checklists.
Creating your own checklist for yourself is a great move -- it makes it immediately relevant so you can't beat that.
I've also found creating checklists to be one of the most effective ways to share knowledge on my teams. I use them to carry lessons learned forward.
I think the real keys in checklist include:
- focus on surprises over obvious
- use stateful condition checks (i.e. fuel full? ... light on? ...)
- use question-driven checklists (it puts your mind in the right place)
- factor checklists by focus (this helps relevancy, maintainability and usability)
One of the other issues with checklists though is the absence of "show me how." I like Wikis for checklist because you can link to more information. We use Guidance Explorer to link checklist to guidelines to code examples ... etc.
Looks like we need a checklist of checklists to navigate the sea of checklists. :-)
Your interactive comments are right on. I've heard all of them many times. Here is a blog I wrote on the subject. Somewhat different environment but exactly the same issues
Thanks for the comments, guys! The article join pointed to is good. Bottom line is that we just have too much stuff in our heads to keep track of all the details. Do they slow you down? Not in the long run if you are about doing quality work up front.