A few weeks ago I took Jerry Weinberg's Problem Solving Leadership (PSL) workshop. Everyone I have heard speak about PSL calls it a life changing experience. I understood Jerry's book Becoming A Technical Leader to be a sort of offline version of PSL. (Turns out it is and it isn't. Although each covers some of the same ground as the other, neither is a replacement for the other.) The only other thing I knew about PSL, whose details seemed to be a mysterious secret, was that I had a deeply felt knowing that I needed to take it. So I did.
Now that I've been through PSL I understand one reason behind this paucity of information about it: many of the exercises would be less effective if you knew their point and their solutions. So I won't say much about them. I will however relate some of what I learned during the workshop and in the time since. PSL was the proverbial "drinking from a fire hose" experience for me, and I am still processing what I took in. Here is what I know so far.
I found the class to be an intense experience. The schedule itself didn't seem unusual: we started at noon on a Sunday and finished at noon the following Friday, with a lunch break each day and most evenings to ourselves. Our sessions were a mixture of interactive lectures, experiential simulations, and debriefings of the simulations. I worked hard during the simulations and almost as hard during the debriefings and lectures. I always looked forward to the lunch and supper breaks for the down time they provided, and I typically spent them doing not much of anything. I found this structure, its ups and downs, intense periods and calm times, long days and long lunches, all compressed into a short span of time, valuable in and of itself. With so much going on, closely followed by time to reflect, right into more activity, and so on without end, I was able to observe how well I did or didn't do when I did and didn't remember to do all of the things I know to do and not do. Staying grounded and centered and present, for example. A shining example of what happens when I don't do this came when we were selecting a leader for one of the simulations. I wasn't completely there and was utterly surprised when someone nominated me to be the leader. Wham! Back in my body in full panic mode, babbling anything that seemed remotely relevant to saying "Thanks, and no thanks". Later on during the simulation itself, however, I stayed grounded and centered and present throughout, and lo and behold I handled unexpected situations with aplomb. Seeing this and other correlations between how I felt and how I did made clear to me what an impact my emotional and energetic states have on my performance.
PSL is intended to help people understand their leadership style. I certainly learned bundles about how I function and how I can best help my team. For example, I have long known that I do not generate raw ideas so well. I excel, however, at taking in bunches of raw ideas, mushing them about, and synthesizing them into something which incorporates the best aspects of each of them. I noticed multiple occasions where my synthesis of the abundant raw ideas seemed to be helpful. I also now see many ways I could have helped differently and/or more effectively. There were several situations, for example, where if I had left the immediate problem to my teammates and instead spent some time looking for the big picture, searching out ambiguities, and identifying questions and unknowns (all strengths of mine), I likely could have simplified the problem. Here again the compressed time frame helped me correlate what I did or didn't do with how well or poorly my team performed; it also helped me identify which of my actions tended to be helpful and which tended to not be helpful.
I noticed oodles of times where I reacted or behaved differently than I likely would have a year ago. This excites me because it means all this work I am doing on myself is paying off. PSL helped me see the progress I am making in this area - a fine gift indeed! One example is my default reaction to puzzlement: being quiet and assuming that I will muddle through somehow. PSL helped me see that this is generally not my best option. Not only does asking clarifying questions help me better understand the situation or problem, it may also help other of my teammates realize they don't understand things as well as they thought they did. This is only one assumption of many I made this week, most of which eventually hurt me. I need a way to identify all these assumptions I don't know I am making!
I even noticed myself reacting and behaving differently over the course of the workshop. More than once a key to a problem with which I was struggling came from someone else. Even though I know that generating ideas is not my strong suit, I often became upset with myself for not having come up with the transforming idea on my own. By the end of the seminar I realized that the source of the idea - that it isn't me - doesn't matter, while what I do with that idea very much matters. I was much happier once I worked this out!
Once I accepted these transforming ideas as gifts rather than reasons to put myself down I was better able to use said ideas to help me. One such idea gifted to me at PSL is using a chime to track the passage of time. I tend to become stressed when I perceive time to be short, and I often take awhile to determine how I feel about something. In part because of this I do better when I allow myself time to mull things over than when I make snap decisions. PSL underscored all of this for me. I felt time constrained during most of the activities and often didn't take think time. I noticed that during certain exercises - ones which used a chime to mark the passing of time - that I did take the think time I need. I believe the chime helps me stay out of time by reminding me that time is passing. A paradox, yes - not my only one of the week! This exemplifies another learning I took away from PSL: that there are always steps I can take to be more in control of my environment.
I mentioned earlier that I had expectations that PSL would change my life. All week I waited for PSL to do so and all week I was disappointed. Although I was learning lots, I didn't feel any different, and I wondered when I would. On the last day, however, I realized I was a completely different person! I didn't have the blinding-flash-of-light transport-me-across-the-room experience I was expecting. Instead, I realized I was in a completely different state! I do not know all of the ways in which I am changed, nor do I completely understand all of the changes I have so far discerned. If you notice changes, please let me know!
I went to PSL to learn about myself, and that I did, in spades. If you're up for a heady, scary, and fun week of learning how you can be a more effective leader, I heartily recommend PSL!
*** Want a fun job on a great team? I need a tester! Interested? Let's talk: Michael dot J dot Hunter at microsoft dot com. Great testing and coding skills required.
[To the tune of The Beatles' Octopus' Garden]
I wish that I could be
A tester who sees
Bugs before they happen
Yes I do
I'd point them out
With joy my dev would shout
Because over that one
He wouldn't stew
My PM would say
"Away you must not stray
Because we need you here
Yes we do"
To his office I would hie
And then I would reply
That I felt his request it
Wouldn't do
Then my boss he would arrive
And say that he would drive
Me anywhere I chose to
Have him go
I'd send him down the street
To purchase lots of sweets
And then I'd send him back out
For some pho
I would teach my tester friends
How to reach this same end
Soon there'd be a cadre
Of us all
We would point and point and point
At every bug in the joint
There'd be no hiding from us
We'd have a ball
We would point out bugs all day
For us it would be play
We would send our boss out
For mac and cheese
(For mac and cheese)
Everyone would look and stare
It might be too much for them to bear
For us testers to be
Celebrities
(Celebrities)
I wish that I could be
A tester who sees
Bugs before they happen
Yes I do
I would show you how it's done
We would have such lots of fun
Pointing out all the bugs
In this zoo
We might finally get an office
With a view
Broken programs would become
So very few
The shareholders would all go
"Woo hoo!"
Statues of us famous artists
Would hew
We would have golden toilet paper
In our loo
(Not familiar with PSL? See my trip report!]
*** Want a fun job on a great team? I need a tester! Interested? Let's talk: Michael dot J dot Hunter at microsoft dot com. Great testing and coding skills required.
A year ago I joined a team building a top secret project. Today we threw off our invisibility cloak and revealed ourselves as as Live Mesh. Come on over and browse around. Subscribe to our team blog, where I'll be posting from time to time. Then come back here, for two important reasons:
- We are hiring! Our Test team has immediate openings for great Macintosh, Web, Windows, and Web Services testers who code as well as they test. Contact me if you are interested.
- I have two Live Mesh invites to give away! Leave a comment describing the devious ways in which you'll test Live Mesh. I'll select two responses at random from all comments posted by Midnight PDT on 4 May 2008 and send their authors invites.
If you find a bug, please post it to our forums. I am focusing on our not-yet-released Mac client and so don't know much about anything else! <g/>
*** Want to work on Live Mesh? I need a tester! Interested? Let's talk: Michael dot J dot Hunter at microsoft dot com. Great testing and coding skills required.
These, then, are the five parallels I see between drawing and testing:
.jpg "A portrait of a tongue-studded woman") |
Work what you see, not what you know. |
.jpg "A portrait of a craggy-faced man") |
Copying and deconstructing the work of people you admire can help you identify the works’ parts and understand how they fit together, at which point you can fit them together in your own unique way. |
 |
A strong vocabulary is one base on which your expertise and effectiveness is built. |
.jpg "A rapidly-drawn sketch of a woman") |
Vary the speed and the level of detail at which you work, and each will bring improvements in the other. |
.jpg "A charcoal sketch of a flamenco dancer") |
Looking to other applications, markets, and roles will help you think outside your box. |
And then there is the most important parallel: The utter necessity of having fun!
.jpg "A happy guy")
[See all my art at http://www.freelyoffered.com.]
*** Want a fun job on a great team? I need a tester! Interested? Let's talk: Michael dot J dot Hunter at microsoft dot com. Great testing and coding skills required.
I mentioned earlier that I started learning to draw so that I could sketch houses I saw in my head. When I went out sketching I noticed all sorts of interesting people whom I sketched as well, and I found these subjects to be a fascinating challenge. Now I draw everything! Similarly, although I started drawing in ink, I have also explored pencil and charcoal and various other media. I have experimented with different styles too, including line art, cartoons, and realistic rendering. Each subject, media, and style has a unique set of possibilities and limitations which I find shed new light on the others. For example, cartooning helps me see the essential lines when I draw from life, and life drawing helps my cartoons seem more alive.
I started learning to test so that I could remove defects from the applications I wrote. As I built my programs I noticed all sorts of interesting problems with software written by others as well, and I found this testing to be a fascinating challenge. Now I test everything! Similarly, although I started testing line of business applications, I have also worked on drawing programs and financials and various other types of programs. I have tested software for different types of people too, such as developers, landscape architects, and bankers. Each application, market, and role has a unique set of goals and restrictions which I find shed new light on the others. For example, thinking like a facilities manager gives me ideas for testing financial packages, and thinking like a banker helps me see alternative ways to test a CAD program.
Looking to other applications, markets, and roles will help you think outside your box.
[See all my art at http://www.freelyoffered.com.]
*** Want a fun job on a great team? I need a tester! Interested? Let's talk: Michael dot J dot Hunter at microsoft dot com. Great testing and coding skills required.
As I learned the vocabulary of drawing I realized that most of my drawings were detailed studies where I attempted to record every last detail. I decided to switch to gesture drawings - quick sketches whose only intent is to get the spirit of the pose. After some time of these I opted for a long study, and I was surprised at how much better I had become. Gesture drawings forced me to identify and record the key details quickly, and long studies gave me time to identify and record a multitude of details. Practicing each brought improvements in the other.
As I learned the vocabulary of testing I realized that most of my tests were studied attacks on my applications where I attempted to identify every last defect. I decided to switch to short timeboxed test sessions, where I invented my test cases on the fly. After some time of these I opted for a time of involved prethought, and I was surprised at how many more issues I found. Rapid-fire testing forced me to think quickly, and long planning sessions gave me time to look for holes I might have missed. Practicing each brought improvements in the other.
Vary the speed and the level of detail at which you work, and each will bring improvements in the other.
[See all my art at http://www.freelyoffered.com.]
*** Want a fun job on a great team? I need a tester! Interested? Let's talk: Michael dot J dot Hunter at microsoft dot com. Great testing and coding skills required.
As I copied and deconstructed art by different people I started to identify commonalities within and across their works. Some of these commonalities were explicitly gathered into visual dictionaries, such as Jack Hamm’s Cartooning The Head & Figure. Even though I didn’t understand exactly how everything worked, by simply combining these pieces parts I was able to create drawings which pleased me. As I became familiar with the vocabulary I started improvising on it, and I became even more pleased with my art.
As I copied and deconstructed the way other testers worked I started to identify commonalities within and across their techniques. Some of these commonalities were explicitly gathered into testing concordances, such as Cem Kaner et al’s Testing Computer Software. Even though I didn’t understand exactly how everything worked, by simply combining these pieces parts I was able to create tests which found important problems. As I became familiar with the vocabulary I started improvising on it, and my tests found even more important problems.
A strong vocabulary is one base on which your expertise and effectiveness is built.
[Facial features copied from Cartooning The Head & Figure, by Jack Hamm. See all my art at http://www.freelyoffered.com.]
*** Want a fun job on a great team? I need a tester! Interested? Let's talk: Michael dot J dot Hunter at microsoft dot com. Great testing and coding skills required.
One technique which has helped me learn to see is to copy and deconstruct the works of artists I admire. My initial copies could hardly be called such. As I continued to inspect other artists’ work, however, I started to understand how the bits and pieces worked individually as well as how they fit together to make the whole. And then I increasingly understood how to build on them to make my own whole.
I used this technique to learn about testing too. My initial tests could hardly be called such. As I watched other testers test and my customers use my applications, however, I started to understand the techniques they used and the things they did individually as well as how they combined these techniques and actions into more involved tests and workflows. And then I increasingly understood how to build on their techniques and actions to create my own tests and scenarios.
Copying and deconstructing the work of people you admire can help you identify the works’ parts and understand how they fit together, at which point you can fit them together in your own unique way.
[Photo from http://www.3d.sk. See all my art at http://www.freelyoffered.com.]
*** Want a fun job on a great team? I need a tester! Interested? Let's talk: Michael dot J dot Hunter at microsoft dot com. Great testing and coding skills required.
What I have had most difficulty learning about drawing is probably also the most important point for me to learn: to draw what I see rather than what I know is there. Many years of experience tell me, for example, that a person’s shoulders are always the same width, regardless of how the person is turned from me. In actuality, however, the angle at which the person is turned affects the length their shoulders appear to be.
What I know is there are shoulders and arms and legs; what I see are lines and curves and shadows. When I draw what I know my mind gets in the way of recording reality; when I draw what I see my mind helps record it.
What I had most difficulty learning about testing was probably also the most important point for me to learn: to test what I see rather than what I know is there. Many years of experience tell me, for example, that all developers tend to make the same mistakes. In actuality, however, developers are different from each other and make different mistakes from each other, and they make different mistakes on each application they write.
What I know is there are boundary errors and duplicated hot keys and failures to validate input; what I see are issues which prevent my customers from doing what they want to do. When I test what I know my mind gets in the way of finding the most important issues; when I test what I see my mind helps find them.
Work what you see, not what you know.
[Original photo reference from http://www.3d.sk. See all my art at http://www.freelyoffered.com.]
*** Want a fun job on a great team? I need a tester! Interested? Let's talk: Michael dot J dot Hunter at microsoft dot com. Great testing and coding skills required.
Two years ago I was your typical product of public school art class. Which is to say I didn’t draw all that well.
Ten years ago I was your typical taught-myself-programming developer. Which is to say I didn’t test all that well.
.jpg "A much better drawing of a church")
I decided to teach myself to draw because I had all these house designs which I wanted to get out of my head and onto paper. I approached learning to draw the way I approach learning anything: I read gobs of books about drawing, and I practiced and practiced and practiced. My first drawings weren't anything to write home about. Eventually, however, I started creating drawings which I thought were worth saving.
I decided to teach myself to test because I wanted to find problems in the applications I wrote before they reached my customers. I approached learning to test the way I approach learning anything: I read gobs of books about testing, and I practiced and practiced and practiced. My first tests didn’t find many problems. Eventually, however, I started finding issues which mattered.
.jpg "A drawing of the Nebraska capital building")
One day, after completing a drawing I was particularly pleased with, I reflected a bit upon the things I had learned. As I did so I discovered many parallels between what I had learned about drawing and what I had learned about testing. Hence this series of posts.
.jpg "A drawing of a cafeteria")
I do not claim to be a great artist. Nor do I claim to be a great tester. One thing I have learned is that being great is not necessary in order to be effective at drawing , nor at testing.
Over the next several posts I will discuss five points I have learned about drawing, and how each of them applies to testing. As I do, look for ways to draw them in to the way you test.
Much of what I am about to say applies to many other endeavors beyond drawing and testing. I make art, and I test software, and so I am drawing parallels between those two pursuits. If you play basketball, or build bridges, or write code, you may find that the parallels I draw apply to those activities as well.
I do not claim these are the only parallels between drawing and testing, or even the most important. They are simply five I thought of as I wrote these posts.
[See all my art at http://www.freelyoffered.com.]
*** Want a fun job on a great team? I need a tester! Interested? Let's talk: Michael dot J dot Hunter at microsoft dot com. Great testing and coding skills required.
Vinayak writes: "I do feel Exploratory Testing has importance in SCRUM-like projects and also scripted testing has its role in specific contexts. I would really like to see your views on how do you think test design, test documentation, and test execution can optimally take place in SCRUM-like situations."
It's not either-or, it's both.
Scripted tests can be useful, especially if they are automated and cover Paths Which Must Always Work. Unit tests tend to be scripted tests, and I imagine that even the most ardent Scripted Tests Are Evil proponent would agree that unit tests are A Good Thing. Especially if you realize and remember that scripted tests will only ever find regressions in your program's behavior, not heretofore unfound bugs. If your scripted tests all pass, that merely means that your application has been trained to pass them. It doesn't (necessarily) mean that your product will pass any other test you might dream up.
Exploratory testing can also be useful, in part precisely because its very nature means your application can't be trained to pass it. I don't see how testers using their brains to identify The One Test Which Seems Most Useful To Run Right Now could ever be a bad thing. Great Testers are always exploratory testing, even when they are supposed to be executing scripted test cases. Their first test may follow the script exactly and ensure that (Presumably) Very Important Path works as expected; after that, however, Great Testers use the script as a starting point for identifying other tests which seem interesting.
By the way, I don't know how to write a scripted test case without doing exploratory testing as I write it.
As for how testing changes when you're doing Scrum, I don't see that using Scrum, or any other program management methodology for that matter, changes The Right Way To Do Testing. Mostly because there is no Right Way! I find that every situation is different and requires a different solution. My current process goes like this:
- Get a feel for the problem however seems to make sense: read specifications, poke around the application for a bit, talk with the developers, talk with whomever tested it last time, and so on.
- Formulate a plan. I almost certainly don't know enough at this point to create The Perfect Plan, so I don't attempt to do so. Instead I build a plan that seems Mostly Good Enough, For Now Anyways.
- Start executing on the plan. As I do this I will likely discover that portions of it are suboptimal.
- Go to Step 2.
Note that this process never reaches The Perfect Plan! Even if I did somehow manage to construct one that is exactly right today, by tomorrow something will have changed and my plan will no longer be perfect. My plan enables me to find issues today, so it is plenty good enough.
Let me know how this process works for you!
*** Want a fun job on a great team? I need a tester! Interested? Let's talk: Michael dot J dot Hunter at microsoft dot com. Great testing and coding skills required.
I find that this Problem Solving Heuristic works when nothing else does.
Situate yourself such that you can easily move about in a circle. Stand, sit on a chair with wheels, grab your kidling's spin toy, whatever.
Now rotate yourself clockwise. If moving clockwise feels wrong, move counter clockwise. If that feels wrong as well, switch back and forth. Move at whatever speed you like - fast is fun, slow is swell, and happy mediums are, well, happy! Different speed have different effects. Experiment and discover what works best for you.
Once you've started, continue spinning until you fall down or are otherwise forced to stop.
At this point something will pop into your head. Meditate on this something however seems appropriate, whether that be dancing, dunking baskets, singing, screaming, painting, producing, assuming the lotus position, assuming the daydreaming position...use your personal route to reverie. Continue meditating until you are done.
This game has never failed to bring me results. Let me know how it works for you!
*** Want a fun job on a great team? I need a tester! Interested? Let's talk: Michael dot J dot Hunter at microsoft dot com. Great testing and coding skills required.
Have you ever reacted to something you noticed by saying "Bizarre!" or "Weird!" or "How strange"? Example: "Hey, Michael's post is tiny this week. Weird!"
Have you ever investigated to learn why what you noticed occurred? Or did you make an assumption about its provenance? Or did you drop it and move on?
The next time you notice yourself reacting in this way, reframe your response and say "How interesting that <what you observed>. I wonder why <that happens>." "How interesting that Michael posted a minute blog entry this week. I wonder why he didn't write something longer."
Now think of five explanations for what you noticed. Then think of five more. And five more. And five more. When you can't think of any more, come up with five more. He didn't have anything interesting to say this week. He self-censored everything he *really* wanted to say. He was pressed for time and didn't have time to write more. He didn't really have time to write anything this week yet didn't want to disappoint his loyal readers. He paid someone to write his blog entry this week and they didn't do a very good job. Michael usually ghostwrites for someone else, and that person is on vacation, and so he had to do it all himself this week. . . .
Once you have really and truly hit the bottom of your imagination barrel (hint: the first three times you think you've hit it, you haven't yet), review your list and pick the explanation you believe is most likely to be correct. Then talk with the person whose action engendered all this activity and Check It Out!
*** Want a fun job on a great team? I need a tester! Interested? Let's talk: Michael dot J dot Hunter at microsoft dot com. Great testing and coding skills required.
Here's a puzzle for you: I am out camping. One day I mark my spot and then walk one mile due south, at which point I turn ninety degrees to my left and walk one mile due east. I pause to admire the scenery, am startled by a bear, and run one mile due north. At this point I am back to my starting place, seem to have lost the bear, and have lunch.
Where am I camping? What color was the bear?
One answer is "obvious". There are many other correct answers as well. How many can you find? Answers below.
I ask this question because I recently I read two books. Both are titled How To Solve It. Both are encyclopedias of heuristics for solving problems. Both are chock full of examples which demonstrate how to apply the plethoras of heuristics they contain to specific hurdles you may or may not find yourself facing in your daily life. Neither is what I consider light reading. Both made my brain hurt. Both have ideas you can apply to your testing. (Even if you aren't inclined to wade through the math!)
The first How To Solve It was written by George Polya decades ago. The first few pages present his oh-so-simple problem-solving algorithm:
- Understand the problem.
- Devise a plan.
- Carry out the plan.
- Look back and validate your answer.
The remaining pages use a series of math problems to explain how to apply this procedure. Regardless of whether you work through the problems or skip them, be sure you understand why George takes the approaches he takes, and consider how to apply them to your testing.
The second How To Solve It was written by Zbigniew Michalewicz and David B. Fogel a few years ago. Their version is effectively a sequel to Polya, presenting and explaining a variety of often-used-today algorithms and heuristics. They note that, despite the massive computing power we now have at our disposal, solving real-world problems is as difficult today as it was when Polya wrote his tome. Solution spaces are still often too large to exhaustively search, and the person solving the problem may still be inadequately prepared to do so.
I won't attempt to summarize the many problem-solving algorithms Zbigniew and David present. I do however want to highlight two points they stress:
- Most problems have more than one solution. Furthermore, the first solution you find isn't necessarily the best answer.
- Thinking about your problem is important; actually working on the problem is important too. The best testers I know often seem to spend more time pondering how to test their product than they do actually testing it. Once they start testing, however, they find flurries of important issues fast. As they do so they use the information they learn to change their approach and triangulate in on a more effective process. (One reason I like Session Based Test Management is that it institutionalizes this think-do-think-do cycle.)
If you find math fun, these books are stuffed to bursting with interesting problems for your puzzle-solving pleasure. If balancing your checkbook is math enough for you, you can skip the gory details and still learn bunches. If you'd rather bypass even the task of searching out pearls of wisdom from masses of math, both George and Z+D summarize the important bits (George first thing, Z+D last thing).
So, d'ya have a solution to my riddle yet? The canonical answer is that I must be camping at the North Pole, because that is the only location where walking one mile south, one mile east, and one mile north would return me to my starting point. Thus the bear must have been a polar bear, thus it must have been white.
This is not however the only solution. I could be camping one mile north of a parallel on the Southern Hemisphere which happens to have a circumference of exactly one mile. So I could walk one mile south, walk one mile east (completely around the globe), and follow my footsteps one mile north back to my campsite.
I could also be camping one mile north of a parallel on the Southern Hemisphere which happens to have a circumference of exactly one-half mile. So I could walk one mile south, walk one mile east (circumnavigating the Earth twice), and follow my footsteps one mile north back to my campsite.
Do you see how there are an infinite number of possible locations for my tent?
The bear could be any color imaginable as well - as there aren't any bears in Antarctica, I must have hallucinated it!
*** Want a fun job on a great team? I need a tester! Interested? Let's talk: Michael dot J dot Hunter at microsoft dot com. Great testing and coding skills required.
If you are looking to attend a testing conference this year, consider CAST 2008. CAST is not your typical conference. While each session does start with you sitting in rows staring at PowerPoints, each session ends with you discussing anything and everything with the presenter and other attendees, all enabled by a trained facilitator. Sessions are separated by generous buffers of time, so energetic discussions can continue and continue and continue. Imagine a that - a conference where the point is to confer!
If you are looking to present at a testing conference this year, consider CAST 2008. Check out the Call For Papers, then submit your proposal outline by 4 February 2008. That's only a week and a weekend away, so get cracking!
Dunno yet whether I'll be attending. However, Michael Bolton will be. Matthew Heusser told me he is going. Pradeep Soundararajan is flying all the way from India to participate. Jerry Weinberg is keynoting. And that's only the tip of the iceberg!
While CAST will be a lot of adjectives, "boring" isn't one of them!