notes and rants

My blog is dead – long live my blog

alanpa — Mon, 02 Nov 2009 20:00:23 GMT

I started blogging about five years ago while I was working on Windows CE. I started the blog as a place to share my conference materials, but it quickly turned into something else. Soon, I was writing this blog to work on my writing skills – practicing many of the tips and techniques I read about effective writing.

Today, I have half a dozen magazine articles under my belt, 300 pages of a (400 page) testing book, contributed a chapter in Beautiful Testing, and a foreword to JW’s latest (and have more secret projects on the horizon).

I started my blog here because it was an easy place to start a blog. For a bunch of reasons I won’t list today (some of which I’m sure will emerge in the coming months), this is the final post of my notes and rants blog.

Bye.

But I’m not done blogging. My new blog is here. Sorry – you’ll have to update your RSS reader to point to the new feed location too.

As far as I know, the content here will remain. I took the liberty of migrating a few of my more popular posts to the new location, but if you have personal favorites from my past, just let me know in the comment section and I’ll transfer them.

PNSQC Slides

alanpa — Tue, 27 Oct 2009 20:10:23 GMT

Thanks to everyone who attended my talk today. Feel free to fire more questions here if you have them.

Slides are available here.

I’m overflowing

alanpa — Thu, 22 Oct 2009 04:25:14 GMT

…literally, I suppose. It’s been a hectic few weeks for me, but some big announcements may be on the horizon…

In the meantime, I strongly encourage you to check out http://testing.stackexchange.com. The cool kids among my readers will know that stackexchange is based on the stackoverflow.com site (and that I love, love, love stackoverflow).

But my love probably isn’t enough to convince you, so I’ll give you two good reasons why I think it’s so cool.

1. First of all, unlike “forum” sites where you have to figure out which forum to ask your question in (or which one to look for answers in), stackexchange discussions are flat – but are sortable in an infinite number of ways through the tags attached to the discussions. For example, over on stackoverflow, this link gives me every post about quality AND testing.

2. Points and badges are cool. Why do world of warcraft players grind on zombies for 6 hours straight? Because they want to get the cool purple hat! As you participate more on stackoverflow/stackexhange, you get points and badges. The badges are purely for bragging rights, but as you earn points, you get more privileges on the site (for the gamers in the crowd, think of it as leveling up).

Check it out – answer a question, or ask a question today.

HWTSAM in China

alanpa — Tue, 20 Oct 2009 00:48:03 GMT

I’m excited to see that the Chinese version of HWTSAM is out (and soon to be followed by a Korean translation).

You can find information here:

http://www.china-pub.com/196002

and on Amazon.

One last virtualization post

alanpa — Sat, 17 Oct 2009 21:38:10 GMT

We (Microsoft, M$ or however you’d like to refer to us) are trying to gather some data on how testers are using virtualization – specifically in the area of lab management of virtual machines.

We’re working with independent analyst firm, voke, to conduct a market study on virtual lab management. If you have a few minutes, tell them what you think about virtual lab management: https://www.surveymk.com//s.aspx?sm=LbWaKloP4hT1PNBIcx834w_3d_3d. All of the data captured is confidential, and information about receiving a complimentary copy of the report based on the survey is available at the end of the survey.

This has been a public service announcement – help out if you can.

My STAR talk

alanpa — Thu, 08 Oct 2009 19:21:23 GMT

My talk’s over – I’ve received a lot of positive feedback (although the people who hate my talks never track me down).

I did have fun though, so at least one person in the room had a great time. My slides (which are nothing like the slides I submitted) are here if you’d like to take a look.

Plane Thoughts and Plain Books

alanpa — Tue, 06 Oct 2009 02:01:00 GMT

I’m somewhere over California on my way to Orange County, home of movie stars, beaches, Disneyland, and a wee bit of the Microsoft mother ship. The flight is half full, and I’m using the increased elbow room to clean out some old email, work on my presentations for the week and to try and figure out if I like the latest Jakob Dylan album or not.

One of the other authors of hwtsam will also be at STAR. Ken Johnston is giving a tutorial and a talk. If you can’t tell from Ken’s writing, he can get a bit excited. So excited in fact, that he’s practically bubbling over at the opportunity to be part of a book signing on Thursday at 10:45am. Of course, I know better about these sorts of things, and don’t really expect anyone to actually show up to have a book they just paid retail price for devalued from potential ebay profits with some scribbling from a couple of writing hacks from a big software company.

But I have a special deal for readers of the blog who find their way to the land of Mickey Mouse this week. If you can track me down any time (outside of my talk Wednesday afternoon) and introduce yourself, you can have one of the two unblemished copies I threw into the vacant space in my bag this morning. Better yet, I promise not to sign it just in case you want to sell it for profit and get a better deal. If you’d like to increase your odds by handing me a beer (preferably cold and full), your odds will increase. Depending on Disney-net access, I’ll update the post when the book is gone.

The other book goes to the best question during my talk. Winner is karentobo - thanks for the support Karen.

Why bugs don’t get fixed

alanpa — Wed, 30 Sep 2009 20:11:00 GMT

I’ve run into more and more people lately who are astounded that software ships with known bugs. I’m frightened that many of these people are software testers and should know better. First, read this “old” (but good) article from Eric Sink. I doubt I have much to add, but I’ll try.

Many bugs aren’t worth fixing. “What kind of tester are you”, I can hear you shout, “Testers are the champions of quality for the customer!” I’ll repeat myself again (redundantly if I need to …) Many bugs aren’t worth fixing. I’ll tell you why. To fix most bugs, you need to change code. Changing code requires both resources (time), and it introduces risk. It sucks, but it’s true. Sometimes, the risk and investment just aren’t worth it, so bugs don’t get fixed.

The decision to fix or not to fix isn’t (or shouldn’t be) entirely hunch based. I like using the concept of user pain to help make this decision. There are 3 key factors I consider to determine user pain. These are:

Severity – what’s the impact of the bug – does it crash the program? Does the customer lose data? Or is it less severe? Is there an easy workaround? Is it just a cosmetic issue?
Frequency – how often will users hit this issue? Is it part of the main flow of the program, or is the issue hidden in an obscure feature. Minor issues in mainline scenarios may need to be fixed, but ugly stuff in an obscure feature may slide.
Customers Impacted – if you’ve done your work up front, you have an idea of who your customers are, and an idea of how many users are in (or how many you would like to be in) each of your customer segments. From there, you need to determine if the issue will be hit by every user, or just a subset. If you have the ability to track how customers are using your product you can get more accurate data here.

From here, make up a formula. Assign a value scale to each of the above and apply some math – you can do straight addition, multiplication, or add weights based on your application and market. For our purposes, let’s just add and use a 10 pt scale for each bug :}.

Bug #1, for example, is a crashing bug (10pts) in a mainline scenario (10pts) impacting 80% of the customer segment (8pts). At 28pts on the user pain scale, I bet we’re going to fix this one.

Bug #2 is an alignment issue (2pts) in secondary window (2pts) in an area used by a few “legacy” users (2pts). At 6 pts, this is a likely candidate to not get fixed.

Unfortunately, they’re not all that easy. Bug #3 is a data loss bug (10pts). It occurs in one of the main parts of the application, but only under certain circumstances (5pts) (btw – numbers are completely made up and subjective). Customer research shows that it’s hardly ever used (2pts). At 17 pts, this one could go either way. On one hand, it’s probably not worth the investment to fix. As long as the issue is understood, and there are no blind spots, leaving the bug in place is probably the right thing to do.

On the other hand, you have to weigh this with the rest of the bugs in the system. The Broken Window theory applies here – if there are too many of these medium threshold bugs in the app, quality (or at the very least, the perception of quality) will suffer. You need to consider every bug in the system in the context of the rest of the (known) bugs in the system and use this knowledge to figure out where the line is between what gets fixed and what doesn’t get fixed.

It sucks that the industry ships software with known bugs – but given the development tools and languages we have today, there isn’t a sensible alternative.

Edit:

As this sits in my head, I think I've missed a fourth factor in the forumla: Ship Date. The proximity of ship date plays into the fix/don't fix decison as much as the above. I'm not sure, however, whether it's a fourth factor in the math, or if the threshold of what "value" of user pain turns into a bug fix as ship dates approach.

7 Influential Books

alanpa — Fri, 25 Sep 2009 05:18:09 GMT

Phil Kirkham recently posted a list of 7 Influential Books and encouraged some of his fellow bloggers to follow suit.

Of course, Phil’s list overlaps mine, so instead, I’d like to present:

7 Influential Books that Phil doesn’t Mention

The Feiner Points of Leadership – Michael Feiner One of the Technical Fellows at Microsoft recommended this book to me a few years ago, and now it’s one that I re-read often. The book is filled with lessons – and more importantly – practical advice.
Peopleware – Tom DeMarco and Anthony Lister Every person who manages teams or people – or cares about how people are productive should read this book
Testing Object-Oriented Systems – Robert Binder – This is probably the largest book I own, but one of my favorite testing books. I have a love for patterns and models, and this book covers both fantastically
The Dilbert Principle – Scott Adams From an anti-pattern perspective, this is one of the best management books in print. This book covers far more than cartoons – Scott Adams understands management far more than many of the managers I know
The Inner Game of Tennis – Timothy Gallwey – I discovered this book when I was a (sort of) musician. The author wrote a follow up for musicians, but the tennis book is the best. The premise is that performance = (potential – distraction). Most people try to improve only by increasing their potential. Gallwey’s premise is that you can improve your performance by decreasing distractions through focus and visualization.
An Introduction to General Systems Thinking – Gerry Weinberg – This is an easy one to mention – I almost feel like I’m cheating for including it, but I can’t leave it off (and I have to take advantage of Phil picking a different Weinberg book for his list). You’d be surprised how you view things differently when you can think of a system as a whole (this happens, of course, after you can identify a system in the first place). If you can’t do either of those, start with this book, practice, repeat.
The Pragmatic Programmer – when I talk to programmers and testers, this is probably the book I mention the most. The writing is good, the concepts are good, and it’s full of great advice. If you aren’t fluent with boiled frogs, broken windows and tracer bullets, you have to read this.

I almost bumped Scott Adams to include Maguire’s Writing Solid Code, but as much as I like it, it just doesn’t come up in conversation as often as the others. For every book on this list, I can name at least a half a dozen people that I know who share my love of the book.

So – that’s my list. What do you think?

Niche Professions

alanpa — Tue, 22 Sep 2009 19:12:00 GMT

I have been thinking about “niche” professions – those that only appeal to a relatively small number of people in the world. Often, these professions are looked down upon by those in similar professions – or by those who don’t know much about the profession in the first place (one example would be the veterinarian who takes care of my dog vs. the guy who has to inseminate an elephant.)

The advantage of being in a niche profession is that there’s plenty of room at the top. The problem, of course, is that a ton of dolts and wannabees self-proclaim themselves as experts and leaders and assume they’re at the top. They come up with new ways to describe what they do (e.g. sanitation engineer), and tell everyone who uses the old terminology that they’re idiots. Meanwhile, the real experts are the ones usually advancing the profession. The connected world we live in enables the loudest voices to be "experts" regardless of ability to do the job.

Interesting world we live in – isn’t it?

Fun with Code and TestApi

alanpa — Mon, 21 Sep 2009 16:59:29 GMT

I took an opportunity to write some code last week. I’ll spare you the full background – short story is that the school my kids go to needed a small web app written. It ended up being 100 lines or so of javascript and about 3 lines of html. The app works, and will save the school hour and hours of time (compared to their previous manual approach).

I was blown away by how much fun I had writing it. Although I try to stay sharp, I just don’t get to write much code these days. It “only” took a few hours to write – it would have been quicker if I knew what I was doing - but it the learning was half the fun.I forgot how much fun it is to start with an idea and make it work. I realized that I really miss and think I better look for more opportunities soon just to keep myself sane.

Sadly, I don’t get to do much testing these days either, but I have been taking some time to play with TestApi (a library of test automation helpers on codeplex.com). TestApi releases incremental updates every few months, and is slowly turning into something quite useful. What I love about it most is that it’s super-lightweight. There’s no install – just put the files somewhere, add a reference to testapi, add a using statement or two, and you’re ready to go!

Now I just need to find something to test with it…maybe a web app some hack put together for my kids school would be a good bet.

Who Owns Quality?

alanpa — Fri, 18 Sep 2009 09:29:19 GMT

On request from Adam Goucher – another excerpt from How We Test Software at Microsoft. BTW – Adam wrote a review of HWTSAM here – although Linda Wilkinson beat him to the clever title.

This is from a section on quality in chapter 16. It’s something I believe strongly in and would love to hear your comments.

Many years ago when I would ask the question, “who owns quality,” the answer would nearly always be “The test team owns quality.” Today, when I ask this question, the answer is customarily “Everyone owns quality.” While this may be a better answer to some, W. Mark Manduke of SEI has written: “When quality is declared to be everyone’s responsibility, no one is truly designated to be responsible for it, and quality issues fade into the chaos of the crisis du jour.” He concluded that “…when management truly commits to a quality culture, everyone will, indeed, be responsible for quality.”[1] A system where everyone truly owns quality requires a culture of quality. Without such a culture, all teams will make sacrifices against quality. Development teams may skip code reviews to save time, program management may cut corners on a specification, or fudge a definition of “done”, and test teams may change their goals on test pass or coverage rates deep in the product cycle. Despite many efforts to put quality assurance processes into place, it is a common practice among engineering teams to make exceptions in quality practices to meet deadlines or other goals. While it’s certainly important to be flexible in order to meet ship dates or other deadlines, quality often suffers because of a lack of a true quality owner.

Entire test teams may own facets of quality assurance, but they are rarely in the best position to champion or influence the adoption of a quality culture. Senior managers could be the quality champion, but their focus is justly on the business of managing the team, shipping the product, and running a successful business. While they may have quality goals in mind, they are rarely the champion for a culture of quality. Management leadership teams (typically the organization leaders of Development, Test, and Program Management) bear the weight of quality ownership for most teams. These leaders own and drive the engineering processes for the team, and are in the prime organizational position for evaluating, assessing, and implementing quality based engineering practices. Unfortunately, it seems that quality software and quality software engineering practices are rarely their chief concerns throughout any product engineering cycle.

Senior management support for a quality culture isn’t entirely enough. In a quality culture, every employee can have an impact on quality. Many of the most important quality improvements in manufacturing have come from suggestions by the workers. In the auto industry, for example, the average Japanese autoworker provides 28 suggestions per year, and 80% of those suggestions are implemented[2].

Ideally within Microsoft engineers from all disciplines are making suggestions to improve quality. Where a team does not have a culture of quality, the suggestions are few and precious few of those suggestions are implemented. Cultural apathy for quality will then lead to other challenges with passion and commitment among team members.

[1] STQE Magazine. Nov/Dec 2003 (Vol. 5, Issue 6)

[2] The Visionary Leader, Wall, Solum, and Sobul

Looking for the “L” word

alanpa — Mon, 14 Sep 2009 08:03:35 GMT

Although my anniversary is coming up, this post is about another “L” word: Leadership. “Senior” people at Microsoft (and I assume at other companies) are expected to exhibit it. This is true for managers and non-managers alike (because, of course, management and leadership are completely different things). The problem is that Leadership is not as easy to find as some people think. I don’t believe that Leadership is something you’re born with – it’s something that you can learn – as long as you work very hard at it. It’s about building credibility and trust. It’s about moving people forward toward a vision (and having that vision in the first place). Good leaders inspire and make those around them better – they worry about reaching the goal much more than individual accomplishments.

Yet, I often see people try to become leaders in ways I can never imagie working. Some of the people I see are:

The Boss – “The Boss” determines that leadership involves a lot of telling people what to do. They are often self-proclaimed experts and use every opportunity they can find to tell people how much they know.
The Demander - “The Demander” expects that people will follow them because of their position or title. They don’t see the need to earn respect or establish credibility. They simply want to be known as a leader without actually leading.
The Micro-Leader - “The Micro-Leader” is like the classic micro-manager. They immerse themselves deeply into any project they can get involved in. They don’t do much, but they often claim they have.

I’m sure there are other ways to “claim” leadership. The only way I know is to work at it. It’s something I consciously work at every day, but an area where I barely know anything. But I keep working at it and keep trying to get better. I doubt I’ll ever figure it out – but I think that’s a good thing.

Flashback

alanpa — Fri, 11 Sep 2009 23:51:30 GMT

I had a bit of a surreal experience this week. One of the groups I work with sent some mail asking me to review an internal report. They noticed an issue and had made some changes. I checked out the report, but it was incorrect. I let them know and got back to work. A few minutes later, I received more email asking me to try again. I did – the report was still broken. This could have gone on forever, but I just went home instead. Eventually they asked someone else, and after a few more iterations, the reports were fixed.

How can any self respecting developer care so little about quality? I was just starting to think that the days of throwing code “over the wall” to testers were long gone. Hey coder-boy - as a tester (or a user), it’s not my job to make sure your basic functionality works – that’s your job. I’m embarrassed for you that you expect me to do this.

I don’t know who reads this blog, but if your job is to “make stuff”, then make sure your stuff works before you give it to a tester (or a user). If you’re a tester, stop accepting broken code from your counterparts. It’s not that hard of a concept…or at least I hope not…

Organizing my Life

alanpa — Tue, 08 Sep 2009 21:55:04 GMT

aka – The secrets of my so called success, part 23.

It’s Tuesday after a long weekend, and my todo list is long. The good news is that I know exactly how long it is, since (for better or for worse), I let outlook drive my daily schedule. Since I’ve mentioned my todo list before, I thought I’d share how I get things done.

If I’m really going to use outlook to help me organize my work, I need to get outlook to make it easy for me. The first things I do is to try to keep my inbox at (or near) zero items. Right now, my inbox has 4 items, but I’ll clean it out when I’m done with this post. I give myself a chance to keep it at zero by re-routing any mail that doesn’t have my name on the to or cc line to another folder (excluding mail sent to my team, or from my boss). I’ve used this system for five or six years now, and it’s worked well for me. The stuff not in my inbox is the stuff I read either at night, or during awkward breaks in the work day.

Of course, restricting the items that make it to the inbox is half the battle – the rest is dealing with the items that do make it through. I am a fan of the “do it or defer it” approach, and I use it daily. When I’m going through my mail, I try to either reply immediately, or – if the reply requires additional research or thought than time will allow, I move it to my todo list. The move to the todo list is literal – I create a new task in the outlook tasklist, add the current mail as an attachment to that task, then delete the item from the inbox (of course, it would be silly for me to manually perform a repetitive non-thinking task such as this, so I wrote a vba macro in outlook to do it for me).

Here’s what the list looks like now. Don’t ask me for details – the list only has to make sense to me.

Then, I just work through the list in priority order. I treat the list like my product backlog, and go through it daily to either accomplish the items in the list, or defer them to another day. I use it for recurring reminders (e.g. I have a reminder to myself to moderate the MSDN testing forums). If I’m in a meeting and someone asks me to do something, rather than ask for them to send me a reminder in email, I just write it down and get it done. It’s a system that works for me

Of course, you need to do whatever works for you – this is my system, and it works for me. ymmv