Scott Oseychik : repro

Help me help you!

Scott Oseychik — Wed, 19 Nov 2008 08:43:00 GMT

While nobody has won the $500 bounty I placed on a set of repro steps (for a nasty bug that I’m still chasing down), that particular blog post sparked a bunch of interest. Sure, one could attribute it to the price tag I had on the bug’s head, but you’d be amazed at the amount of responses along the lines of “while I don’t have the answer, this piece of info might be helpful.”

Which got me thinking about a few things ...

Judging from both the quantity & quality of responses I received, there’s genuine interest not only in the OCS community, but the Microsoft community as a whole in terms of providing quick, relevant feedback. While we have structured programs in place that recognize these contributions (MVP for example), perhaps there’s a more “informal” approach that doesn’t have any barriers to entry other than providing relevant feedback via a blog form. Something as simple as “Hey, I’m hitting this same issue, too” goes a long way for me in terms of driving compelling discussions with our Developers, Testers, and Program Managers as for why they should fix a particular bug.

While I’m probably the last to the crowdsourcing party (but perhaps the first from Microsoft), I’ve become increasingly fascinated by the “Power of the Crowd.” But wait Scott: don’t you debug code? Yes, and I recognize this is in stark contrast to how programs are debugged. Usually, it’s mono e mono. Insert Don LaFontaine/Pablo Francisco voice here ... “One man, one debugger.” Aside: I’ve opened for Pablo; he’s out of his mind, but a very cool person and it was a blast watching him wreck a room (and an honor sharing the stage with him). Back to my point ... while I still haven’t connected the dots in terms of how I can combine the two (collective crowd knowledge & debugging), my gut tells me there’s value in exploring this further. Feedback encouraged.

There’s a huge emphasis on an initiative within Microsoft Customer Support Services dubbed “Front of the Funnel” work. This is essentially work performed by support engineering staff that is customer-facing, and equips the customer with the knowledge needed to resolve their issue (without having to pickup the phone & call Microsoft for help). Think blogs, whitepapers, KBs, Webcasts, Online Crash Analysis, etc. Along these lines, I’m going to make a concerted effort to write-up post-mortem reports on issues we’ve resolved, and get these posted to our blogs. With the exception of hotfix-specific Knowledge Base articles, KB articles are pretty much dead. And publishing these “resolution posts” via blogs cuts right through the red tape ... so it only makes sense. If I can post the resolution & save you a phone call to MSFT / support incident, we both win.

I guess my point is that I learned a lot from my “Wanna win $500” post; way more than I ever expected! So don’t be surprised if you start seeing more “request for repro steps” via my blog, and the more you give me, the more I’ll give back. Ultimately, I’d hope this trend would catch on not only within my team, group, and org, but across all of Microsoft Customer Support Services. But that’s the optimist in me.

Wanna win $500?

Scott Oseychik — Wed, 17 Sep 2008 01:08:00 GMT

I need your help getting an issue to reproduce consistently, and the first person to give me verifiable repro steps will win *$500.00 (USD)!

While this problem has reared its ugly head sporadically, I can't get a consistent repro, and therefore, have little to go on in terms of debugging. However, more and more customers are reporting this issue, and I'm quite sure we have a bug somewhere. But this son of a beesting is quite the elusive one, and I'm turning to the OCS community at large for help!

Issue/Symptoms:

When attempting to shut down Windows (XP SP2, XP SP3, Vista, or Vista SP1), Microsoft Office Communicator 2007 will occasionally hang on shutdown, and the user will be prompted with the following dialog box:

End Program - communicator.exe

This program is not responding. To return to windows and check the status of the program click cancel. If you choose to end the program immediately, you will lose any unsaved data. To end the program click END NOW or the other option is to CANCEL.

Environment:

· Office 2003 SP3 (or Office 2007 SP1)

· Outlook running in Cached Mode against Exchange Server 2007

· Office Communicator 2007 (builds 2.0.6362.0 - 2.0.6362.64)

· Outlook is set as the Personal Information Manager in Communicator 2007

· "Save my instant message conversations" checkbox can be either enabled or disabled

Notes:

This problem typically manifests itself after Outlook has been closed, yet Communicator is running (then the user shuts down Windows)

Contest Details:

If you can reproduce this error 100% of the time, and can furnish detailed steps (with an accompanying video demonstrating the repro) that will enable me to build out a lab topology causing this issue to reproduce consistently in-house here at Microsoft, I will personally pay the person who provided the repro steps $500 USD out of my own pocket. The key here is so long as I'm able to reproduce it 100% of the time based of the steps you provided me, I will declare you the winner, and will gladly pay you your winnings.

*Please note that this is not a Microsoft-sponsored contest nor sanctioned event. However, the self-inflicted price tag is worth it, as I've already burned way too many hours (both my own, and my customers) trying to get this issue to manifest itself consistently. Oh, and Microsoft employees (that is, full-time employees, contingent staff, and their respective families) are considered ineligible for this contest.

UPDATE: We finally fixed this! I'll create a new post shortly with full details ...

The value of repro steps and why you should care

Scott Oseychik — Fri, 22 Aug 2008 16:28:00 GMT

I'm gonna skip right over the Tier 1 & Tier 2 troubleshooting stuff, and get into the meat. When an issue lands on my plate, more often than not, it means I have to debug something (and as dry as that sounds, I truly enjoy my job). On top of that, 99% of the time, it's code I've never seen before. So in order for me to isolate where the problem lies, I need a [fairly] consistent path of execution that I can focus on. Otherwise, I'm performing the debugging equivalent of a car salesman randomly trolling the lot looking to fill his monthly quota.

The ability to reproduce the bug not only allows me to narrow my scope and get to the bug quicker, but it gives the Test team concrete steps to use to validate the bug was fixed properly. Oh, and let's not forget the reason we're all here: resolving the customer's issue.

Here's a small example of "Good Repro Steps" (which include both Actual and Expected Results):

1. Start > All Programs > SomeBuggyApp

2. From the File menu, select, Add New ..., then select Participant

3. Enter an e-mail address that contains an underscore (for example: doogalis_bigglesworth@thethird.com)

Actual Results:

SomeBuggyApp will crash, and the following event will be logged in the Application Log:

"Event ID 0; Something went crazy wrong here."

Expected Results:

As per RFC, SomeBuggyApp should properly handle e-mail addresses containing underscores

And now for the "Horrible Repro Steps" ...

Launch SomeBuggyApp and it'll crash

I guess what I'm asking for is that when bringing me an issue, you (the Customer, Engineer, Dev, PM, etc.) supply very specific steps to reproduce the exact problem you're having. I know it sounds like an obvious request, but there are many misconceptions around what we can and cannot do. One thing I definitely can't do is fix a bug I can't get to.

I've worked with many silly geese in my career, and along the lines of this post, here are some of my favorite quotes from 'em:

Silly Goose #1: "Can't you just attach a debugger and fix my problem?"

No, and unfortunately, it would take too long for me to explain why not. Let me put in terms we all understand: you've got a better chance of taking down Chuck Norris.

Silly Goose #2: "How about just reading the code and figuring it out that way?"

This approach might fly if all we shipped was HelloWorld.exe, but more often than not, we tend to have more than five lines of code in the products we ship.

Silly Goose #3: "It crashed, and I gave you a dump file. What more do you want from me?!"

Give me your tears, gypsy. Actually, the crash dump file is quite good, and you gave me a starting point. But if this is going to get fixed, we need to know the series of events that lead up to this.

Silly Goose #4: "But I even gave you a Time Travel Trace file! C'mon ... that should have everything you need!!"

Sure, it might have the info I'm after, but without any breakpoints, background, or context to at least give me a starting point, your expectations are ridiculous. At a minimum, give me SOMETHING from the trace logs, eventvwr, etc. Without data to key off of, you're accomplishing two things: frustrating me *and* getting your name added to my whiteboard's "Noob Blacklist."