As you may know, I recently gave a presentation on Testing in Production at TestBash 2.0 . The videos from this event will be available in 1-2 weeks, and I will post the link here when available. In my talk I discussed how high volume unstructured data streams can be used by software professionals to...

TestBash 2.0 will be Friday March 22, 2013 in Brighton England Sure you get to hear me speak about Testing in Production (TiP) and Data-Driven Quality Assessment, but there is an all-star line-up of speakers – a full day’s worth so you’ll get your money’s worth. A...

  If you are interested in hearing my popular STPCon talk ( last delivered in October ) A to Z Testing in Production: Industry Leading Techniques to Leverage Big Data for Quality …then you have one more chance to see it online. Just sign up for the STP Online Summit: Hottest Testing Topics in 2012...

    If you would like to hear the sound of my voice, Mark Tomlinson was kind enough to interview me at STPCon, you can hear that here .

  Monitoring : What Ops does, using data from production + Testing : What Test Does = TestOps , augmenting the signal for assessing quality with Big Data More details on TestOps here Big Data is often defined by the three V’s Volume Velocity Variability But let’s not get hung up on that. Sometimes...

Download the Slide Deck: PPT PDF The folks at STPCon had asked me to give an encore presentation of my March 2012 talk A to Z Testing in Production: TiP Methodologies, Techniques, and Examples . The new talk has some substantial updates including more focus on the use of Big-Data and a simplified categorization...

This ad, recruiting software testers for Microsoft, ran in 1985 As a Software Tester you will design execute and document tests of application software. You will generate test scripts and automatic test packages. This is a challenging and highly visible position within a fast growing division of...

Download the Slide Deck PPT PDF Listen to the audio (requires free registration with STP – they’re good folks, so go ahead and register) A to Z Testing in Production: TiP Methodologies, Techniques, and Examples Testing in production (TiP) is a set of software testing methodologies that utilizes real...

Cool….I am seeing more hits on Bing and other search engines (I comparison shop my queries) for “Testing in Production”.  Here’s one: Why testing in production is a common and costly technical malpractice Yipes. Not the message I’ve been trying to drive, but even within Microsoft, especially among...

Testing in Production (TiP) really happens, and yes that it s a good thing! Here are some examples from Facebook, Amazon, Google, and Microsoft. The Methodologies This is not a complete list of TiP methodologies, but explains the ones used in the examples given A/B Testing (aka Online Controlled...

  It’s About Software Quality Not sure what you were expecting here at Your Software Has Bugs , but ignore that picture on the left, and learn how Exposure Control let’s you roll out dangerous software while minimizing risk to your users by giving them only a peep into the your new service. peep...

No this is not a piece on job security for Software QA Professionals. I was recently at a Test Team sprint-end demo. For at least two of the projects shown, the metric of interest to Dev managements was, “how many bugs did you find?” and when the answer was either a high number or a high severity (“Three...

My talk at the Better Software Conference was a lot of fun. It was well received. If you want to check out the slides, take a look here . Evidence shows than more than half of the ideas that we think will improve the user experience actually fail to do so—and some actually make it worse. Instead of guessing...

While I previously made some comparisons between Amazon.com and Microsoft's different approaches to software testing in Building Services: The Death of Big Up-Front Testing (BUFT)? , I think now would be a fun and interesting time to do a deeper dive on this. Before I joined Amazon.com in 2005 as an...

Testing with Real Users - User Interaction and Beyond, with Online Experimentation Evidence shows than more than half of the ideas that we think will improve the user experience actually fail to do so—and some actually make it worse. Instead of guessing, why not measure what your real users like and...

If I need to defeat a group of men terrorizing a circus family , then Chuck Norris is the man to call. The tough guy, martial artist, action hero has become an internet meme – Chuck Norris Facts attest to the true bad-a**-ness of the one called Chuck. For example one of my favorites is: There is...

Let's start with some definitions: Death Demise, end, termination, the state of no longer existing BUFT "Big Up-Front Testing" an activity or "Big Up-Front Test" a group that carries out that activity (the latter sounds cool, like "Big Pharma") It's an acronym of my own invention. At least I believe...

[Going to try a mini-post…keeping it short and focused] In my previous post I introduced you to Online Experimentation. The general use case for experimentation is to take two or more websites or services, each with different features or designs, and test them against each other using real users....

One topic I hope to return to many times in this blog is that of TiP , or Testing in Production . For a good introduction to this check out Ken Johnston ’s blog entry TIP-ING SERVICES TESTING BLOG #1: THE EXECUTIVE SUMMARY which is an updated version of the executive summary of the ThinkWeek paper that...