Welcome to MSDN Blogs Sign in | Join | Help

The Ultimate Desktop Reference

I have a library of books and white papers on software testing, engineering processes and management, and software development that I have read and reference quite often. For new testers I generally recommend A Practitioner's Guide to Software Test Design by Lee Copeland, and How to Break Software: A Practical Guide to Testing by James Whittaker. There are 5 books I highly recommend (not including How We Test Software at Microsoft which I co-authored and also highly recommend).

In my current role as a teacher, trainer, and mentor of new testers the 2 books that are constantly on my desktop are Testing Object-Oriented Systems: Models, Patterns, and Tools, by Robert V. Binder, and Software Testing Techniques, 2nd edition by Boris Beizer. Not that I don't frequently reference other books, but to me these are the quintessential books on the foundational knowledge of software testing techniques and methodologies for intermediate to advanced testers with a strong technical background.

But, the booklet that I would keep in my shirt pocket if I tested products on a day-to-day basis would be Josh Poley's Black Book. Josh's Black Book is the ultimate desktop reference for software testers (and developers). While this book is primarily intended to aid those who work on projects developed in C/C++, it has loads of information that is valuable to any tester working on just about any technology. From decimal and named entities of ISO characters to error codes for DOS, VB, JScript, HTTP, and of course Windows Errors this book is jammed packed with great information and quick reminders for both developers and testers.

Published Wednesday, December 24, 2008 3:55 AM by I.M.Testy

Comments

# re: The Ultimate Desktop Reference

I would have loved to see some of Jerry Weinberg's books including the most recent one

"Perfect software and other illusions about testing"

1. Introduction to General systems Thinking

2. 3 volumes of "Quality software Management"

The beauty of these books is that first they shatter many myths about testing ... so for someone who is new to this field - Jerry's book would be right beginning... then they can easily understand other software testing books well. For many so called experienced testers, these books can be "eye-openers"

I agree that each of these books are written for some serious reading and are not for day-today tactical,prescriptive recipes (as you find in the books you mentioned especially Beizer and Copeland - how do I do boundary testing? What tool should I use etc).

I my opinion, book on General systems thinking is a must ... if you can read and understand the stuff in that book... you are ready for any real challenges in the world of testing ...

Shrini

Wednesday, December 24, 2008 5:50 AM by Shrini

# re: The Ultimate Desktop Reference

Gerald Weinberg's books are good books; especially for managers. I think his 4 volumes of Quality Software Management are important reads for any anyone in software management. Introduction to General Systems Thinking and similar books on engineering theory are typically required reading in many universities and I agree should be read by anyone coming into the software industry.

Many of Weinberg's books provide interesting philosophical perspectives on engineering paradigms, and his books are excellent at providing alternative insights and inspiring additional thought. But, to me there is a big difference between strategic metaphysical discourses software engineering and the tactical and pragmatic application of software test design and execution.

I suspect the part that you are missing from Beizer, Binder, Copeland, et. el. is that they provide examples of test design based on historical patterns of failures. But, there are three caveats. One is that people can learn from the examples and understand how to apply them in the correct context. Secondly, the examples are based on the presumption that testers have an in-depth understanding of the domain space and the 'system' (meaning their target audience is not business analysts who simply 'test' a product by executing 'tests' from the user interface and a general understanding of how a particular piece of software should work and learning or trying something based on the reaction of the software. (Good test design means actually thinking about the problem space ahead of time, and generally forming a test hypothesis from a macro perspective rather than a micro or myopic view.) Thirdly, it assumes the developers are at least somewhat competent.

By the way, one book by Weinberg (and co-authored by Freedman) that provides real tactical value to testers (and developers and managers) is his book entitled Handbook of Walkthroughs, Inspections, and Technical Reviews. This is important for testers because as Weinberg points out in his book "reviews have proved to be the most cost-effective method in use for error detection and removal." This is one reason why I teach a course on inspections at Microsoft for both testers and developers and why we emphasize hiring testers with strong technical skills and knowledge.

You ask "how do I do boundary testing?" I would recommend that you read chapter 5 in How We Test Software at Microsoft or any of my previous blog posts on the subject. 

As to your question "What tool should I use?" the answer is "your brain!" Rational, logical, and analytical thought is the best tool in a tester's toolbox! Understanding how to design practical tests requires an incredible amount of cognition (mental process of knowing, including aspects such as awareness, perception, reasoning, intuition; knowledge, and judgment) within the appropriate context.

I know that some 'experts' promote the notion of testers sitting around debating such trivialities as whether or not the down arrow in a drop down control is 'correct.' But, I really suspect the majority of software testers in the industry are a bit more pragmatic and realize they provide value to their employer via their cognitive abilities and skills, not by their incessant philosophical diatribes and completely out-of-context 'what-if' scenarios. That is why I suggested this booklet as the ultimate desktop reference for professional software testers.

Wednesday, December 24, 2008 3:25 PM by I.M.Testy

# re: The Ultimate Desktop Reference

>>> Gerald Weinberg's books are good books; especially for managers.

I beg to disagree of the second part. In the contrary they are like learning alphabets for every software engineer (more so for a tester). While managers can really benefit from them, it is required for those newbies. Reading Jerry's books would protect them from being brain washed by lots of testing folklore and other false notions of about software testing. It is like getting vaccinated against common diseases at the time of birth of a new kid.

>>>But, to me there is a big difference between strategic metaphysical discourses software engineering and the tactical and pragmatic application of software test design and execution.

Yes there is a difference. To me, those strategic metaphysical discourses and other philosophical perspectives are like "root" and are as important as foundation to a building. Once you conquer them, understand and assimilate them - you have won more than half of the battle. From that point, a tester will not only be able to quickly understand the tactical and pragmatic application of software test design and execution but also be able to go beyond and be able to design new things and critique them as well. I have personally experience these things so I am writing from that experience. Reading and understanding books like Beizer, Copeland and Binder will be lot more quicker and easier once you can digest the heavy yet basic stuff from Jerry.

>>> I suspect the part that you are missing from Beizer, Binder, Copeland, et. el. is that they provide examples of test design based on historical patterns of failures.

While I do not deny the value of reading books from these folks ... what they teach you "test design" and I am looking first learning from a very high level (general systems thinking in simplistic terms helps one to understand and tame the beast of complexity) at modeling the problem space, understanding interacting parameters, non linearity of the system, sociological interactions, software team dynamics, psychology of software people and many more. Armed with an analysis of this magnitude, test design is a some where deep down the line that I can easily pick up - well I can understand (and critique at times) the concepts better. And, you would know that I am not one of these testers who would view testing only from GUI.

>>> You ask "how do I do boundary testing?" I would recommend that you read chapter 5 in How We Test Software at Microsoft or any of my previous blog posts on the subject.

Yes, I am eager to read your book ... BTW my question of "how do I do boundary testing" was posed with an intention of showing an example where books that you quoted will be helpful (including your book probably).

>>> I know that some 'experts' promote the notion of testers sitting around debating such trivialities as whether or not the down arrow in a drop down control is 'correct.'

While I have not worked with or aware of any such experts who focus "ONLY" on trivialities (real experts will also look at trivialities among other things). You must also agree that in some contexts it might be really matter of serious concern when bug around such trivialities might cost the business millions of dollars. So I would not generalize any thing associated with test observations as "trivial". Moreover, as tester my job never is to pass a judgment about "correct/wrong", "important/trivial" - that would the decision product owners make, I can only highlight the inconsistencies.

>>> their employer via their cognitive abilities and skills, not by their incessant philosophical diatribes and completely out-of-context 'what-if' scenarios.

We have debated this times before on this blog... I must say that you can not see incessant philosophical diatribes as necessarily bad and unwanted things and term them always as "out of context". Thinking about philosophical perspectives, creative modeling of problem space, thinking about "in context" what-if scenarios is very important way to hone and improvise cognitive abilities and skills. You seem to value technical and tactical knowledge, in-depth understand of software systems over Philosophy and other abstract forms of thinking. That is OK ... we are different in our approaches to software testing ...

Shrini

Wednesday, December 24, 2008 6:32 PM by Shrini

# re: The Ultimate Desktop Reference

Shrini,

Weinberg's books are good books. So are books by Maguire, Minsky, and a host of others who discuss various theories and ideas of software engineering. And, I agree that learning theory is important in any field because it helps one think abstractly and approach a problem from pluralistic perspectives.

Strategic thinking and understanding are important. But, you have again taken this post completely out of context and went off on a tangent.

The purpose of this post was not to promote books on strategic thinking such as team dynamics, psychology of software, etc. The purpose of this post was on providing people with a very useful tactical reference.

Unfortunately I cannot attest to your abilities to approach software testing other than from an end user's perspective. But, if you have anything meaningful about the value of, or logical critique of Josh Poley's Black Book I am sure you input would be appreciated.

Wednesday, December 24, 2008 7:49 PM by I.M.Testy
Anonymous comments are disabled
 
Page view tracker