<?xml version="1.0" encoding="UTF-8" ?>
<?xml-stylesheet type="text/xsl" href="http://blogs.msdn.com/utility/FeedStylesheets/rss.xsl" media="screen"?><rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:slash="http://purl.org/rss/1.0/modules/slash/" xmlns:wfw="http://wellformedweb.org/CommentAPI/"><channel><title>Data Tools and Software Testing : General Software Testing</title><link>http://blogs.msdn.com/nihitk/archive/tags/General+Software+Testing/default.aspx</link><description>Tags: General Software Testing</description><dc:language>en-US</dc:language><generator>CommunityServer 2.1 SP1 (Build: 61025.2)</generator><item><title>Spec Explorer 2010</title><link>http://blogs.msdn.com/nihitk/archive/2009/12/02/spec-explorer-2010.aspx</link><pubDate>Wed, 02 Dec 2009 16:51:03 GMT</pubDate><guid isPermaLink="false">91d46819-8472-40ad-a661-2c78acb4018c:9931487</guid><dc:creator>nihitk</dc:creator><slash:comments>0</slash:comments><comments>http://blogs.msdn.com/nihitk/comments/9931487.aspx</comments><wfw:commentRss>http://blogs.msdn.com/nihitk/commentrss.aspx?PostID=9931487</wfw:commentRss><description>&lt;p&gt;I am very excited to see the latest version of &lt;a href="http://msdn.microsoft.com/en-us/devlabs/ee692301.aspx" target="_blank"&gt;Spec Explorer&lt;/a&gt; (which is available even for &lt;a href="http://msdn.microsoft.com/en-us/vstudio/dd582936.aspx" target="_blank"&gt;Visual Studio 2010 Beta 2&lt;/a&gt;) now available publicly for everyone in the world to leverage. It is available via the &lt;a href="http://msdn.microsoft.com/en-us/devlabs/ee692301.aspx" target="_blank"&gt;MSDN DevLabs&lt;/a&gt; page and the Spec Explorer team (the fact that there is a team dedicated to pushing this tool makes me immensely optimistic about it’s future) even has a &lt;a href="http://blogs.msdn.com/SpecExplorer/" target="_blank"&gt;wonderful new MSDN blog&lt;/a&gt; dedicated to Spec Explorer.&lt;/p&gt;  &lt;p&gt;Kudos to the Spec Explorer team and I look forward to wonderful improvements coming from them in the years ahead to help leverage &lt;a href="http://blogs.msdn.com/nihitk/pages/144664.aspx" target="_blank"&gt;Model Based Testing&lt;/a&gt; in a more pervasive way across the software industry.&lt;/p&gt;&lt;img src="http://blogs.msdn.com/aggbug.aspx?PostID=9931487" width="1" height="1"&gt;</description><category domain="http://blogs.msdn.com/nihitk/archive/tags/General+Software+Testing/default.aspx">General Software Testing</category><category domain="http://blogs.msdn.com/nihitk/archive/tags/Model+Based+Testing/default.aspx">Model Based Testing</category></item><item><title>Future of Testing from James Whittaker</title><link>http://blogs.msdn.com/nihitk/archive/2008/12/26/future-of-testing-from-james-whittaker.aspx</link><pubDate>Sat, 27 Dec 2008 02:55:19 GMT</pubDate><guid isPermaLink="false">91d46819-8472-40ad-a661-2c78acb4018c:9253973</guid><dc:creator>nihitk</dc:creator><slash:comments>1</slash:comments><comments>http://blogs.msdn.com/nihitk/comments/9253973.aspx</comments><wfw:commentRss>http://blogs.msdn.com/nihitk/commentrss.aspx?PostID=9253973</wfw:commentRss><description>&lt;p&gt;Found this exceptionally &lt;a href="http://www.utest.com/webinar_james_whittaker.htm" target="_blank"&gt;inspiring, interesting and informative talk&lt;/a&gt; (I saw an internal version of the same talk) by &lt;a href="http://blogs.msdn.com/james_whittaker" target="_blank"&gt;James Whittaker&lt;/a&gt; (please subscribe to his blog if you remotely care about software quality), who is a pioneer in the software testing world. He also has a &lt;a href="http://blogs.msdn.com/james_whittaker/archive/tags/Future+of+testing/default.aspx" target="_blank"&gt;series of blog posts on this topic&lt;/a&gt; in which he covers more details.&lt;/p&gt;  &lt;p&gt;In fact it was so inspiring, that when I tried to save this post in &lt;a href="http://windowslivewriter.spaces.live.com/" target="_blank"&gt;Windows Live Writer&lt;/a&gt;, I got the following dialog + a crash:&lt;/p&gt;  &lt;p&gt;&lt;a href="http://blogs.msdn.com/blogfiles/nihitk/WindowsLiveWriter/FutureofTestingfromJamesWhittaker_D71C/image_2.png"&gt;&lt;img style="border-right: 0px; border-top: 0px; border-left: 0px; border-bottom: 0px" height="225" alt="image" src="http://blogs.msdn.com/blogfiles/nihitk/WindowsLiveWriter/FutureofTestingfromJamesWhittaker_D71C/image_thumb.png" width="332" border="0" /&gt;&lt;/a&gt; &lt;/p&gt;  &lt;p&gt;instead of a simple message followed by a SaveAs dialog which would have been so much better if there was simply no space to save this on disk.&lt;/p&gt;  &lt;p&gt;In defense of the Live Writer test team though there were quite a few &amp;quot;different&amp;quot; things that I was doing and it just makes the same point of how diverse the usage scenarios end up being from our labs.&lt;/p&gt;  &lt;ol&gt;   &lt;li&gt;I have my &amp;quot;My Documents&amp;quot; setup to automatically redirect to a share for backup purposes.&lt;/li&gt;    &lt;li&gt;I also had the blog I was targeting set to a different one instead of my MSDN blog. This is my personal site blog and since I blog&amp;#160; on it from home, I cannot use the externally visible form of &lt;a href="http://www.sitename.com"&gt;www.sitename.com&lt;/a&gt; (since I am behind my router on which my server is also located), I was forced to use the &lt;a href="http://server"&gt;http://server&lt;/a&gt; format for that blog setup. &lt;/li&gt;    &lt;li&gt;I was creating the draft from work, where the &lt;a href="http://server"&gt;http://server&lt;/a&gt; format does not work for my personal blog anyway.&lt;/li&gt;    &lt;li&gt;I tried to save and switch the blogs (from personal to MSDN) at the same time.&lt;/li&gt; &lt;/ol&gt;  &lt;p&gt;Ah, but I shouldn't be surprised. First I used to think that I was an innate tester who just naturally found bugs. But over the years, I have sadly realized that almost all our users are just as gifted as me. Thanks to the quality of our software.&lt;/p&gt;  &lt;p&gt;Just today morning as well I got all excited reading about &lt;a href="http://research.microsoft.com/en-us/um/cambridge/projects/autocollage/" target="_blank"&gt;Microsoft Research AutoCollage&lt;/a&gt; software and took it out for a spin. But at the end when I tried saving and exiting the app it crashed on me. The weird part was that all along, I was &lt;strong&gt;&lt;em&gt;expecting&lt;/em&gt;&lt;/strong&gt; it to crash so it didn't bother me. Why? I think subconsciously for a couple of reasons - it was MSR so I was not expecting released product quality from it, even though we are &lt;a href="http://store.microsoft.com/microsoft/AutoCollage-2008/product/8D6DDFB5?WT.mc_id=ecomaircover_autocollage" target="_blank"&gt;selling the software&lt;/a&gt; on our brand spanking new &lt;a href="http://www.MicrosoftStore.com"&gt;www.MicrosoftStore.com&lt;/a&gt; and secondly because it was doing a lot of fancy, CPU intensive image processing which made me just hold my breath and wait for the inevitable crash (though to their credit it didn't quite occur during the processing). Oh! We have &lt;strong&gt;such&lt;/strong&gt; a long way to go.&lt;/p&gt;  &lt;p&gt;Anyway, this talk is all about the future and not the present. The present sucks, we all know that. But fear not, for we brave, intrepid testers will fix all of that (or at least find all of that - we'll need some devs to fix it for us - we can't touch the product code you know!) :)&lt;/p&gt;&lt;img src="http://blogs.msdn.com/aggbug.aspx?PostID=9253973" width="1" height="1"&gt;</description><category domain="http://blogs.msdn.com/nihitk/archive/tags/General+Software+Testing/default.aspx">General Software Testing</category></item><item><title>Elements of MBT - Part 3 – Actions and Action Parameters  </title><link>http://blogs.msdn.com/nihitk/archive/2007/07/05/elements-of-mbt-part-3-actions-and-action-parameters.aspx</link><pubDate>Fri, 06 Jul 2007 02:45:00 GMT</pubDate><guid isPermaLink="false">91d46819-8472-40ad-a661-2c78acb4018c:3715851</guid><dc:creator>nihitk</dc:creator><slash:comments>1</slash:comments><comments>http://blogs.msdn.com/nihitk/comments/3715851.aspx</comments><wfw:commentRss>http://blogs.msdn.com/nihitk/commentrss.aspx?PostID=3715851</wfw:commentRss><description>&lt;P&gt;In the last 2 posts we have looked at what we mean by a &lt;A href="http://blogs.msdn.com/nihitk/archive/2007/03/23/elements-of-mbt-part-1-what-is-a-model.aspx" mce_href="http://blogs.msdn.com/nihitk/archive/2007/03/23/elements-of-mbt-part-1-what-is-a-model.aspx"&gt;model of a system&lt;/A&gt; and defined what &lt;A href="http://blogs.msdn.com/nihitk/archive/2007/06/27/elements-of-mbt-part-2-states-and-state-variables.aspx" mce_href="http://blogs.msdn.com/nihitk/archive/2007/06/27/elements-of-mbt-part-2-states-and-state-variables.aspx"&gt;States and State Variables&lt;/A&gt; are. In this post we'll look into Actions and Action Parameters. &lt;/P&gt;
&lt;P&gt;We can define an &lt;STRONG&gt;Action&lt;/STRONG&gt; as something that modifies or exercises the system. Thus "TurnIgnitionOn" (in the car model being used in this series) can be an action which causes the system to change in some way. When considering Actions you need to think about what exactly is it that changes in the system and which all state variables are impacted by those changes. An action is therefore simply something that causes the system to change states. &lt;/P&gt;
&lt;P&gt;Actions become much more powerful and useful when combined with the concept of &lt;STRONG&gt;Action Parameters&lt;/STRONG&gt; which can be considered to be similar to method parameters. Like a method (in a programming sense) does some work which can be tweaked or controlled by the method parameters, the effect of an Action can be affected by Action Parameters. In the car driving model an example of an Action Parameter would be whether the ignition could be turned on remotely (via the keychain) or locally. In that case we might want to add an Action Parameter to the model such as TurnIgnitionOn(Mode) where the "Mode" action parameter could take values "Remote" or "Local" similar to the way state variables can have different values. &lt;/P&gt;
&lt;P mce_keep="true"&gt;&lt;IMG style="WIDTH: 460px; HEIGHT: 102px" height=102 src="http://blogs.msdn.com/photos/nihitk/images/3715908/original.aspx" width=460 mce_src="http://blogs.msdn.com/photos/nihitk/images/3715908/original.aspx"&gt;&lt;/P&gt;
&lt;P&gt;A really interesting point that emerges here is that there is no &lt;STRONG&gt;one&lt;/STRONG&gt; way to model the system. In the example above if for some reason you actually wanted to &lt;STRONG&gt;track&lt;/STRONG&gt; how you had opened the car, maybe for a later stage in the model when you were actually locking the car then you would need to store this information in a state variable. Anytime you find yourself using the word "track" you should think about having that value as a state variable. We could therefore break our TurnIgnitionOn action into 2 different types of actions, &lt;EM&gt;TurnIgnitionOnRemotely&lt;/EM&gt; and &lt;EM&gt;TurnIgnitionOnLocally &lt;/EM&gt;and track how we opened the car as a new state variable, say S3, with values Remote or Local. Our model would therefore look something like this: &lt;/P&gt;
&lt;P mce_keep="true"&gt;&lt;IMG src="http://blogs.msdn.com/photos/nihitk/images/3715910/original.aspx" mce_src="http://blogs.msdn.com/photos/nihitk/images/3715910/original.aspx"&gt;&lt;/P&gt;
&lt;P&gt;I will end this post with a word of caution. Even though it is quite easy to add a state variable to the model to track some aspect of the system, you should be very careful when adding state variables to your model and strive to minimize this number. The reason for this guidance is to limit the amount of state explosion that takes place as a result of defining these additional state variables. Remember that every time you define another state variable the number of possible states in the model is increasing exponentially. To keep your model manageable you should not add a state variable till it is absolutely critical to the behavior being captured.&lt;/P&gt;
&lt;P&gt;UPDATE: Just noticed that this is actually my 100th post on this blog - WOW! Never thought I'd see the 100th post when I started out...it's been fun! :)&lt;/P&gt;&lt;img src="http://blogs.msdn.com/aggbug.aspx?PostID=3715851" width="1" height="1"&gt;</description><category domain="http://blogs.msdn.com/nihitk/archive/tags/General+Software+Testing/default.aspx">General Software Testing</category><category domain="http://blogs.msdn.com/nihitk/archive/tags/Model+Based+Testing/default.aspx">Model Based Testing</category><category domain="http://blogs.msdn.com/nihitk/archive/tags/Elements+of+MBT/default.aspx">Elements of MBT</category></item><item><title>Elements of MBT - Part 2 – States and State Variables  </title><link>http://blogs.msdn.com/nihitk/archive/2007/06/27/elements-of-mbt-part-2-states-and-state-variables.aspx</link><pubDate>Thu, 28 Jun 2007 03:52:00 GMT</pubDate><guid isPermaLink="false">91d46819-8472-40ad-a661-2c78acb4018c:3573496</guid><dc:creator>nihitk</dc:creator><slash:comments>5</slash:comments><comments>http://blogs.msdn.com/nihitk/comments/3573496.aspx</comments><wfw:commentRss>http://blogs.msdn.com/nihitk/commentrss.aspx?PostID=3573496</wfw:commentRss><description>&lt;P&gt;This is a continuation of the Elements of MBT series of posts which I started last time with &lt;A href="http://blogs.msdn.com/nihitk/archive/2007/03/23/elements-of-mbt-part-1-what-is-a-model.aspx" mce_href="http://blogs.msdn.com/nihitk/archive/2007/03/23/elements-of-mbt-part-1-what-is-a-model.aspx"&gt;Part 1&lt;/A&gt; and covered what is meant by a model of a software system. In this post I shall be covering additional concepts which you will need to know to gain a better understanding of Model Based Testing. &lt;/P&gt;
&lt;P&gt;One of the most fundamental concepts required to define a model of a system is that of &lt;STRONG&gt;State Variables&lt;/STRONG&gt; which in turn are needed to define what a State is. A State Variable is simply some aspect of the system which changes over time. For e.g. in the car driving model, one aspect that changes (and which you care about tracking for the purpose you are building the model for) is whether the car is moving or not. So this can be a state variable, say "CarMotionState", which can have values such as "Moving" and "NotMoving". Couple of important things to keep in mind is that a state variable can have multiple values (not just 2 as in the example above) and that these "values" are really just semantic terms to help us define the model. What the actual values of these do and result in (especially from an automation point of view) is tied in later when you are implementing the code for the model. &lt;/P&gt;
&lt;P&gt;A &lt;STRONG&gt;State&lt;/STRONG&gt; then becomes much easier to define as simply a combination of all the State Variables in the model with particular values assigned to each state variable. Thus if your model contains 3 state variables S1, S2 and S3 with multiple values for each state variable, a state will be defined by a &lt;STRONG&gt;unique&lt;/STRONG&gt; set of values for each of the state variables S1, S2 and S3. You can therefore think of a state as simply a snapshot of all the state variable values at a given point in time. From the Part 1 car driving model we can now see that the 3 states in the model are comprised of the following unique combinations of state variable values. &lt;/P&gt;
&lt;P&gt;&lt;SPAN style="COLOR: #632423"&gt;S1 = { IgnitionState = IgnitionOff; CarMotionState = NotMoving; } &lt;/SPAN&gt;&lt;/P&gt;
&lt;P&gt;&lt;SPAN style="COLOR: #632423"&gt;S2 = { IgnitionState = &lt;STRONG&gt;IgnitionOn&lt;/STRONG&gt;; CarMotionState = NotMoving; } &lt;/SPAN&gt;&lt;/P&gt;
&lt;P&gt;&lt;SPAN style="COLOR: #632423"&gt;S3 = { IgnitionState = IgnitionOn; CarMotionState = &lt;STRONG&gt;Moving&lt;/STRONG&gt;; } &lt;/SPAN&gt;&lt;/P&gt;
&lt;P&gt;One thing to notice here is that not &lt;STRONG&gt;all&lt;/STRONG&gt; possible states need to be defined for a particular model. Depending on the system and the reason you are creating the model or the abstraction level you want to capture, you might not include certain states in your model. Or certain states simply won't make any sense to model such as the state "IgnitionOff" + "Moving" if you do not want to capture the behavior of the car sliding down hill when the ignition has not been turned on. The great thing though is that it forces you to think of that scenario and to consider if that is something you want to capture and if so then what the expected behavior would be. &lt;/P&gt;
&lt;P&gt;In the next post I shall cover Actions and Action Parameters and update the model to add an Action Parameter to one of the actions.&lt;/P&gt;&lt;img src="http://blogs.msdn.com/aggbug.aspx?PostID=3573496" width="1" height="1"&gt;</description><category domain="http://blogs.msdn.com/nihitk/archive/tags/General+Software+Testing/default.aspx">General Software Testing</category><category domain="http://blogs.msdn.com/nihitk/archive/tags/Model+Based+Testing/default.aspx">Model Based Testing</category><category domain="http://blogs.msdn.com/nihitk/archive/tags/Elements+of+MBT/default.aspx">Elements of MBT</category></item><item><title>Elements of MBT - Part 1 - What is a model?</title><link>http://blogs.msdn.com/nihitk/archive/2007/03/23/elements-of-mbt-part-1-what-is-a-model.aspx</link><pubDate>Sat, 24 Mar 2007 03:54:00 GMT</pubDate><guid isPermaLink="false">91d46819-8472-40ad-a661-2c78acb4018c:1939974</guid><dc:creator>nihitk</dc:creator><slash:comments>2</slash:comments><comments>http://blogs.msdn.com/nihitk/comments/1939974.aspx</comments><wfw:commentRss>http://blogs.msdn.com/nihitk/commentrss.aspx?PostID=1939974</wfw:commentRss><description>&lt;P mce_keep="true"&gt;In this series of "Elements of MBT" posts, I will be covering some basic concepts used when trying to build models of software systems for use in &lt;A class="" href="http://blogs.msdn.com/nihitk/pages/144664.aspx" mce_href="http://blogs.msdn.com/nihitk/pages/144664.aspx"&gt;Model Based Testing&lt;/A&gt;. The aim is to give an explanation of the various elements involved in defining a model of a software system and hopefully this should get us on the same page and allow us to explore the complexities in more detail. &lt;/P&gt;
&lt;P&gt;Let us start by going over what we mean by a "model" of a system. Put simply, a model is a description of the system and is much simpler than the actual system itself. What this means is that the model hides a lot of the details about the system and hiding these details allows us to work at a higher level of abstraction. For e.g. a simple model of driving a car could be: 
&lt;P&gt;&lt;A href="http://blogs.msdn.com/blogfiles/nihitk/WindowsLiveWriter/ElementsofModelBasedTesting_C20B/image%7B0%7D%5B3%5D.png" atomicselection="true" mce_href="http://blogs.msdn.com/blogfiles/nihitk/WindowsLiveWriter/ElementsofModelBasedTesting_C20B/image%7B0%7D%5B3%5D.png"&gt;&lt;/A&gt;&lt;A href="http://blogs.msdn.com/blogfiles/nihitk/WindowsLiveWriter/ElementsofModelBasedTesting_C20B/image%7B0%7D%5B6%5D.png" atomicselection="true" mce_href="http://blogs.msdn.com/blogfiles/nihitk/WindowsLiveWriter/ElementsofModelBasedTesting_C20B/image%7B0%7D%5B6%5D.png"&gt;&lt;IMG style="BORDER-RIGHT: 0px; BORDER-TOP: 0px; BORDER-LEFT: 0px; BORDER-BOTTOM: 0px" height=95 src="http://blogs.msdn.com/blogfiles/nihitk/WindowsLiveWriter/ElementsofModelBasedTesting_C20B/image%7B0%7D_thumb%5B2%5D.png" width=433 border=0 mce_src="http://blogs.msdn.com/blogfiles/nihitk/WindowsLiveWriter/ElementsofModelBasedTesting_C20B/image%7B0%7D_thumb%5B2%5D.png"&gt;&lt;/A&gt; 
&lt;P&gt;One thing to keep in mind is that there is no perfect&amp;nbsp;model of a system, since the same system can have completely different models depending on what you plan to &lt;STRONG&gt;use&lt;/STRONG&gt; the model for. Are you creating the model of a system simply to capture the behavior and requirements of the system? Are you creating the model to auto-generate the code for the model implementation itself? To describe the data flowing through the system? Or are you generating the model to actually test a particular portion of the system? 
&lt;P&gt;A quote which captures this unusual thing about models really well is as follows: 
&lt;BLOCKQUOTE&gt;
&lt;P&gt;&lt;STRONG&gt;All models are wrong; some models are useful. &lt;/STRONG&gt;
&lt;P&gt;&lt;EM&gt;- George Box&lt;/EM&gt;&lt;/P&gt;&lt;/BLOCKQUOTE&gt;
&lt;P&gt;What is worth hanging onto from this definition is that a model that you create should strive to be "useful" more than anything else. It might be rough round the edges and might not tell you everything about the system (for e.g. you don't know from the above model how the gear is to be changed or what happens when a gear is changed) but if it is &lt;STRONG&gt;useful &lt;/STRONG&gt;for the purpose it was created it is a correct model. 
&lt;P&gt;Another aspect of defining a model is the notion that the model should help &lt;STRONG&gt;predict&lt;/STRONG&gt; what the behavior of the system would be under certain conditions. So if you are in the "Not Moving" state and you "ChangeGearAndAccelerate" then you reach the "Moving" state. Are there certain assumptions in the model? You bet there are! Such as assuming that the ignition is on, that the car has wheels, that the car is not hanging in the air, that there is oxygen in the air, we can go on and on and on. The hiding of these details and assumptions is what helps us look at just this behavior we are capturing. If anything is important for the &lt;STRONG&gt;purpose&lt;/STRONG&gt; (say testing) for which we are creating the model, we should consider adding that detail into the model. So if we thought that it was important to know whether the ignition was on or off we would change the model to capture this as well. 
&lt;P&gt;&lt;A href="http://blogs.msdn.com/blogfiles/nihitk/WindowsLiveWriter/ElementsofModelBasedTesting_C20B/image%7B0%7D%5B12%5D.png" atomicselection="true" mce_href="http://blogs.msdn.com/blogfiles/nihitk/WindowsLiveWriter/ElementsofModelBasedTesting_C20B/image%7B0%7D%5B12%5D.png"&gt;&lt;IMG style="BORDER-RIGHT: 0px; BORDER-TOP: 0px; BORDER-LEFT: 0px; WIDTH: 511px; BORDER-BOTTOM: 0px; HEIGHT: 96px" height=120 src="http://blogs.msdn.com/blogfiles/nihitk/WindowsLiveWriter/ElementsofModelBasedTesting_C20B/image%7B0%7D_thumb%5B6%5D.png" width=736 border=0 mce_src="http://blogs.msdn.com/blogfiles/nihitk/WindowsLiveWriter/ElementsofModelBasedTesting_C20B/image%7B0%7D_thumb%5B6%5D.png"&gt;&lt;/A&gt; 
&lt;P&gt;Now we know that if you are in the "Ignition &lt;STRONG&gt;Off&lt;/STRONG&gt;" and "Not Moving" state then if you "TurnIgnitionOn" you reach the "Ignition &lt;STRONG&gt;On&lt;/STRONG&gt;" and "Not Moving" state since simply turning on the ignition does not cause the car to start moving in the model that we have created. To start moving you will need to "ChangeGearAndAccelerate" which should take you to the state of having the "Ignition On" and the car "Moving". 
&lt;P&gt;Hope this gives you an idea of what a model contains and what it can be used to capture. In the following posts we will look into each element of a model in more detail.&lt;/P&gt;&lt;img src="http://blogs.msdn.com/aggbug.aspx?PostID=1939974" width="1" height="1"&gt;</description><category domain="http://blogs.msdn.com/nihitk/archive/tags/General+Software+Testing/default.aspx">General Software Testing</category><category domain="http://blogs.msdn.com/nihitk/archive/tags/Model+Based+Testing/default.aspx">Model Based Testing</category><category domain="http://blogs.msdn.com/nihitk/archive/tags/Elements+of+MBT/default.aspx">Elements of MBT</category></item><item><title>How to test a delete operation?</title><link>http://blogs.msdn.com/nihitk/archive/2006/11/17/how-to-test-a-delete-operation.aspx</link><pubDate>Sat, 18 Nov 2006 02:08:00 GMT</pubDate><guid isPermaLink="false">91d46819-8472-40ad-a661-2c78acb4018c:1096618</guid><dc:creator>nihitk</dc:creator><slash:comments>8</slash:comments><comments>http://blogs.msdn.com/nihitk/comments/1096618.aspx</comments><wfw:commentRss>http://blogs.msdn.com/nihitk/commentrss.aspx?PostID=1096618</wfw:commentRss><description>&lt;P mce_keep="true"&gt;Question to the community: &lt;STRONG&gt;How can you best test a delete operation?&lt;/STRONG&gt; &lt;/P&gt;
&lt;P&gt;For e.g. When you&amp;nbsp;test the following API - DeleteObjectFoo("ABCD") -&amp;nbsp;what should we test to verify that this function actual deleted "ABCD"? This API is just a placeholder for an API that deletes something. The question is that, in general, how do we verify that "something" was indeed deleted? 
&lt;P&gt;Should we simply test that "ABCD" is deleted from the database (or whatever the underlying storage is)? That is typically what is done. The question then becomes how do we catch a bug which not only deletes "ABCD" but also "XYZ"??? 
&lt;P&gt;A colleague some time back had raised this issue since we ran into exactly this sort of a bug and found to our dismay that the existing tests were not catching this bug. 
&lt;P&gt;What do we change to test for a delete operation in general so that the testing becomes foolproof - or can it even be made foolproof? &lt;/P&gt;&lt;img src="http://blogs.msdn.com/aggbug.aspx?PostID=1096618" width="1" height="1"&gt;</description><category domain="http://blogs.msdn.com/nihitk/archive/tags/General+Software+Testing/default.aspx">General Software Testing</category></item><item><title>Process Monitor for Debugging</title><link>http://blogs.msdn.com/nihitk/archive/2006/11/09/process-monitor-for-debugging.aspx</link><pubDate>Thu, 09 Nov 2006 21:09:00 GMT</pubDate><guid isPermaLink="false">91d46819-8472-40ad-a661-2c78acb4018c:1045905</guid><dc:creator>nihitk</dc:creator><slash:comments>1</slash:comments><comments>http://blogs.msdn.com/nihitk/comments/1045905.aspx</comments><wfw:commentRss>http://blogs.msdn.com/nihitk/commentrss.aspx?PostID=1045905</wfw:commentRss><description>The &lt;A class="" href="http://www.microsoft.com/technet/sysinternals/processesandthreads/processmonitor.mspx" mce_href="http://www.microsoft.com/technet/sysinternals/processesandthreads/processmonitor.mspx"&gt;Process Monitor&lt;/A&gt; looks like a great tool to help in debugging issues and includes a bunch of features from a diverse set of sysinternal utilities such as RegMon, FileMon etc. I think this is the first tool offering to come out since the acquisition of &lt;A class="" href="http://www.microsoft.com/technet/sysinternals/default.mspx" mce_href="http://www.microsoft.com/technet/sysinternals/default.mspx"&gt;Sysinternals.com&lt;/A&gt; and more importantly perhaps, &lt;A href="http://blogs.technet.com/markrussinovich/about.aspx"&gt;Mark Russinovich&lt;/A&gt;&amp;nbsp;(who has bunch of interesting posts about debugging issues). Looks like there is even a dedicated &lt;A class="" href="http://blogs.technet.com/sysinternals/" mce_href="http://blogs.technet.com/sysinternals/"&gt;Sysinternals Blog at TechNet&lt;/A&gt;&amp;nbsp;and if you are not familiar with the sysinternal family of tools there is a free &lt;A class="" href="http://www.solsem.com/Video/TourofSysinternalsTools.wmv" mce_href="http://www.solsem.com/Video/TourofSysinternalsTools.wmv"&gt;Tour of the Sysinternals&lt;/A&gt; video available.&lt;img src="http://blogs.msdn.com/aggbug.aspx?PostID=1045905" width="1" height="1"&gt;</description><category domain="http://blogs.msdn.com/nihitk/archive/tags/Geeky/default.aspx">Geeky</category><category domain="http://blogs.msdn.com/nihitk/archive/tags/General+Software+Testing/default.aspx">General Software Testing</category></item><item><title>Model Based Testing – The Yellow Brick Road</title><link>http://blogs.msdn.com/nihitk/archive/2006/07/28/YellowBrickRoad.aspx</link><pubDate>Sat, 29 Jul 2006 04:04:00 GMT</pubDate><guid isPermaLink="false">91d46819-8472-40ad-a661-2c78acb4018c:682122</guid><dc:creator>nihitk</dc:creator><slash:comments>1</slash:comments><comments>http://blogs.msdn.com/nihitk/comments/682122.aspx</comments><wfw:commentRss>http://blogs.msdn.com/nihitk/commentrss.aspx?PostID=682122</wfw:commentRss><description>&lt;P class=MsoNormal style="MARGIN: 0in 0in 0pt"&gt;The &lt;A href="http://blogs.msdn.com/nihitk/archive/2004/05/30/144667.aspx"&gt;future was here&lt;/A&gt; – now it’s the past. I am, however, wiser about the ways-of-the-model-based world. Still taking baby steps really, but starting to stand up on my feet. The real-world turned out to be a whole different ball-game from what I had envisioned (and read and expected) but it has been a fun, FUN ride and the road ahead looks fascinating – though long.&lt;/P&gt;
&lt;P class=MsoNormal style="MARGIN: 0in 0in 0pt"&gt;&lt;?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" /&gt;&lt;o:p&gt;&amp;nbsp;&lt;/o:p&gt;&lt;/P&gt;
&lt;P class=MsoNormal style="MARGIN: 0in 0in 0pt"&gt;In the eons since I posted my original &lt;A href="http://blogs.msdn.com/nihitk/articles/144664.aspx"&gt;introduction to model based testing&lt;/A&gt;, a lot has happened to the world. For e.g. Wikipedia now has &lt;A href="http://en.wikipedia.org/wiki/Model_based_testing"&gt;an article on MBT&lt;/A&gt;. There is an &lt;A href="http://www.mdqa.org/aamdqa.php"&gt;Association for the Advancement of Model Driven Quality Assurance&lt;/A&gt; (though they don’t actually state how one can join them!). There are a host of companies dedicated to doing only model based testing (for e.g. &lt;A href="http://www.reactive-systems.com/"&gt;Reactive Systems&lt;/A&gt; and &lt;A href="http://www.conformiq.com/"&gt;Confirmiq&lt;/A&gt;). There is an IEEE workshop on &lt;A href="http://www.csc2.ncsu.edu/conferences/issre/workshops/AMOST06/"&gt;Advances in Model-based Software Testing&lt;/A&gt; (scheduled for Nov 2006 and looks really interesting – would love to hear from someone attending it!). There is &lt;I style="mso-bidi-font-style: normal"&gt;even&lt;/I&gt; a new acronym for the technology – MDQA for &lt;B style="mso-bidi-font-weight: normal"&gt;M&lt;/B&gt;odel &lt;B style="mso-bidi-font-weight: normal"&gt;D&lt;/B&gt;riven &lt;B style="mso-bidi-font-weight: normal"&gt;Q&lt;/B&gt;uality &lt;B style="mso-bidi-font-weight: normal"&gt;A&lt;/B&gt;ssurance (that’s when you know something has &lt;I style="mso-bidi-font-style: normal"&gt;really&lt;/I&gt; taken off – when people call the same thing by different names!). I find all this amazing and reflective of the direction in which the testing industry will be moving in the next 5, 10, 20 years. &lt;/P&gt;
&lt;P class=MsoNormal style="MARGIN: 0in 0in 0pt"&gt;&lt;o:p&gt;&amp;nbsp;&lt;/o:p&gt;&lt;/P&gt;
&lt;P class=MsoNormal style="MARGIN: 0in 0in 0pt"&gt;So what have I been doing all this time? Well, in a nutshell, simply using MBT to actually test something that gets shipped – in this case that happens to be portions of &lt;A href="http://www.microsoft.com/commerceserver/"&gt;Commerce Server 2007&lt;/A&gt; that I worked on (that should be another selling point used by the sales team and on our info sheet – “Tested Using Bleeding Edge Model Driven Testing Technology”). Along the way I cleared the hurdles and burnt my fingers while trying to keep the eye on the ball – which is always simply to test as best as you can. I now plan to expound in future posts on how to adopt MBT better, what to expect and more importantly, what &lt;B style="mso-bidi-font-weight: normal"&gt;not&lt;/B&gt; to expect from this fascinating methodology.&lt;/P&gt;
&lt;P class=MsoNormal style="MARGIN: 0in 0in 0pt"&gt;&lt;o:p&gt;&amp;nbsp;&lt;/o:p&gt;&lt;/P&gt;
&lt;P class=MsoNormal style="MARGIN: 0in 0in 0pt"&gt;For this post, I will just mention the most startling thing I discovered – that this dream is going to take a long time to realize. By “this dream” I refer to that futuristic world where model based approaches are entrenched in our processes and systems. Where a tester thinks first in a model based approach whenever he/she is given something to test (even a regression!). Where the metrics are tailored to encourage testers to adopt a model based approach. Where the feature teams contribute to and strive to keep their models up-to-date. And where the tools have the features and reliability that is crucial to be able to develop models and have high levels of confidence in them.&lt;/P&gt;
&lt;P class=MsoNormal style="MARGIN: 0in 0in 0pt"&gt;&lt;o:p&gt;&amp;nbsp;&lt;/o:p&gt;&lt;/P&gt;
&lt;P class=MsoNormal style="MARGIN: 0in 0in 0pt"&gt;All of these are non-trivial problems and there are no easy answers that anyone can give. But maybe that’s part of the challenge? So, all in all, buckle up and put on some good music – it’s gonna be a long, FUN ride and we’ll change the world on the way!&lt;o:p&gt;&lt;/o:p&gt;&lt;/P&gt;&lt;img src="http://blogs.msdn.com/aggbug.aspx?PostID=682122" width="1" height="1"&gt;</description><category domain="http://blogs.msdn.com/nihitk/archive/tags/General+Software+Testing/default.aspx">General Software Testing</category><category domain="http://blogs.msdn.com/nihitk/archive/tags/Model+Based+Testing/default.aspx">Model Based Testing</category></item><item><title>Buggy Test Code</title><link>http://blogs.msdn.com/nihitk/archive/2006/04/20/580399.aspx</link><pubDate>Fri, 21 Apr 2006 08:40:00 GMT</pubDate><guid isPermaLink="false">91d46819-8472-40ad-a661-2c78acb4018c:580399</guid><dc:creator>nihitk</dc:creator><slash:comments>3</slash:comments><comments>http://blogs.msdn.com/nihitk/comments/580399.aspx</comments><wfw:commentRss>http://blogs.msdn.com/nihitk/commentrss.aspx?PostID=580399</wfw:commentRss><description>&lt;P&gt;All the technical jargon of the world put together cannot compensate for raw stupidity. I recently had this epiphany when I ran into one of those “DUH!” bugs in my test code. Made me feel like a fool and so I thought I would redeem some of my hurt ego by confessing and discussing general test code bugs and why they occur, in the hope that it brings me some closure!&lt;/P&gt;
&lt;P&gt;Here’s the nasty little not-so-secret of the testing world that is rarely talked about – the test code is full of bugs. Big, bad, ugly BUGS. The product code is blessed since it will have people beating it to death and using it in different ways thus exposing most (hopefully!) of the flaws contained. But the story of the test code is quite sad – it starts out on any regular afternoon when a tester codes up a few test cases for an API or a particular scenario. The code is run once (depending on time constraints might be run ½ a time only – what that means, we’ll get to in a second) and if everything looks nice and dandy, it is checked in. This is a big moment for the test code, because my guestimate is that for a huge percentage (I would speculate almost 75%) of the code will never be looked at again. I know, I know – that’s not the way things should be. We should have every line of code reviewed (hell – let’s have it reviewed multiple times while we are day dreaming) and periodic re-reviews to ensure the test code is in sync with the API behavior. But then we should also have world peace. But we don’t. So let’s suck it up, get real and move on.&lt;/P&gt;
&lt;P&gt;So our story continues with the checked in code that contains a bug as follows.&lt;/P&gt;
&lt;P dir=ltr&gt;&lt;EM&gt;Bool myBuggyTestCase&lt;BR&gt;{&lt;/EM&gt;&lt;/P&gt;
&lt;BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px"&gt;
&lt;P dir=ltr&gt;&lt;EM&gt;&amp;nbsp;bool testPassed = false;&amp;nbsp;&amp;nbsp;//Default should be test has failed&lt;/EM&gt;&lt;/P&gt;
&lt;P&gt;&lt;EM&gt;&amp;nbsp;//Do a bunch of things for setup&lt;BR&gt;&amp;nbsp;…&lt;/EM&gt;&lt;/P&gt;
&lt;P&gt;&lt;EM&gt;&amp;nbsp;//Invoke the API(s) we are interested in&lt;BR&gt;…&lt;/EM&gt;&lt;/P&gt;
&lt;P&gt;&lt;EM&gt;&amp;nbsp;//Check the thing worked&lt;BR&gt;&amp;nbsp;testPassed = VerifyThingsAreFine();&lt;/EM&gt;&lt;/P&gt;
&lt;P&gt;&lt;EM&gt;&amp;nbsp;//Do the required cleanup&lt;BR&gt;&amp;nbsp;…&lt;BR&gt;&lt;BR&gt;&amp;nbsp;&amp;nbsp;//This could be a lot of cleaning – hang in there&lt;BR&gt;&amp;nbsp;…&lt;BR&gt;&amp;nbsp;&lt;BR&gt;&amp;nbsp;return true;&lt;/EM&gt;&lt;/P&gt;&lt;/BLOCKQUOTE&gt;
&lt;P dir=ltr&gt;&lt;EM&gt;}&lt;/EM&gt;&lt;/P&gt;
&lt;P&gt;The problem here is that we do correctly obtain the pass / fail in the line:&lt;/P&gt;
&lt;BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px"&gt;
&lt;P&gt;&amp;nbsp;&amp;nbsp;&lt;EM&gt;returnValue = VerifyThingsAreFine();&lt;/EM&gt;&lt;/P&gt;&lt;/BLOCKQUOTE&gt;
&lt;P&gt;however by the end of the test module, we just return a constant value of “true”. In the dry run that we do of this test case, where we step through the code, we find the API does what it supposed to and the returnValue = true as expected. We also step through and verify that the value actually returned by the module and logged in the automation harness is true (of course missing that fact that it is actually the hardcoded “true”). Now we are all setup for trouble. All that needs to happen now is a regression – an API has to stop doing things the way it is actually doing it currently and then all of a sudden we find (or rather not-find) that even though returnValue is correctly calculated as “False” the value returned is still “True”. This test case will happily go on logging this scenario as “Pass” till the end of eternity (or a breaking API change causing an exception or compile time failures – whichever comes first).&lt;/P&gt;
&lt;P&gt;We sort-of have started avoiding this problem by throwing exceptions in the test code when a failure case is hit, expecting the automation harness to actually catch and log those as failures. This works much better at avoiding errors such as the one above, but has it’s own gotchas since many times you might be interested in catching exceptions and logging some information about the state you are in. Ideally you should rethrow the exception after you’re done logging, but humans have a tendency to make the same mistake in wonderfully different ways – so we might not.&lt;/P&gt;
&lt;P&gt;Another approach that can help ameliorate this situation is the use of intentional programming which would make the code much more lean and mean, thus making the bug a little more easier-to-spot (currently I think the long cleanup parts of the code can cover up this issue as well).&lt;/P&gt;
&lt;P&gt;But wait there is more – remember when I mentioned that you might just run some test cases only ½ a time (and you mumbled to yourself “whatever that means!”)? Well what I was referring to was the fact that when you tend to have a lot of similar test cases which are using well factored-out code (so that they all look terribly similar), you will not (and I’m ready to bet on this!) run through each of these cases. In fact you will, like any other tester with his head-screwed on tight, put break-points at key places and run through the cases by ensuring things at those break-points are as expected. But that does mean that a lot of code paths will be running “unanalysed” – with the usual baggage that comes with that word (such as unintended goof-ups).&lt;/P&gt;
&lt;P&gt;This phenomenon is exacerbated in Model Based tests, since the test cases are by definition dynamic and so many that you cannot step through each one of them. You step through the action handlers and try and make sure things will work as designed. Plus you tend to get lost in all the jargon such as Action Handlers and Transition Rules. Net effect is that the test code which will be executed in millions of different ways, would have been stepped through for only a handful. Of course, this is exactly what we want for model based tests as well, but it just goes to highlight that the few we walk through and take a hard look at, need to be looked at REALLY HARD!. &lt;/P&gt;
&lt;P&gt;DCRs don’t help in the least since you end up patching code on top of existing code creating the classic sphagetti (with no marinara sauce).&lt;/P&gt;
&lt;P&gt;So, that brings me to the main agenda (I knew we’d get there someday). Here is a small bullet-list of things I believe can be useful when doing a test plan/code/coverage review (we’ll leave the differences between these for another day).&lt;/P&gt;
&lt;UL&gt;
&lt;LI&gt;Make the test case FAIL. I think we don’t do this enough and it is the most important aspect. It is in fact the whole purpose of creating the test code – you hope that someday it fails and catches a regression. So make sure you cause failures and check if the net of your test code can catch those fish or not. Sometimes this is too easy (you just say “of course it will fail if I do that”) and therefore doesn’t get done and sometimes the excuse is that it is too hard (“how the hell do I get that to occur!").&lt;BR&gt;
&lt;LI&gt;Run through test cases in different buckets. Run through them end-to-end – right from setup, invoking the API(s), clean up, logging etc.&lt;BR&gt;
&lt;LI&gt;Refactor and modularize code so that we hopefully do the things where goof up is most costly only in a single place.&lt;BR&gt;
&lt;LI&gt;Look at refactored, modularized code that is called again and again and again&amp;nbsp;using different data sets. The potential damage that bugs in these areas have, is much larger, so spend more time on them.&lt;BR&gt;
&lt;LI&gt;Spend time on algorithmic and mathematical sections. If you doing something fancy in your test code, make sure it gets looked at more than other parts.&lt;BR&gt;
&lt;LI&gt;This one would be my favorite but something I don’t see happening in the near future - &amp;nbsp;injecting product code with bugs to see if the test harness can catch those bugs. This would of course entail not checking in the bugs, the “buggy” version of the software is purely a test exercise to check the effectiveness of the test cases being automated. This also goes by the name of Mutation Testing and was recently covered in &lt;A href="http://msdn.microsoft.com/msdnmag/issues/06/04/MutationTesting/default.aspx"&gt;this MSDN Magazine article&lt;/A&gt;.&lt;/LI&gt;&lt;/UL&gt;
&lt;P&gt;One more solution that I know some teams do take is to hold the test code to the same bar and same vigorous processes that product code has to go through. I haven’t tried this first-hand but my gut feeling is that it will bog down testing more than the value it will provide. But it would be interesting to hear from someone who has actually tried this and found this to be “the fix”.&lt;/P&gt;
&lt;P&gt;What other things would you do to make your test code more reliable? Are these steps even practical to do in the real world? What would be practical?&lt;/P&gt;&lt;img src="http://blogs.msdn.com/aggbug.aspx?PostID=580399" width="1" height="1"&gt;</description><category domain="http://blogs.msdn.com/nihitk/archive/tags/General+Software+Testing/default.aspx">General Software Testing</category></item><item><title>Spot the Bug Contest #1 Solution: The mystery of the Decimal.Parse FormatException</title><link>http://blogs.msdn.com/nihitk/archive/2005/03/26/402611.aspx</link><pubDate>Sun, 27 Mar 2005 05:31:00 GMT</pubDate><guid isPermaLink="false">91d46819-8472-40ad-a661-2c78acb4018c:402611</guid><dc:creator>nihitk</dc:creator><slash:comments>0</slash:comments><comments>http://blogs.msdn.com/nihitk/comments/402611.aspx</comments><wfw:commentRss>http://blogs.msdn.com/nihitk/commentrss.aspx?PostID=402611</wfw:commentRss><description>&lt;p&gt;Well - as promised here is the solution for the &lt;A href="http://blogs.msdn.com/nihitk/archive/2005/01/23/359076.aspx"&gt;Spot the Bug&lt;/a&gt; (contest?) posting which was correctly answered first by &lt;a href="http://geekswithblogs.net/mrnat"&gt;Nat&lt;/a&gt; (Congratulations!).&lt;/p&gt; &lt;p dir="ltr" style="MARGIN-RIGHT: 0px"&gt;&lt;strong&gt;The Bug&lt;/strong&gt;:&lt;br /&gt;The following line of code will fail intermittently when executed and most of the time rerunning will work. Sadly, these are the sort of bugs most easily overlooked.&lt;/p&gt; &lt;p dir="ltr" style="MARGIN-RIGHT: 0px"&gt;&lt;font color="#0000ff"&gt;&lt;font&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;return&lt;/font&gt;&lt;/font&gt; &lt;font color="#008080"&gt;Decimal&lt;/font&gt;&lt;font&gt;&lt;font&gt;.Parse(rand.NextDouble().ToString());&lt;/font&gt;&lt;/p&gt;&lt;/font&gt; &lt;p dir="ltr" style="MARGIN-RIGHT: 0px"&gt;where rand is a &lt;a href="http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfSystemRandomClassTopic.asp"&gt;Random&lt;/a&gt; class instance. The exception thrown is:&lt;/p&gt; &lt;p dir="ltr" style="MARGIN-RIGHT: 0px"&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&lt;a href="http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfsystemformatexceptionclasstopic.asp"&gt;FormatException&lt;/a&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; Input string was not in a correct format.&lt;/p&gt; &lt;p&gt;What is bewildering is that on the face of it this seems like a perfectly legitimate line of code - we are simply converting a value from a &lt;a href="http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfsystemdoubleclasstopic.asp"&gt;Double&lt;/a&gt; in the range 0.0 - 1.0 (since that is what NextDouble() returns) to a Decimal value. The boundary conditions are both well contained within the Decimal range.&amp;nbsp; &lt;/p&gt; &lt;p&gt;The only way out is to write a quick and dirty app to simulate this failure and here is what I wrote (no comments for sake of brevity):&lt;/p&gt; &lt;p&gt;&lt;font color="#0000ff"&gt;static&lt;/font&gt; &lt;font color="#0000ff"&gt;void&lt;/font&gt; Main(&lt;font color="#0000ff"&gt;string&lt;/font&gt;[] args)&lt;br /&gt;{&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&lt;font color="#008080"&gt;Decimal&lt;/font&gt; decNum; &lt;font color="#008080"&gt;Random&lt;/font&gt; rand = &lt;font color="#0000ff"&gt;new&lt;/font&gt; &lt;font color="#008080"&gt;Random&lt;/font&gt;(); &lt;font color="#008080"&gt;String&lt;/font&gt; str = &lt;font color="#800000"&gt;""&lt;/font&gt;;&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&lt;font color="#0000ff"&gt;for&lt;/font&gt; (&lt;font color="#0000ff"&gt;long&lt;/font&gt; counter = 0; counter &amp;lt; &lt;font color="#0000ff"&gt;long&lt;/font&gt;.MaxValue; counter++)&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;{&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&lt;font color="#0000ff"&gt;try&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&lt;/font&gt;{&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;decNum = &lt;font color="#008080"&gt;Decimal&lt;/font&gt;.Parse(rand.NextDouble().ToString());&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&lt;font color="#0000ff"&gt;if&lt;/font&gt; (counter % 1000 == 0) &lt;font color="#008080"&gt;Console&lt;/font&gt;.WriteLine(&lt;font color="#800000"&gt;"Iteration: "&lt;/font&gt; + counter);&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;}&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&lt;font color="#0000ff"&gt;catch&lt;/font&gt; (&lt;font color="#008080"&gt;FormatException&lt;/font&gt; ex)&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;{&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&lt;font color="#008080"&gt;Console&lt;/font&gt;.WriteLine(&lt;font color="#800000"&gt;"Iteration: "&lt;/font&gt; + counter + &lt;font color="#800000"&gt;"\nDouble value string: "&lt;/font&gt; + str);&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&lt;font color="#008080"&gt;Console&lt;/font&gt;.WriteLine(&lt;font color="#800000"&gt;"Exception occured: "&lt;/font&gt;&lt;font&gt;&lt;font&gt; + ex.ToString());&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&lt;/font&gt;&lt;font&gt;}&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&lt;/font&gt;&lt;font&gt;}&lt;br /&gt;&lt;/font&gt;&lt;font&gt;}&lt;/font&gt;&lt;/p&gt;&lt;/font&gt; &lt;p&gt;Running this console app with a breakpoint set inside the catch block, immediately bubbles up the root cause. The crux of the issue turns out to be that this exception occurs when the random Double value generated is so small as to need scientific notation. E.g. the double 0.000083304941692997209&lt;br /&gt;&amp;nbsp;will return "8.33049416929972E-05" when you do a Double.ToString() and this scientific representation is in turn not an acceptable input to the Decimal.Parse method, causing it to throw the FormatException. Mystery solved.&lt;/p&gt; &lt;p&gt;&lt;strong&gt;The Fix:&lt;/strong&gt;&lt;/p&gt; &lt;p&gt;&lt;font color="#0000ff"&gt;&lt;font&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;return&lt;/font&gt;&lt;/font&gt; (&lt;font color="#008080"&gt;Decimal&lt;/font&gt;)rand.NextDouble();&lt;/p&gt; &lt;p&gt;Seems like the simpler and more logical way to write this in the first place, but that's only because this is a simplified case for illustration of this bug.&lt;/p&gt; &lt;p&gt;I already have another spot the bug lined up - just need some time to write it up. In case you have any such interesting bugs that you have hit as well (solved or not) that you would like to share, let me know!&lt;/p&gt;&lt;img src="http://blogs.msdn.com/aggbug.aspx?PostID=402611" width="1" height="1"&gt;</description><category domain="http://blogs.msdn.com/nihitk/archive/tags/Geeky/default.aspx">Geeky</category><category domain="http://blogs.msdn.com/nihitk/archive/tags/General+Software+Testing/default.aspx">General Software Testing</category></item><item><title>SDET Memes</title><link>http://blogs.msdn.com/nihitk/archive/2005/01/23/359113.aspx</link><pubDate>Sun, 23 Jan 2005 22:56:00 GMT</pubDate><guid isPermaLink="false">91d46819-8472-40ad-a661-2c78acb4018c:359113</guid><dc:creator>nihitk</dc:creator><slash:comments>27</slash:comments><comments>http://blogs.msdn.com/nihitk/comments/359113.aspx</comments><wfw:commentRss>http://blogs.msdn.com/nihitk/commentrss.aspx?PostID=359113</wfw:commentRss><description>&lt;p&gt;Starting to see a lot of good discussion about what it means to be a test developer and the practices, ideas and challenges that go into being a successful test developer. In this post I want to talk about some insights about what I find challenging about this role and a question as old as the chicken and the egg conundrum:&lt;/p&gt; &lt;blockquote dir="ltr" style="MARGIN-RIGHT: 0px"&gt; &lt;p&gt;Are test developers, just wannabe developers? &lt;/p&gt;&lt;/blockquote&gt; &lt;p&gt;This is a thought that crosses the mind of most people who have worked as or with test developers (also known as Software Design Engineers in Test or SDETs) within Microsoft. &lt;/p&gt; &lt;p&gt;Being someone who originally came from that traditional school of thought that testers are just developers who couldn't make it as devs (similar to dentists being labeled as doctors who couldn't make it), it has taken me some time to wrap my head around the SDET job description and grow an appreciation for it. You also have to pacify your ego and let go of the prejudices. Well I did and now I am hooked onto it and think it offers a lot of great incentives not available to a traditional SDE, such as flexibility in coding, high-level view of the system and a more egalitarian workload across the product cycle. All these and more are tackled by &lt;A href="http://blogs.msdn.com/steverowe/"&gt;Steve Rowe&lt;/a&gt; in his blog postings about why a &lt;A href="http://blogs.msdn.com/steverowe/archive/2005/01/19/356361.aspx"&gt;test developer role is better&lt;/a&gt; than a traditional developer role and why &lt;A href="http://blogs.msdn.com/steverowe/archive/2004/03/26/96663.aspx"&gt;test developers are just as "real"&lt;/a&gt;. I liked the posts and ran into them through the Careers@Microsoft &lt;A href="http://blogs.msdn.com/jobsblog/archive/2005/01/19/356412.aspx#358126"&gt;posting&lt;/a&gt;.&lt;/p&gt; &lt;p&gt;Other than these, one additional challenge that I have just begun to realize, is that as a tester, you are always trying to cope with a lot of theoretically hard problems (testing space in most cases is unbounded) and I find dealing with this computational complexity especially rewarding. It's like trying to tame a wild horse...:)&lt;/p&gt; &lt;p&gt;You can also read up &lt;A href="http://blogs.msdn.com/chappell/archive/2004/10/06/239001.aspx"&gt;over here&lt;/a&gt;, about differences between the STE / SDET positions and how vague and thin the line really is at times. &lt;/p&gt;&lt;img src="http://blogs.msdn.com/aggbug.aspx?PostID=359113" width="1" height="1"&gt;</description><category domain="http://blogs.msdn.com/nihitk/archive/tags/General+Software+Testing/default.aspx">General Software Testing</category></item><item><title>Spot the Bug</title><link>http://blogs.msdn.com/nihitk/archive/2005/01/23/359076.aspx</link><pubDate>Sun, 23 Jan 2005 20:40:00 GMT</pubDate><guid isPermaLink="false">91d46819-8472-40ad-a661-2c78acb4018c:359076</guid><dc:creator>nihitk</dc:creator><slash:comments>23</slash:comments><comments>http://blogs.msdn.com/nihitk/comments/359076.aspx</comments><wfw:commentRss>http://blogs.msdn.com/nihitk/commentrss.aspx?PostID=359076</wfw:commentRss><description>&lt;p&gt;Ran into this recently and thought it might be interesting. &lt;/p&gt; &lt;p&gt;Can you try and spot the bug and the possible error that can occur with the following lines of code?&lt;/p&gt;&lt;font color="#008080" size="3"&gt;&lt;font size="3"&gt; &lt;blockquote dir="ltr" style="MARGIN-RIGHT: 0px"&gt; &lt;p&gt;&lt;/font&gt;&lt;/font&gt;&lt;font color="#008080" size="3"&gt;Decimal&lt;/font&gt;&lt;font size="3"&gt; randomDoubleAsDecimal()&lt;br /&gt;{&lt;br /&gt;&lt;/font&gt;&lt;font color="#008080" size="3"&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;Random&lt;/font&gt;&lt;font size="3"&gt; rand = &lt;/font&gt;&lt;font color="#0000ff" size="3"&gt;new&lt;/font&gt;&lt;font size="3"&gt; &lt;/font&gt;&lt;font color="#008080" size="3"&gt;Random&lt;/font&gt;&lt;font size="3"&gt;();&lt;br /&gt;&lt;/font&gt;&lt;font color="#0000ff" size="3"&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;return&lt;/font&gt;&lt;font size="3"&gt; &lt;/font&gt;&lt;font color="#008080" size="3"&gt;Decimal&lt;/font&gt;&lt;font size="3"&gt;.Parse(rand.NextDouble().ToString());&lt;br /&gt;}&lt;/font&gt;&lt;/p&gt;&lt;/blockquote&gt; &lt;p dir="ltr"&gt;Solution in the next post.&lt;font size="3"&gt;&lt;/p&gt;&lt;/font&gt;&lt;img src="http://blogs.msdn.com/aggbug.aspx?PostID=359076" width="1" height="1"&gt;</description><category domain="http://blogs.msdn.com/nihitk/archive/tags/Geeky/default.aspx">Geeky</category><category domain="http://blogs.msdn.com/nihitk/archive/tags/General+Software+Testing/default.aspx">General Software Testing</category></item><item><title>Grads Getting Good Guidance</title><link>http://blogs.msdn.com/nihitk/archive/2005/01/16/354221.aspx</link><pubDate>Mon, 17 Jan 2005 04:55:00 GMT</pubDate><guid isPermaLink="false">91d46819-8472-40ad-a661-2c78acb4018c:354221</guid><dc:creator>nihitk</dc:creator><slash:comments>2</slash:comments><comments>http://blogs.msdn.com/nihitk/comments/354221.aspx</comments><wfw:commentRss>http://blogs.msdn.com/nihitk/commentrss.aspx?PostID=354221</wfw:commentRss><description>&lt;p&gt;&lt;a href="http://www.joelonsoftware.com/"&gt;Joel&lt;/a&gt; offers some &lt;a href="http://www.joelonsoftware.com/articles/CollegeAdvice.html"&gt;great tips&lt;/a&gt; to Computer Science students (or really anyone in college who is a geek and likes software and programming) who will soon be graduating - very practical advice. In addition, it would be great to see fresh graduates trying to incorporate testing into their goals as well. I took a look at one of my undergraduate projects (Static Analyzer for C++) and was amazed by how little testing we did on it. Maybe if graduates spend more time on testing and automating some of the tests for their projects, they will also develop an appreciation for the challenges and rewards that testing holds. Well - guess it's never too later too start. Would love to hear from anyone who actually used some automated testing for a college project of theirs?&lt;/p&gt; &lt;p&gt;Take a look at the &lt;a href="http://www.microsoft.com/college/ip_overview.mspx"&gt;internship options at Microsoft&lt;/a&gt; - definitely one of the most pampered and exhilarating internships (can vouch since I have been one myself) - and don't forget to ask for a test position if you think it might be your cup of tea!&lt;/p&gt;&lt;img src="http://blogs.msdn.com/aggbug.aspx?PostID=354221" width="1" height="1"&gt;</description><category domain="http://blogs.msdn.com/nihitk/archive/tags/Geeky/default.aspx">Geeky</category><category domain="http://blogs.msdn.com/nihitk/archive/tags/General+Software+Testing/default.aspx">General Software Testing</category></item><item><title>Are we there yet?</title><link>http://blogs.msdn.com/nihitk/archive/2004/12/31/345062.aspx</link><pubDate>Sat, 01 Jan 2005 01:14:00 GMT</pubDate><guid isPermaLink="false">91d46819-8472-40ad-a661-2c78acb4018c:345062</guid><dc:creator>nihitk</dc:creator><slash:comments>0</slash:comments><comments>http://blogs.msdn.com/nihitk/comments/345062.aspx</comments><wfw:commentRss>http://blogs.msdn.com/nihitk/commentrss.aspx?PostID=345062</wfw:commentRss><description>&lt;p&gt;So have we reached the top of the graph predicted by &lt;a href="http://en.wikipedia.org/wiki/Moores_Law"&gt;Moore's law&lt;/a&gt;? Seems like it (if not the top, very close to it), especially with respect to performance of a single, lonely processor. &lt;a href="http://pluralsight.com/blogs/hsutter/"&gt;Herb Sutter&lt;/a&gt; has written a &lt;a href="http://www.gotw.ca/publications/concurrency-ddj.htm"&gt;very nice article&lt;/a&gt; on how we are reaching the end of the road, or rather a bend in the road, which will require us to be much more aware of concurrency than we are at present. The article is slated to be printed in &lt;a href="http://www.ddj.com/"&gt;Dr. Dobb's Journal&lt;/a&gt; and the &lt;a href="http://www.cuj.com/"&gt;C/C++ Users Journal&lt;/a&gt;, in 2005 - but you can steal a sneak right now.&lt;/p&gt; &lt;p&gt;Interestingly, this article on the &lt;a href="http://www.pcworld.com/news/article/0,aid,119020,00.asp"&gt;PC of 2005&lt;/a&gt;, in &lt;a href="http://www.pcworld.com/"&gt;PCWorld&lt;/a&gt;, also mentions this exact same phenomenon, stating that few major hardware changes are expected in 2005. It even starts off with "Consumers thinking about buying a new computer in 2005 might be better off putting off their purchase until 2006" which is a very brave thing to say for a magazine which relies mainly on the health&amp;nbsp;of the&amp;nbsp;PC Industry!&lt;/p&gt; &lt;p&gt;So this might be a good time to&amp;nbsp;read up on&amp;nbsp;software concurrency&amp;nbsp;and how it relates to&amp;nbsp;development and testing of software. I know I need to.&lt;/p&gt; &lt;p&gt;P.S. - Wishing everyone a &lt;strong&gt;Happy New Year&lt;/strong&gt;! May it be a peaceful and joyous one for all.&lt;/p&gt;&lt;img src="http://blogs.msdn.com/aggbug.aspx?PostID=345062" width="1" height="1"&gt;</description><category domain="http://blogs.msdn.com/nihitk/archive/tags/Geeky/default.aspx">Geeky</category><category domain="http://blogs.msdn.com/nihitk/archive/tags/General+Software+Testing/default.aspx">General Software Testing</category></item><item><title>ZDrop for eBay Sellers</title><link>http://blogs.msdn.com/nihitk/archive/2004/10/05/238312.aspx</link><pubDate>Tue, 05 Oct 2004 22:46:00 GMT</pubDate><guid isPermaLink="false">91d46819-8472-40ad-a661-2c78acb4018c:238312</guid><dc:creator>nihitk</dc:creator><slash:comments>2</slash:comments><comments>http://blogs.msdn.com/nihitk/comments/238312.aspx</comments><wfw:commentRss>http://blogs.msdn.com/nihitk/commentrss.aspx?PostID=238312</wfw:commentRss><description>&lt;p&gt;Nice to &lt;a href="http://home.businesswire.com/portal/site/google/index.jsp?ndmViewId=news_view&amp;amp;newsId=20041004005904&amp;amp;newsLang=en"&gt;read about&lt;/a&gt; this very cool tool, &lt;a href="http://www.ztradingindustries.com/products/zdrop.aspx"&gt;zDrop&lt;/a&gt;,&amp;nbsp;running on &lt;a href="http://www.microsoft.com/commerceserver/"&gt;Commerce Server&lt;/a&gt; which helps eBay sellers to easily and cost effectively post and manage items on eBay.&lt;br /&gt;&amp;nbsp;&lt;br /&gt;Would love to hear about any other publicly available&amp;nbsp;tools you know of that utilize Commerce Server.&lt;br /&gt;&amp;nbsp;&lt;br /&gt;P.S. - Sorry for not posting in a while – have been working heads-down on the next release of Commerce Server. If you have any requests for features or any questions about the next release, feel free to ping me.&lt;br /&gt;&amp;nbsp;&lt;br /&gt;P.P.S – I am getting my first real-world experience with &lt;A href="http://blogs.msdn.com/nihitk/articles/144664.aspx"&gt;model based testing&lt;/a&gt; and loving it. It is like having a genius work for you (a very diligent and monotony-loving genius)…but there is quite a steep learning curve involved. Will write more about the lessons learnt, when I have actually learnt them…:)&lt;/p&gt;&lt;img src="http://blogs.msdn.com/aggbug.aspx?PostID=238312" width="1" height="1"&gt;</description><category domain="http://blogs.msdn.com/nihitk/archive/tags/Commerce+Server/default.aspx">Commerce Server</category><category domain="http://blogs.msdn.com/nihitk/archive/tags/General+Software+Testing/default.aspx">General Software Testing</category></item></channel></rss>