In the last post we decomposed the set of characters in the ANSI character set into valid and invalid class subsets for use in a base filename component on the Windows Xp operating system. The second part of the testing technique of equivalence class partitioning is to then use this information in tests that will adequately evaluate the functional capabilities of the base filename parameter used by COMDLG32.DLL in Windows Xp.
(Note: It is appropriate here to differentiate between a test case and a test. A test case is the hypothesis or assertion that I am either trying to prove or disprove. A test is the process or procedure I am using to prove or disprove my hypothesis. In other words, a test case in this example might be "Validate a base filename can be saved using valid characters in the ANSI 1252 character set." In order to prove or disprove this hypothesis, I will execute multiple tests using the various subsets outlined in the previous post, and as described below. If any test fails, the test case fails.)
There are basically 2 procedures for defining the tests that use the data from an equivalence class table. Paul Jorgensen describes a procedure for robust equivalence class testing in which each valid class subset is tested individually. Glenford Myers suggests a procedure in which valid class subsets are combined in a test until all valid class subsets have been covered. (Jorgensen refers to this as weak equivalence class testing.) Both authors agree that subsets in the invalid subsets must be tested in a way that only one parameter is invalid and any other parameters use valid variables. Using Myers' approach (or what Jorgensen describes as weak equivalence class testing) the tests for the base filename parameter are illustrated in the table below.
[Edited 11/30] I appearantly made too many assumptions about conducting these tests, so I must clarify some of those assumptions.
The total number of possible tests of valid string combinations for just the base filename parameter using only characters within the ANSI 1252 character set is 214251 + 214250 + 214249 + ...2142 + 2141. This number of tests, of course, is a virtual impossibility, so by employing the equivalence class partitioning testing technique we are able to systematically produce a minimum baseline set of tests that has a high probability of both proving or disproving our hypothesis or test purpose, as well as providing great variability in the test data to increase breadth of data coverage. The minimum possible number of valid tests determined by combining at least one element from each valid class subset is only 4 tests. But, let's look at each valid test a little more closely.
Test #3 is probably a red-herring! This is only an invalid filename on Windows NT 4.0 and below. So, if your application is ported from that time frame, and you are using a custom function for your file save functionality rather than Windows APIs then you might consider running this test once. If if passes, you can probably forget running this again ever again on that product. Test #1 evaluates the literal strings in valid subset V1 can be listed in an array or enumeration and one element can be selected at random throughout the development lifecycle, or each literal string can be tested once and the probability of failure in a later build is most likely less than .001%. Test #4 is also a test that probably doesn't require a great deal of retesting of various combinations of elements from subsets V2, V3, V4, V5 , & V7. Elements from the valid class subsets described in Test #2 are most interesting and this is the test that we will probably want to automate and run repeated throughout the development lifecycle because it provides great breadth of coverage. Remember this is the minimum number of valid tests. What isn't covered in this set is common or 'real-world' data sets which we would certainly want to include. Additionally, Test #2 relies on at least one element from each indicated subset. We might also want to consider additional tests that focus on subsets V4 and V5 only. Also, we might also consider testing valid class subset V6 as a special case if we suspected a private function excluded code point values that were not assigned character glyphs. However, if these 4 valid tests pass the probability of failure of any combination of these data sets used in this parameter is minimal. Random selection of elements for Test #2 (and possibly Test #4) may slightly increase the probability of exposing a defect in the base filename parsing routine. Tests #5 through #26 are tests for invalid filenames, or in the case of Test #19 and #20 where the expected result is to truncate the leading or trailing space character.
This of course only analyzes (or tests) the base filename parameter and assumes a nominal valid 3-letter extension, valid filenames do not preexist on the file system, and within the specific context described in the first post. If the context changes, (e.g. this example does not apply to Unix platforms, or Windows Vista, or other parameters) then this set of tests (assuming we would include at least Tests #2, and #5 through #26 as part of our regression suite ran on each new build) would provide a high level of confidence in the functional capability of this specific parameter.
Next, we would also decompose the data for the file extension parameter (which is different than the base filename parameter) because in the File name textbox we can enter either the base filename or the base filename and an extension, and once we verify the functionality of the base filename component, we can then proceed to the next step in analyzing the functionality of the arguments passed to the File name textbox parameter which we shall examine at a later date.
It is important to note the this and any other techniques are simply systematic procedures designed to help us wrap our minds around complex problems. They are not the only approach to testing (only a fool would advocate a single approach to testing), but when used in conjunction with various other techniques, methods, and approaches EQ can help to establish a strong foundation of confidence in low level operations. Of course, as has been previously stated, the limiting factor of this and other functional techniques is the knowledge of the professional tester to think critically and rationally analyze the "overall system" within the appropriate context.