Welcome to MSDN Blogs Sign in | Join | Help

Test Automation Foibles: Hard coded paths

Anyone who has listened to me talk about automated test design knows that I loathe hard-coded strings in test automation. Yes, I know that static test data is important, but it is almost never a good practice to hard-code strings or other test data in the body of your test code (or any code for that matter). 

But, as I started porting some of my demos to run on Windows Vista I realized that I violated my own axiom by hard-coding local paths for test artifacts (test files) in my own test code. Granted this code was for demonstration and training, but that is not an excuse for slovenly written code. Besides, as a general principle we should develop all code following which ever best practices you adhere to because we never really know who is going to critique that code or use it as an example elsewhere. 

Unfortunately, I am not alone in committing this mistake and I occasionally encounter hard coded paths in various automated test projects. Perhaps this occurs because it takes less effort (and lines of code) to hard code a path rather than get the path using a .NET method or Win32 API, or perhaps because we might assume the code will never run on a different machine (both of which are probably bad assumptions). No matter what the reason hard-coded paths are generally not a good practice.

In some automated tests I generate dynamic test data during runtime and create a file to temporarily store that data for use in the test, or save other user files with information that doesn't necessarily have to persist after the completion of that automated test. (In generating dynamic test data I only have to log or record the seed value used to generate the data.) In one demonstration I created test artifacts (test files) and saved them to the root of the C: drive. (Yes, I know the old PC-98 architecture used in Japan might not have a C: drive because it had a floating drive architecture, but on the PC/AT architecture running the Windows OS environment the default drive is almost always C:.)  Writing to the root directory of the C:\ drive was easy and relatively safe in strictly controlled environments, but it is certainly not the best thing to do (and generally bad from a security perspective). But, fast forward to Windows Vista and my code breaks. Even though I have administrator privileges when I run the demo code and attempt to save a file to the root of C:\ on the Vista platform the system gives me an error message indicating I don't have permission to save in that location. Unexpected dialog == unexpected behavior == code breaking. File not found by the test oracle in the expected location == false positive. Demonstration falling apart in front of an audience == embarrassment (because it really does no good to say "well, it worked on my other machine!").

Avoiding hard-coded paths for storing temporary test artifacts (e.g. disposable test files generated during the automated test) in automated test cases is easy with C# and the .NET framework. If we need the path to the desktop folder, or program files folder (or other special folders created by the Windows operating system) the GetFolderPath method in the System.Environment class gets the path to "special" system folders. The special folders include folders such as the system folder \windows\system32), the Program Files folder, the Desktop folder, the MyDocuments folder, etc. and are listed on MSDN under the Environment.SpecialFolder enumeration.)

For example, the following snippet returns a path to the current user's Documents folder ("C:\Users\[logged in account name]\Documents") which is a good place to store temporary test artifacts.

            string desktopFolderPath = Environment.GetFolderPath(
                Environment.SpecialFolder.MyDocuments);

It would probably be better to use a StringBuilder object as illustrated in the following example because a string is an immutable object in C#, and we will probably want to append the path separator and eventually a filename for the test artifact.

            StringBuilder desktopFolderPath = new StringBuilder(
                Environment.GetFolderPath(
                Environment.SpecialFolder.MyDocuments) +
                Path.DirectorySeparatorChar);

This returns a StringBuilder object of "c:\\users\\[logged in account name]\\desktop\\".

Another useful location to temporarily store disposable test artifacts might be the current user's temporary directory ("C:\\Users\\[logged in account name]\\AppData\\Local\\Temp\\") as illustrated in the following example:

            StringBuilder tempFolderPath = new StringBuilder(
                Path.GetTempPath());

Using the Environment.SpecialFolder enumeration or the .GetTempPath method are easy ways to get the path to a folder to store disposable test artifacts on the local machine without hard-coding a path in the automation code. It is likely this will be reused, so it is a good idea to wrap some methods for inclusion in your test automation framework or library. Also, don't forget to delete or move any test artifacts from the test machine after the automated test completes. It is generally a best practice to restore a system to its' pre-test state after your automated test executes.

Published Thursday, June 21, 2007 4:44 AM by I.M.Testy

Comments

# re: Test Automation Foibles: Hard coded paths

I generally prefer using Path.Combine rather than Path.DirectorySeparatorChar as I feel Path.Combine better describes what is happening. If I have a bunch of folders to string together, I'll do a String.Format with Path.DirectorySeparatorChar as one of the formatting arguments.

For a small number of string concatenations, StringBuilder can actually be less efficient than stringing together a series of "string + string + string". Short sequences of concatenations are so common the string concatenation operator is optimized for those scenarios.

Friday, June 22, 2007 2:58 PM by micahel

# re: Test Automation Foibles: Hard coded paths

Hi Michael,

Good point on using Path.Combine when combining two strings. For example, the following will return a string pointing to the logged in user's Documents folder then we can use Path.Combine to simply add the filename.

string myDocumentsFolderPath = Environment.GetFolderPath(   Environment.SpecialFolder.MyDocuments).ToString();

I agree with you that StringBuilder can be less efficient from a performance perspective than concatenating 2 or three strings. Here is a great article comparing StringBuilder and String. And if I am working simple strings then I also usually concatenate them because it is easy and fewer lines of code. :-)

But, I often work with char arrays and I find it easier to work with a StringBuilder object. Also since a string is immutable each time a string is concatenated the original string is discarded. That means there is a lot of memory deallocation and allocation going on. (Here is a good article illustrating this.) StringBuilder doesn't even cause a new allocation when converting ToString(). So, I am thinking that I am willing to sacrifice a few nanoseconds in perf time for simple concatenations in order to keep the automated test memory usage as small as possible so the test is less intrusive on the system. But, I guess it is six of one and half a dozen of the other. :-)

Sunday, June 24, 2007 2:07 AM by I.M.Testy

# re: Test Automation Foibles: Hard coded paths

you said on the PC-AT architecture there is always a C drive, but that Vista doesn't require that to be the case. Well, to be honest, Win2k didn't require it to be the case either, nor did XP or W2k3. just FYI... I had a C drive as a secondary disk, which was later removed, and since the D: drive was the active partition, the C drive never existed unless I put a USB flash drive in from time to time. And the funny thing is, sometimes, when an app installs, it creates C:\program files\... on that usb flash drive.

but I digress.

Passing parameters is good, but making sure library tasks in automation systems all pass the right parameters all the way down, can be a very big source of headaches....

Tuesday, July 10, 2007 3:41 PM by seanick

# re: Test Automation Foibles: Hard coded paths

Hi Seanick,

I think you are confusing the PC-AT hardware architecture and the location of the software boot loader. For example, even with Windows 3.0 I could install the OS on a D: drive; however the boot files (MSDOS.SYS, IO.SYS for example) lived on the boot sector of the C: drive.

Later changes in the BIOS allowed the active partition (where the boot loader lived) to float to a user defined active partition, and around 2000 Intel introduced the extensible firmware interface (EFI) as an abstraction layer between the hardware and firmware and the OS boot loader.

The reason your flash drive becomes C: is because the hardware is assuming that is a hard disk. (The PC-AT architecture that also reserves A and B drive letter assignments for floppy drives (although there are utilities to modify drive letters), and why when we plug in a flash drive to a computer similar to the setup you describe above and with only one floppy disk that flash drive is assigned C and not B for example.

Yes, the C:\Program Files problem has been around for a long time and is a classic setup testing bug.

For example, on the PC-98 floating drive architecture the disk the boot sector lived on became the A drive. If we only had one drive on that hardware platform, and launched Win95 we would see setup crashes all day long with various 3rd party apps that hard coded the C:\program files path.

Tuesday, July 10, 2007 8:24 PM by I.M.Testy

# re: Test Automation Foibles: Hard coded paths

"on the PC-AT architecture there is always a C: drive"

Wrong, as seanick pointed out.  The first time I installed an XP system without a C: drive it was accidental, but later I've done it intentionally in XP and Vista.  You described correctly that Windows 3.0 (also 95 and its successors) force letter C: (except on the PC98 architecture) onto the active partition that was loaded by the MBR, but seanick and I are right about NT and its successors.

"Later changes in the BIOS allowed the active partition (where the boot loader lived) to float to a user defined active partition,"

That is also wrong on the PC-AT architecture.  The BIOS loads the MBR.  PC-AT BIOSes know nothing or only a bit about partitions.  The MBR contains code that searches for the active partition, but even that doesn't assign a drive letter.  Different Windows series have different rules about how forcefully they assign drive letters.  Linux doesn't assign drive letters at all.

(Many PC-AT BIOSes know how to hibernate into a dedicated partition, and a few know how to hibernate RAM into a file on a FAT partition, but even those don't know how various Windows systems will assign drive letters.)

Thursday, February 28, 2008 9:28 PM by ndiamond

# re: Test Automation Foibles: Hard coded paths

Hi Mr. Diamond,

You're correct. I incorrectly stated "the BIOS allowed the active partition...". On the original PC/AT computer on start up the computer runs the basic input/output (BIOS) from ROM. The BIOS initializes devices such as disk drives. The BIOS loads the boot loader from the MBR of any controller card containing a BIOS extension. Today, the BIOS is a simple bootstrap to the more powerful 32-bit OS drivers that now control most of the hardware interactions.

Today, the CMOS passes info to the BIOS on startup; which by the way also controls such things as hybernation. BIOSes don't know how to hibernate...it's actually the CMOS.

Windows 3.0 or Windows 95 didn't force drive c: at all. In fact, I installed many EN-Win3.0 to a c: drive and JPN-Win3.0 to a D: drive, and installing Windows 95 to a D or other partitioned drive was a common test.

Some versions of Linux can be ran from flash memory (which looks like a BIOS). Linux was designed originally without an boot loader because it placed its kernel on a specific location on the disk (boot sector).

Thursday, February 28, 2008 11:48 PM by I.M.Testy

# re: Test Automation Foibles: Hard coded paths

"On the original PC/AT computer on start up the computer runs the basic input/output (BIOS) from ROM."

That still happens.  It does locate disk drives.  In the PC-AT architecture it chooses which disk drive to read the MBR, but after that the code in the MBR takes over.

The CMOS contains settings (values) but not executable code.

"Windows 3.0 or Windows 95 didn't force drive c: at all."

They forced drive letter C: onto the active partition of the booted drive.  Your article was right about those versions of Windows depending on files such as IO.SYS and MSDOS.SYS to be on the C: drive.  Of course the rest of the installed system could be in another partition but there still had to be a C: drive.

The NT series doesn't force that, though it usually occurred by default.  The active partition has to contain a few crucial files (NTLDR and friends, or Vista's equivalent) but the drive letter doesn't have to be C:.  Of course the rest of the installed system can be in another partition and there doesn't have to be a C: drive.

Friday, February 29, 2008 3:52 AM by ndiamond

# re: Test Automation Foibles: Hard coded paths

Great. Not sure I wrote anything about the CMOS containing executable code, and I am very well aware of the boot sector issues for DOS, Windows 3.x and 9x.

My example was an over-generalization because most people in the industry today simply do not have your superior knowledge and understanding of the system.

So, just to make you happy I have edited my statement in the post body to "...on the PC/AT using the Windows OS environment the default drive is almost always C:."

Of course, this discussion it really tangential and adds no relative value to the point of the post which is to avoid hard-coded strings in one's code.

Friday, February 29, 2008 12:11 PM by I.M.Testy

# re: Test Automation Foibles: Hard coded paths

"Not sure I wrote anything about the CMOS containing executable code"

This:  'Today, the CMOS passes info to the BIOS on startup'.  I guess you meant that the BIOS passes info to the OS on startup, but your fingers didn't type what you meant.  OK, but last week I only saw what I saw.

Your main post still says:  "the old PC-98 architecture used in Japan might not have a C: drive" which is true but which is a big understatement.  Seanick and I are still right that PC-AT architecture with Vista or XP or Server 2003 might not have a C: drive at all.  I guess the reason why I pounced in this discussion was that Seanick was right and you accused her/him of being confused.

This can still be an issue.  Here are a few anecdotes.  All of these are on PC-AT architecture within the last 5 years.

Since there are still a lot of defective programs that write stuff to C: where it doesn't belong, that might damage an unrelated Windows installation on C:, I learned not to install any OS to C:.  (On preinstalled systems on C:, if I don't reinstall, then I still use the OS on C:.)  A friend's system indeed ended up with a few directories on C: and mostly I could figure out which defective products had put them there.  Well next, that friend's system ended up with pagefile.sys on C: and warnings from XP about the pagefile being too small.  Some Knowledge Base articles say to install Intel Application Accelerator but Intel doesn't make an IAA for that friend's chipset.  Next I ended up with pagefile.sys on my own C: and the same warnings from XP.  Well guess what, Intel doesn't make an IAA for a Crusoe CPU and ALi chipset.  Eventually I got it cured on my machine but not on my friend's.

When my friend got a boot sector virus I zeroed his MBR and reinstalled.  At that time he had a media reader/writer, and although no flash media cards were inserted at the time, XP's installer assigned it drive letters starting with C:.  So his active partition was somewhere around E: and his Windows installation was somewhere around F:.  I don't recall the effect of buggy software that he reinstalled after that.

On a more modern machine, I got Vista and Server 2008 installed without C: drives at all.  The active partition gets D: and it only took two rounds of installation to have no C:.

On another more modern machine, I got XP installed without a C: drive at all.  The active partition gets D: and XP gets E:.  This one took some effort to create.  Now on this machine, the sound chip's vendor ships a user mode app that cooperates with their driver, and during an earlier installation when I had a C:, the app created a directory on C: where it didn't belong.  I haven't decided yet whether to try it again now that I have no C:.

Sunday, March 02, 2008 8:24 PM by ndiamond
Anonymous comments are disabled
 
Page view tracker