Re-inventing The Wheel

Unit Tests for the EnableLaunchApplication Task

Let's start to adress the negative thoughts I was having last time around: what happened to all the unit testing love I developed a little while back?

The truth is, I was having a hard time coming up with a good place to start writing tests. First of all, I wanted to write tests that tested my code, and not the Windows Installer or msbuild functionality, which presumbly is rock solid. Second, I'm still not very pleased with my code: its not very object-oriented, and (for some reason) I think that unit tests are best written against objects. Who knows, though: maybe writing unit tests will force me to structure my code better. We'll see.

So, now where to begin? I believe that tests and production code (if you can call this "production code") should be thoroughly de-coupled. But doesn't that mean that I can only reliably test things that are public? It seems like there is a potential problem: limiting surface area is great, but not at the expense of testing! Hopefully, the Team System folks thought about this. At the very least, we could probably use friend assemblies.

The last psychological hurdle I have to overcome is how to deal with the logistics behind the very nature of my tasks: updating a largish MSI. I was worried about how to get an MSI that I could run my tests against. But I was able to take some principles from Test-Driven Development in Microsoft .NET and their unit tests against code which writes to and from a database (I admit, I made reference to this very book in my first TDD example; but I've become a bigger fan after some re-reading over Frosted Mini-Wheats with my son). It makes sense there would be lessons to be learned from this: an MSI is a sort of database. We'll get to the solution after going through some logistics here.

What do we want to test first? We should probably come up with a complete list of test tasks, but let's start at the code I wrote most recently: the code to insert a row into a table, and code to modify a column within a row.

We're going to make some slight changes to the structure of this code. I don't like having this code tucked into a private method of the EnableLaunchapplication task; I think these functions have some serious potential to be used by other tasks. I think some modifications are also important if we want to move these changes into a more object-oriented world; lets set ourselves up so that this transition will hopefully be a little easier. Finally, the logistics behind testing this private method is a little screwy: the task opens the database and it will be hard having our test go through this code path, especially since I'm not sure I can trust the code in the task yet. Towards these ends, here is my first stab at a modification of this method:

public static void InsertIntoMsi(Database msi, string sql)
{
}

The signarute for this method is similar to what I had before with some notable differencecs:

• I added a new parameter: the Database that the code will be modifying.
• The method is public and static

I would think that if I ever did write a class to wrap the WindowsInstaller Database object, "Insert" is the sort of method that I would add to it. And it would  definitely be public there. For now, I don't have that wrapper class; instead I'll add this method to a Utilities class that I will add to the SetupProjects.Tasks namespace.

Let's create a project for the tests to live in by running through the Unit Test wizard. We will choose C# again as our language of choice. Let's create unit tests for SetupProjects.Tasks.Utilities. This generates a very basic test for us:

/// 

///A test for InsertIntoMsi (Database, string)
///
[TestMethod()]
public void InsertIntoMsiTest()
{
    Database msi = null; // TODO: Initialize to an appropriate value

    string sql = null; // TODO: Initialize to an appropriate value

    SetupProjects.Tasks.Utilities.InsertIntoMsi(msi, sql);

    Assert.Inconclusive("A method that does not return a value cannot be verified.");
}

Not a very useful test; let's see if we can perk it up a little. What do we want our test to accomplish? Fundamentally, we need to do the following:

• Create a database for our function to work against
• Insert a row somewhere in the database
• Verify that the row was added to the database

Sounds easy enough. Let's get started.

First and foremost, we're going to need an MSI to insert into. We could generate one by using a setup project to build an MSI, but I prefer an even blanker slate. The msi schema that setup projects use as a basis for its generated MSI is a good start. I will add this as a resource to my testing project, so that I can always extract the clean file from the assembly resources.

Open the project properties, select the Resources tab, and add the default resources file. Then move to the Add Resources chooser button and select Add Existing File..., and navigate to the schema.msi file (Program Files\Microsoft Visual Studio 8\Common7\Tools\Deployment\VsdSchema\schema.msi). This adds the schema to our project's resources with the name "schema".

The code to extract the file resource to a temporary file is pretty easy, and we've already seen how to open an Msi Database:

// Create a database for our function to work against
string databaseFileName = Path.GetTempFileName();
FileStream fout = new FileStream(databaseFileName, FileMode.Open);
fout.Write(SetupProjects.Tasks.UnitTests.Properties.Resources.schema, 0, SetupProjects.Tasks.UnitTests.Properties.Resources.schema.Length);
fout.Close();

Type classType = Type.GetTypeFromProgID("WindowsInstaller.Installer");
Object installerClassObject = Activator.CreateInstance(classType);
Installer installer = (Installer)installerClassObject;
Database msi = installer.OpenDatabase(databaseFileName, MsiOpenDatabaseMode.msiOpenDatabaseModeTransact);

Next on the agenda is to insert into the database. We need a good table to insert into. Ultimately, we should probably be super-thorough and test all plausible data types and configurations. However, I don't think a table like that exists in our schema, so I would have to create one myself. Let's wait on that until we need it: to me, that feels too much like testing the Windows Installer API directly, when we really want to test how well our code uses the Windows Installer API. Instead, let's pick a table that we expect to be pretty easy, like the Error table. Opening the schema in the MSI, we see that the Error table has two columns: Error, which takes a number, and Message which takes a string. Let's have a value of 1 for our error, and "Hello world" as our message:

// Insert a row somewhere in the database
int error = 1;
string message = "Hello world";
string sql = string.Format("INSERT INTO `Error` (`Error`, `Message`) VALUES ('{0}', '{1}')", error, message);
SetupProjects.Tasks.Utilities.InsertIntoMsi(msi, sql);

Finally, we need to verify that the entry was actually added. We will do this by querying the msi's to look in the Error table for our specific error number. Once we get the row from the database, make sure the columns are what we expect:

// Verify that the row was added to the database
string select = string.Format("SELECT `Error`, `Message` FROM `Error` WHERE `Error`={0}", error);
View view = msi.OpenView(select);
view.Execute(null);
Record record = view.Fetch();
Assert.IsNotNull(record);
Assert.AreEqual(error, record.get_IntegerData(1));
Assert.AreEqual(message, record.get_StringData(2));

Running the test gives us a failure, as we might expect: the body of our method is empty, so obviously the test won't pass. Here is the exact failure:

Failed InsertIntoMsiTest SetupProjects.Tasks.UnitTests Assert.IsNotNull failed.

Let's write some code to get this test passing:

public static void InsertIntoMsi(Database msi, string sql)
{
    View view = msi.OpenView(sql);
    view.Execute(null);
    view.Close();

    System.Runtime.InteropServices.Marshal.FinalReleaseComObject(view);
}

This is essentially the same code we had written before. And running the tests (after remembering to rebuild our test target. Have I mentioned before that this is really lame? Good.):

Passed InsertIntoMsiTest SetupProjects.Tasks.UnitTests

Looks like our test passed. Huzzah! Let's refactor our EnableLaunchapplication task to use this new code in Utilities by replacing
 InsertIntoMsi(
with
 Utilities.InsertIntoMsi(Msi,

Lets move on to the next method: ModifyTableData. Let's follow a pattern similar to what happened with InsertIntoMsi. Lets make our method signature:

public static void ModifyTableData(Database msi, string sql, int column, object value)

Lets add a unit test for this method. We can do that by right-clicking on our test project and select Add->Unit Test... and select the ModifyTableData method. This again generates some pretty lame test code for us. Before we fix this up, lets try to think of what our test case should do.

We will again use a simple table; the Error table worked for us last time, let's try it again. We will want to test changing 2 different types of values: an int as well as a string. But the structure should look similar to the previous test:

• Create a database for our function to work against
• Insert a row somewhere in the database
• Modify the contents of that row (using the code we want to test)
• Verify that the row has been modified.

Let's write our first test for this: a test which modifies an int value. We start by renaming the generated test to  ModifyIntegerTableDataTest(). We then start knocking off the goals above one step at a time. Creating a database to work against should be easy enough; we did that with the last test. Let's copy that into the beginning of this one:

public void ModifyIntegerTableDataTest()
{
    // Create a database for our function to work against
    string databaseFileName = Path.GetTempFileName();
    FileStream fout = new FileStream(databaseFileName, FileMode.Open);
    fout.Write(SetupProjects.Tasks.UnitTests.Properties.Resources.schema, 0, SetupProjects.Tasks.UnitTests.Properties.Resources.schema.Length);
    fout.Close();

    Type classType = Type.GetTypeFromProgID("WindowsInstaller.Installer");
    Object installerClassObject = Activator.CreateInstance(classType);
    Installer installer = (Installer)installerClassObject;
    Database msi = installer.OpenDatabase(databaseFileName, MsiOpenDatabaseMode.msiOpenDatabaseModeTransact);
}

Next, we insert into the database. It would be tempting to simply use the Utilities function we previously tested. However, I want to keep these tests isolated, so I will perform my insertion by actually doing the insertion myself:

// Insert a row somewhere in the database
int error = 1;
string message = "Hello world";
string sql = string.Format("INSERT INTO `Error` (`Error`, `Message`) VALUES ('{0}', '{1}')", error, message);

View view = msi.OpenView(sql);
view.Execute(null);
view.Close();

System.Runtime.InteropServices.Marshal.FinalReleaseComObject(view);

Now, modify the 1 to a 2 in the Error table:

// Modify the contents of that row (using the code we want to test)
sql = string.Format("SELECT `Error`, `Message` FROM `Table` WHERE `Error`={0}", error);
int newError = 2;
Utilities.ModifyTableData(msi, sql, 1, newError);

And finally vaidate that the table has actually been modified. I will do this by testing 3 different things:

• There is no row in our Error table with 1 as the Error value
• We have a row in our Error table with 2 as the Error value
• The Message for that row has "Hello world" as the message

// Verify that the row has been modified. // Look for the original row (should not find this one) string select = string.Format("SELECT `Error`, `Message` FROM `Error` WHERE `Error`={0}", error); view = msi.OpenView(select); view.Execute(null); Record record = view.Fetch(); Assert.IsNull(record); System.Runtime.InteropServices.Marshal.FinalReleaseComObject(view); // Look for the new row (should find this one) select = string.Format("SELECT `Error`, `Message` FROM `Error` WHERE `Error`={0}", newError); view = msi.OpenView(select); view.Execute(null); record = view.Fetch(); Assert.IsNotNull(record); Assert.AreEqual(newError, record.get_IntegerData(1)); Assert.AreEqual(message, record.get_StringData(2));

Running this test fails:

Failed ModifyIntegerTableDataTest SetupProjects.Tasks.UnitTests Assert.IsNull failed.

Because ModifyTableData is empty, the call to it doesn't modify that value, and therefore the record is found (and is not null). Hopefully, this will be fixed by actually implementing the method. We will move the code from the original location to the new Utilities area. After re-naming some variables, we end up with:

public static void ModifyTableData(Database msi, string sql, int column, object value)
{
    View view = msi.OpenView(sql);
    view.Execute(null);

    Record record = view.Fetch();
    if (value is int)
    {
        record.set_IntegerData(column, (int)value);
    }

    view.Modify(MsiViewModify.msiViewModifyReplace, record);
    view.Close();

    System.Runtime.InteropServices.Marshal.FinalReleaseComObject(record);
    System.Runtime.InteropServices.Marshal.FinalReleaseComObject(view);
}

Note that I removed the code for setting string data: we don't want to get ahead of our tests. Our new test passes. Just to be safe, re-run all tests and make sure they pass as well (which they do).

Before moving on to the next test, which will modify a string value, lets refactor the original tests by getting rid of the duplication of extracting our schema. We could refactor this into a new method and call that at the beginning of the test. But that's not nearly cool enough: by adding the TestInitialize attribute, it is possible to automatically dictate methods that get called prior to running a test. Let's see an example of this:

[TestInitialize()]
public void TestInitialize()
{
    // Create a database for our function to work against
    string databaseFileName = Path.GetTempFileName();
    FileStream fout = new FileStream(databaseFileName, FileMode.Open);
    fout.Write(SetupProjects.Tasks.UnitTests.Properties.Resources.schema, 0, SetupProjects.Tasks.UnitTests.Properties.Resources.schema.Length);
    fout.Close();

    Type classType = Type.GetTypeFromProgID("WindowsInstaller.Installer");
    Object installerClassObject = Activator.CreateInstance(classType);
    Installer installer = (Installer)installerClassObject;
    msi = installer.OpenDatabase(databaseFileName, MsiOpenDatabaseMode.msiOpenDatabaseModeTransact);
}

This code is essentially the same as we had before; the difference is that I fixed the scoping problem we would have had by making msi a member variable of this class. I'd show how we can now modify our tests to take advantage of this functionality, but the next test should demonstrate that as well. Our tests continue to pass.

Let's write our string modification test. It will look a lot like our integer modification test. The code is written below:

[TestMethod()]
public void ModifyStringTableDataTest()
{
    // Insert a row somewhere in the database
    int error = 1;
    string message = "Hello world";
    string sql = string.Format("INSERT INTO `Error` (`Error`, `Message`) VALUES ('{0}', '{1}')", error, message);

    View view = msi.OpenView(sql);
    view.Execute(null);
    view.Close();

    System.Runtime.InteropServices.Marshal.FinalReleaseComObject(view);

    // Modify the contents of that row (using the code we want to test)
    sql = string.Format("SELECT `Error`, `Message` FROM `Error` WHERE `Message`='{0}'", message);
    string newMessage = "A new message";
    Utilities.ModifyTableData(msi, sql, 2, newMessage);

    // Verify that the row has been modified.
    // Look for the original row (should not find this one)
    string select = string.Format("SELECT `Error`, `Message` FROM `Error` WHERE `Message`='{0}'", message);
    view = msi.OpenView(select);
    view.Execute(null);
    Record record = view.Fetch();
    Assert.IsNull(record);
    System.Runtime.InteropServices.Marshal.FinalReleaseComObject(view);

    // Look for the new row (should find this one)
    select = string.Format("SELECT `Error`, `Message` FROM `Error` WHERE `Message`='{0}'", newMessage);
    view = msi.OpenView(select);
    view.Execute(null);
    record = view.Fetch();
    Assert.IsNotNull(record);
    Assert.AreEqual(error, record.get_IntegerData(1));
    Assert.AreEqual(newMessage, record.get_StringData(2));
}

This test fails until we change ModifyTableData to account for strings:

public static void ModifyTableData(Database msi, string sql, int column, object value)
{
    View view = msi.OpenView(sql);
    view.Execute(null);

    Record record = view.Fetch();
    if (value is int)
    {
        record.set_IntegerData(column, (int)value);
    }
    else
    {
        record.set_StringData(column, value.ToString());
    }

    view.Modify(MsiViewModify.msiViewModifyReplace, record);
    view.Close();

    System.Runtime.InteropServices.Marshal.FinalReleaseComObject(record);
    System.Runtime.InteropServices.Marshal.FinalReleaseComObject(view);
}

And now the tests pass.

Now to eliminate duplication within the tests (I'd do the code first, but I think that's already done). The are all sorts of modifications that can be done. For example, the initial insertions in the Modify tests are EXACTLY the same; clearly this could be extracted into a method. Of course, those changes are also functionally equivalent to what happens in the Insert test. Maybe its time to review my feelings about using this method in our tests; I wish there was someone out there who could tell me what to do, instead of having me bumbling along looking for solutions (of course, if there was someone out there, I'd probably be quickly discouraged after they pointed out all of the things I had done wrong). Screw it, I'm going to use the Utilities function.

Making that change (which I won't include) leads to compiler errors, because I took out the declaration of view. But that's okay: I'm going to write a helper function in our test that retrieves a record based on a select statement:

private Record RetrieveRecord(string select)
{
    View view = msi.OpenView(select);
    view.Execute(null);
    Record record = view.Fetch();
    System.Runtime.InteropServices.Marshal.FinalReleaseComObject(view);
    return record;
}

And re-work our code so that this helper function is used.

There isn't too much I want to do here: I like my tests being explicit about the values inserted, modified, and tested, and I feel that further hiding these in helper functins would not be ideal. I like the tests where they are. I will show ModifyStringTableDataTest to show a typical example.

public void ModifyStringTableDataTest()
{
    // Insert a row somewhere in the database
    int error = 1;
    string message = "Hello world";
    string sql = string.Format("INSERT INTO `Error` (`Error`, `Message`) VALUES ('{0}', '{1}')", error, message);
    SetupProjects.Tasks.Utilities.InsertIntoMsi(msi, sql);

    // Modify the contents of that row (using the code we want to test)
    sql = string.Format("SELECT `Error`, `Message` FROM `Error` WHERE `Message`='{0}'", message);
    string newMessage = "A new message";
    Utilities.ModifyTableData(msi, sql, 2, newMessage);

    // Verify that the row has been modified.
    // Look for the original row (should not find this one)
    string select = string.Format("SELECT `Error`, `Message` FROM `Error` WHERE `Message`='{0}'", message);
    Record record = RetrieveRecord(select);
    Assert.IsNull(record);

    // Look for the new row (should find this one)
    select = string.Format("SELECT `Error`, `Message` FROM `Error` WHERE `Message`='{0}'", newMessage);
    record = RetrieveRecord(select);
    Assert.IsNotNull(record);
    Assert.AreEqual(error, record.get_IntegerData(1));
    Assert.AreEqual(newMessage, record.get_StringData(2));
}

One last bit of cleanup I do want to accomplish, though, is cleaning up our files when we're done. That's just good citizenship. This is easily accomplished through a method marked with the TestCleanup attribute:

[TestCleanup()]
public void TestCleanup()
{
    if (msi != null)
    {
        System.Runtime.InteropServices.Marshal.FinalReleaseComObject(msi);
    }
    if (File.Exists(databaseFileName))
    {
        File.Delete(databaseFileName);
    }
}

Once again, we are careful to release the msi prior to file deletion; otherwise, we get problems deleting the file as there is still an active handle on it.

There is still lots I would like to do to this code. First and foremost, I would like to make these utilitiy functions more user-friendly: having to write an entire SQL statement to insert or modify an entry is pretty lame. Its also time to consider turning our helper functions into something more: maybe put an object model in here somewhere, or turn the inserts and modifications into tasks.