Random Musings of Jeremy Jameson : Core Development

The "Copy Local" Bug in Visual Studio

Jeremy Jameson — Wed, 18 Nov 2009 12:21:00 GMT

If you've ever worked with me on a Microsoft Office SharePoint Server (MOSS) 2007 project -- or if you've read my Sample Walkthrough of the DR.DADA Approach to SharePoint -- then you've probably seen the following comment:

Note: Referenced assemblies must be specified with a path corresponding to the build configuration. If the path is not specified to the referenced assembly, then the build works fine as long as the referenced assembly is not in the GAC.

However, when the referenced assembly is in the GAC (i.e. after a deployment) then MakeCAB will not be able to find the referenced assembly (since it is no longer copied to the current project's bin\Debug or bin\Release folder).

It turns out that I was using a rather elaborate workaround for a problem that is actually much easier to solve.

To workaround the "Copy Local" bug and force a referenced assembly to always be copied to the output folder (regardless of whether the referenced assembly is in the GAC):

In the Solution Explorer window in Visual Studio, expand the References folder for the project and then select the referenced assembly.
In the Properties window, change the value of Copy Local to False, and then change it back to True.

Following these two simple steps explicitly adds <Private>True</Private> to the project file, as shown in the following example:

    <ProjectReference Include="..\CoreServices\CoreServices.csproj">
      <Project>{01C58D27-9818-45D6-A0B6-8EF765CA9397}</Project>
      <Name>CoreServices %28CoreServices\CoreServices%29</Name>
      <Private>True</Private>
    </ProjectReference>

When you add a referenced assembly in Visual Studio using a project reference, Copy Local defaults to True, but Visual Studio doesn't explicitly state this in the MSBuild project file. Toggling the value of Copy Local forces this element to be added to the project file and consequently you no longer need any hacks to reference the assembly in its original output folder.

At this point, you might be wondering why do I bring this up after all this time? After all, hasn't the hack I came up with for building SharePoint Web Solution Packages (WSPs) been working for several years? Well, yes, in most cases it works just fine.

However, there's one fundamental problem that I only recently discovered back in early October: you can't use Team Foundation Build to build the WSP when specifying relative paths to assemblies in the DDF file.

I'll cover this in my next post.

SketchPath - The XPath Tool

Jeremy Jameson — Wed, 18 Nov 2009 11:40:00 GMT

I added another tool to my Toolbox yesterday: SketchPath.

The SketchPath site labels it as "The XPath Tool" but I'd say it more like "The XPath Tool."

I've seen a few other tools for quickly building and testing XPath expressions against an XML document, but they pale in comparison to SketchPath. There are a number of online tools that unquestionably require less effort to get started with, but you'll likely find them very limiting as well.

I know I'm not the only one who thinks SketchPath rocks. Scott Hanselman included it in his 2009 Ultimate Developer and Power Users Tool List for Windows.

If you do any significant amount of work with XML, I recommend you download SketchPath today.

Compiling C++ Projects with Team Foundation Build

Jeremy Jameson — Sat, 07 Nov 2009 16:24:00 GMT

As I mentioned in my previous post, this week I incorporated Password Minder into my "Toolbox" Visual Studio solution that is scheduled to build daily through Team Foundation Server (TFS).

It's not that I really need daily builds of Password Minder; rather it's just been something on my "TO DO" list for a long time and I finally got around to doing it. Unfortunately, it wasn't without issue.

In case you are not familiar with Password Minder, it is mostly written in C#, but it includes one C++ project (for the NativeHelpers.dll).

Thus when I added the Password Minder projects to my "Toolbox" solution, I woke up the next morning to a notification from Team Foundation Server that my build failed. [No, I don't have some sort of alarm that goes off when one of my builds break. By "notification" I am referring to the e-mail message that I found in my inbox when I sat down at the computer.]

Clicking the build log referenced in the e-mail message, I quickly discovered the following:

Using "VCBuild" task from assembly "Microsoft.Build.Tasks.v3.5, Version=3.5.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a". Task "VCBuild" Locating vcbuild.exe: not found at "c:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\..\..\vc\vcpackages\vcbuild.exe". Locating vcbuild.exe: Visual C++ Express is not installed on this computer. Locating vcbuild.exe: falling back to the system PATH variable. C:\Users\svc-build\AppData\Local\Temp\Toolbox\Automated Build - Main\Sources\Source\Toolbox.sln : error MSB3411: Could not load the Visual C++ component "VCBuild.exe". If the component is not installed, either 1) install the Microsoft Windows SDK for Windows Server 2008 and .NET Framework 3.5, or 2) install Microsoft Visual Studio 2008.

That certainly is one of the best error messages I've seen in a long time. It told me exactly what I needed to do to fix the problem. Well, almost...

Note that DAZZLER (a VM in the "Jameson Datacenter" that is my dedicated build server) does not have the full installation of Visual Studio 2008. Rather it only has the Team Foundation Build install. In keeping with best practices, I try to keep the build server as "clean" as possible. That means no Visual Studio, no SharePoint, etc.

Following option #1 from the log file, I proceeded to install the Microsoft Windows SDK for Windows Server 2008 and .NET Framework 3.5 -- but, of course, I didn't bother to review the ReadMe file that comes with it (at least not at first).

Consequently, I was greeted with the following error on the next build attempt:

c:\Windows\Microsoft.NET\Framework\v3.5\Microsoft.Common.targets : warning MSB3428: Could not load the Visual C++ component "VCProjectEngine.dll". To fix this, 1) install the Microsoft Windows SDK for Windows Server 2008 and .NET Framework 3.5, 2) install Microsoft Visual Studio 2008 or 3) add the location of the component to the system path if it is installed elsewhere. System error code: 126. c:\Windows\Microsoft.NET\Framework\v3.5\Microsoft.Common.targets : warning MSB3425: Could not resolve VC project reference "..\NativeHelpers\NativeHelpers.vcproj".

That's when I discovered the following from the release notes for the SDK:

5.1.1 VCBuild fails to compile or upgrade projects

In order for VCBuild to run properly, vcprojectengine.dll needs to be registered. If vcprojectengine.dll is not registered, VCBuild.exe will fail with errors such as:

On compile: warning MSB3422: Failed to retrieve VC project information through the VC project engine object model. System error code: 127.

On upgrade: Failed to upgrade project file ‘foo.vcproj'. Please make sure the file exists and is not write-protected.

To workaround this issue, vcprojectengine.dll must be manually registered. From a Windows SDK command line window (as administrator in Vista:

On an X86 machine, run:
cd %mssdk%\VC\bin
regsvr32 vcprojectengine.dll
On an X64 machine, run:
cd %mssdk%\VC\bin\X64
regsvr32 vcprojectengine.dll

Unfortunately, these instructions aren't quite right -- or at least they didn't work verbatim in my environment. The workaround stated above makes you think there's an environment variable (%mssdk%) that refers to the path where the SDK is installed. However, this wasn't configured on DAZZLER.

Also, I didn't find my copy of VCProjectEngine.dll in an x64 folder, but rather in an amd64 folder:

C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\amd64\VCProjectEngine.dll

I know most of you out there are not doing much (if any) C++ work anymore, but I thought I should share this just in case you need it at some point.

Recommendations for Code Analysis

Jeremy Jameson — Sat, 31 Oct 2009 13:44:00 GMT

In my previous post, I briefly mentioned the Code Analysis feature of Visual Studio in the context of using check-in policies with Team Foundation Server (TFS). However, there's a lot more to talk about with regards to using Code Analysis.

If you are ever find yourself "starting from a clean slate" on a software development project, I hope you take advantage of the opportunity to not only enable Code Analysis, but also to treat all Code Analysis warnings as errors. That's right, go ahead and check all of those boxes (for both Debug and Release configurations, of course) so that whenever potential issues are reported by Code Analysis, it stops developers dead in their tracks and considers the build to be broken.

While you're at it, go ahead and select the All option in the Treat warnings as errors section on the Build tab (something I definitely recommend as a best practice). Again, be sure to do this for all build configurations.

Yes, this is going to be really painful for some developers.

For example, you can say goodbye to using the overload of string.Format() that doesn't specify a CultureInfo:

            string foo = "here";

            string logMessage = string.Format(
                "Imagine something interesting {0}.",
                foo);

Attempting this will now result in a broken build:

Error 2 CA1305 : Microsoft.Globalization : Because the behavior of 'string.Format(string, object)' could vary based on the current user's locale settings, replace this call in 'Program.Main(string[])' with a call to 'string.Format(IFormatProvider, string, params object[])'. If the result of 'string.Format(IFormatProvider, string, params object[])' will be displayed to the user, specify 'CultureInfo.CurrentCulture' as the 'IFormatProvider' parameter. Otherwise, if the result will be stored and accessed by software, such as when it is persisted to disk or to a database, specify 'CultureInfo.InvariantCulture'.

To avoid the CA1305 error, you will need to use something like this instead:

            string foo = "here";

            string logMessage = string.Format(
                CultureInfo.InvariantCulture,
                "Imagine something interesting {0}.",
                foo);

You will also need to enable signing of all assemblies with a strong name key, and also specify various assembly level attributes (e.g. [assembly: CLSCompliant(true)]). However, if you follow the steps I've provided in a previous post, this should only take you at most 1-2 minutes.

Like I said before, this will probably cause some heartburn in the beginning, but trust me, developers will quickly get used to it and the code they write will be much better as a result.

Now what about the other 99% of the time when you aren't starting from a clean slate, but instead are dealing with a substantial code base that causes the Code Analysis feature in Visual Studio to "light up like a Christmas tree" when you turn it on?

Well, as with all software bugs, it all comes down to triage. You obviously cannot simply halt all feature development in order to resolve code analysis issues. Instead, you should go for the "low hanging fruit" (meaning the issues that occur frequently but can be resolved with relatively little effort or risk of changing). You should also try to enable the Code Analysis rules that are most important, such as potential security issues.

The following post provides some great information about which Code Analysis warnings to focus your attention on first:

What rules do Microsoft have turned on internally? 2007-08-09.

http://blogs.msdn.com/fxcop/archive/2007/08/09/what-rules-do-microsoft-have-turned-on-internally.aspx

This article is obviously a little dated, but still very relevant over two years later.

Note that you will undoubtedly need to suppress some specific Code Analysis warnings from time to time, and therefore you'll have to choose how to suppress the warnings (whether inline within the source, or in a GlobalSuppressions.cs file). However, I hope you'll choose to fix the underlying issue that causes the Code Analysis warning (whenever practical) instead of simply suppressing a bunch of warnings.

So, in conclusion, think of the Code Analysis feature in Visual Studio as a way of choosing a more strict compiler. For example, back in the days when I used to program in C on Unix, the compiler wouldn't complain when I wrote something like this (even though this is obviously very wrong):

I was relieved when I switched to C++ and these kinds of bugs were a thing of the past. [Note that I never switch my coding style to use something like "1 == i" in order to avoid these errors.]

Fortunately, in C# you don't have a choice about whether or not this compiles (it doesn't).

I like to think of enabling all of the Code Analysis rules in a similar fashion. In other words, I want the compiler to tell me whenever I am doing something potentially wrong, because then I can spend just a little bit of time fixing my code to do it right. By forcing the Code Analysis warnings as errors, I don't have a choice -- meaning I have to fix them before check-in.

Recommended Check-In Policies for Team Foundation Server

Jeremy Jameson — Sat, 31 Oct 2009 12:14:00 GMT

I love using Team Foundation Server (TFS). There's just an amazing amount of "goodness" for software development that comes out-of-the-box; and there's even more available from Microsoft and other sources in the form of add-ons (many of which are free).

From a source control perspective, one of my favorite features is check-in policies. Anything that improves the quality of software development with minimal effort is really a "no-brainer" in my opinion.

The check-in policies that I prefer to have configured on all TFS projects are listed in the following table.

Recommended TFS Check-In Policies
Policy Type	Description
Builds	This policy requires that any code changes have been compiled and the last build was successful.
Work Items	This policy requires that one or more work items be associated with every check-in.
Changeset Comments Policy	Reminds users to add meaningful comments to their check-ins.
Code Analysis	This policy requires that Code Analysis is run with a defined set of rules before check-in.
Testing Policy	Ensures that tests from specific test lists are successfully executed before checking in.

The first two check-in policies are virtually painless and can be applied to any existing project at any time. These policies ship with Visual Studio Team System 2008, and assuming your developers have good software development discipline already, there's absolutely no reason not to enable these policies.

After all, why would any developer want to check-in code that hasn't been verified to at least compile? Similarly, why would a developer make a code change that isn't specifically related to a work item (even if that work item is simply something like "Refactor X" or "Code cleanup (M5)")?

However, the last three policies in the table above require a little more effort.

The Changeset Comments Policy is included in the Team Foundation Server Power Tools, which means that every member of the Development team will need to download and install this in order to use it. (Note that if you don't have the Power Tools installed, you can still override the check-in policy in order to avoid being blocked.)

The Code Analysis policy can definitely cause a little heartburn for your Development team, depending on how many code analysis rules you enable and whether or not you treat these warnings as errors. I'll talk more about Code Analysis in a separate post.

The Testing Policy is not one that I consider essential for all projects. For projects in which you have a good set of Build Verification Tests (BVTs) that give you a signficant amount of code coverage and don't take very long to execute, then enabling this policy just makes sense. However, keep in mind that you should definitely be running BVTs as part of your automated build, so if these tests require a substantial amount of time to execute (for example, they require signficant setup or teardown), then forcing developers to run these tests before every check-in could definitely impede their productivity.

Of course, you can always separate your tests into "quick tests" and "long running tests" like I've mentioned in a previous post, and consequently only require the "quick tests" to be executed before check-in.

If you are fortunate enough to be using TFS on your project, I hope you are reaping the benefits of check-in policies.

Build and Deployment Overview

Jeremy Jameson — Sat, 26 Sep 2009 19:40:00 GMT

This post provides a high-level walkthrough of the build and deployment process that I typically recommend on projects that I am involved with. It includes a series of illustrations that capture key concepts at various points in time. Specific parts of the process -- such as the development and build environments, versioning, and source control -- have already been covered in previous posts.

The following figure illustrates the initial build and deployment of the solution.

Figure 1: Initial build and deployment of the solution

See full-sized image.

Note the following key steps (corresponding to the various arrows in Figure 1):

Using their local environments, members of the Development team add new code – or modify existing code – and subsequently check the code into the source control system.
On the Build Server, the entire solution is labeled in the source control system to “snapshot” the exact state of the solution at the point in time just prior to the build. The label includes the version number (for example, 1.0.1.0 in the case of the initial build).
Using the label established in the previous step, all of the source code for the solution is copied from the Source Control Server to the Build Server, thereby ensuring that what is compiled into a specific version of the solution is based on the corresponding version of the source code in the source control system.
The solution is then compiled on the Build Server.
Next, the output of the build is copied from the Build Server to the Release Server (in a subfolder on the Builds share). The location of the build output on the Release Server includes the build number (e.g. 1.0.1.0) as well as the build configuration such as Debug or Release (note that the build configuration is not illustrated in Figure 1). The initial build often contains nothing more than the compiled code, which may be a group of assemblies (i.e. a number of EXE and/or DLL files) or a setup package (i.e. an MSI or CAB file) but most likely does not have an installation guide or installation scripts.
The solution is then installed by the Development team into the Development Integration Environment (DEV). This initial installation is typically performed manually (for example, by copying files, registering assemblies, modifying the server configuration, and/or interactively stepping through a setup program).

Figure 2 shows a subsequent build of the solution.

Figure 2: Subsequent build and deployment (with draft of installation guide)

See full-sized image.

The steps performed in the initial build (illustrated in Figure 1) are repeated in this subsequent build. The key difference illustrated in Figure 2 is the inclusion of the installation guide (which documents the manual steps performed by the Development team during the initial deployment). The new build (in this example, 1.0.2.0) is then deployed to the DEV environment by following the installation guide.

While this deployment process is functional -- in that a new build of the solution is successfully deployed to the DEV environment -- it requires too much manual effort to sustain the process. Figure 3 shows yet another subsequent build in which the manual installation to DEV has been replaced by an automated process.

Figure 3: Automated build and deployment (with install scripts and updated installation guide)

See full-sized image.

Note that in Figure 3, the automated installation process actually installs version 1.0.3.0 but installs it from the Latest folder on the Release Server. At the point in time represented in Figure 3, the contents of the Latest and 1.0.3.0 folders are identical. Using this mirrored structure greatly simplifies the automated installs of the latest build into the DEV environment. Also note that the build illustrated in Figure 3 includes install scripts, in addition to the build output (i.e. compiled code) and the installation guide (which has been updated to specify the install scripts to be used for installing the solution). These install scripts are a key piece of the automated installation process -- and subsequent “manual” installs into other environments (as described below).

Note that the process illustrated in Figure 3 does not require any manual intervention and can therefore be performed on a scheduled basis (à la, the “daily build”) or ad hoc, simply by executing the scheduled task as desired. Each subsequent build includes the source code changes checked-in to the source control system -- thus incorporating new features and bug fixes -- and is automatically installed into the DEV environment -- thus providing visibility to all team members on the progress of development. This is particularly useful for the Test team (in order to create test cases).

Note

The daily build provides a “heartbeat” of the development effort that can be monitored regularly by all team members – not just the Development team.

As this cycle repeats (as more features are implemented in the solution), the Test team – in conjunction with the Development team -- identifies a specific build of the solution to promote to the TEST environment. This is illustrated in Figure 4.

Figure 4: Installing the Beta 1 version to TEST

See full-sized image.

Note the following key concepts illustrated in the previous figure:

The build selected by the Test team does not necessarily correspond to the latest build -- as indicated by the ellipsis (…) after 1.0.57.0 in Figure 4. It is common for a later build of the solution to be running in the DEV environment, as a result of the daily build.
At one point in time, the version of the solution selected by the Test team was automatically installed to the DEV environment – as indicated by the dashed line in the previous figure.
Before installing the selected build to the TEST environment, the corresponding label in the source control system is annotated – as indicated by (Beta 1) in Figure 4. This allows the Development team to quickly get the version of the source code in order to debug problems encountered in the TEST environment.
The Test team installs the solution in the TEST environment by following the installation guide (which details the install scripts and configuration steps to be used).

Just as the daily build process (illustrated in Figure 3) is repeated throughout the development process, the promotion of selected builds to TEST is also periodically repeated, albeit much less frequently. This is illustrated in Figure 5 with the first “release candidate” of the solution.

Figure 5: Installing the RC1 version to TEST

See full-sized image.

Note that if issues are discovered with the RC1 build, it may be necessary to promote yet another build to the TEST environment, as illustrated in Figure 6.

Figure 6: Installing the RC2 version to TEST

See full-sized image.

Assume that the second release candidate (version 1.0.379.0 in Figure 6) passes all of the test cases. The final step is to install the solution into the production environment, as illustrated in Figure 7.

Figure 7: Installing the V1.0 release to PROD

See full-sized image.

Note the following key concepts illustrated in the previous figure:

Release Management – not the Test team – is responsible for installing the solution into the PROD environment.
The build installed into PROD was previously installed and evaluated in the DEV and TEST environments.
Before installing the selected build to the PROD environment, the corresponding label in the source control system is annotated – as indicated by (V1.0) in Figure 7. The Development team also creates a branch in the source control system from this version in order to support “hotfixes” (patches) and minor releases (e.g. V1.1).
Release Management installs the solution in the PROD environment by following the installation guide (which details the install scripts and configuration steps to be used) just as the Test team did when installing the solution into the TEST environment.

Best Practices for SCM and the Daily Build Process

Jeremy Jameson — Sat, 26 Sep 2009 19:12:00 GMT

In a previous post, I briefly discussed a simple branching strategy for Team Foundation Server (TFS). This was somewhat of a follow-up to another post in which I briefly referenced a great article titled The Importance of Branching Models in SCM. If you haven't read this article, I highly recommend it.

However, while branching is certainly an important aspect of Software Configuration Management (SCM) -- and perhaps one of the most important aspects -- it's certainly not the only one that warrants discussion. In this post, I want to cover some other best practices for SCM as well as the daily build process.

For example, if you are using TFS, you know that each TFS project typically has a source control repository as well as a project team site in SharePoint (a.k.a. the "Project Portal"). You also probably know that all SharePoint document libraries can be configured to support versioning similar to TFS source control, and thus allow you to, for example, view an old version of an installation guide stored on the Project Portal. However, just because you can do this doesn't mean that it is recommended as a best practice.

Note that I'm not saying you should not store any documents on the Project Portal, nor am I saying that you should never enable versioning on document libraries in the Project Portal.

On the contrary, the Project Portal provides a great place to store documents that various project stakeholders need to access from time to time (without going through Visual Studio or TFS Web Access), and I almost always recommend enabling versioning on all document libraries in your SharePoint sites (provided you have plenty of storage on the backend SQL Server, of course).

Enabling versions on document libraries provides the ability to go back and see who changed a document and when (which is sort of like an "audit trail" on a document). While you could use the SharePoint versioning feature to go "back in time" in order to view the state of a document (e.g. an installation guide) for a particular version or build of your solution, this would obviously involve a good deal of considerable effort based on timestamps.

A much easier way of correlating a version of a particular document to a particular build of your solution is to actually store the document in source control within TFS. This allows you to quickly retrieve the document version using a changeset number or, more generally, a build label.

Build Labels

Back in the days of using Visual SourceSafe (VSS) on customer projects, at the beginning of the build process, a label would always be applied to the project in VSS. Since the build label was applied at the top-level project within a branch, it applied to all files in the solution – including the source code, setup files, and automated tests. This ensures that the installation and tests could be repeated for any particular build.

Note

While VSS certainly provided the ability to retrieve a project based on a particular timestamp, it definitely wasn't easy (the only way that I was ever to do it was through the SS command line utility). Thus build labels provided a quick way of getting a snapshot of the code for a particular build.

With the concept of changesets in TFS -- and the ability to quickly get the code for a specific changeset -- build labels are obviously not as important as they were in VSS. However, regardless of which particular source control system you are using, build labels provide an easy way to identify and retrieve important builds (such as Beta 1, Beta 2, RC1, and v1.0) or simply specific versions of the solution (e.g. 1.0.57.0).

Source Code

Applying the build label to the entire solution allows the Development team to obtain all of the source code for a particular build, thereby enabling them to step through the code to debug an issue that may not be reproducible in the latest version of the code.

Setup Files

In addition to the source code, the files supporting the installation of the solution (such as the install scripts and the installation guide – but not the compiled setup packages) should also stored in the source control system. In this way, the changes to the installation can be tracked from one version to another.

Automated Tests

Automated tests change as features are added to the solution. Consequently the tests must be matched to a specific build and therefore need to be checked into the source control system (and therefore labeled as part of the correponding build). Automated tests typically include unit tests as well as Build Verification Tests (BVTs).

Daily Build Process

The build process is a fundamental activity within any software project. In order to ensure success, the build and deployment process needs to conform to a number of basic requirements. If the solution cannot be built directly from its source code then it is not possible to integrate many of the key processes required for successful delivery.

The following sections describe the steps in the build process and who is responsible for each part of the process:

Check-Ins

All code included in a build must be checked into source control before the build process is initiated. All checked-in code must compile and it is the responsibility of the developer who checks in the code to ensure that the solution builds and all files that are needed to build the solution are checked in.

Important

The source control must not be left in a broken state at any time. If a build breaks, resolving the problem becomes the highest priority.

Tip

You can use the Builds check-in policy for TFS to ensure the solution compiles before a developer is allowed to check-in a changeset.

Each member of the Development team is responsible for ensuring the following:

Check-ins must not break functionality in the solution. If a developer checks in a change in their area of the code, he or she must not expect another developer to resolve errors caused by the check-in; either the person checking in the code must fix the dependant code when they check-in or the developer must work with the other developer(s) to coordinate the check-in.
The installation process must also be kept in a functional state at all times. Whenever a developer adds a feature, he or she is responsible for making sure that any setup changes are completed as well. Feature owners are ultimately responsible for the installation and configuration of their features.
The developer is responsible for writing any custom installation actions needed to deploy their feature, for knowing what files need to be deployed on which server, and what permissions the feature needs to run.
The installation of the feature should be completed when the code is checked into source control. The development of any custom installation actions needs to be done in conjunction with the team lead(s), and should not be done in isolation.
Changes to the installation should be checked in to give the team sufficient time to test the changes before the next scheduled build.
Unit tests must pass in the local developer’s environment (LOCAL) before a check-in is performed. If the unit tests are broken because of changes to the feature’s functionality, the developer must resolve the failures -- or reach agreement with the rest of the Development team to disable certain tests until they can be fixed (which should be a rare occurrence).

Tip

You can use the Testing Policy check-in policy for TFS to ensure that specific unit tests pass before a developer is allowed to check-in a changeset.

If any of the above conditions cannot be met, the code should not be checked into source control.

Automated Build Process

The automated build process should:

Automatically increment the Build Number or Revision portion of the assembly version (depending on which branch is being built)
Apply a build label (if appropriate)
Retrieve the code from the source control system
Compile the solution
Run any associated unit tests
Copy the build to the Release Server

If an error occurs during the build -- including execution of the unit tests -- the entire process stops. In this event, an email detailing the error should be mailed to the appropriate distribution list. Once the error is fixed, the build can be restarted. Note that the build/revision number should be incremented for the restarted build.

Build Verification Tests

Build Verification Tests (BVTs) are a quick, automated test run on the solution to catch defects in the code. The BVTs should cover the major functionality of the solution but are not meant to be a full regression of all the functionality in the solution.

The time required to run the entire set of BVTs should be on the order of minutes -- not hours. Otherwise the tests become too cumbersome to run on a regular basis. [Note that this guideline obviously depends on the size of your solution and the resources involved. For example, I believe the BVTs used by the Windows team actually run for several hours, due to the size of the code base. However, it is still short enough to repeat on a daily basis.]

BVTs are owned by the Test team in conjunction with the Development team to verify the build has completed successfully and to catch any regressions that are introduced as a result of changes to the solution. BVTs should be launched immediately after the automated build and deployment process. The BVTs must return an error to indicate if they have succeeded or failed. Only after the BVTs have passed should the build be considered ready for additional testing by the Test team.

As noted before, BVTs are owned by Test. The Test team should work closely with Development to keep the BVTs in sync with the changes in the solution. By having the BVTs owned by Test, it ensures that there is a knowledge transfer between Development and Test for any changes to the behavior or architecture of the solution.

Smoke Tests

In order to successfully deploy the solution, automated tests should be written to survey the environments before and after the application has been deployed. The smoke tests should check that the correct components have been installed, the configuration of the solution has been completed correctly, and that the network connectivity between machines is functioning correctly. The smoke tests must result in a clear pass or fail.

Smoke tests are generally considered to be more exhaustive than BVTs. The key differentiator is that BVTs are intended to identify whether a build is worthy of additional testing. Ideally, a substantial portion of the testing performed by the Test team is automated -- not manual. So whether you refer to your automated tests as BVTs and consider them part of a "smoke test" is really just a matter of semantics.

Investigating Failures

When a failure is found with the BVTs or automated smoke tests, the Test team should be the first point of contact to ensure that the tests are correct and the failure is valid.

Build Hand-off

The Release Management team should always lead the deployment of the solution, with Test and Development provided support as necessary.

Note that the Development team owns the DEV environment. Consequently, developers will often resort to “tweaking” the environment in order to get the solution deployed and operational. However, in order for the solution to be deployed to TEST, the configuration changes made in DEV must either automated as part of the installation or thoroughly documented as part of the installation guide.

Troubleshooting Guide

When problems are discovered with the deployment of the solution, a troubleshooting guide should be created (or updated) to capture the experience learned in resolving problems in the solution. The troubleshooting guide is not meant to be exhaustive in terms of troubleshooting all aspects of the solution; however it should provide a baseline for developing the operational support documents for maintaining the solution in the Production environment.

The troubleshooting guide should capture the process and tools used for investigating the problems with the solution and where to look for logs, events, etc. in the system. It should also capture patterns that have been seen in the solution that point toward a particular fault in the system.

Tip

A SharePoint site -- or even just a simple SharePoint list -- provides an excellent alternative to a Troubleshooting Guide document. Think of this site --or list -- as a simple "Knowledge Base" for your solution.

Debugging Symbols -- They're Not Just for Debug Builds Anymore

Jeremy Jameson — Sat, 26 Sep 2009 16:20:00 GMT

I started another new project this week.

Typically one of the first tasks on any new development project is to create a Development Plan that provides consistent guidelines and processes for the Development team. On this new project, another Microsoft consultant had already created a draft of the Development Plan, but in the process of reviewing it, I added some content from a Development Plan that I had created a few years ago.

In the process of reviewing my old document, I came across the following:

Installation
...
Debug Symbols

All Debug builds should create symbol files for debugging purposes. These symbols are included as part of the setup to facilitate debugging in other environments such as DEV.

Important

Do not include Debug symbols in the Release configuration of the setup projects.

When I read this, I actually let out an audible laugh (okay, I suppose it was more of a chuckle). It must have been the old C++ developer in me that originally put this in the Development Plan (thinking you should never provide PDB files in your Release builds because it makes it all too easy for an outsider to understand your code).

Well, any .NET developer who has ever fired up Reflector on somebody else's assembly (which hasn't been obfuscated, obviously) knows very well how easy it is to decompile -- er, I mean disassemble -- source code. In fact, regardless of whether you have the PDB files, you can actually debug .NET code -- including setting breakpoints and examining variables -- using tools like WinDbg. I've had to do a little of this in the past and while it's not exactly easy -- especially for a developer who doesn't use WinDbg frequently -- it does help out in a pinch when troubleshooting some nasty problem in Production.

However, including debugging symbols (i.e. PDB files) in Release builds certainly makes debugging .NET code easier. This is a key point that John Robbins makes in Debugging Microsoft .NET 2.0 Applications. In fact, here's a direct quote from page 38:

Build All Builds with Debugging Symbols
...build all builds, including release builds, with full debugging symbols. [...]

In other words, the Development Plan should say:

Important

Always include Debug symbols in the Release configuration of the setup projects -- or, preferably, make them available from a symbol server.

Regarding John's book...I strongly recommend this book to anyone who considers himself or herself a "serious developer." It is chock full of great tips and recommendations for developing .NET solutions.

Development and Build Environments

Jeremy Jameson — Fri, 25 Sep 2009 15:57:00 GMT

In a previous post, I briefly touched on the "DEV-TEST-PROD" triad of environments that I typically recommend (at a minimum) for every organization doing any form of software development.

This post describes, in greater detail, the various environments used for developing, building, testing, and deploying a solution. [If you've ever worked on a project with me, you'll find this content is typically found in the Development Plan deliverable.]

When developing custom solutions, I typically recommend a minimum of four environments:

Since it is assumed that members of the Development team already have individual workstations or laptops, I typically refer to the abbreviated “DEV-TEST-PROD” triad when referencing the various environments.

Note

For very large development efforts, there may be multiple physical environments corresponding to DEV, TEST, and PROD. For example, numerous “labs” are used within Microsoft for testing various configurations of Windows. However this is not always necessary depending this size of a project. PROD might also refer to multiple physical locations (i.e. for disaster recovery) and depending on release schedules, an additional Maintenance Environment (MAINT) might also be necessary.

In addition to describing each environment, this post includes descriptions of the additional servers that are necessary to support the development process. These additional servers include the following:

Local Developer Environments (LOCAL)

The “LOCAL” environment refers to the standard machine configuration (or virtual machine configuration) for each developer. It is recommended that this environment have all the applications required for the solution and all the tools required for building and debugging the solution, as well as the standard tools that are needed on the project (such as Microsoft Office).

Note

Developers must have administrative privileges on their own local environments in order to facilitate development and debugging.

Most of the functionality for the solution should be able to run in a LOCAL environment. This allows the team members to work independently on their areas of the solution, without having to compete for a single environment resource. The majority of a developer’s assigned functionality should be able to be coded and unit tested in his or her own LOCAL environment. This reduces the contention in larger teams over access to the environments.

There should be a minimum number of differences between the solution running in a developer’s LOCAL environment and the solution running in the DEV, TEST, or PROD environments. Certain differences are expected, however, as there may be some changes required in order to reduce costs or to emulate the behavior of a shared resource.

Some organizations use the term “sandbox” instead of LOCAL when referring to this environment. I prefer the term LOCAL since it more accurately reflects the intent of the environment and does not imply that the environment is completely isolated (for example, when integrating with external systems, each developer’s LOCAL environment is often configured to connect to the systems corresponding to the DEV environment).

Development Integration Environment (DEV)

The “DEV” environment is similar to each developer’s LOCAL environment. The primary purpose of DEV is to provide an integration environment where the latest build of the complete solution can be evaluated, including all of the various pieces contributed by various members of the Development team. DEV should have all the applications used in the solution and the tools to debug them.

Note

This environment is owned by the Development team. All members of the Development team (or some subset, depending on the size and experience of the team) have administrative privileges in the DEV environment. Having administrative privileges allows developers to easily debug and reconfigure the environment.

After the Build Server finishes compiling the source code, the solution is installed on the DEV environment. This is the first location where the installation process is tested. It is also an important platform for ensuring the basic functionality of the solution works.

The DEV environment is important for catching problems early on. Any project stakeholder, not just members of the Development team, can use the DEV environment to evaluate the progress of the solution.

Important

No development should occur on the DEV environment (in other words, no files should ever be checked out directly to the DEV environment). The DEV environment is shared by the entire team and must be treated appropriately as a shared resource. Careful consideration must be made when debugging in this environment (since this will likely interrupt anyone else accessing the DEV environment). Generally speaking, debugging should only take place on the DEV environment when the issue cannot be reproduced in a LOCAL environment. Notification should be provided to the entire team prior to initiating a debugging session directly on the DEV environment (this will also greatly reduce the time required to investigate the issue by avoiding concurrent activities).

Build Server

The Build Server compiles all the code in the solution and creates the installation. It should have the same configuration as a Local Developer Environment (LOCAL). The Build Server should be dedicated to building the solution and should not contain any unnecessary applications or tools on it. The machine should be kept as clean and stable as possible so that it is in a good state to perform the builds.

All official builds that are distributed to be tested must be built on the Build Server.

Note

Depending on the size of the organization and sophistication of the automated build process, a subset of the Development team may need administrative privileges on this server.

Source Control Server

The source control system should be deployed on a server whose primary purpose is to be the central repository for the source code. This should be dedicated hardware that has sufficient resources to avoid bottlenecks when numerous team members are working concurrently (e.g. 100Mb network connectivity between the source server and the development machines).

The source control server should perform maintenance checks on a regular basis. Backups should be scheduled at least once daily to ensure that the files can be recovered in the event of a failure. Backups should be kept in a separate location to ensure they can be recovered in case of hardware failure.

Note

The Development team has limited read-write privileges on this server (i.e. the minimum required for the source control system to function properly).

Release Server

The Release Server keeps an archive of the output from each successful build. Under the share that is used to archive the solution, there is a folder corresponding to each build. Each build folder contains all of the files necessary to install the solution. There is typically a folder for each project configuration in the solution (e.g. Debug, Release, etc.).

For example:

file://BuildServer/Builds/Fabrikam/Project1/1.0.1.0/Debug

In addition to the folders corresponding to each build, there is a folder named Latest that always contains the most recent successful build. This simplifies the process of installing the solution to the DEV environment.

Note

All team members should have read-only access to the Builds share. Only the service account used for the build process should have read-write access to the Builds share.

The shared folder on the Release Server should be protected by ACLs to ensure that only members of the solution team can access the shares.

Important

The Test and Release Management teams should only install the solution from the Release Server (or, preferrably, a CD or DVD containing the files from the Release Server). Furthermore, only explicit build folders should be used by the Test and Release Management teams. In other words, the Latest folder should only be used for automated installs into the DEV environment – it should not be used for installations in the TEST or PROD environments.

Test Environment (TEST)

This environment should mirror the same configuration as the production environment as much as possible. Having an exact replica of the deployed environment ensures that the deployment is correct and repeatable when it goes live. In order to minimize costs, TEST may not always have the same level of fault tolerance as the production environment.

The primary purpose of TEST is to provide the Test team with an isolated environment for validating test cases. In addition to functional testing, TEST is also used for performance testing and “soak” testing (long-running test scenarios primarily intended to identify resource leaks). Provided the differences between TEST and PROD are only regarding true fault tolerance (and not scale), then performance testing in TEST should accurately predict the performance of PROD.

Note

This environment is owned by the Test team. No member of the Development team should have privileges in TEST above those of the end users of the solution. Members of the Test team should have the additional privileges necessary to install the solution and assist the Development team with debugging issues in the TEST environment.

Typically, testers create their test cases and initially regress defects using the DEV environment, however the formal validation of each test case must be performed using the TEST environment.

Having a separate environment is valuable because it allows the Test team to decide which specific build of the solution should be tested at any given time. “Smoke” tests are typically performed on each build in DEV. Periodically the Test team selects a successful build to promote to TEST.

Production Environment (PROD)

The Production environment is where the solution is ultimately deployed for end users. Strict security procedures must be enforced in this environment.

Note

Only the Release Management team should have privileges in PROD above those of the end users of the solution.

Comparing Source Code Branches

Jeremy Jameson — Wed, 16 Sep 2009 15:54:00 GMT

During the more than three years I spent helping Agilent Technologies migrate their Internet site from their legacy, proprietary platform to Microsoft Office SharePoint Server (MOSS) 2007, we unfortunately never used Team Foundation Server (TFS). Instead, we used Visual SourceSafe (VSS) in combination with a "Work Items" list in SharePoint that I've described in previous posts.

While I certainly prefer TFS over VSS, sometimes you simply have to concede that you can't have everything you would like on a customer project and move on to actually getting the work done.

However, just because we used VSS doesn't mean we didn't follow good Software Configuration Management (SCM) principles. For example, as I've described in the past, I insist on using branching right from the start. Thus when I setup the Visual Studio solution for the Agilent project, I created a Main branch and subsequently created branches for the various releases (e.g. v1.0, v2.0, and v3.0).

The particular branch that a developer uses would thus depend on whether the changes are for the next major release or a QFE (hotfix) for the version running in Production. For example, after deploying the Technical Support site (i.e. v2.0), we began working on the "General Site" (i.e. v3.0). [Note that in Agilent's terminology, the "General Site" essentially refers to everything outside of Technical Support, the Literature Library, and the Online Store (i.e. the "Buy" tab).]

However, since we didn't create the v3.0 branch until shortly before the v3.0 release, all v3 ("General Site") development was initially done on the Main branch -- just like v2 (Tech Support) development was initially done in Main prior to creating the v2.0 branch.

This branching strategy works really well, regardless of which particular SCM system you actually use. The key thing to remember is that all of the changes should eventually make it into the Main branch (since that branch will eventually be used to create another branch for the next major release).

The problem with VSS is that while it certainly supports branching, the merging features, um...well, let's just say that they leave a lot to be desired ;-)

Personally speaking, I've never felt comfortable using the out-of-the-box merging features in VSS, and instead always insisted on merging the changes from one branch to another manually. TFS is obviously years ahead of VSS in terms of branching and merging, but as I said before, sometimes you simply have to deal with what you've got.

Fortunately, long before the Agilent project, I had previously created my own process that takes a great deal of the "pain" out of manually merging source code. Here is what I came up with.

In my Toolbox, I have two simple scripts: DiffBranches.cmd and CopyBranch.cmd.

Here are the contents of DiffBranches.cmd:

@echo off

setlocal

REM set DIFFTOOL=Windiff.exe
set DIFFTOOL=C:\NotBackedUp\Public\Toolbox\DiffMerge\DiffMerge.exe

set BRANCH1=%1
set BRANCH2=%2

if ("%BRANCH1%") == ("") set BRANCH1=Main

if ("%BRANCH2%") == ("") set BRANCH2=v3.0

call CopyBranch.cmd "%BRANCH1%" "%BRANCH1%_tmp"

call CopyBranch.cmd "%BRANCH2%" "%BRANCH2%_tmp"

"%DIFFTOOL%" "%BRANCH1%_tmp" "%BRANCH2%_tmp"

As you can see, there's not much to it. I simply make temporary copies of the two branches (i.e. by copying the branch folder into a new folder appended with "_tmp") and then use my "Diff Tool" to compare the two folders. Originally, I used WinDiff, but once I discovered DiffMerge, I quickly switched to using it exclusively for all of my "diff'ing" activities.

The real "magic" lies in CopyBranch.cmd:

@echo off

setlocal

set BRANCH1=%1
set BRANCH2=%2

if ("%BRANCH1%") == ("") set BRANCH1=Main

if ("%BRANCH2%") == ("") set BRANCH2="%BRANCH1%_tmp"

robocopy "%BRANCH1%" "%BRANCH2%" /E /MIR /XD bin obj TestResults /XF *.scc *.suo *.user *.vspscc

Note that I put the word magic in quotes because it's really not magic at all. When copying a source code branch, I simply use robocopy.exe and specify the following options:

robocoby.exe command-line options used in CopyBranch.cmd
Command-Line Option	Comments
/E	Copy subdirectories, including empty ones
/MIR	Mirror the directory tree, thus ensuring that when I subsequently run CopyBranch.cmd, any deleted files from the original branch are removed from the "temporary" copy of the branch)
/XD bin obj TestResults	Exclude directories matching the given names/paths, thus skipping the compiled output (i.e. bin and obj) as well as the test results generated by running unit tests from within Visual Studio
/XF .scc .suo .user .vspscc	Exclude files matching the given names/paths/wildcards, thus skipping source code control files (.scc and .vspscc), and Visual Studio solution/project user-specific options (i.e. .suo and .user)

For example, let's suppose that I've checked in some changes to the v3.0 branch that need to be propagated to the Main branch. I would open a command prompt and run the following:

C:\NotBackedUp\Agilent>DiffBranches.cmd v3.0 Main

After the two branches are copied to their respective temporary folders, DiffMerge.exe is launched to compare the two branches. Since I exclude many of the directories and files that are expected to differ between the two branches, I can quickly view only the differences that I am interested in. I can even use DiffMerge.exe to apply the changes interactively (checking out the files to be updated beforehand as necessary). This greatly reduces the effort involved in manually merging changes from one branch into another.

Note that it takes a little bit of time to copy a branch the first time, but this is lightning fast on subsequent runs. Hence why I don't delete the "_tmp" folders after comparing two branches.

I occasionally use this process even on projects where we've used Team Foundation Server. For example, I sometimes create a new workspace in TFS to represent a "temporary branch" where I want to experiment with some substantial changes to the source code (while I still have pending changes in my main workspace, that I may have shelved but don't want to check-in just yet). Using a little robocopy.exe and DiffMerge.exe, I can more easily compare two branches or workspaces.

Using SysPrep'ed VHDs for New Hyper-V Virtual Machines

Jeremy Jameson — Thu, 13 Aug 2009 18:16:00 GMT

As noted in my previous post, I spent 7 days in "alpha" training last week for SharePoint 2010.

Consequently, one of my goals for this week was to update my "dogfood" VM with the CTP build of SharePoint Server 2010, so that I can continue building on the foundation of what I learned last week and dive deeper into the new features of the product.

This is the quintessential scenario for why I love virtual machines. I can quickly "spin up" a new VM and then proceed to load various products such as SQL Server 2008, SharePoint Server 2010, and Visual Studio 2010; and, since it's isolated in a VM, there's no risk with exploring early, pre-release builds of software.

Note that I don't start from scratch each time I need to create a new VM. Rather, I start from a SysPrep'ed image of the particular operating system that I want to use.

In this case, I copied my ws2008-std-x64.vhd (Windows Server 2008 Standard x64 with Service Pack 2) over to the folder for my DOGFOOD VM and renamed it -- yep, you guessed it -- dogfood.vhd.

I then proceeded to start the DOGFOOD VM using Hyper-V Manager. Unfortunately, I encountered the following error:

Virtual Machine Connection

The application encountered an error while attempting to change the state of 'dogfood'.

'dogfood' failed to start.

Microsoft Emulated IDE Controller (Instance ID {GUID}): Failed to power on with Error 'General access denied error'

IDE/ATAPI: Could not attach 'C:\NotBackedUp\VMs\dogfood\dogfood.vhd' to location 0/0 of IDE Controller. Error: 'General access denied error'

The file 'C:\NotBackedUp\VMs\dogfood\dogfood.vhd' does not have the required security settings. Error: 'General access denied error'

Using the cacls.exe utility, I examined the permissions specified for a VHD used by one of the other VMs that was already running on the same server. I found the following:

NT VIRTUAL MACHINE\48FEF7FD-04D0-4AFF-B6F F-4F0D159DC40E:(special access:)
- READ_CONTROL
- SYNCHRONIZE
- FILE_GENERIC_READ
- FILE_GENERIC_WRITE
- FILE_READ_DATA
- FILE_WRITE_DATA
- FILE_APPEND_DATA
- FILE_READ_EA
- FILE_WRITE_EA
- FILE_READ_ATTRIBUTES
- FILE_WRITE_ATTRIBUTES
NT AUTHORITY\SYSTEM:(ID)F
BUILTIN\Administrators:(ID)F
BUILTIN\Users:(ID)R

Note that the list of discrete permissions listed for the NT VIRTUAL MACHINE service account essentially equates to Read and Write, as observed by running the new icacls.exe tool introduced in Windows Server 2003 SP2, and also included in Windows Vista and Windows Server 2008.

Consequently, you can specify a command like the following to grant permissions on the VHD to the service account used by Hyper-V:

icacls dogfood.vhd /grant "NT VIRTUAL MACHINE\C60995EB-8B7D-46DB-BD1E-3638CD1AEC32":(R,W)

Note that the GUID varies with each VM, so you'll need to tweak this accordingly for your environment. To find the GUID to use, examine the Virtual Machines folder for the VM.

Also note that you may need to use the VM settings to remove the VHD from the IDE controller and subsequently re-add it (in other words, it looks like Hyper-V does not recognize that the file permissions have been updated in the background).

It takes about 10 minutes to copy the SysPrep'ed VHD, which is obviously much, much faster than the time it takes to install Windows Server 2008 from scratch (especially when you take patching into account). Overall, it takes me about 25-30 minutes to spin up a new Windows Server 2008 VM, which includes joining it to my Active Directory domain and getting all of the latest patches applied using Windows Server Update Services (which, in the "Jameson Datacenter" is enforced through Group Policy).

Configuring Logging in ASP.NET Applications (and SharePoint)

Jeremy Jameson — Fri, 19 Jun 2009 04:38:00 GMT

This post continues on the original post for my simple, but highly effective approach to logging and the follow-up post which introduced configuring logging for console applications.

Obviously not all solutions are simple console-based applications. With ASP.NET Web services and applications -- including Microsoft Office SharePoint Server (MOSS) and Windows SharePoint Services (WSS) -- you can still view log messages from the Logger class very easily on a per-request basis.

Note that the System.Web.WebPageTraceListener can be specified in Web.config to enable logging to the ASP.NET tracing feature:

  <system.diagnostics>
    <sources>
      <source name="defaultTraceSource" switchName="allTraceLevel">
        <listeners>
          <add name="webPageTraceListener" />
        </listeners>
      </source>
    </sources>
    <sharedListeners>
      <add type="System.Web.WebPageTraceListener, System.Web, Version=2.0.0.0,
Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"
        name="webPageTraceListener" traceOutputOptions="None" />
    </sharedListeners>
    <switches>
      <add name="allTraceLevel" value="All" />
      <add name="verboseTraceLevel" value="Verbose" />
      <add name="infoTraceLevel" value="Info" />
      <add name="warningTraceLevel" value="Warning" />
      <add name="errorTraceLevel" value="Error" />
      <add name="offTraceLevel" value="Off" />
    </switches>
  </system.diagnostics>

Also note that ASP.NET tracing must be enabled using the <trace> element within <system.web>:

  <system.web>
    <trace enabled="true" pageOutput="false" requestLimit="50" localOnly="true" />
  </system.web>

Be sure to set the requestLimit high enough to enable access to the page trace you are interested, but also be aware that you can easily clear the captured traces and then browse to the page of interest again.

Once configured, browse to the page on the site. Then modify the URL to browse to Trace.axd (e.g. http://fabrikam-local/Trace.axd) to display the Trace Viewer.

Figure 1: ASP.NET Trace Viewer

See full-sized image.

Locate the request that you want to view log messages for and click the corresponding View Details link.

Figure 2: ASP.NET Trace Viewer

See full-sized image.

Notice that the WebPageTraceListener even formatted the warning message in red. How cool is that?! [For all you SharePoint developers out there, compare this with "diving" into the ULS logs to find an error or warning for a particular page request!]

Be aware that the Web.config file you use for SharePoint applications varies depending on whether you want to view log messages for a content page or for an application (i.e. _layouts) page. This is covered in my next post.

Configuring Logging in a Console Application

Jeremy Jameson — Fri, 19 Jun 2009 03:50:00 GMT

In my previous post, I introduced my simple, but highly effective approach to logging. As promised, this post introduces how to configure logging (based on a simple console application).

As noted in my previous post, one or more trace listeners can be configured for a particular System.Diagnostics.TraceSource.

To understand the configuration of various trace listeners, consider the following console application:

using System;
using System.Diagnostics;
using System.Globalization;
using System.Text;

using Fabrikam.Demo.CoreServices.Logging;

namespace Fabrikam.Demo.AdminConsole
{
    class Program
    {
        static void Main(
            string[] args)
        {
            try
            {
                LogCommandLine("AdminConsole.exe", args);

                // Parse the command line arguments and initialize the work
                // ...

                // Process the work
                Process();
            }
            catch (Exception ex)
            {
                Logger.LogCritical(ex.Message + "\n" + ex.StackTrace );
                Logger.Flush();
                Environment.Exit(1);
            }

            Logger.LogInfo("Program complete.");
            Logger.Flush();
        }

        private static void LogCommandLine(
            string program,
            string[] args)
        {
            Debug.Assert(string.IsNullOrEmpty(program) == false);
            Debug.Assert(args != null);

            StringBuilder commandLineBuilder = new StringBuilder(program);
            for (int i = 0; i < args.Length; i++)
            {
                commandLineBuilder.Append(" ");
                commandLineBuilder.Append(args[i]);
            }

            string commandLine = commandLineBuilder.ToString();
            Logger.LogInfo(commandLine);
        }        

        private static void Process()
        {
            Logger.LogDebug("Starting processing...");
            
            // Simulate the processing of a work item queue.
            for (int i = -1; i <= 4; i++)
            {
                ProcessWorkItem(i);
            }            

            Logger.LogInfo("Successfully completed processing.");
        }
        private static void ProcessWorkItem(
            int workItemId)
        {
            // If an error occurs processing a work item, we want to
            // log a warning, but continue processing the next work item.
            try
            {
                // Simulate processing of the work item by simply
                // showing the square root of the work item Id.
                ShowSquareRoot(workItemId);
            }
            catch (ArgumentException ex)
            {
                Logger.LogWarning(
                    CultureInfo.InvariantCulture,
                    "Unable to process work item {0} ({1}).",
                    workItemId,
                    ex.Message);
            }
        }

        private static void ShowSquareRoot(
            double d)
        {
            if (d < 0.0)
            {
                throw new ArgumentException(
                    "The value must be greater than or equal to zero.",
                    "d");
            }


            Logger.LogDebug(
                CultureInfo.InvariantCulture,
                "Computing square root of {0}...",
                d);

            double sqrt = Math.Sqrt(d);
            Debug.Assert(double.IsNaN(sqrt) == false);

            Console.WriteLine(
                "The square root of {0} is {1:0.000}.",
                d,
                sqrt);
        }
    }
}

Running this program without configuring any trace listeners results in the following output to the console:

The square root of 0 is 0.000. The square root of 1 is 1.000. The square root of 2 is 1.414. The square root of 3 is 1.732. The square root of 4 is 2.000.

Notice that none of the log messages are displayed in the console output.

However, if we start DebugView before running the program, then we can easily view all of the log messages.

Figure 1: DebugView (a.k.a. Dbgview.exe)

See full-sized image.

To show the log messages in the console, we simply need to configure a listener. For a console application, this is done using the application configuration file (which is located in the folder with the application executable and has the name of the application with the ".config" file name extension added).

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <system.diagnostics>
    <sources>
      <source name="defaultTraceSource" switchValue="All">
        <listeners>
          <add name="consoleListener" />
          <!--
            Remove the Default listener to avoid duplicate messages
            being sent to the debugger for display
          -->
          <remove name="Default" />
        </listeners>
      </source>
    </sources>
    <sharedListeners>
      <add
        type="System.Diagnostics.ConsoleTraceListener"
        name="consoleListener"
        traceOutputOptions="None" />
    </sharedListeners>
  </system.diagnostics>
</configuration>

With this configuration, the console output shows:

defaultTraceSource Information: 0 : AdminConsole.exe defaultTraceSource Verbose: 0 : Starting processing... defaultTraceSource Warning: 0 : Unable to process work item -1 (The value must be greater than or equal to zero. Parameter name: d). defaultTraceSource Verbose: 0 : Computing square root of 0... The square root of 0 is 0.000. defaultTraceSource Verbose: 0 : Computing square root of 1... The square root of 1 is 1.000. defaultTraceSource Verbose: 0 : Computing square root of 2... The square root of 2 is 1.414. defaultTraceSource Verbose: 0 : Computing square root of 3... The square root of 3 is 1.732. defaultTraceSource Verbose: 0 : Computing square root of 4... The square root of 4 is 2.000. defaultTraceSource Information: 0 : Successfully completed processing. defaultTraceSource Information: 0 : Program complete.

Notice that System.Diagnostics.ConsoleTraceListener outputs the name of the TraceSource (which in this case is "defaultTraceSource") as well as the event Id (which, for my simple Logger class is always 0). It can also be configured to show other fields, such as thread Id.

A TraceSwitch can also be configured to control tracing and debug output. For example, the following configuration specifies that the trace source should ignore low-level debugging and information messages, and only show warning and error messages.

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <system.diagnostics>
    <sources>
      <source name="defaultTraceSource" switchName="warningTraceLevel">
        <listeners>
          <add name="consoleListener" />
          <!--
            Remove the Default listener to avoid duplicate messages
            being sent to the debugger for display
          -->
          <remove name="Default" />
        </listeners>
      </source>
    </sources>
    <sharedListeners>
      <add
        type="System.Diagnostics.ConsoleTraceListener"
        name="consoleListener"
        traceOutputOptions="None" />
    </sharedListeners>
    <switches>
      <!-- Use this switch to output all log messages -->
      <add name="allTraceLevel" value="All" />
      <!-- Use this switch to output low-level debugging, informational,
        warning, and error messages -->
      <add name="verboseTraceLevel" value="Verbose" />
      <!-- Use this switch to output informational, warning,
        and error messages -->
      <add name="infoTraceLevel" value="Info" />
      <!-- Use this switch to output warning and error messages -->
      <add name="warningTraceLevel" value="Warning" />
      <!-- Use this switch to output error messages -->
      <add name="errorTraceLevel" value="Error" />
      <!-- Use this switch to output no tracing messages -->
      <add name="offTraceLevel" value="Off" />
    </switches>
  </system.diagnostics>
</configuration>

With this configuration, the console output is:

defaultTraceSource Warning: 0 : Unable to process work item -1 (The value must be greater than or equal to zero. Parameter name: d). The square root of 0 is 0.000. The square root of 1 is 1.000. The square root of 2 is 1.414. The square root of 3 is 1.732. The square root of 4 is 2.000.

With just a little more configuration, we can specify that we want all log messages to go to the console (for example, so the person running the utility can easily monitor the progress of some long-running operation) but only output information, warning, and error messages to a file (possibly for later review).

In the following configuration, note the new logFileTraceListener and the corresponding <filter> element used to limit the file output to level "Information" (and above -- which includes warning and error messages).

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <system.diagnostics>
    <sources>
      <source name="defaultTraceSource" switchName="allTraceLevel">
        <listeners>
          <add name="consoleListener" />
          <add name="logFileTraceListener" />
          <!--
            Remove the Default listener to avoid duplicate messages
            being sent to the debugger for display
          -->
          <remove name="Default" />
        </listeners>
      </source>
    </sources>
    <sharedListeners>
      <add
        type="System.Diagnostics.ConsoleTraceListener"
        name="consoleListener"
        traceOutputOptions="None" />
      <add
        type="System.Diagnostics.TextWriterTraceListener"
        name="logFileTraceListener"
        initializeData="AdminConsole.log"
        traceOutputOptions="DateTime">
        <!--
          Use a filter so that log file excludes low-level debugging messages
        -->
        <filter type="System.Diagnostics.EventTypeFilter"
          initializeData="Information" />
      </add>
    </sharedListeners>
    <switches>
      <!-- Use this switch to output all log messages -->
      <add name="allTraceLevel" value="All" />
      <!-- Use this switch to output low-level debugging, informational,
        warning, and error messages -->
      <add name="verboseTraceLevel" value="Verbose" />
      <!-- Use this switch to output informational, warning,
        and error messages -->
      <add name="infoTraceLevel" value="Info" />
      <!-- Use this switch to output warning and error messages -->
      <add name="warningTraceLevel" value="Warning" />
      <!-- Use this switch to output error messages -->
      <add name="errorTraceLevel" value="Error" />
      <!-- Use this switch to output no tracing messages -->
      <add name="offTraceLevel" value="Off" />
    </switches>
  </system.diagnostics>
</configuration>

This produces a file named AdminConsole.log with the following content:

defaultTraceSource Information: 0 : AdminConsole.exe
    DateTime=2009-06-18T16:57:54.4488429Z
defaultTraceSource Warning: 0 : Unable to process work item -1 (The value must be greater than or equal to zero.
Parameter name: d).
    DateTime=2009-06-18T16:57:54.4798429Z
defaultTraceSource Information: 0 : Successfully completed processing.
    DateTime=2009-06-18T16:57:54.4818429Z
defaultTraceSource Information: 0 : Program complete.
    DateTime=2009-06-18T16:57:54.4828429Z

Unfortunately the TextWriterTraceListener class appends the timestamp onto a separate line, but you can easily substitute a custom listener that formats results like the following, if necessary:

2009-06-18T17:17:43 Information: AdminConsole.exe
2009-06-18T17:17:43 Warning: Unable to process work item -1 (The value must be greater than or equal to zero.
Parameter name: d).
2009-06-18T17:17:43 Information: Successfully completed processing.
2009-06-18T17:17:43 Information: Program complete.

When I wrote the Logger class a few years ago, I also wrote a SimpleTraceListener class to do just this:

using System;
using System.Collections;
using System.Diagnostics;
using System.Globalization;
using System.IO;
using System.Runtime.InteropServices;
using System.Security.Permissions;

namespace Fabrikam.Demo.CoreServices.Logging
{
    /// <summary>
    /// 
    /// </summary>
    [HostProtection(SecurityAction.LinkDemand, Synchronization = true)]
    public class SimpleTraceListener : TextWriterTraceListener
    {
        /// <summary>
        /// Initializes a new instance of the <see cref="SimpleTraceListener"/>
        /// class.
        /// </summary>
        public SimpleTraceListener()
        {
        }

        /// <summary>
        /// Initializes a new instance of the <see cref="SimpleTraceListener"/>
        /// class, using the specified stream as the recipient of the debugging
        /// and tracing output.
        /// </summary>
        /// <param name="stream">A <see cref="Stream"/> that represents the
        /// stream the <see cref="SimpleTraceListener"/> writes to.</param>
        public SimpleTraceListener(
            Stream stream)
            : base(stream, string.Empty)
        {
        }

        /// <summary>
        /// Initializes a new instance of the <see cref="SimpleTraceListener"/>
        /// class, using the specified writer as the recipient of the debugging
        /// and tracing output.
        /// </summary>
        /// <param name="writer">A <see cref="TextWriter"/> that receives the
        /// output from the <see cref="SimpleTraceListener"/>.</param>
        public SimpleTraceListener(
            TextWriter writer)
            : base(writer, string.Empty)
        {
        }

        /// <summary>
        /// Initializes a new instance of the <see cref="SimpleTraceListener"/>
        /// class, using the specified file as the recipient of the debugging
        /// and tracing output.
        /// </summary>
        /// <param name="fileName">The name of the file the
        /// <see cref="SimpleTraceListener"/> writes to.</param>
        public SimpleTraceListener(
            string fileName)
            : base(fileName)
        {
        }

        /// <summary>
        /// Initializes a new instance of the <see cref="SimpleTraceListener"/>
        /// class with the specified name, using the specified stream as the
        /// recipient of the debugging and tracing output.
        /// </summary>
        /// <param name="stream">A <see cref="Stream"/> that represents the
        /// stream the <see cref="SimpleTraceListener"/> writes to.</param>
        /// <param name="name">The name of the new instance.</param>
        public SimpleTraceListener(
            Stream stream,
            string name)
            : base(stream, name)
        {
        }

        /// <summary>
        /// Initializes a new instance of the <see cref="SimpleTraceListener"/>
        /// class with the specified name, using the specified writer as the
        /// recipient of the debugging and tracing output.
        /// </summary>
        /// <param name="writer">A <see cref="TextWriter"/> that receives the
        /// output from the <see cref="SimpleTraceListener"/>.</param>
        /// <param name="name">The name of the new instance.</param>
        public SimpleTraceListener(
            TextWriter writer,
            string name)
            : base(writer, name)
        {
        }

        /// <summary>
        /// Initializes a new instance of the <see cref="SimpleTraceListener"/>
        /// class with the specified name, using the specified file as the
        /// recipient of the debugging and tracing output.
        /// </summary>
        /// <param name="fileName">The name of the file the
        /// <see cref="SimpleTraceListener"/> writes to.</param>
        /// <param name="name">The name of the new instance.</param>
        public SimpleTraceListener(
            string fileName,
            string name)
            : base(fileName, name)
        {
        }

        private bool IsEnabled(
            TraceOptions options)
        {
            return ((options & this.TraceOutputOptions) != TraceOptions.None);
        }

        /// <summary>
        /// Writes trace information, a message, and event information to the
        /// listener specific output.
        /// </summary>
        /// <param name="eventCache">A TraceEventCache object that contains the
        /// current process ID, thread ID, and stack trace information.</param>
        /// <param name="source">A name used to identify the output, typically
        /// the name of the application that generated the trace event.</param>
        /// <param name="eventType">One of the TraceEventType values specifying
        /// the type of event that has caused the trace.</param>
        /// <param name="id">A numeric identifier for the event.</param>
        /// <param name="message">A message to write.</param>
        [ComVisible(false)]
        public override void TraceEvent(
            TraceEventCache eventCache,
            string source,
            TraceEventType eventType,
            int id,
            string message)
        {
            if ((this.Filter == null)
                || this.Filter.ShouldTrace(
                    eventCache, source, eventType, id, message, null, null, null))
            {
                this.WriteHeader(source, eventType, id, eventCache);
                this.WriteLine(message);
                this.WriteFooter(eventCache);
            }
        }

        /// <summary>
        /// Writes trace information, a formatted array of objects and event
        /// information to the listener specific output.
        /// </summary>
        /// <param name="eventCache">A TraceEventCache object that contains the
        /// current process ID, thread ID, and stack trace information.</param>
        /// <param name="source">A name used to identify the output, typically
        /// the name of the application that generated the trace event.</param>
        /// <param name="eventType">One of the TraceEventType values specifying
        /// the type of event that has caused the trace.</param>
        /// <param name="id">A numeric identifier for the event.</param>
        /// <param name="format">A format string that contains zero or more
        /// format items, which correspond to objects in the
        /// <paramref name="args"/> array.</param>
        /// <param name="args">An object array containing zero or more objects
        /// to format.</param>
        [ComVisible(false)]
        public override void TraceEvent(
            TraceEventCache eventCache,
            string source,
            TraceEventType eventType,
            int id,
            string format,
            params object[] args)
        {
            if ((this.Filter == null)
                || this.Filter.ShouldTrace(
                    eventCache, source, eventType, id, format, args, null, null))
            {
                this.WriteHeader(source, eventType, id, eventCache);
                if (args != null)
                {
                    this.WriteLine(string.Format(
                            CultureInfo.InvariantCulture, format, args));
                }
                else
                {
                    this.WriteLine(format);
                }
                this.WriteFooter(eventCache);
            }
        }

        private void WriteFooter(
            TraceEventCache eventCache)
        {
            if (eventCache == null)
            {
                return;
            }

            this.IndentLevel++;
            if (this.IsEnabled(TraceOptions.ProcessId))
            {
                this.WriteLine("ProcessId=" + eventCache.ProcessId);
            }
            if (this.IsEnabled(TraceOptions.LogicalOperationStack))
            {
                this.Write("LogicalOperationStack=");
                Stack logicalOperationStack = eventCache.LogicalOperationStack;
                bool flag = true;
                foreach (object obj2 in logicalOperationStack)
                {
                    if (!flag)
                    {
                        this.Write(", ");
                    }
                    else
                    {
                        flag = false;
                    }
                    this.Write(obj2.ToString());
                }
                this.WriteLine(string.Empty);
            }
            if (this.IsEnabled(TraceOptions.ThreadId))
            {
                this.WriteLine("ThreadId=" + eventCache.ThreadId);
            }
            if (this.IsEnabled(TraceOptions.Timestamp))
            {
                this.WriteLine("Timestamp=" + eventCache.Timestamp);
            }
            if (this.IsEnabled(TraceOptions.Callstack))
            {
                this.WriteLine("Callstack=" + eventCache.Callstack);
            }
            this.IndentLevel--;
        }

        private void WriteHeader(
            string source,
            TraceEventType eventType,
            int id,
            TraceEventCache eventCache)
        {
            if (eventCache != null)
            {
                if (this.IsEnabled(TraceOptions.DateTime))
                {
                    DateTime localTime = eventCache.DateTime.ToLocalTime();
                    string formattedDate = localTime.ToString(
                        "s",
                        CultureInfo.InvariantCulture);

                    this.Write(formattedDate);
                    this.Write(" ");
                }
            }

            this.Write(
                string.Format(
                    CultureInfo.InvariantCulture,
                    "{0}: ",
                    eventType.ToString()));
        }
    }
}

Depending on your specific needs, you could modify the WriteHeader and WriteFooter methods to output additional fields on the same line as the log message. (In my scenario, I only needed to support the TraceOptions.DateTime option.)

In my next post, I discuss logging to the ASP.NET tracing feature.

A Simple, but Highly Effective Approach to Logging

Jeremy Jameson — Fri, 19 Jun 2009 03:01:00 GMT

A common question that frequently arises both with customers and fellow consultants is what do I recommend for logging? As experienced software developers, we know that there are going to be errors in our solution -- as well as other important events that we want to monitor -- and therefore we know we need a robust way of logging these.

However, with numerous options available -- e.g. log4net, Enterprise Library, Common.Logging, SharePoint ULS, etc. -- it's no wonder this is frequently a "hot" topic.

Several years ago, I created a simple, but highly effective approach to logging based on the (then) new System.Diagnostics features in the .NET Framework version 2.0.

Before introducing my logging feature, it is first important to understand the goals (and non-goals) of the solution.

Goals and Non-Goals

The primary goals of the logging feature are:

Provide solution components a way of logging messages of various levels (e.g. errors, warnings, information) with minimal custom code.
Ensure log messages can be enabled in all environments (e.g. DEV, TEST, and PROD); in other words, in both Debug and Release builds (unlike the Debug.WriteLine method, which relies on the DEBUG conditional compilation constant).
Through configuration, enable log messages to be routed to various outputs such as a text file, the Windows Event Log, or ASP.NET tracing.
Ensure that different levels of log messages can be filtered and potentially routed to different outputs; for example, log all messages to the console, but only log errors and warnings to a text file.

Non-goals of the logging feature include:

Providing the fastest possible logging implementation; rather the performance impact of logging should be insignificant when compared with the "real" work performed by the solution components.
Changing logging configuration without reinitializing the solution; for example, to change the logging for an ASP.NET application, it is acceptable to restart the corresponding application pool.
Filtering log messages based on subsystems (or feature areas) of the solution; for example, when logging is enabled for debug messages, then log messages from all components are output.

Introducing the Logger Class

With the custom Logger class, logging a debug message is simply a matter of calling a static method, specifying nothing more than a string containing the message :

Logger.LogDebug("Successfully loaded search results into DataSet.");

This example shows how the Logger class achieves the primary design goal. Note that there is no need to explicitly create objects within each class -- or create additional classes within an assembly -- for logging purposes.

Also note that the Logger class provides additional overloads to easily format log messages:

    Logger.LogDebug(
        CultureInfo.InvariantCulture,
        "Successfully loaded embedded resource ({0})"
            + " ({1:n0} bytes) from assembly ({2}).",
        resourceName,
        resourceContent.Length,
        resourceAssembly.FullName);

Note that the Logger.LogDebug method is simply a convenient alternative to the Logger.Log method:

    /// <summary>
    /// Logs an event to the trace listeners using the specified
    /// event type and message.
    /// </summary>
    /// <param name="eventType">One of the System.Diagnostics.TraceEventType
    /// values that specifies the type of event being logged.</param>
    /// <param name="message">The message to log.</param>
    public static void Log(
        TraceEventType eventType,
        string message)

Other methods such as LogInfo and LogError provide similar overloads for convenience.

The simplicity of the Logger class is made possible by the improved tracing functionality introduced in the .NET Framework version 2.0. Specifically, the Logger class is simply a "wafer-thin" wrapper around the System.Diagnostics.TraceSource class.

The Logger class declares a singleton TraceSource that is used to log all messages:

    private static TraceSource defaultTraceSource =
        new TraceSource("defaultTraceSource");

Various listeners can then be configured to output log messages. Each type of listener derives from TraceListener.

Note that the .NET Framework includes listeners for logging to a file, the Windows Event Log, as well as ASP.NET tracing. Consequently, with just a little bit of custom code (i.e. the Logger class) combined with all the "goodness" baked into the core .NET Framework, we have a logging feature that meets all of the established goals.

In my next post, I introduce how to configure logging (starting out with a console application).

Here is the complete source for the Logger class:

#define TRACE

using System;
using System.Diagnostics;
using System.Globalization;
using System.Text;

namespace Fabrikam.Demo.CoreServices.Logging
{
    /// <summary>
    /// Exposes static methods for logging various events (e.g. debug,
    /// informational, warning, etc.). This class cannot be inherited.
    /// </summary>
    /// <remarks>
    /// All methods of the <b>Logger</b> class are static
    /// and can therefore be called without creating an instance of the class.
    /// </remarks>
    public sealed class Logger
    {
        private static TraceSource defaultTraceSource =
            new TraceSource("defaultTraceSource");

        private Logger() { } // all members are static

        /// <summary>
        /// Flushes all the trace listeners in the trace listener collection.
        /// </summary>
        public static void Flush()
        {
            defaultTraceSource.Flush();
        }

        /// <summary>
        /// Logs an event to the trace listeners using the specified
        /// event type and message.
        /// </summary>
        /// <param name="eventType">One of the System.Diagnostics.TraceEventType
        /// values that specifies the type of event being logged.</param>
        /// <param name="message">The message to log.</param>
        public static void Log(
            TraceEventType eventType,
            string message)
        {
#if DEBUG
            // Some debug listeners (e.g. DbgView.exe) don't buffer output, so
            // Debug.Write() is effectively the same as Debug.WriteLine().
            // For optimal appearance in these listeners, format the output
            // for a single call to Debug.WriteLine().
            StringBuilder sb = new StringBuilder();

            sb.Append(eventType.ToString());
            sb.Append(": ");
            sb.Append(message);

            string formattedMessage = sb.ToString();
            Debug.WriteLine(formattedMessage);
#endif

            defaultTraceSource.TraceEvent(eventType, 0, message);
        }

        /// <summary>
        /// Logs a debug event to the trace listeners using the specified
        /// format string and arguments.
        /// </summary>
        /// <param name="provider">An System.IFormatProvider that supplies
        /// culture-specific formatting information.</param>
        /// <param name="format">A composite format string.</param>
        /// <param name="args">An System.Object array containing zero or more
        /// objects to format.</param>
        public static void LogDebug(
            IFormatProvider provider,
            string format,
            params object[] args)
        {
            string message = string.Format(
                provider, format, args);

            LogDebug(message);
        }

        /// <summary>
        /// Logs a debug event to the trace listeners.
        /// </summary>
        /// <param name="message">The message to log.</param>
        public static void LogDebug(
            string message)
        {
            Log(TraceEventType.Verbose, message);
        }

        /// <summary>
        /// Logs a critical event to the trace listeners using the specified
        /// format string and arguments.
        /// </summary>
        /// <param name="provider">An System.IFormatProvider that supplies
        /// culture-specific formatting information.</param>
        /// <param name="format">A composite format string.</param>
        /// <param name="args">An System.Object array containing zero or more
        /// objects to format.</param>
        public static void LogCritical(
            IFormatProvider provider,
            string format,
            params object[] args)
        {
            string message = string.Format(
                provider, format, args);

            LogCritical(message);
        }

        /// <summary>
        /// Logs a critical event to the trace listeners.
        /// </summary>
        /// <param name="message">The message to log.</param>
        public static void LogCritical(
            string message)
        {
            Log(TraceEventType.Critical, message);
        }

        /// <summary>
        /// Logs an error event to the trace listeners using the specified
        /// format string and arguments.
        /// </summary>
        /// <param name="provider">An System.IFormatProvider that supplies
        /// culture-specific formatting information.</param>
        /// <param name="format">A composite format string.</param>
        /// <param name="args">An System.Object array containing zero or more
        /// objects to format.</param>
        public static void LogError(
            IFormatProvider provider,
            string format,
            params object[] args)
        {
            string message = string.Format(
                provider, format, args);

            LogError(message);
        }

        /// <summary>
        /// Logs an error event to the trace listeners.
        /// </summary>
        /// <param name="message">The message to log.</param>
        public static void LogError(
            string message)
        {
            Log(TraceEventType.Error, message);
        }

        /// <summary>
        /// Logs an informational event to the trace listeners using the specified
        /// format string and arguments.
        /// </summary>
        /// <param name="provider">An System.IFormatProvider that supplies
        /// culture-specific formatting information.</param>
        /// <param name="format">A composite format string.</param>
        /// <param name="args">An System.Object array containing zero or more
        /// objects to format.</param>
        public static void LogInfo(
            IFormatProvider provider,
            string format,
            params object[] args)
        {
            string message = string.Format(
                provider, format, args);

            LogInfo(message);
        }

        /// <summary>
        /// Logs an informational event to the trace listeners.
        /// </summary>
        /// <param name="message">The message to log.</param>
        public static void LogInfo(
            string message)
        {
            Log(TraceEventType.Information, message);
        }

        /// <summary>
        /// Logs a warning event to the trace listeners using the specified
        /// format string and arguments.
        /// </summary>
        /// <param name="provider">An System.IFormatProvider that supplies
        /// culture-specific formatting information.</param>
        /// <param name="format">A composite format string.</param>
        /// <param name="args">An System.Object array containing zero or more
        /// objects to format.</param>
        public static void LogWarning(
            IFormatProvider provider,
            string format,
            params object[] args)
        {
            string message = string.Format(
                provider, format, args);

            LogWarning(message);
        }

        /// <summary>
        /// Logs a warning event to the trace listeners.
        /// </summary>
        /// <param name="message">The message to log.</param>
        public static void LogWarning(
            string message)
        {
            Log(TraceEventType.Warning, message);
        }
    }
}

Note that the Logger.cs file actually includes #define TRACE at the top of the file. This is because I originally wrote this class with an old version of Visual Studio (which did not define this compilation constant by default when creating new projects). Visual Studio 2008 projects include this in the project options by default (for both Debug and Release configurations), so this is superfluous.

As you can see, there's really not much to this class (which hopefully means there isn't much that can go wrong with it). Most, if not all, of it should be very straightforward. I'll explain the importance of the Logger.Flush method in my next post.

Environment Naming Conventions

Jeremy Jameson — Wed, 10 Jun 2009 02:42:00 GMT

One of the challenges I see in organizations that I work with is the lack of naming conventions for various environments -- or sometimes naming conventions that provide little or no value.

For about the last ten years, I've been a strong proponent of a simple "-suffix" naming convention that is not only very easy to learn (and remember!) but also makes things incredibly obvious to all team members and stakeholders on a project.

Most enterprise IT organizations typically have a naming convention for servers that includes things like the Active Directory domain the server is a member of, the role of the server, and usually some unique identifier to distinguish multiple servers in a farm. Other "ingredients" such as the datacenter the server is physically located in may also be included.

For example, my favorite fictitious manufacturing company, Fabrikam Technologies, has Production servers named:

FAB-DC-01 (the first domain controller for the FABRIKAM domain),
FAB-DC-02,
FAB-WEB-01 (the first Web server in a farm),
FAB-WEB-02,
FAB-SQL-01A (the first node in a SQL Server cluster, for which the cluster is named FAB-SQL-01),
FAB-SQL-01B,
etc.

This is a great start. As you can see from these examples, it's pretty easy to tell what the role of each server is.

However, in any organization of reasonable size, we certainly need more than just the Production environment (PROD). At a minimum, I recommend using a Test environment (TEST) and a Development environment (DEV). Note that this is in addition to individual developer environments -- which I typically refer to as "local" evironments (LOCAL). Depending on how much parallel development is planned -- and the corresponding release schedule -- you might also need a Maintenance environment (MAINT). However, many organizations can function effectively with just the DEV-TEST-PROD triad.

So this begs the question, what names should we use for DEV and TEST?

Here is what I recommend:

Monikers in the Development environment utilize a “-dev” suffix. For example, the Web server in DEV corresponding to the FAB-WEB-01 server in PROD is named FAB-WEB-01-DEV. Likewise, the SQL Server in the Development environment is named FAB-SQL-01A-DEV. Note that typically Development environments do not have a cluster, but I still recommend following the same naming convention and even adding a DNS entry for the DEV "cluster" name (e.g. FAB-SQL-01-DEV). Test environments often do have a SQL Server cluster configuration (which is a great place to validate your failover configuration and load testing).

Host headers in the Development environment also follow the “-dev” naming convention. For example, if the internal name of the Fabrikam Web site in Production is http://fabrikam, then http://fabrikam-dev is the corresponding URL used in the Development environment.

In order to distinguish sites on the local development VMs, the “-local” suffix is used. For example, http://fabrikam-local is the URL on an individual developer’s VM corresponding to http://fabrikam in Production. The %WINDIR%\System32\Drivers\etc\hosts file is then used to associate these URLs with the loopback address (127.0.0.1). [Be sure you are aware of the workaround in KB 896861 when using the loopback address.]

Similarly, monikers and host headers in the Test environment utilize a “-test” suffix (e.g. FAB-WEB-01-TEST, FAB-SQL-01A-TEST, and http://fabrikam-test).

Also note that organizations typically create separate service accounts within Active Directory for the Development, Test, and Production environments (and if you are not currently doing this, I strongly recommend it). For example, suppose FABRIKAM\svc-web is the service account for the application pool running http://fabrikam in Production. Then we would expect the Web site in DEV to be running as FABRIKAM\svc-web-dev and the site in TEST to be running as FABRIKAM\svc-web-test.

This ensures that only the minimal number of people need to know the passwords for a particular environment. For example, either the whole Development team -- or just a small subset of team leads -- needs to know the password for FABRIKAM\svc-web-dev, but they certainly don't know the password for FABRIKAM\svc-web-test (since that environment is managed by the Test team). Likewise, the Test team had better not know the password for the service account in Production (since -- at least hopefully -- that environment is owned by a separate Release Management team).

It is also important to point out how this naming convention is applied to fully qualified domain names (FQDNs). For example, suppose the external address of http://fabrikam is http://www.fabrikam.com. What then should we use for the corresponding FQDN in DEV, or should we even configure an FQDN for DEV? The answer to that latter question is a resounding "Yes!" and the answer to the former is http://www-dev.fabrikam.com. Note that we almost certainly won't to expose DEV on the Internet -- although we may choose to expose TEST ( http://www-test.fabrikam.com)

The primary reason I recommend using these FQDNs (aside from the fact that we are following the simple "-suffix" naming convention) is that browsers behave differently between intranet names (e.g. http://fabrikam-dev) and FQDNs (e.g. http://www-dev.fabrikam.com). In other words, when you are trying to simulate a problem on the external site, you should ensure that you are simulating the same security "zone" in DEV, or even in your local environment (e.g. http://www-local.fabrikam.com).

Lastly, I want to mention documentation. Whenever I sit down to write an Installation Guide -- or similar documentation -- I typically only specify the values for the Production environment, even though I fully expect the document to also be used to install and configure the Test and Development environments. I simply include a brief section at the beginning of the document describing the naming convention and that it is the responsibility of the developer or tester to modify the server names, host headers, and service accounts appropriately as he or she is following along when configuring DEV or TEST.

Since we typically have "infrastructure models" (i.e. Visio diagrams showing the physical architecture of DEV, TEST, and PROD), identifying servers and host headers in various environments is really straightforward.