Automated Continuous Integration and the Ambient Orb™

Automated Continuous Integration and the Ambient Orb™

  • Comments 43

How often have you or one of your fellow developers checked code into your source control system that causes a build failure for someone else? And how many times have you heard that “it builds just fine on my machine?” More importantly, how much time does it take to backtrack through all of the changes to uncover the root cause of the failure? If the last good check-in was two days ago, it can be difficult to figure out not only what caused the failure, but whose check-in did it.

Integration problems like this happen every day on software development projects. Back in 1996, Steve McConnell published an article in IEEE Software magazine about the benefits of performing a Daily Build and Smoke Test that addressed this specific issue. Since then, the concept of a daily build has become more accepted in the development community, but it still amazes me how many projects don’t take advantage of this simple, yet powerful technique.

Well, if we agree that integrating and building a project once a day is good, are there any benefits to continually integrating and building many times a day? How about every 15 minutes?

Continuous Integration

Continuous integration is a technique that encourages developers to check-in their code more frequently, have it compiled by an automated process, run a suite of unit tests, and report on the status. The idea is to tighten the feedback loop so that the effect of an integrated change is communicated back to the developer as soon as possible. By reducing the time between check-in and build status, developers find it much easier to identify faulty code.

A continuous integration server is responsible for monitoring the source code repository for changes that are made during check-in. When a change is detected, the server automatically:

  • Performs a full check-out
  • Cleans the build output folder
  • Forces a complete rebuild of the entire project
  • Executes all unit tests (optional)
  • Reports on the build and unit testing status

Of course, for this to work, all source code needs to be stored in a central location that is shared by the development team. Fortunately, source control tools like Visual Studio 2005 Team System, Visual SourceSafe, CVS, Perforce, Subversion, Rational ClearCase, SourceGear Vault, etc. are found on many projects and provide a single repository.

For the continuous integration server to build the project, a build script needs to be created. NAnt is a popular and free Open Source project that is supported by both CruiseControl.NET and Draco.NET, two popular continuous integration systems for .NET development. Before configuring the continuous integration server, it is worth spending the time to create a functional build script. Although NAnt is not difficult to use, plan to invest a little time reading through the documentation before generating your first script.

Last, although unit tests are optional, they are highly recommended. On the NxOpinion project, we have over 600 individual NUnit tests that run during every build. The success of this suite of tests tells us that a majority of the basic functionality that is provided by our system is working as-expected. We consider the failure of a single unit test to be equal to a full build failure, and we work to immediately resolve any issues.

With CruiseControl.NET, when a build or test failure is detected, an e-mail message is sent to the entire development team notifying them of recent check-ins along with the specific failure. When a check-in is successful, an e-mail is sent only to those developers who checked-in code for that build. This way, everyone is always kept in-the-loop on build status. CruiseControl.NET also provides a web site and system tray application to monitor the status of the continuous integration server.

For our current project, we have configured the server to poll for changes every 15 minutes. If there have been no check-ins since the last build, the system sleeps for another 15 minutes. This cycle continues until new changes are detected. When this happens, the entire system is completely rebuilt. Because of the relatively short polling period, developers only have to wait a few minutes before learning the status of their most recent check-in.

CruiseControl.NET can also automate FxCop. FxCop is a code analysis tool that uses reflection and IL parsing to check managed assemblies for conformance to the Design Guidelines for Class Library Developers. Think of it as an "after-the-fact code review" that doesn't depend on the original source code. The current version of FxCop evaluates over 200 individual rules, and custom rules can be created. FxCop is very comprehensive, and it is very rare to receive a completely clean report. You may need to ignore rules that don't apply to your project. Although we run FxCop as part of our build, we do not currently have it configured to cause a build failure if any violations are found.

For a very thorough look at continuous integration, I recommend reading Continuous Integration with CruiseControl.NET and Draco.NET by Justin Gehtland.

The Ambient Orb™

Webster defines ambient as an adjective that means “existing or present on all sides.” When used as a noun, it means “an encompassing atmosphere.” Ambient science is an evolving field that attempts to convey low-bandwidth information by embedding it into our surrounding environment. The idea is that certain information isn’t worthy of interruption, so instead, it should available at a glance.

Ambient Devices is a company that manufactures products that convey ambient information. The Ambient Orb is a $150 sphere of frosted glass that glows a nearly unlimited variety of colors. The Orb is capable of some simple color “animations” like pulsing and an effect called “crescendo” that slowly brightens a color, then immediately dims. Ambient Devices also makes a Beacon that is similar in concept to the Orb, and a recently-available Dashboard.

The Orb is commonly sold as a device to monitor the stock market, local weather, or any of a number of other free channels of information. Fortunately, for the true geek, a premium account is available for around $20 a quarter that allows you to send your own custom information to the Orb. By calling a specific URL at Ambient’s web site, you can control both the color and animation of your own Orb.

You might ask: “why do I need to call a URL to change the color of something that’s sitting on my desk?” Good question…this is where the geek magic steps in. You see, each Ambient device has wireless pager electronics built-in that allow it to receive signals over the wireless network (I’m not talking about 802.11x wireless…I’m talking about a wireless pager like the kind you’d wear on your belt to monitor those servers that like to go down in the middle of the night). This means that you only have to plug the Orb into a power outlet, register its device ID with Ambient Devices, and from that point on, the Orb will change colors and animations when its information changes. Pretty cool.

The Orb is imminently touchable, and visitors always stop to feel its warm glow. It’s definitely a conversation piece. After they ask if you bought it at Target for $19.95, you can proceed to twist their brain by explaining how you’ve configured your system to send information over the Internet that the Orb wirelessly receives through the pager network. The look on their face is similar to the look you’d get if you said: “I’m an alien from Alpha Centauri and that’s my invisible spaceship parked out there on the lawn.” Priceless.

Raising Build Visibility

So I had this idea that we could configure an Ambient Orb to reflect the current status of our NxOpinion continuous integration build. A slowly pulsing green would mean that the build is currently okay, and a quickly pulsing red would indicate a build failure. I planned to put the Orb in the middle of our project team so that everyone would be aware of the build status. I hoped that by raising its visibility, everyone on the project team (including the customer) would be more aware of the project “health.”

Now, when the build breaks and the Orb pulses red, it’s like a fire alarm around here. The first question out of everyone’s mouth is “who broke it?” After appropriate developer guilt has been piled on by the development team (all in good fun, of course), it’s usually a relatively trivial matter to discover and fix the problem. Because we continuously integrate our code and the automated build potentially runs every 15 minutes, determining what caused the failure is as simple as looking at what has been checked-in since the last successful build. Fortunately, CruiseControl.NET includes this information (along with check-in comments) in its e-mail and web page summaries.

To-date, our solutions contain approximately 175,000 lines of C# code and over 600 unit tests. Since we consider the failure of a single unit test to be a failure of the entire build, if one test fails, the Orb pulses red. As you’d guess, CruiseControl.NET also includes unit test results in its e-mail and web page summaries which makes it easy to identify the problem.

Although we haven't done it on the NxOpinion project, it would be trivial to configure multiple Orbs to reflect the build health at any location within wireless pager range. You could have an Orb in every development office. And even one at home. Okay, maybe I'm going overboard with that last suggestion, but you know what I mean.

Configuration

To configure the automated process to send build status information to the Ambient Orb, we need to add some properties and targets to the NAnt build script that CruiseControl.NET uses. NAnt has two built-in properties that can be leveraged to execute a task on build success and failure. The properties are called nant.onsuccess and nant.onfailure, and they need to be set to point to valid target elements in the build file. In our case, we define targets called OnSuccess and OnFailure, although any valid names will work just fine.

To send information to the Ambient Orb, query string parameters are passed to an Ambient JavaServer Page. The format of the request is as follows:

    http://myambient.com:8080/java/my_devices/submitdata.jsp?

      devID=###-###-###&anim=#&color=#&comment=Comment+here

Where:

devID The device ID (serial number) of your Ambient Orb
color A number representing the color (0-36)
anim A number representing the animation style (0-9)
comment A short comment that is logged at the Ambient web site


For more information on the available colors, animation styles, and formatting requirements, see the Ambient Orb WDK documentation.

We use the NAnt <get> task to send our build status. The <get> task queries a URL and copies the response to a specified file. In our case, we copy the response to a file in the temporary folder, then immediately delete it with the next task. This isn’t our entire NAnt script, but it does contain enough for you to figure out how to incorporate this into your own process:

<?xml version="1.0" ?>

<project name="Example">

 

    <!-- Load environment variables -->

    <sysinfo />

 

    <!-- Define targets for build status -->

    <property name="nant.onsuccess" value="OnSuccess" />

    <property name="nant.onfailure" value="OnFailure" />

 

    <!-- Set Ambient Orb for successful build -->

    <target name="OnSuccess" description="Build success">

        <get src="http://myambient.com:8080/java/my_devices/submitdata.jsp?

          devID=###-###-###&anim=4&color=12&comment=Build+success"

          dest="${sys.os.folder.temp}\delete.me" failonerror="false"/>

        <delete file="${sys.os.folder.temp}\delete.me" failonerror="false"/>

    </target>

 

    <!-- Set Ambient Orb for failed build -->

    <target name="OnFailure" description="Build failure">

        <get src="http://myambient.com:8080/java/my_devices/submitdata.jsp?

          devID=###-###-###&anim=6&color=0&comment=Build+failure"

          dest="${sys.os.folder.temp}\delete.me" failonerror="false"/>

        <delete file="${sys.os.folder.temp}\delete.me" failonerror="false"/>

    </target>

</project>

Conclusion

We’ve been using CruiseControl.NET for automated continuous integration for the past year-and-a-half, and it has been a fantastic addition to the project. Although continuous integration is typically associated with the Agile development community, it is a technique that can provide major benefits to teams using any project methodology (even if the developers aren’t writing unit tests).

It is rare that our build is in a failed status for more than 30 minutes, because CruiseControl.NET makes it so easy to determine what has changed since the last successful build. Plus, because our Ambient Orb is in a highly visible location within the project team, it is easy to see the health of our source code with just a quick glance. We can go home in the evening confident that our offshore team is working with a healthy build, and we have that same expectation when we come in to work the next morning.

I firmly believe that once you’ve worked on a project with automated continuous integration, you won’t want to work on a project without it. To get started, download either CruiseControl.NET or Draco.NET. It might take a bit of effort to create your first build script, but once you have it up and running, you'll find that it requires relatively little care and feeding to maintain.

Oh, and don’t forget to let me know about your experience!

Resources

Daily Build and Smoke Test by Steve McConnell: http://www.stevemcconnell.com/bp04.htm
Martin Fowler's introduction to Continuous Integration: http://www.martinfowler.com/articles/continuousIntegration.html
Continuous Integration with CruiseControl.NET and Draco.NET by Justin Gehtland: http://www.theserverside.net/articles/showarticle.tss?id=ContinuousIntegration
CruiseControl.NET: http://ccnet.thoughtworks.com/
Draco.NET: http://draconet.sourceforge.net/
NAnt: http://nant.sourceforge.net/
NUnit: http://www.nunit.org/
FxCop: http://www.gotdotnet.com/team/fxcop/
Ambient Orb: http://www.ambientdevices.com/cat/orb/
Ambient Orb WDK: http://www.ambientdevices.com/developer/OrbWDK.pdf

Leave a Comment
  • Please add 6 and 8 and type the answer here:
  • Post
  • I'd like to get continuous integration (CI) working for my TFS server here at the office as my next step.&amp;nbsp;...
  • A few days off from work.

    An employee urgently needed a few days off from work but he knew his boss would not allow him to take a leave. So, he thought that maybe if he acted "crazy" then his boss would tell him to take a few days off.

    So he hung himself upside down on the ceiling and made funny noises. His co-worker saw him and asked what he was doing. He told her that he is just pretending to be a light bulb so that the boss will think he is crazy and give him a few days off.

    A few minutes later the boss came into the 0ffice and asked "What are you doing?" The employee told him "I am a light bulb". Then the boss said to him: "You are clearly stressed out. Go home and recuperate for a couple of days."

    The employee jumped down and walked out of the 0ffice. When the co-worker followed him, the boss asked her "And where do you think you're going?"

    She answered, "I'm going home too, I can't work in the dark."
  • After learning about the concept of continuous integration, I became&amp;nbsp;interested in building a continuous...
  • good related article
  • Automated Continuous Integration and using an Ambient orb for notification was very well described in...
  • Nearly two years ago, while I was still working for Microsoft Consulting Services in the field, I posted...
  • This is too cool.  I am sitting here shaking my head.
  • I've been using Dragon NaturallySpeaking since 2000.  Although I am seriously contemplating upgrading to version 9, I have recently experienced a problem with dictating in Microsoft Word.  In fact, version 8 will no longer dictate into Microsoft Word.  Have you had a problem with this, and do you have any clue how to correct this issue?

  • TrackBack,

    Really nice work, on the orb part of it.

  • Based upon some ideas from this article I have created a Visual Studio controlled light bulb. On the following URL is an article containing a demonstration video. The arcticle is dutch but contains a link to the english translation:

    http://www.inchl.nl/blog/

    Kind regards,

    Stephan

  • Great product but the technical support is terrible.  No return emails or phone calls.  Technical support is quick to get off of the phone without resolving the problm.  My device hasn't worked in over a month of repeated calls to technical support who assured me that they were fixing the problem.  

  • I am a lead developer in a very complex Java development environment for a major Global Financial institution.  We are strong believers in the benefits of some type of continuous integration(CI), whether it is based on automatically building based on changes or just scheduled periodically.  The benefits are just as Mike explained.

    We have development teams that utilize both methods using the centralized workflow tool Mojo Max from OpenMake Software along with ClearCase.  Personally I prefer the scheduled builds for my team of 9 developers because we closely communicate and collaborate. We have a 11:30 A.M, 4:30 P.M and 10:0O P.M scheduled builds.  We used to use the automatic build every 15 minutes if there were changes, but there are often times when one developer would deliver changes that also required changes from another developer that hadn't yet been delivered, this caused unnecessary broken builds, emails warning of broken builds that should be broken.  That was annoying for my team but other teams are okay with it.

    Ultimately each of our workflows are configured centrally and stored centrally on the Mojo Max machine but execute on dedicated Build Servers using the Mojo remote agents.  Our workflows are pretty standard.

     1. Start ClearCase View

     2. Synchronize view with stream in case of any stream config updates were necessary do to updates of common modules.

     3. Clean Build directory

     4. Execute Ant build.xml file

     5. run junit test on success of build

     6. The nightly build actually has steps to deploy the build to the application server and start the applicaiton.

     7. We receive notification of the result of the workflow execution

     8. All steps are logged in a single html formatted log for the workflow execution and is stored centrally on the Mojo Max server.  The logurl is also sent in the email in step 7.

    Some of our teams are evaluating utilizing the Meister product from OpenMake Software which takes thing a step further.  It give us more than workflow automation, but it does everything that Mojo Max does but also provides build automation in the since that it understand the full dependency relationship of your application, can report on the dependencies that went into the build, and most important it generates our Ant scripts on the fly.  We no longer have to code those things.  We are building a business case to purchase the product, so my ultimate purpose here was to see if anyone else has experience with Meiser formally known as OpenMake.  We will most likely be purchasing meister 7.3.1 the latest version.  Any comments will be helpful.

    www.openmakesoftware.com

    Thanks

    Channing W.

Page 3 of 3 (43 items) 123