jeffcal's blogland

rem koolhaas couldn't build better than us!

debugging a build process - what works for you?

building a large piece of software can take a very long time.  combine the times to compile and link with syncing sources, running test suites and unit tests, mailing a build report, copying the bins over the network, etc. and you have a process that can easily take hours.  what happens when some step in this process fails? 

i'm curious what those of you out there do. 

how do you track the problem down?  how do you correct it?  how do you carry on with the build?  do you rely solely on the build's logs?  do they normally contain the information you're interested in?  what information are you interested in?  do you try to repro the problem?  does dependency analysis save you the trouble of again running those parts of the build which were successful?

let me know :)

jeff.

Published Thursday, September 02, 2004 6:33 PM by jeffcal
Filed under:

Comments

 

Scott Sargent said:

Well, we're a small team but we build a diverse mix of software that all in all spans probably about 50K SLOC. That's vb (6.0, com objects & exes), asp pages, crystal reports, C++ Dlls and a slew of database updates. We've never really built an automated build system (although I've wanted to for quite a while). We use a series of batch scripts to build everything. With some simple options you can tell the batch script whether to get the latest from source control, or to use the source on the current hdd. We also let the builder choose which components to rebuild and which not to, this ultimately is how we debug what goes wrong as you're not constantly rebuilding everything and it simplifies the process. Its unfortunately a pretty manual process.

Scott
September 2, 2004 6:50 PM
 

Steve Loughran said:

ooh, that is an interesting question. Let's stop before deploying to make
it tractable.

A build process should be simple enough to be
comprehensible and manageable. Admittedly, what we have in smartfrog.org is
pushing the limit there, which is a sign that ant1.6's import task makes
things tricky.

1. builds create output, be they messages or tangible artifacts in the file system.

2. these can be tested. Assertions in the build process can probe for
things existing

3. you can also make assertions that stuff is there before you proceed.

<pre>
&lt;target name="assert-smartfrog" depends="init-common"
description="verify smartfrog is present, fail if not"&gt;
&lt;available
classname="org.smartfrog.SFSystem"
classpathref="smartfrog.classpath"
property="sfSystem.present"/&gt;
&lt;property name="sfjars.fullpath" refid="smartfrog.classpath"/&gt;
&lt;fail unless="sfSystem.present"&gt;
Smartfrog entry point not found in
${sfjars.fullpath}
&lt;/fail&gt;
&lt;echo level="verbose"&gt;Smartfrog found in ${smartfrog.dist.lib.dir}&lt;/echo&gt;
&lt;/target&gt;
</pre>

Here we also put something else out on the log for extra diags. So when
someone wants to find out what is going on they run ant -verbose, but
when it doesn't

One thing people have in make that ant doesnt replicate is the make -n
command, the dry run. Ant doesnt do it as dependency logic is not in the
runtime but in the tasks. If msbuild has sole ownership of the
dependencies and knows exactly which files will be created, then it
could replicate the -n behavior, which we sometimes get asked for.

The other technique is to do a clean build. That stops you worrying
about any dependency problems, and get on with creating stuff cleanly.
It also avoids the subtle problem of non-build artifacts getting into
the output tree, so nothing is replicable.

Finally, IntelliJ idea 4.5 does a great job of highlighting live syntax errors in a
build, offers basic refactoring, etc. That build files need refactoring
scares me, but IJ4.5 make it possible.

-Steve Loughran, Ant development team.

ps, hours? builds dont take hours. we are not talking C++ here :)

September 3, 2004 5:37 AM
 

David Levine said:

Our process only takes about 1/2 hour, but it includes all the steps you mentioned and a number of others as well.

The basic information we need is typically the step the build process was in, what file(s) it was operating on, and the cause of the failure. The build program captures standard output and standard error, so as much info as possible is presented to aid in trouble-shooting.


Currently the build process stops where it failed. All the information is there as to what it was doing at the instant of the failure and the step it was in. It also can rollback certain changes (such as auto-incrementing the assembly version in a shared AssemblyInfo.cs file).

September 3, 2004 7:03 AM
 

Jeff Atwood said:

Our build process is the "F5" key.

Sometimes I accidentally hit SHIFT+F5 though. Doh!
September 3, 2004 2:36 PM
 

debugging a build process - what works for you? said:

November 26, 2007 2:05 PM
 

jeffcal s blogland debugging a build process what works for you | storage bench said:

June 14, 2009 6:25 AM
Anonymous comments are disabled

© 2009 Microsoft Corporation. All rights reserved. Terms of Use  |  Trademarks  |  Privacy Statement
Microsoft
Page view tracker