In my previous posts, I have been arguing the point that throwing away source code and starting over from scratch is a notably bad idea in general. In this, I am echoing what Joel Spolsky says so eloquently in his post Things You Should Never Do.

(Incidentally, I just observed the zero-filled naming convention that Joel is using in the URLs of his posts. This is post number 0000000069. Either Joel is expecting to be astonishingly prolific, on the order of tens of thousands of posts per day, he is expecting to be outlandishly long lived, or he really likes the number zero.)

I think it would be unfortunate to interpret this sentiment as being absolute in any way. Quite simply, there are some situations where it is, indeed, appropriate to completely write from scratch. The only question is, how do you decide?

A good example to consider is the revolution that took place in aircraft engines. Originally, we used propellers, and now we have the option of jet engines. The move to jet engines was anything but an evolution. Rather, their designers began with a clean slate. In fact, had they worked under the constraint that they must begin with a propeller engine, and gradually make changes until it became a jet engine, it would have been much more costly overall. Biological life has the constraint of natural selection. You begin with a certain organism, and you can gradually evolve it. At each step, the organism must have equal or better survival characteristics as the step before it. Had we applied the same constraint to jet engines, then engineers would have been required to develop a series of engines evolving the propeller to the jet, each of which would succeed in the market place.

Fortunately, they did not operate under that constraint. Your software does not need to either. So, evolution and revolution are both tools that you have available to work with the software organism, and making intelligent decisions regarding the best approach is absolutely critical.

This truly is the trick, then. If the code you are writing actually exhibits an identical phenotype as code that already exists, then you are fundamentally wasting your time. Observe what happens, in the world of natural selection, when two genes exhibit identical phenotypes. The genes that are present in the majority of organisms, no matter how small that majority is, will eventually have a dominant position in the gene pool. By creating new software DNA to exhibit the same phenotype, you are most likely wasting your time because you not only divert resources from creating new functionality, but that code is doomed to obscurity and eventual demise. You lose an opportunity for achieving immortality through your software - and isn't that part of the fun?

Propellers have fundamental limitations that justified the investment in developing jet engines. No matter how carefully the designers worked to evolve the design, they simply could not achieve the results they needed without taking a different approach altogether. Is there a fundamental limitation in the nature of the software you are creating? Can it be evolved and refactored, or should it be completely reengineered? And, if you need to engineer a different solution altogether, you should choose the smallest necessary piece to re-engineer. With the advent of the jet engine, parts of the plane needed to evolve to support the new engine design, but the concept of the plane itself could be evolved, rather than re-developed from scratch.

I continue to be surprised by suggestions that an entire body of code - one which has proven its ability to survive in the software ecosystem, should be completely disposed of and replaced with new, less "buggy" code. I read another treatise on this recently, and I still fail to understand the logic. Why would humans, with the same human imperfection, suddenly start writing completely bug-free code now? Instead, we should evolve this imperfect design.

As an analogy, consider the human eye. If you look at the structure, what you will find is exactly the opposite of what the typical engineer would expect (and, in fact, exactly the opposite of what exists in cephalopods): the light-sensing portion of rods and cones is actually tucked away behind several neuronal layers, as well as the cell's own nucleus.

(The image above taken from wikipedia.org, and has been released to the public domain.)

One consequence of this design is that the light-sensing portion of these cells need to be more sensitive in order to process a smaller amount of light, given that this light is filtered. This design has evolved to compensate for a shortcoming in the original design. It was simply more expensive to replace the existing design than to make this modification, in order to achieve phenotypically similar results.

Another consequence is that these nerves must find their way out of the eye, which entails going through the layer of light-sensitive cells. That leads to a blind spot, and the eye can't compensate much for this on its own. Here, another system entirely - the brain - is the compensating agent. This is important to consider, particularly in large enterprises with existing infrastructure.

When you are faced with designs that are less than perfect, throwing them all away is certainly always an option, and often times it will be the most expensive option. So, when considering your software organism, there is always the potential to evolve that system in such a way to compensate or make irrelevant that design shortcoming. There is additionally the option of using system integration to allow another system to handle the compensation. Of course, if your systems are not designed with integration in mind (which is, of course, a shortcoming that additional evolution can address), then your options are limited.

In every case, considering how to evolve software is a complex decision. Yes, completely beginning again is always an option. Evolving the existing software is another option. Compensating with a connected system is another option. In fact, this analogy (the ability of disparate systems to serve as the point of evolutionary adaption) only begins to scratch the surface of the power of integration.