Selection is the basis by which evolution can develop the enormously complicated systems that exist today. The underlying principal is non-random selection of random mutations. In any organism, there is some degree of genetic drift. Despite the built-in error correction of DNA replication, mutations still occur, and they occur randomly. I should be explicit here - they occur randomly with regards to the phenotype of that mutation. Some mutations will produce a phenotype that is non-selective. In other words, an individual who possesses that phenotype is neither more likely nor less likely to survive. Other mutations will be beneficial, increasing the likelihood of survival. Still others will be detrimental, decreasing the likelihood of survival. Because selection is non-random, it will usually select the phenotype that has superior survivability characteristics. So, a single mutation that increases, however slightly, the ability of an individual to survive, it is far more likely that the organism will, in fact, survive to reproductive age, and that this mutation will continue into the next generation.

For example, assume that an organism randomly develops a mutation that makes a cell on its face more sensitive to light. That sensitivity allows it to see, however, vaguely, that there is a shadow moving overhead, and run away. Such an organism is likely to have sufficiently higher survivorship than its companions, be more likely to reproduce, and eventually this selection will drive the majority of the population to have this mutation. Each successive mutation that refines this detection will be selected for, and eventually you end up with an eye.

This has a couple of assumptions, mind you. The most important one: that there are too many organisms for a given environment. A given group of organisms will produce too many offspring to ensure the survival of all of them. As a result the remaining organisms must compete to survive. Because only some of them survive, most often it will be those organisms who possess the superior phenotype - filtering out mutations that are less beneficial. The same is also true between species: if two species are competing for the same set of limited resources, then mutations that either make them more effective at competing, or else mutations that cause them to seek out a different (and less hotly contested) set of resources will be selected.

The idea of selection is just as important for software. This process is what drives the continued evolution of software, just as it drives the continued evolution in the natural world. The only difference is that mutations are non-random (or, at least, most of them are, we hope!), but selection is the process that determines which characteristics will survive into the next generation of software, and which will not.

The importance of selection can be seen in a couple of ways. The first is in determining the evolutionary rate. Consider, first, the amount of competition for a limited set of resources. In this case, these resources will likely be either money or time. If there is a great deal of competition, then your software is likely to evolve fairly quickly. If there is not much competition, then the evolutionary rate is likely to slow. Mutations may still take place, but without a set of selection criteria these mutations will not survive based on merit, but will simply be part of the genetic drift.

You can find an excellent example with web browsers. Once people began to realize that they really needed to have a web browser, then resources opened up to consume them. However, this set of resources was limited: people generally were content to have a single web browser. So, the competitors were vying for this resource, to become the choice. This spurned rapid innovation in the web browser market, as competitors struggled for their very survival against other web browsers. Eventually, one browser came to be the victor, consuming the vast majority of the resources (installed desktops) available in the market. The competition died down, and so did the innovation. Even assuming that development continued at the same rapid pace on the product, without contention for resources, there would have been no way to select specifically for those features which are of the greatest tangible benefit to users, against those features that are pure genetic drift. The product simply cannot evolve as quickly.

However, once competition emerged again, selection could come back into play. Features could be added to enhance the product, and those features that drove adoption would be selected for, driving evolution for more rapidly. Non-random selection is critical.

It is important to keep in mind, however, that all mutations in a software organism that the environment is currently selecting for are not necessarily the unit of causation of that selection. If you come up with a new browser that supports tabbed browsing, a new security model, and the ability to render all pictures in shades of purple instead of full color, can you necessarily say that rendering all pictures in shades of purple were instrumental in the success of the product? In much the same way, can you say that tabbed browsing is a mutation that is selected for, or was it actually security that was selected for? It is important not to conflate the issues; the environment, given a degree of competition, will provide selection. However, you can not always interpret any one factor as being key to the success of an adaptation.

An even more extreme scenario comes with evolutionary arms races. Competition for a limited set of resources drives rapid evolutionary change. Competition for immediate survival compels even more rapid evolutionary change. Consider the evolution of the cheetah and the antelope. The antelope will evolve to run away more quickly. An antelope with a mutation that will allow it to run a little bit faster will have a much greater chance of survival into the next generation. By the same token, a cheetah with a mutation that enables it to run a little bit faster will catch more food, thus increasing the chance of this mutation surviving to the next generation. Here, the success of one organism definitively means the failure of the other. This is an arms race: in this scenario, evolution takes place very quickly.

The most direct analogy here is towards security. Software that evolves to be more secure is more likely to survive. So, the attackers become more sophisticated, creating new malicious software able to overcome these obstacles. Both sides evolve their software more rapidly than in any other evolutionary environment. But, in this case, there is some degree of hope for everyone who is trying to develop secure software while facing attacks from continually more sophisticated attackers. To quote Richard Dawkins, "if the predator loses the race, he simply loses a meal. If the prey loses the race, he loses his life." The pressure is on everyone developing software to make it more secure, or they will lose altogether. Thus, it seems likely that many will be able to maintain this accelerated evolutionary rate, because they simply have no choice.

Understanding the environment your software is critical to guiding its evolution. What are the resources? Who is competing for them? And what features are actually being selected for? This is a powerful principal that has done amazing things with nature, and can do similarly amazing things for software when they are leveraged.