|
Software program |
Scientific theory |
| What is it? |
A description of some desired computer behavior |
A postulated description of reality |
| Who creates it? |
Programmer |
Theorist |
| Who first validates it? |
Tester |
Experimentalist |
| Provability |
Usually can't ever be proven 100% correct. The longer we go without finding any bugs, the more we suspect it may be bug-free. |
Can never be proven correct, just proven wrong. The more it resists being proven wrong, the more we believe it is probably 100% correct. |
| Testability |
Good software is designed from the ground up to be tested effectively. If you build a large system first, and then later start thinking about how you might test it, you're likely to end up with a lot of bugs that are hard to find. |
For something to qualify as a "scientific theory" it must be "falsifiable" - that is, there needs to be a way in theory you could prove it doesn't work. In general, the more ways you might be able to show that a theory is wrong, the better the theory is. For example, many people argue that most forms of string theory are not sufficiently testable to be seriously considered as science. |
| Predictive power |
If you write a program precisely to pass a certain set of tests, you don't really know if it's correct until you test it against some new test cases.
|
Theories are generally designed to fit known experimental results. Properly predicting the results of experiments that have already been done is important, but the real test comes when the theory predicts results which aren't yet known. |
| Reproducibility |
No matter how much testing you do, you ultimately need to get some real customers to try your software and see if they confirm your belief of correctness.
Often the best tests come from users which are most critical of you. For example, seeing positive comments on slashdot about Silverlight has been a confirmation to me that my team is making some good decisions. |
No matter how much one group's experiments agree with theory, little weight is attributed to the experiments unless other independent groups are able to reproduce the results.
Often the best tests of a theory come from people who believe it to be false. The history of quantum mechanics is full of scientists who disbelieved it, but whose arguments and experiments ultimately strengthened it (such as Einstein and the EPR paradox). |
| Simplicity |
Correct software tends to have a simpler, smaller, and/or more elegant implementation. This doesn't mean it's behavior needs to be simple. |
All things being equal, simpler theories tend to be the correct ones. This is known as Occam's razor. There are many examples in science of theories which are conceptually and mathematically very simple, but whose implications are very complex and non-obvious. A classic example is an explanation for the complex movements of the planets in the sky. Ptolemy's model with the earth at the center was very complex, but Copernicus presented a much simpler model with the sun at the center. |
| Believability |
Good software tends to be well structured and documented so that a person can reason about it's correctness. |
Good theories tend to have a logical and believable explanation for why they should be correct. |
| Specificity |
Good software tends to have a concrete specification for what is considered correct behavior. We try to avoid the temptation to build something, write some tests for it, and call the results of those tests "correct".
When the tests are first executed, there is ideally only one possible correct output. |
Good theories tend to have fewer "free variables", which are parameters determined by experiment. For example, each planet in Ptolemy's model of the solar system had a number of concentric rings with various sizes associated with it. In the Newtonian model, the movement it determined entirely by each planet's mass, orbital radii (major and minor), and the universal gravitational constant.
When new experiments are performed, there is ideally just one result that would be consistent with the theory. |
| Generality |
The field of software (and often individual large systems) advances by replacing special-purpose components with general-purpose frameworks. Operating systems and managed runtimes like the CLR are obvious examples here. |
Good theories often supercede many previous (apparently unrelated) results, encompassing them all under one larger umbrella. A great example here is the realization that electricity, magnetism, radio waves, and light were all properties of the same electromagnetic force (and eventually even just one aspect of the electroweak force). |
| Reusability |
Good software tends to build on previous successes, re-using components that are known to be of high quality, but avoiding dependencies on high-risk pieces. It's extremely difficult (and wasteful) to build a completely new large system from scratch. Personally, I believe this is an area we could do better on the CLR. |
The progress of science has obviously only been possible by building on previous successes. |
