Steve Rowe recently talks about who you're really writing for when you write code. The argument he makes is essentially that your primary audience is not the compiler, but rather your primary audience is other developers. This is something I believe strongly.
Steve also makes a point about premature optimization, and how it affects readability. This reminded me of a list of the Rules of Optimization that I try to live by:
Optimizing code has a negative effect on readability and maintainability. The vast majority of code doesn't need to run particularly fast, but it all needs to be maintainable. Even programs you think are one-off and temporary have a tendency to become permanent.
Most businesses want to keep costs down, but as a rule, hardware is cheap. Developer time is expensive. Don't optimize for the wrong variable.
If you believe that the First Rule doesn't apply to you, (by the way, you're probably wrong. Go read the first rule again) then it's time to consider the second rule.
Designing code and optimizing code are two very different operations. Your maintenance programmers will thank you not to mix the two. For both optimizing and maintaining code, it is far easier to start from a good solid readable design, than to start from code that was "written optimized". (and that's assuming you can even figure out what the latter was doing). Do not optimize code as you write it, unless you like rewriting over and over.
Optimizing all of the code is almost never feasible, and is never worth it. It you're going to spend three developer days to squeeze 10 milliseconds of performance out of your code, it's probably better that you get those out of the inner loop of a column lookup, rather than the startup splash screen display code.
So which code is best to optimize? You don't know. You can generally make educated guesses, but there's your problem. They're just guesses. If you guess wrong (your first guess is almost always wrong), then you'll end up doing more harm than good by optimizing (remember the maintainability hit). So how do you know? Profile. Run tools that will tell you which code is spending most of the time. Optimize that code. Often, you'll find an answer you never expected.
Never optimize without profiling first.
Code optimization is sometimes a necessary evil, but do not forget that it is an evil. It obfuscates the true meaning of the code, and it takes developer time that could be spent on bug fixing or new features. Some applications (such as real-time audio) can't get by without optimizing, but most applications (card games, mail programs, web apps, etc) can get along just fine without unnecessarily complicating the code. And when you do need to optimize, have hard data to back up that you're optimizing the right place.