One of the things that irritates me is the fact that most universities teach software engineering the DOD-way. In other words they teach you how to run software projects like if you were doing contract work for the military. The problem with this is that most computer science graduates will never work on DOD-projects, but have been made to belief that everything revolves around documenting everything and that the measure of the project success is how many pages of documentation or how many UML diagrams you have made.
One of the consulting jobs I did after leaving MS was to do technical due dillegence work for a venture capital fund. I would visit the companies they were considering to invest in and determine if the company was on the right track with regards to their software development effort. Out of the 30 companies I looked at, only 2 did have their software development effort in good standing. Most of the time the companies did have some processes in place but quite often these were not followed due to their complexity and need to generate hundreds of pages of documentation.
Lets take an example of one company I visited. This was a small startup, where almost everyone that worked there had graduated 2 years earlier. I started my due dillegence by chatting with them about various aspects of the software development. We then moved to the stage where they started showing me various UML diagrams of the system. When we started going through the UML diagrams I realized they had created every type of UML diagram in the book. You did not only have class diagrams and sequence diagrams, but also use case diagrams, state diagrams, collaboration diagrams, etc. In fact they had used all twelve diagram types defined by UML. Pretty impressive documentation effort and I was looking forward to seeing the end-result of all this documentation work. We therefore moved to a computer where they had their product running. I was very disappointed when I saw the fruit of their labor. It was a rather badly looking UI running in what I would say very non-user friendly manner. If only they had spent a little more time working out usability issues instead of drawing endless diagrams showing the same things from different aspects. Another thing I noticed was that their product was not too stable - when I asked him to show me various things, the system sometimes crashed. So I asked them about their testing effort. They replied that they had some very good beta testers that got new versions from them weekly and told them about the bugs. In other word, there was no internal testing effort (this was actually very common problem with the software development companies I looked at). I also asked them what they had felt about the process they had been using (Rational Unified Process). They said that it had been quite hard to follow it, lots of documents to produce and everything quite rigorous. So they said they had been reading up about eXtreeme Programming (which was relatively new at that time) and that they were considering to switch to that for the next version of the product. Here you had a company that was switching from one extreeme methodology to another.
Over the next few weeks I'm planning to write about various aspects of the software development methodology that is used within Microsoft - at least the way I experienced it during my stint there. Some things vary between product groups within Microsoft, but the same underlying philosophy is used - philosophy that was developed from running into the wall 2000 times and then gradually learning where to turn in order to get high quality products out to customers.