a. Management or administration marked by hierarchical authority among numerous offices and by fixed proceduresb. The administrative structure of a large or complex organization
a. Management or administration marked by hierarchical authority among numerous offices and by fixed proceduresb. The administrative structure of a large or complex organization
I was able to spend the day today with students at Harvard Business School. I was fortunate enough to meet a number of second year students and was invited to participate in a class teaching a case on the development of Office 2000. I spent a semester teaching at HBS and it was great to be back in a classroom where the students bring incredible insight to the problems we face in building Office. This post is about a discussion I had a number of times today—the topic of bureaucracy. The topic applies equally to undergraduate computer science majors and to MBAs, and is certainly one that based on your interest will generate further posts on the topic.
Up front, it is of course impossible to defend bureaucracy. So any attempt to justify rules, process, hierarchy, etc. are met with a groan at best or a complete rejection at worst. In fact it is common to just assume that anyone brave enough to defend such structure is either oblivious or stupid, or both, and in all cases probably a pinhead you would never want to work for. After all, in the world of technology and the internet the one who is out there with no rules, no process, no hierarchy is the one who is going to win big while all those sloths with their spreadsheets and dashboards are all bunched up trying to plan their way out of a paper bag. OK, maybe I went too far. But the basic challenge in talking about this topic is how do you say that Microsoft is not bureaucratic when there are articles out there saying that the company has become too bureaucratic? How do you talk about this topic without at the same time sounding like you like something which everyone obviously loathes?
It is worth noting that I am sure people can share with me stories about how bureaucracy has stifled their progress at Microsoft. We make mistakes and we have dumb things. But I also heard a stories from HBS students today about how difficult the HBS administration is to work with (just ask them about recruiting!). Of course I have friends at all sorts of companies that tell me (in private) stories of how bureaucratic their organizations are as well—some of these companies are even famous for claiming not to have any bureaucracy. Organizations of more than about 100 people are all capable of dumbness—once you grow beyond grabbing money from the petty cash drawer you have process and once you have process it is a matter of time before you don’t understand what is going on. If you don’t believe me then you just haven’t worked with more than a 100 people or you just never happened to stumble across the processes.
In class today we talked about the development of Office 2000. This is a case that “Describes the history of Microsoft's Office product suite. Discusses evolution of the Office 2000 project. Set at the end of the project the team must decide upon the direction for the next version of Office, as well as make changes to the process.” This is a case that goes into detail about how we decide what features to put in the product and the overall engineering process. It was written in 2000 after the release. What is fascinating for me is seeing over time how students in different classes react to the case in the classroom—believe it or not even though the case is unchanged and the facts are the same, students have different views of the important issues of the case based on the perceptions of Microsoft in the market. I didn’t expect that for sure. [Note: business school is often taught by the case method—this is a way of learning through narratives with the goal of discussion the situation faced and the possible alternatives, but not necessarily about finding the right answer since most of the time there is no single answer.]
When the case was first taught, a lot of the focus fell to Microsoft’s plans for being successful in the market and how the product was another release of a successful product. There was always a lot of talk about the big plans Microsoft had for the software and how it would lead to further success of an already wildly successful product. And while the case brings up some issues relative to the challenges the team faced, most of the focus was on the challenges the business faced—was the product late, was it the right set of features, did the company do a good job listening to customers. That was an era where our success probably shaped the perception quite a bit.
Today, the students picked up on issues in the case related to the efficiency of the development team. Now of course that is an issue. In fact the reason this was in the case was because shortly after the project was completed we did our planned (and rigorous) postmortem on the project. From that we identified that we had probably pushed too much on to developers to keep the “daily build” of the whole Office product stable. But in class, there was definitely a feeling of “see this is that Microsoft bureaucracy that we have read about”. But wait a minute this all happened starting in 1998!
What changed? Well very recently a perception has developed that Microsoft has become “slow” or that there is too much “process” around getting things done. Certainly we have talked quite a bit about our efforts on some products and how we probably tried to get too much done and that caused us to take longer than expected and caused us to make changes to products. But that is something that we have been guilty of from the earliest days of Microsoft—we’ve always had a pretty aggressive appetite for signing up for a lot of work (perhaps more than we could get done!). Like nearly every software project, Microsoft has had projects fail to meet their initial ship dates. As we saw in the Office 2000 case we were about 8 months late from our original schedule.
In class a number of students, not computer people, said that it is crazy and we should just pick the features and pick the schedule and just meet it. Interestingly, the discussion was then “how do you do that” and of course the way you do that is planning. And of course the more planning you do the more you probably introduce bureaucracy. Oops.
Here’s a quick quiz. Which project will get done faster and have better quality at the end:
Project A – starts off with some developers that have an idea and they just start coding.
Project B – starts off with some developers that have an idea and they spend 3 months not writing code, but designing the architecture and interface and then they start coding.
Project A might “finish” faster but it will likely not be finished by any stretch, will almost certainly have a set of features that are not coherent, and probably won’t be very easy to sell unless the developers happened to spend a bunch of that time doing some research to understand how to price, position, and promote the product. Now are there examples of Project A being wildly successful—of course there are. Remember, this is a social science we’re talking about so there is a bell curve and anything is possible. The question is how repeatable is it. About the only time this is repeatable is when you’re cloning something that already exists or building an identical second version of an existing system (many open source projects have the advantage of building based on existing, running systems).
Project B on the other hand has a good chance of being coherent. It has a good chance of meeting a customer need. And most importantly has the best chance of not having to go through a major rewrite one or more times during development. If you’ve ever tried to build a house or remodel a kitchen, you know that you don’t just bring in tools and start sawing and hammering away. Software is no different. Now are there examples of Project B being gummed up and a total failure—of course there are. On the other hand, Project B can be a repeatable process and does not depend on super human programming skills combined with psychic market knowledge.
Returning to our case and the recent change in perception that students have had does raise an issue for me. I certainly don’t think we’ve gotten more bureaucratic. In fact if we were to look at the number of new features, the lines of code, and the breadth of products we have offered with each release of Office we are definitely doing more with the same or fewer developers on the project teams. The question students asked to that point was “yes, but don’t developers feel like they have too much to do to ‘check in’ their code?” The answer is of course—because you should just be able to type a line and then boom, everyone should see it.
But that turns out to be rather difficult. The baseline comparison for this, especially for college hires, is the “edit-compile-debug” cycle we’re all familiar with in college. You are the master of the machine. You have no dependencies on anyone else. You know every line of code. This is an awesome way to write code. It is also the best way to write projects that are the work of one person. Anything more than that and you have a dependency. At the same time, if you are working well as a team you can quickly have a 1+1+1 = 4 or better. The challenge is that you have to put in some process to make sure that you get the benefit of using each other’s code and the benefit of working as a team. It does not come for free. Putting in the right process is very hard work. It is super [sorry I used that word] easy to put in process that makes 1+1+1 =2. Process can sap the life out of a team. On the other hand, it seems absurd that you should be able to change “on a whim” the code of a system that is used by hundreds of millions of people—customers expect and demand that you have some process for deciding what to do. Anything of any material complexity must have that rigorous view. If it doesn’t then the system is a toy or the system just isn’t valuable.
Everyone who has built a software project knows that once you have customers you have an installed base and therefore you have to be careful about changing things. But you also have customers that come to expect features in a certain way. You have customers that might want to have say in how things evolve. You can’t just be an “enlightened engineer” and speak for every possible customer, every compatibility issue. It is likely you’ll need some help. Imagine for example, you wrote the code that decides what advertising to show on a web site (like we have on many sites at Microsoft). You have a great idea to make it better and make a change and check that in—but oops, you didn’t know that product management had come up with a clever, revenue positive, pricing scheme based on how different ads appear. You just changed the company revenue with that “flexible” development process. You then find yourself backing the change out in the middle of the night. Perhaps then the team will put some safeguards in place.
Once you put in a process that “slows” down that work there is now an official bureaucracy. So you have to minimize that so that people can spend the time they have doing the things they like—developing, doing marketing campaigns, etc. Pick any profession—writing for a newspaper, selling cars, being a surgeon, airplane pilot, banking, etc.—in every case people do not get to do their profession 100% of the time. In all professions there is some notion of checks and balances or some notion of planning, arguing, agreeing, deciding, revisiting, fighting again, etc. This is all a natural part of groups of people working together—the editor gets to make you go back and rewrite the article, the FAA makes the pilot take certifications, surgeons have to see patients outside the OR, bankers need to report on their returns, etc. And surprisingly every profession when they have complaints they refer to this as the bureaucracy.
As Peter Parker’s father said (sort of), “with power comes responsibility.” So if you have the ability to put something on the front page of the NY Times then your editor is probably going to get in the way. If you want to change the user interface of Office you bet that a lot of people are going to have opinions and you are going to have to spend the time planning and developing an architecture before you just start checking in code.
But you know through all of this discussion I kept thinking about the developments we have done in Office over the years (since the Office 2000 case) and Office12 in particular. We decided to change the file format to XML in Office – actually the team decided to do it and we just did it. We did not have any corporate oversight, no approval process. We just did the work to make it real. The new user interface in Office was decided on 4 levels down from BillG – it was done so without a big approval meeting, without “top down” forcing of a specific design. Perhaps in a future blog, if I don’t get too many flames on this whole topic, I’ll go through the bottom-up process that led to the new UI. Of course there were many big “fights” about *how* the UI should be done, but if it should be done, or who should do the work, or should we change our mind—none of those were mucked with unskilled by middle managers :-)
One thing we did in developing Office 12 was take our organization and break the work down into very small “design/build teams” that go below the feature team organization I talked about earlier. These “feature crews” allow essentially 1 or 2 developers to work side-by-side with test and pm on their feature areas with a private branch of the source code for as long as they need to in order to get the work done. The “isolation” this affords worked very well for many teams. It is a new process and we’re still learning. But if you look at the feedback from the PDC and TechEd (and soon you can hear from our MVPs) we made a lot of progress in this product cycle that really impressed folks (we still have a lot of work to do).
Another thing that came up in the discussions today was the notion of “it sure takes a long time to do one product cycle”. This is definitely a choice we make for Office. Actually, we release over 100 product changes every month for Office XP/2003—these are customer requests, maintenance updates, and robustness fixes based on our instrumentation. We just released a service pack as well, which rolls up ~6 months of these product changes into one installation. We could release these “automatically” but we choose to do this on a more regular interval because that is what most of our customers request of us (obviously if we had a high priority fix we would be more aggressive). For the full product releases, customers tell us different things. Corporate customers sometimes say they would like a new version approximately every…10 years or so J At the other end of the spectrum, popular computer magazines could use a new release twice a year since subscribers like that J We choose about every 24 months or so since that is a good midpoint.
There’s been a lot written about moving very fast and getting new things out there. That is all well and good. For new markets this is definitely something that Microsoft and many companies do—we released big updates for InfoPath and OneNote less than a year after the first release (about the time between releases of things like the search bars that are popular these days). But the truth is that as a developer you know very well that you can only write so much code in say 6 months. And this is even harder if you don’t have any time to plan. And of course if you are an MBA you know that you can only make a big deal in the marketplace about a very significant innovation—and doing that is hard if there is not a lot of engineering to support that. Every once in a while a small amount of engineering can yield a whole marketing campaign (red squiggles or AutoCorrect in Word). But as we all know, most new features are no massive breakthroughs upon which you can build a whole marketing campaign. So you have to build a set of features that solve a customer scenario or theme. This is an area where in hindsight people always find counter examples—but the question is not can you find counter examples in hindsight, but can you define today a feature that will hit it out of the park in six months? If you can, you’re hired—send me your resume!
Despite this defense [I know it comes across like that, but I didn’t mean it to be] of bureaucracy it is obvious that like any organization we’ve done some silly things. We’ve put in processes that make no sense. We’ve decided things as an organization that are plain dumb. How do we excuse these? We don’t. MS people send me mail. I want to know about them.
Microsoft is a learning organization. It is in our DNA. We are hypercritical about our own work. At the end of every milestone and project we look back and then change things going forward. We take out process as much as we can. We change processes. And most of all, if something isn’t working this is a company where you can send an email rant to the very top leader and I guarantee you that you will be heard (heard is different than necessarily agreeing or acting, though). I know when I get mail like that I am in your office right away. I know many companies have a suggestion box. I don't know of many that have one that is near instant.
Building software that 400M end-users depend on and pay you for is a big responsibility. We’re always balancing that responsibility with trying to push the envelope of new technologies.
I personally like definition (b) of bureaucracy and can't stand definition (a).