What's good for you isn't always good for your group. Obvious, right? You can call it local versus global optimization. You can get geek philosophical about it and say, "The needs of the many outweigh the needs of the few…or the one." Or you can simply notice the difference you feel between zany ideas from the intern (cool) versus zany ideas from your general manager (scary).

For example, spontaneity in an individual is a good thing and unvarying predictability makes Jack a dull boy. But when Jack is running a large enterprise, unpredictability can wreak havoc. There are managers who grow up and learn this lesson, and there are managers who are Randomizing Ambiguous Nimrods Causing Incessant Distraction, or rancid for short.

I despise rancid managers. They think they are responsive and flexible when in reality they are the fastest means to team dysfunction and failure. If you are a manager and you tell your team, "Hey, I've got a great idea," and they look back at you seemingly saying, "Does it involve tying yourself to a tractor and driving off a bridge?", then you might be rancid.

It's not that bad

What's so bad about being a rancid manager? Isn't consistency and predictability boring? The answer lies in Brownian motion, which describes the movement of particles under random bombardment. Particle movement in Brownian motion could best be described as erratic.

Consider a large team of engineers who are constantly being pushed in different directions at random intervals by their general manager. You'd expect the same erratic movement from that team. Their chances of actually accomplishing anything as an organization are negligible.

Alternatively, if the team is consistently pushed in the same direction they will gather momentum and make significant progress toward their goal. Therefore, a successful manager sets a clear direction, points the team in that direction, and consistently pushes in that same direction until their goal is achieved. Course corrections that naturally come along should translate to gradual team nudges, until the correction is made. Only severe circumstances should prompt major changes in course.

I’m talking about high-level direction, not the day-to-day details which require more agility and flexibility. The key is aligning the day-to-day detailed decisions to the high-level direction. You can’t do that if your high-level direction constantly changes.

Do I look all rancid and clotted?

If teams run by rancid managers are erratic and make little progress, why are there so many rancid managers? Here are three reasons:

§  Being rancid seems responsive. Instead of always pushing in the same direction, rancid managers respond quickly to outside influences, giving their management a nice warm phony feeling of agility.

§  Being rancid means never having to achieve anything. Since the world of a rancid manager's team is always changing, there's a self-fulfilling excuse for why nothing gets accomplished ("Hey, a bunch of stuff happened!"). If these managers weren't rancid they'd risk being accountable. They'd have to do the hard work of setting an achievable vision and direction for their team and then achieve it.

§  Being rancid keeps you occupied. Where's the fun in consistency? What do you do all day to stay busy? What value do you bring to the team if you're always saying the same things? Making a call and sticking to it is difficult. It's scary and you might need to do real work to support it.

Of course, rancid upper management breeds rancid middle management. It's hard to escape. Often people don't even know how good consistent management can be until they finally experience it.

It's a path made of principle

When managers stop being rancid and start being consistent, a number of wonderful things happen. First and foremost, teams start achieving results. Managers move from creating havoc to preventing havoc. And team members actually understand what they are supposed to do each day, and know they have to do it because it won't be different tomorrow.

All this leads to a higher sense of purpose and higher morale. At first, the team may struggle to get fully aligned, but soon the job of a manager becomes easier. You're not always changing your story. Once the team is aligned there are fewer problems to resolve. It's easier to track and show progress.

What challenges does this leave for managers? The biggest is to paint the picture of where the team is going. You need to be clear and understand it deeply in order to describe it consistently and repeatedly to the team. Once you have your story of the future, the challenge is to stay on course at a high level as team members and external factors change. You must discover and communicate adjustments as needed, and prevent upper management from inappropriately disrupting your course.

It's not easy. You need confidence in your convictions and strong thinking behind your direction. It helps to have principles you follow and communicate. They let people know what you expect and how you make decisions. Doing so helps them align their decisions with yours and gets the whole team behind you.

Oh, the noise! Noise! Noise! Noise!

"But things change!" protest the rancid managers, "Is your team supposed to blindly follow some ancient plan? Our agile competitors will beat us every time." Of course things change and plans, architectures, and designs must adjust. Details and understanding constantly evolve, driving continuous iteration of our work. But if your goal today is to build a social computing experience, no detail or external influence should be switching you to build tractor tires.

Strong managers know that all kinds of variation in team structure, competitive landscape, market fluctuations, and technical challenges precipitate changes to the best laid plans and designs. However, it's rare for those variations to necessitate a complete shift of direction. A strong manager welcomes change and contextualizes it for their team so that their momentum toward a shared goal stays strong through many corrections in course.

All progress has resulted from people who took unpopular positions

Being consistent as a manager may seem dull, but it's the kind of dull teams truly appreciate—the kind of dull that leads to results. While it sounds easy to be consistent, it requires courage. Courage to stand your ground and stand behind your words. Courage to be accountable for the direction you've set.

I'm talking about commitment. I'm talking about integrity. I'm talking about shaping the world to fit your vision rather than mindlessly following the latest trend or circumstance.

Our vision should be shaped by the needs and aspirations of our customers and the imagination of our collective minds, but in the end our vision belongs to our leaders as individuals. Our success depends on them having the courage to speak it and steadfastly pursue it. Do you have that courage?

Looking for that perfect candidate to fill a role? Good, that means you'll never steal a great candidate away from me. I love it when industrial-strength stupidity renders my competition comatose. You can't hire the perfect candidate, but please keep trying. Maybe after six months I'll even get your open headcount.

This isn't a case of letting the perfect be an enemy of the good. Idiotic hiring managers aren't trying to create perfect candidates—they are waiting for them to appear. It's like a romantic awaiting their soul mate or a home buyer searching for their "dream house." Guess what? Dreams aren't reality.

If someone tells you on your first or second date that you are the man or woman of their dreams—run. They aren't dating you. They are dating their dreams. Sooner or later, they'll realize the two don't match. The same thing happens when hiring managers get caught up in seeking their dream candidate.

Hiring people instead of pipe dreams

It's natural for a hiring manager and their team to imagine what the new hire will be like. When you spend so much time checking resumes, doing phone screens, informationals, and interviews, you can't help but think about the potential outcome.

Unfortunately, there are three dangers with getting caught up in fantasy:

§  You may not interview a great candidate even if they meet your needs because they don't match your preconceived notions.

§  You may hold off on an offer to a great candidate because they lack attributes that were imagined but not essential.

§  You may regret hiring candidates who turn out different from your projection of perfection, compromising their chances for success.

Of course, you shouldn't hire people who don't meet the high bar we set, but it's awful to lose out on strong hires because you were busy hallucinating. Let's dive deeper into each case.

I found myself much more reasonable

If you have a preconceived notion of whom you're hiring (worse case—someone just like you), then you're certain to subconsciously filter applicants based on that template. You may get candidates that match your musings, but you'll miss a crowd of candidates that meet your needs. If a good hire is one in a hundred (not far from the truth), the sooner you review a hundred qualified candidates, the sooner you'll hire someone. In addition, less filtering brings more diversity, which leads to better balance, better products and services, and better customer experiences.

If you hold off on an offer to a great candidate thinking that the next candidate might be even more ideal, you're likely to lose both. Surely another team will want your great candidate, and your next candidate will be at least as flawed. They say, "A bird in the hand is worth two in the bush." Thus, the next candidate better be twice as good if you hold off. Otherwise, make an offer now before your great candidate becomes your great regret.

Speaking of regrets, ever order something only to regret it shortly after? That feeling of buyer's remorse can afflict managers as well. Hire's remorse happens when you've imagined who might fill an important position, fill it with a real person, and then regret the decision because that person isn't precisely who you wanted. Quit sulking and get over it. Only movie directors occasionally get what they imagine, and that only lasts while the camera is rolling. You've hired a great person, now give them a chance.

Well, what do you need?

How do you differentiate what is really required for your open position from your unintentional biases? Consider the must-have results you seek from the role, and consider how soon you need those results. Then insist on candidates that have achieved those results in some general way. The sooner you need results, the more specifically the candidate needs to have achieved them before.

For example, say you need someone who can develop and ship code for a service. If you can afford to have them figure it out over nine months, hire someone who has demonstrated they can write code of any kind and ship it. It could be a game or a numerical algorithm for a thesis. All you need is the development skill and the ability to finish something. That could be almost anyone from any field—lots of possibilities.

However, say you're in SQL Server and need an engineering lead to step in right away. In that case, you need someone who has recently been an engineering lead. Your need for SQL Server expertise might not be as immediate though, so that's not an essential skill and shouldn't bias your thinking. You'd hate to pick a lousy lead with SQL experience over a great lead who can gain SQL experience over time.

The key is to focus on must-have results and the core skills essential to achieving those results. Then look for anyone who demonstrates those essential skills and gets analogous results. You can be more specific if your need is more immediate, but don't kid yourself. Getting picky may cost you candidates who'll deliver great results for years if you support them and give them a chance.

You could even say that he has principles

One item you generally won't find on a resume is a candidate's principles. Principles don't tend to align with appearance, skill set, or even achievements. Thus, when you imagine your ideal candidate you often don't think of principles, which is yet another reason not to waste your time imagining your ideal candidate.

What principles are important to you and your team? Integrity? Transparency? Accountability? Selflessness? Loose coupling? Screen for them. They are just as important as skills sets and demonstrated results. Many would argue that principles are more important. People can learn skills and work together to achieve results. Principles can be harder to attain or change.

Don't know what principles matter to you and your team? Figure it out—now. Let your team know. Being principled is itself a principle. For me, it matters the most and is the first trait I seek when talking to candidates.

I'm trying to tell you something about my life

Life is not a novel you get to write where everything turns out the way you conceive it. It's full of twists and turns for you to ride and hopefully enjoy in the end. Hiring and building a team is no different.

You have no idea who will respond to your open positions at what times, so don't try to guess. Instead, keep your mind open to all possibilities. Remember, you want a team full of differing viewpoints that are all aligned toward a common goal.

While you don't want unpredictable ships dates and quality, you do want unpredictable ideas and interplay on your team. They are the source of innovation and improvement. The more your team is filled with variety instead of clones, the more your team will enjoy learning from each other—a result reflected in more thoughtful designs and improved customer experiences.

So, consider the needs of your team, but stop short of projecting who might fill them. Keep your mind open to the possibilities, give offers immediately to great candidates unless you know the next candidate is twice as good, stick by your principles, and avoid hire's remorse. Everyone complains about insufficient resources—the sooner you fill your open positions the happier and more productive your team will be.

My older son can now drive. This adds two new worries to my life—how ancient I feel and thoughts of my son in a ditch somewhere. To mitigate the second worry, my wife and I enforce a curfew and insist my son call if he's running late. The other night, he arrived home 20 minutes late without notice. My wife was furious that he was late. I was furious that he didn't call.

Why didn't my son call to say he was running late? Because, like my wife, he was focused on the schedule. He avoided facing conflict until he got home. He said, "I got home as fast as I could"—presumably breaking numerous traffic regulations along the way. My son completely missed the point. The purpose of the rules was to mitigate risk, yet his response to them was to drive recklessly.

Software engineers do this all the time. They come up with a development schedule, unexpected issues come up, and they end up being late. Instead of informing their managers of the delay, they avoid facing conflict, rush the work, sacrifice quality, and slip the schedule, all with little control or visibility. It's the opposite of what managers should want, yet those same managers insist on following the schedule precisely. Why? Because most managers and engineers don't distinguish between the two types of scheduling—meeting a commitment and managing risk.

Those who understand binary and those who don't

Yes, that's right. There are two types of scheduling and project management.

§  Meet a commitment. You made a commitment to customers or partners and you must meet it at the quality and time period promised. Period.

§  Manage risk. There is a mix of critical and desirable work. People can make bad choices. Issues can arise. You must manage risk to ensure critical work gets done.

These two approaches to scheduling and project management often get confused. Why?

§  They typically appear together. The overall project has commitments, but it is made up of smaller tasks that require risk management.

§  They both use dates. The difference is that dates for commitments are untouchable and drive everything. Dates for risk management are simply checkpoints to make sure work stays on track.

§  They both are called scheduling. Most people simply don't know the difference.

§  Meeting commitments is the only type most people are taught. From the time kids enter school, they are exposed to inflexible due dates and commitments they must meet. When they later learn project management, it is all about Gantt charts and milestones, with some risk management thrown in as an aside.

§  Managing risk is usually self-taught and informal. A small number of people are formally taught risk management. Most of us learn it from peers in college as we juggle large workloads. Instead of completing everything on-time, we form workgroups, focus on the critical work, and minimize the damage to our grades.

This tragic lack of understanding leads to horrible decisions, poor engineering, and scheduling disasters. You need to know the difference and apply the right scheduling to the right problems. Let's start with meeting commitments.

That's the only thing you're committed to

Meeting commitments is essential to working with partners, which in turn is essential to running most businesses. You can't coordinate work across internal dependencies or external agreements without synchronizing on dates and deliverables. Because commitments cascade, you must meet them or face catastrophe.

Say your daughter's birthday is coming and you've promised her a new game she wants. How would you feel if it wasn't delivered to you when promised? There is a chain of handoffs between the developer, the manufacturer, the seller, and you that all must be met to ensure your daughter's happiness on her birthday.

Of course, this is much easier to do for Web-based products, but the same problems come up with handoffs between teams or departments. Delivering on commitments builds trust and lasting relationships between partners. Missing commitments does the opposite. While many software projects require little or no coordination across groups, running large and mature projects typically involves meeting commitments, and thus formal project management techniques.

Don't you think it's a little risky?

Risk management is about making sure the critical work gets done properly, even in highly variable environments. Scrum and Feature Crews are excellent examples of software development practices that focus on risk management—in particular, the risk that you won't efficiently ship value and quality to the customer.

As I pointed out in my very first column, Dev schedules, flying pigs, and other fantasies, dev schedules and test schedules are in the risk management category. All the feature dates are checkpoints to mitigate risk. Only the small number of cross-group synchronization points (major milestones) are commitments.

What matter in risk management are focus, order, and status—not precision. Focus on what's important, do those items in priority order, and track the status as the situation changes. Notice that hitting precise dates for tasks isn't important, so long as you finish the critical tasks at the expected quality on time. Everything else can be cut.

That's why you must tell your engineers the same thing I told my son, "Coming home right on time isn't important. Telling us you aren't coming home on time is." You can only manage the risks if you know about them. The dates are there only to alert you when plans need to change.

Of course, you can't slip a critical task past a commitment date (a major milestone), and my son can't stay out past 1:00 A.M. or his license will be suspended. But curfew is well before then, as it should be, to avoid any chance of catastrophe.

You pick the one right tool

So before you start working on a schedule, stop and think about what's on it. Is it a set of dates and deliverables you've committed to a partner? Or is it a set of tasks with various priorities that you need to track and avoid messing up?

In the case of my son coming home by curfew, it was a task we didn't want him to wreck, literally. Thus, we were doing risk management and the focus needed to be on giving timely status as opposed to coming home at a precise time. Most software development tasks fit that mold. You just want your engineers to tell you promptly if they are running late so you can adjust. Precision is unnecessary and potentially counter-productive.

However, if you are running a large project with multiple teams and partnerships, then commitments and synchronization are critical at a high level. The high-level schedule is full of milestones and classic project management tools.

Just don't confuse the high-level schedule with the low-level tasks. If you treat the low-level tasks like your high-level commitments, your engineers will take shortcuts and drive too fast. Instead of managing risk, you might cause them to crash one of your critical tasks, which in turn breaks your high-level commitments. Use the right tool for the right level. You'll sleep better at night.

It's the end of the fiscal year. Most engineers associate this time with performance review season, but for principal-level engineers and higher it's also executive review season. Time to waste weeks of your life writing slides for executive presentations that will be rewritten five times before they are never presented.

Executive reviews aren't a waste of time—occasionally you need an experienced, authoritative voice to blow apart your assumptions and refocus your efforts on desired business results. Preparing isn't a waste of time—being forced to explain yourself to others always helps your thinking, and I've got no desire to look like an idiot in front of the person who approves my compensation. The real waste of time is the focusing on getting the right slides for every season and situation instead of getting the right strategy.

Smart, high-level people simply don't know how to cope with executive reviews. They think it's a time to show off instead of a time to listen. They respond inappropriately to executive criticism of their badly presented, unsuitable slides. I've done it too—it's a trap set by our superiors filling out the poor templates dictated by their superiors. It's the misinformed leading the uninformed. Well, now it's time to break that cycle, avoid the pitfalls, and focus on what matters—valuable feedback on your clear and concise plans.

Wisdom to know the difference

Why do so many otherwise intelligent people bungle executive reviews? I believe there are two reasons—exuberance and confusion.

§  Exuberance over the importance of the moment. It's so important that we must cover every detail rather than focus on what's important. Twisted huh? Who defines what's important? The executive. What's important to the executive? Ask. Press for clarity. Don't accept second-hand smoke from an assistant. It's not good for your health.

§  Confusion between executive reviews and presentations. Both use slide decks. Both involve presenting. The difference is that in presentations the presenter is in control. In executive reviews, executives and their posses are in control. Failure to recognize this difference leads to failure in your review.

The secret of my success

How do you handle executive reviews successfully, obtaining all their potential benefits? Here's a three-step guide:

1.      Learn what is important to your executive.

2.      Present that information in three slides or less.

3.      Respond to questions with insight and to feedback with thanks.

That's it. Let's break it down.

A riddle, wrapped in a mystery, inside an enigma

You first must learn what's important to your executive, from both an informational and a philosophical perspective.

From an informational perspective, what does your executive want to know about your project? Alignment with other projects? Financial contribution? Market share impact? Value proposition? Competitive response? You want to learn how your plans are being evaluated.

From a philosophical perspective, what principles are most important to your executive? Transparency? Alignment? Loyalty? Integrity? Self-confidence? You want to learn how you are being evaluated.

How do you learn your executive's informational and philosophical perspectives? Ask peers who've already been through a review with your executive. Ask your boss and your skip-level boss. You can even get thirty minutes on your executive's calendar. Be sure not to believe any one individual, but see patterns in feedback from multiple sources.

Easy as 1, 2, 3

Now you are ready to create the slide deck. You need only three slides—the current situation, the desired situation, and the tactics to get from the first to the second. That's all you should have. Remember, you are not in control of the review—the executive is. All you can hope to do is frame the discussion. All the other slides can either be cut or left as appendix slides for reference.

The current situation slide may be a problem statement, a current scorecard, or a recap of progress to date. The proper context and information depend on what's important to your executive, which you learned earlier.

The desired situation slide may be a solution, a target scorecard, or a going-forward strategy. It should align with your first slide and resolve the issues it raised.

The tactics slide may be a timeline, bullet list, or table of next steps, typically also indicating risks and mitigations with associated asks. Careful what you say on this slide. It tends to convert directly to commitments.

I read the instructions

Many executive reviews require you to follow a template with predetermined slides for you to fill-out. Having a template is terrific—it adds consistency and clearly sets expectations. Unfortunately, most templates are hideous with four or five times the number of slides needed. The templates are either ancient or produced by assistants.

How do you handle a fifteen-slide template? Pick out the three crucial slides—the ones that describe the current situation, the desired situation, and the tactics to get from the first to the second. Get those three slides right—then ensure the other slides align consistently with your crucial slides. This keeps you on message no matter where the executive takes the conversation. If possible, skip past the other slides and stick to the crucial three.

The purpose of the review is to get valuable feedback on your clear and concise plans. By aligning your slides and staying focused, you can frame the discussion and get the input you need.

Oh behave!

The last step in my three-step guide to executive reviews is about how to behave. First and foremost, do not be intimidated or enchanted by the executive. Executives used to have your job and they miss it (just ask if you don't believe me). They still use the bathroom. They still embarrass their kids. Do yourself a favor and get over yourself. Act like you've done this before.

The executive will typically take two actions during your review—ask questions and make comments. Most of the time you want to take notes and keep your mouth firmly shut. Use duct tape if you must. Abraham Lincoln said, "It is better to remain silent and be thought a fool than to open one's mouth and remove all doubt."

When should you say something? When you have something insightful and relevant to say. "I agree" doesn't cut it. "We're doing that" doesn't cut it. "When we fixed that issue our support calls dropped 67%" might be worth mentioning if it's relevant. When you open your mouth, be sure you are adding value, staying respectful, and being concise. "Yes," "No," and "I don't know" are often sufficient. The rest of the time you should listen.

As I mentioned in "I'm listening," the single proper response to all feedback is, "Thank you." In the case of executive feedback, be sure to write the comments down and consider each point. You don't have to address everything the executive mentions, but you do need to consider it.

Remember, executive reviews aren't the time to show off. They are the time to receive valuable feedback on your plans. The time to show off is when you deliver on those plans.

How'd I do?

When the review is finally over, you'll likely feel awful. The executive asked many tough questions and made a bunch of pointed comments. Was it a disaster? How can you tell?

Luckily, it's easy to tell how the review went. The key is the level of detail the executive discussed, not in the number of positive or negative comments. If the executive's questions and comments were all general and high level, then the review did not go well. The executive was questioning your basic strategy and assumptions. If the questions and comments were in the details, then the review went quite well. The executive agreed with your strategy and approach and was giving you feedback on small pieces.

Either way, executive reviews aren't about giving you an ego boost. They are about getting valuable feedback on your plans. Learn the information and principles the executive cares about, present your plans as simply as possible within that context, and be a professional during the review, and you'll get all the feedback you need to succeed.

The annual engineering awards are being given out this week at the Microsoft Engineering Forum. Annual reviews will soon follow. These are great opportunities to recognize impactful work. It's too bad most managers are tragically ignorant of how to recognize their employees or truly why they should.

If you are a manager, you're probably relishing this opportunity to heckle all those bad managers. Guess what? I'm talking about you. You don't know how to use recognition properly. You don't know why you should. "But I'm great at recognizing my employees," cries a clueless manager. "I'm always congratulating my team—I even shaved my head once." Let's call this manager, "Chaos."

Chaos thinks recognition is all about morale and motivation. Certainly, there are real benefits for Chaos being a team cheerleader. But if that's the depth of his use of recognition, then chaos is what he'll get.

Everybody wants results

What Chaos fails to recognize is that recognition is a form of reinforcement. Reinforcement drives behavior. Behavior drives results. Results are king at Microsoft. Good recognition focuses on reinforcing desired results (and correcting undesirable ones).

If you aren't thoughtful about the results you seek, recognition can easily drive detrimental behavior. For example, Chaos shaved his head when his team met a tough milestone. To meet that milestone, team members cut corners on quality and deferred fixing structural issues. Those issues increased the "bug debt," prolonged stabilization, and reduced release quality. More importantly, Chaos' team members learned that Chaos rewards cutting corners and doesn't respect quality.

The end may justify the means

"Yeah, but we hit the milestone. We built team unity. That means something!" claims Chaos. Do the ends justify the means? No, not with your narrow view of the ends.

The ends for Chaos were a united team hitting a tough date. However, others ends were achieved—increased bug debt. If the ends had been defined as a united team hitting a tough date with zero bug debt, then the means can be left to the team.

Chaos is getting cynical. He says, "So if the team robbed a bank to get money to pay a vendor to reduce bug debt then that would be okay?" No, it wouldn't. Chaos has introduced another end—jail time. Define the ends as a united team legally and responsibly hitting a tough date with zero bug debt, and that problem is avoided.

It's not easy to carefully define and communicate the ends you seek. It's much simpler to follow Chaos and reward haphazard results, ignoring the unintended consequences. However, setting clear expectations and recognizing your truly desired results pays huge dividends:

§  You micromanage less.

§  Your team innovates more.

§  You get the results you desire.

§  You have more time to focus on developing and recognizing the great work of your team.

The time has come to act and act quickly

Do you wait till the ends are achieved before recognizing them? Recognition is most effective when given immediately after achieving a goal. But you also want to recognize all the intermediate results that led to the end result.

For example, on the way to your united team legally and responsibly hitting a tough date with zero bug debt, your lead ran a great design review. You should immediately recognize her effort saying, "That was a terrific design review today. I love how you listened to everyone's opinion without critique. I also liked how you summarized that feedback and improved the design. Correcting those errors and misunderstandings early will save us significant time and lead us toward hitting our date with zero bug debt."

Notice the keys to great recognition:

§  It's immediate—the same day, or even better, the same minute.

§  It describes precisely what you liked—as opposed to the generic "nice job."

§  It's tied to the desired end result—reinforcing the real value of the effort.

You may not have the time or opportunity to recognize every positive step toward your end goal. However, you should keep your eyes open and address as many small victories as you can. Your team will love it, they'll better appreciate your expectations, and they'll be better aligned toward your desired result.

Let us celebrate

Chaos wonders, "Day-to-day recognition is great, but doesn't it detract from the big celebration at the end? Was I wrong to shave my head?" Chaos' grooming habits aside, a celebration marking a major achievement is just as important as day-to-day recognition. However, because it's a culmination of a long-term effort it's not as simple as sharing a few comments or shaving your head.

First and foremost, recognition should be for a carefully defined desired result, not something arbitrary like "great effort" or "someone said something nice." For example, don't give out an award for "teamwork." Give out an award for "teamwork across divisions and geographies that led to delivering a complete customer scenario."

Once you have clearly defined criteria for the recognition, plan the celebration. Here's what makes for a lasting positive experience:

§  Make it personal—if you defined the goal, the recognition should come from you. Include some personal remarks that express the spirit behind your goals and the way individuals or the team embodied that spirit.

§  Make it public—public ritual is important. It carries meaning, builds community, spreads the message, and creates a shared sense of purpose.

§  Make it lasting—the associated award should be substantial to the senses—heavy, bold, eye-catching, and tactile. Shaving your head meets many of these criteria, but so do heavy physical awards (like the Oscars) and giant signed banners on heavy poster board.

These guidelines give your celebration enduring emotional impact. Food or money don’t cut it.

I'd like to thank the Academy

The last element to consider is, "Do you give the recognition to the team or to the individual?" While you can certainly give the whole team a celebration and a day off, recognition for carefully defined desired results should be called out to a small number of recipients (typically, no more than five). Any more than that dilutes the impact.

Why is the impact diluted when given to more than five recipients? Because when you recognize a larger group, there are sure to be individuals who didn't embody the spirit of your goals. The message you send is that tagging along is just as good as being the driver. It isn't. Who originated the action? Who drove it to fruition? Those are the people at the center and in the lead. Those are the people you recognize and encourage others to emulate.

All right, let's review

Now that you know how to recognize your employees and why you should, how can you apply this to the annual review process?

§  Describe what you liked (and disliked) about the results your employees achieved and how they achieved them.

§  Talk about the small results and the big results. Tie them together.

§  Set carefully defined goals for the future that will drive personal development and stronger results for your business and customers.

Clarity around expectations tied to consistent feedback described in plain language is critical to getting great results, and a happy and secure team. Recognition requires careful thinking and deliberate action, but it's not difficult to do. Avoid the chaos and you can even keep your hair.

Call me "old school" but I believe in shipping. Trying isn't enough. Getting close isn't enough. Good ideas aren't enough. You've got to ship.

It used to be that interviews started with, "What have you shipped?" If you hadn't shipped recently, "Why?" Why? Because you can't deliver customer value if you don't deliver. You can't iterate and improve without finishing an iteration. You can't get customer feedback without customers.

People used to complain that promotions and rewards were disproportionally distributed to those who shipped. I say, "Absolutely, that's how it should be." Does this hurt quality? No, you set a high minimum quality bar and ship. Does it hurt innovation? No, innovators have always risked an initial drop in pay to receive a big payoff should they deliver.

It all starts with shipping. This is particularly apt with services, where everything literally starts with shipping, and where I'm focusing the rest of this column. Our critics claim that in the new world of services Microsoft has forgotten how to ship. Perhaps, but Microsoft has forgotten more about shipping than most companies will ever know. We just need some reminders and reeducation, especially when it comes to services.

I offer you my service

How much about shipping services has Microsoft forgotten or doesn't get, according to critics? Not as much as they would have you believe, but enough to make you think. Let's go over the herrings and the heartaches, mixed with a little happiness.

The red herrings:

§  Services make you think about everything differently.

§  Services center on data while packaged products center on functionality.

§  Services have greater security concerns than packaged products.

§  Services have serious issues with dependencies.

§  Services demand higher quality and faster iterations than packaged products.

The heartaches (and happiness):

§  Services run across hundreds of machines, not on a single client.

§  Services must scale out automatically.

§  Services are easier to switch than packaged products.

§  Service upgrades hit everyone instantly.

§  Services are living, changing things.

Let's break these down, starting with the red herrings.

What is that smell?

The first services red herring is a big one, "Services change everything." As I addressed in At your service, this is total bovine fertilizer. Services start and end with helping customers achieve their goals, just like all products ever. You focus on the customer experience and what they hope to accomplish or you lose. End of story.

The next three red herrings—centering on data, security concerns, and dependency issues—all apply just as well to shipping packaged products, though it may have taken us longer to realize it. You can't expose data format changes to customers without chasing them away, on the client or the server. There isn't a computer product or service today that isn't vulnerable to attack—you must secure them all. Finally, if you think external dependencies aren't problematic on the client, you clearly don't use many drivers. I'm not saying these aren't real issues—I'm saying they aren't new or specific to services.

The last red herring is among the most common concerns raised about why shipping services differs from shipping packaged products—high availability and Internet time. Look, it's not okay for packaged products to never work or require a reboot every time you use them; at least it hasn't been for quite some time. The quality bar is no different for services, though there are plenty of services that fail constantly.

As for Internet time, that hit packaged products a decade ago with the introduction of Windows Update. And if you think that those patches are just security fixes, you haven't been paying attention. More and more we are fixing all kinds of experience issues shortly after customers report them, for services and packaged products. That's a great thing for customers.

However, gradually improving the customer experience every month or every day isn't enough. Both services and packaged products need to ship significant, orchestrated updates to deliver breakthrough customer value. Facebook wasn't going to gradually update itself into Twitter any more than Vista would gradually update itself into Windows 7. You must focus on what the customer is trying to accomplish, and sometimes that isn't a quick change.

There are too many of them

However, not everything about shipping packaged products applies to shipping services. There are mental, process, and team adjustments that you need to make.

First and foremost is that services run across hundreds or thousands of machines dispersed in multiple data centers worldwide. Sometimes functionality and data are replicated. Sometimes functionality and data are specialized. Usually, it's a combination of both for scale and reliability. Naturally, this presents design and synchronization problems but plenty of books have been written about that (read don't rediscover). The less obvious challenges are around debugging and deployment.

Why is debugging a service so tough? Timing issues are killer given multiple threads on multiple processors across multiple machines. Yikes! However, that's not even the toughest challenge.

What's the first thing you do when debugging an issue? Analyze the stack, right? With services the stack is split across servers and requests, making it nearly impossible to trace a specific user action. The good news is that there are new tools that help tie user actions together across machines. The bad news is that this isn't the toughest challenge either. The toughest challenge is that you're always debugging in the live environment. You don't get symbols, breakpoints, or the ability to step through code.

So let's recap. Debugging services means debugging nasty timing issues across multiple machines with no stack, symbols, or breakpoints on live code. There's only one solution—instrumentation—and lots of it, designed in from the beginning, knowing you'll soon be debugging across live machines with no stack, symbols, or breakpoints.

They're multiplying too rapidly!

Solving debugging brings us to the other huge challenge—deployment. Deployment needs to be completely automated and lightning fast. We're talking file copy installation, with fast file copy. No registry, no custom actions, and no manual anything.

Why does deployment need to be so fast and simple? Two reasons:

§  You're installing onto hundreds or thousands of machines worldwide while they are live. Installation must work and work fast with zero human intervention ever. The slightest bit of complexity will cause failures. Remember, five minutes times 1,000 machines equals three-and-a-half days. It had better just work.

§  The number of servers needs to grow and shrink dynamically based on load. Otherwise, you are wasting hardware, power, cooling, and bandwidth in order to meet the highest demand. Because your scale depends on load, it can change any time. When it changes, you need to build out more systems automatically and instantly.

The happiness around deployment is that Azure will do most of the heavy lifting for you (so let it, don't reinvent). However, you still need to design your services to support file copy installation.

Life is so uncertain

Enough of the challenges you can predict, how about the unpredictable ones? The services landscape is in constant change. While some services are sticky because they hold your data (like Facebook or eBay), many aren't sticky at all (like search or news). A few minutes of downtime can cost you thousands of customers. Data compromise or loss can cost you millions of customers. They'll just switch. Our competitors will be happy to accept them. It cuts both ways so you need work hard to both welcome and keep new users.

When you update a service everyone gets the new version instantly, not over years. If there's a bug that only one customer in a thousand experiences, then that bug will hit thousands of customers instantly (law of truly large numbers). That means you need to resolve the issue quickly or rollback. Either way, it's a bad idea to update a service on a Friday and a good idea to have an emergency rollback button always at the ready.

Finally, it's important to realize that services are living, changing things. You'd think that because the servers are all yours with your image and your configuration that it would be a controlled environment—and it is until you turn on the switch. Once the server goes live, it changes. The memory usage changes, the data and layout on the disks change, the network traffic changes, and the load on the system changes. Services are like rivers not rocks. You can't ship and forget services. They need constant attention. To make your life easier, bake resilience in by automating the five Rs—retry, restart, reboot, reimage, and replace (though replace may require human hands at some point).

The happiness that comes with these heartaches are customers willing to switch; an ideal idea testing platform because you can show customers different ideas and see which they prefer on a daily basis; and the ability to ship now and find the tricky intermittent Heisenbugs later (using your five Rs resilience to keep up availability).

Back to basics

There you have it. Some food for thought mixed in with the old basics of writing solid code that focuses on customers and their goals.

However, none of this is worth anything without shipping. Make shipping a priority and we all win. Sure, the quality bar has gone up, but we're not kids selling lemonade anymore. We need to ship quality experiences regularly, on both long and short time scales. We need to ship on the Internet, on the PC, and on the phone. We need to serve our customers well and delight them into sticking with us. It's a long journey, but it doesn't start until we ship.

During difficult economic times like these, people tend to whine less about common complaints that now seem trite. Mostly, I'm relieved not to hear how much e-mail is in Ingrid's Inbox, how Brian broke the build again, and how Suresh's service schedule slipped successive sprints.

However, it's during difficult days that we should patch plaguing problems. When are you going to be more motivated to mend malignant maladies? Surely, no additional alliteration is advisable.

A surprising number of common issues can be solved using two simple techniques—single-piece flow and checklists. There's a ton of behavioral and process theory behind why these simple methods are effective. The point is that they are effective and you and your team are less effective without them.

All too easy

Take Ingrid's Inbox. Like most Inboxes, Ingrid's is overflowing. She spends tons of time on it, yet it only gets bigger. She constantly loses track of mail, discovers mail, and revisits mail. It's hopeless.

Ingrid would have her mail under control if she followed single-piece flow. Single-piece flow tells her to handle one piece (one message) at a time till it's done. By "done" I mean she'll never look at that message again (except to answer a related message).

Here's how it works. Each time after Ingrid reads an e-mail message she does one of four actions:

1.      She deletes the message (my favorite).

2.      She files it away in a folder.

3.      She forwards the message to someone else and then deletes or files it.

4.      She answers the message and then deletes or files it.

That's it. She never opens a message and then leaves it in her Inbox. By the end of each day, every day, her Inbox is empty. She never misses a mail message, loses a message, or revisits a message.

Single-piece flow is efficient because it removes overhead and rework. There's overhead every time you context switch to look at a new message, and there's rework when you re-read a message. In single-piece flow, overhead is minimized and rework is eradicated.

Deja vu – all over again

Brian keeps breaking the build. You can punish Brian till he's afraid of checking in code regularly, but doing so only causes other problems.

A better solution is to give Brian a checklist. Checklists are wildly misunderstood, improperly developed, and underutilized. Regrettably, well-meaning compulsive people list everything possible Brian should check before he submits code to the build. That's not only a waste of time, it's also ineffective.

Brian's checklist should list only common causes of build breaks (less than one page’s worth). It should be in Brian's sight when he submits code. The goal is to be quick, easy, and effective.

Too long or complex and Brian won't follow it. Too short and it's not effective. Luckily, most mistakes are common mistakes. Thus, all the team needs to do is collect a list of the common or critical causes for build breaks and turn that into a checklist. The same is true for design review lists, code review lists, and all checklists.

Remember to update your checklists as your failure patterns change or they will become stale. Checklists prevent common errors and promote good habits. Any structured manual process you follow should have a simple checklist—unless you like being Brian the build breaker.

Slip sliding away

Suresh's software squad slipped their schedule on successive sprints. Bad news for a service—or any project. Is it time to work weekends? No. Is it time to slap the squad silly? No. Is it time for single-piece flow and checklists? Yes.

Suresh's squad is made up the usual way—a few PMs including a service engineer, a bunch of developers, and a similar-sized bunch of testers. The PMs write specs, the developers code them, and the testers test them. The squad is using Scrum-like Sprints with a nicely prioritized backlog of features.

Unfortunately, the PMs are creating specs faster than can be implemented. They waste time and effort on specs that change or are cut before they see daylight. The developers and testers jump from feature to feature as they get blocked. Nothing is in sync. Nothing gets finished. The schedule slips incessantly. In retrospectives, all Suresh's squad talks about is unblocking people.

Even though Suresh's squad is using Scrum-like Sprints, they aren't using single-piece flow. They aren't splitting into cross-discipline teams, with each team working on one feature at a time till it's done. They don't even have a checklist that defines done. It's doomed.

Once they create feature teams that spec, code, and test one feature at a time (sometimes called, feature crews) and a checklist that defines done, there's a chance for progress. The single-piece flow removes blocking issues because everyone is working on the same problem. Instead of jumping ahead, the checklist keeps the team honest, motivating them to work together, finish, and stay focused. Now the team doesn't waste time context switching and can tell how long it really takes to complete a feature, leading to confident scheduling and higher quality finished products and services.

Our two weapons are

So there you have it—single-piece flow and checklists. Two enormously useful, remarkably simple, and yet woefully underutilized techniques for managing workload and building quality software.

Single-piece flow and checklists can be applied to individuals, small teams, and large divisions. They aren't controversial when used pragmatically, and have years of documented case history supporting their effectiveness.

Sure, you could create a whole grassroots movement around single-piece flow and checklists, but that seems a bit overblown for such simple ideas. Maybe you should just use them wisely. Enjoy the time you get back and the improvement in your results. The best things in life are often the most basic and simple.

It's Midyear Career Discussion time at Microsoft. Perhaps you just finished, but more than likely you're still trying to squeeze yours in. How'd it go? How will it go? For you? For your manager? Well, that depends.

It depends a bit on your prior performance and your manager's prior performance. It depends a bit on the feedback itself and how that feedback is given. It depends a bit on how your parents raised you and the comfort of your chair. But the biggest influence on the lasting impact of your Midyear Career Discussion is the way you and your manager respond to feedback.

Let me put this delicately to you. You have no frigging idea how to give and take feedback. Seriously—not one frigging idea. Think I'm wrong? You are only proving me right. If you actually knew how to give and take feedback your response would be a sincere and polite, "Thank you."

Thanks for the advice

In fact, there are only two valid responses to feedback, "Thank you" and "Go on."

The "Thank you" is simple and self-explanatory. Too bad most people don't use it. Most people defend themselves, explain their behavior and results, and describe how they are already taking the right steps.

Please, slowly and carefully shut your mouth, empty your mind, and listen. Perhaps you can even take notes. Then, when the generous soul is finished, say, "Thank you."

You don't say it to be polite. You say "Thank you" because you mean it. Your relationships, your life, and our products and services would reek far beyond their current stench if people were not kind enough to provide an outside perspective and help us improve. Thank goodness they are willing to do it. To ensure they continue it's essential to sincerely appreciate it.

Tell me more about me

In addition to "Thank you," a valid response to feedback is "Go on." As in:

§  "Could you talk more about that?"

§  "I don't quite understand—could you describe that further?"

§  "Thank you, that's helpful, what else can I do differently?"

Anything that encourages clear and continued feedback is appropriate.

Back off, man. I'm a scientist

What's inappropriate is anything that questions or cuts off feedback. This includes:

§  "I'm working on it." So what? You're doing the wrong thing, you haven't made much progress, or you are actually improving. Regardless, the feedback is valid and your comment is irrelevant and self-serving.

§  "I was trying to …" So that makes it better? Never confuse reasons with excuses. If you can get better, you should get better. No excuses.

§  "I disagree." So this is news? You're getting feedback. It's opinion. The fact that you like your current approach is not a revelation. When the feedback seems wrong, you've either missed something or left the wrong impression—that’s precious information.

Keep in mind that you don't have to follow whatever advice you get. All you are obligated to do is listen, consider the advice carefully, and thank the person for helping you.

Now it’s my turn!

Now that you know how to take feedback, it's time to learn how to ask for it and provide it. When asking for feedback and when providing it, there are three basic questions:

§  What is good [about what I'm doing or the work I've done]?

§  What could be better?

§  Any further comments?

You can structure feedback more, but the simplest, complete ask are those three questions. And it is those three questions that you want to answer when you provide feedback.

We have come full circle

When providing your feedback, start with what's good, talk about improvements, add on your other comments, remind about improvements, and then reiterate what's good. That order is important.

§  You start with what you like. It sets up the conversation on an upbeat note and prevents the impression that all is lost. If you start with what's wrong, your listener may never hear what's right.

§  Next, you talk about ways to improve. Ideally, your listener should focus on just one change. One change is all that most people can handle at a time. Pick the most impactful improvement and emphasize it.

§  Of course, you'll have plenty of other thoughts that aren't as important. Feel free to mention those in the context of "a few other comments."

§  Then come back to your main message—the one or perhaps two improvements that would make the most difference.

§  Finish with what is going well. It's important to end on a positive note.

We don't have much time

That's it. Being concise is important. If you want your feedback to matter it should be clear, consumable, considerate, concise, and centered on the receiver. Your feedback isn’t about you and your glorious knowledge; it is about helping the recipient.

If you are on the receiving end of the feedback you should be just as concise. Feedback is precious, whether it's from a customer, a peer, or your manager. Don't get in the way. Encourage it. Savor it. Appreciate it. Thank you.

As I said in Nailing the nominals, the two keys to successful big projects (100K+ LOC) are thinking ahead and defining done. Thinking ahead is about design and planning. Defining done is about setting a quality bar and sticking to it. Yet many big projects go astray even when people think ahead and define done. Why?

Often failure is due to poor executive decisions that place their own agendas above shipping. (Given you hit your quality bar, shipping is better—much better.) However, an even more frequent form of failure comes from engineering teams over thinking and over generalizing—trying to solve world hunger instead of feeding the kids in front of you.

It's so tempting, so intellectually pleasing, and so self-serving to examine the customer problem you're solving and see the bigger problem, the more general problem, the problem you can solve for all time, all peoples, and all nations. Oh spare me, and spare your customers. Cultivating the green fields of broad ideas is not only self-serving, it's a recipe for feature-rich and value-poor products and services your customers will use begrudgingly, despise utterly, and abandon gleefully at the first opportunity. Green fields are full of maggots.

The horror

Yet, I can't walk down a hallway without hearing people trying to come up with ideas, algorithms, or class structures that "will work for this problem, and all problems like it." Evil. Evil! Warning! Watch out. This line of thinking is EVIL!

What's so evil about general solutions? After all, your code could be both a floor wax and a dessert topping. There are three primary pitfalls:

§  You rarely work through the full general solution in one ship cycle. The unfinished framework isn't quite right, but you've shipped it. Now you are stuck with an unworkable legacy code base—forever.

§  You introduce a broader test surface and a broader security attack surface. Neither is desirable.

§  You put the problem at the center instead of the customer. When the customer isn't at the center, your code loses its soul. It goes from being astounding to being adequate, from marvelous to mediocre.

You saved me from this fanatic

Before we break down the three primary pitfalls of general solutions, I need to calm down my agile zealot readers. Because agile methods put a premium on working software and customer collaboration, they tend to avoid the over-thinking trap. In particular, test-driven development and emergent design deliberately focus on keeping solutions as simple as possible and laser-targeted at customer requirements.

Because agile methods avoid the pitfalls of general solutions and general thinking, many agile zealots believe all big design up-front is vile. They are wrong. Sure, the regular refactoring and rework needed for emergent design isn't a problem for small code bases. However, when you get above 100,000 lines of code serious rework becomes extraordinarily expensive. That's why customer-focused, up-front architecture is essential for large projects.

The good news is that many agile methods like Scrum, continuous integration, and test-driven development (TDD) work well within a large project with a customer-focused architecture. These great techniques can keep a team locked onto the customer, instead of straying off into the green fields of self-gratifying intellectual exercises in futility.

Who will save your soul?

Okay, back to the three primary pitfalls of general solutions— incomplete, unworkable legacy code, expanded test and attack surface, and losing your soul.

Say you are developing a big product like Halo or SharePoint. In either case, you can start with a general framework or you can start with a customer story.

The green fields approach is to start with a general framework. The framework supports all forms of weapons, landscapes, and content viewers. Putting together a game or a Web site is simply a matter of choosing the desired combination of weapons, landscapes, and content viewers. Bingo! You've got yourself a cruddy game or Web site.

The game or Web site has no soul because you focused on the framework instead of staying with the story. In addition, to test the framework you've got to consider every combination of weapon, landscape, and content viewer—each of which also presents an attack vector for hackers. And, should you be foolish enough to expose the framework as an "extensible interface," all these problems become orders of magnitude more diabolical.

It's not gonna get any easier

What's even worse is when you design and develop Halo II and SharePoint 2.0. Inevitably, there are many considerations you didn't address in the prior release that invalidate significant portions of your framework. Unfortunately, those portions are already built and shipped, so you get all the enjoyment of legacy backward compatibility with little of the benefit. Aren't you glad you were just so very clever to focus on the general problem instead of the customer and the story?

"But frameworks and extensible interfaces are critical to our platform!" scream the clueless cretins. Yes, they are when the customer is a developer and the story involves building upon our platform. However, the customers of Halo and SharePoint aren't primarily developers. They are consumers and enterprise customers. The story line for them involves crushing the Covenant and sharing information. Focus on the customer, not on the framework.

Can I tell you a story?

What does it mean to focus on the customer and the story? It means don't solve the general problem—solve the customer problem. The problem customers are trying to solve.

It means knowing the story line (the scenario) for customers. Who are they? What are they trying to accomplish? How are they used to accomplishing it? How might our software help? What would that look like to the customer?

Remember, many of our customers are developers. It's no different for them. Who are they? What are they trying to accomplish? How are developers used to accomplishing it? How might our software help? What would that look like to the developer using our platform and tools?

Once you are focused on the customer and the story, design, develop, test, and deploy that story for those customers—nothing more, nothing less. Make that story compelling and delightful.

Along the way, certain generalizations will emerge. Avoid them unless they are needed for the story. Only engineer what's needed to realize the story for the customer. Everything more general is wasted effort because by the time you need it the story will be different.

Temptations always come along

So you code up one story (scenario) and start working on the next. Perhaps it's even the next release. Aren't you going to wish you had generalized? No. No, you aren't. If you had generalized you would have done it wrong. You wouldn't have known what you know now.

Say while working on the next story, you realize you need to add a new weapon, landscape, or content viewer. No problem—refactor the design and add what you need now that you know what it is and why you need it. The result is better code, tighter code, and better tested code because the tests can focus on the story line.

Focusing on the customer and their story line isn't difficult conceptually or even in practice, but it does require restraint. It is so tempting to solve the big problem for humanity. Resist that temptation. You can't please everyone, so do your best to at least please your customers. In time, humanity will sing your praises.

Plumbing channels waste water into a series of larger and larger pipes till it is expelled. That's because sewage flows downstream, which explains the quality of goods that test, operations, and sustained engineering teams receive. After all, they are downstream of design and development.

I've written about pushing quality upstream for testers in "Feeling testy" (chapter 4), and making services resilient to failure for operations using the five Rs in Crash dummies. Like most engineers, I've neglected sustained engineering (SE), also known as the engineering sewage treatment center. No, on second thought, that analogy implies that what we release to customers has been cleansed. SE is more akin to environmental cleanup after an oil spill—thankless, difficult, and messy.

Imagine what must go through the minds of those cleanup crews as they wash oil from the feathers of sea birds. Naturally, there's empathy for the birds (customers). There's frustration at the inevitability of mistakes that lead to tragedy (buggy software). And there's a palpable desire to have the jackasses who caused the spill be forced to take your place (the engineers who let the bugs slip by).

You make the call

Should the engineers who design, construct, and test the code be the same engineers who fix the bugs found after release? This is the quintessential question of SE.

If the engineers who built the release fix the post-release bugs, you typically get better fixes, the engineers feel the pain of their mistakes, and the fixes can be integrated into the next release. Then again, the next release may not happen because its engineers are being randomized.

If you have a dedicated SE team you build up knowledge of the code base outside the core engineering team, you can potentially pay a vendor to sustain old releases, and you don't distract or jeopardize progress on new releases. Then again, SE teams get little love, their fixes can be misinformed, you duplicate effort, and the core engineering team isn't held accountable for their mistakes.

Tough call, huh? Nope, not at all. While both models can work, having the engineers who build the release also fix post-release bugs is far better. Only idiots believe a lack of accountability leads to long-term efficiency and high quality. Of course, the world is full of idiots, but I digress.

Someone's got to take responsibility

Yes, a dedicated SE team can work, but long term it will only cause grief for team members and customers. Why? Because you can mitigate post-release fixes distracting the core team, but you can't mitigate the problems with a dedicated SE team.

Let's go through those dedicated SE team problems again.

§  Little love. What would it take for the dedicated SE team to be appreciated as much as the core engineering team? A disaster, right? And what would it take on a day-to-day basis? Non-stop disasters. In other words, the conditions for loving the SE team are undesirable.

§  Misinformed fixes. To get a fix right, recognizing all the implications of changes, you need to deeply understand the impacted portion of the code base. Let's fantasize that the core engineering team has that level of depth. The core team is always considerably larger than the SE team. The SE team has no hope of truly appreciating the impact of fixes. Reality is only worse. Sure you can have the SE team consult with the core team, but doing that all the time defeats the purpose.

§  Duplicate effort. Whenever you have two teams fixing issues in the same code you duplicate effort, by construction. You've got two teams learning the same code, debugging the same code, changing the same code, and testing the same code. There's no getting around it, unless you neglect to incorporate the fixes into the next release, which is even worse.

§  Accountability for mistakes. The whole point of the dedicated SE team is to avoid derailing the core engineering team, protecting them from dealing with fixes. The core team doesn't correct their mistakes in the old code, and doesn't know to prevent those mistakes from recurring in the new code. What's worse is that there's no reinforcement of good and bad behavior. Conscientious heroes don't get to write more quality code, while careless villains fix past mistakes. Thus, we can never expect to improve. A great recipe for joyful competitors  and sorrowful customers.

What do I do now?

In contrast, there's plenty you can do to avoid jeopardizing future releases while the core engineering team fixes prior mistakes. Let's run through the relentless, randomizing requests and resolve them.

§  Triviality. How do you avoid wasting the core team's time with issues that aren't software bugs, or have trivial workarounds? You have a small dedicated team triage the issues. Note this team isn't a development team. It's purely an evaluation team that determines which issues are worth fixing. That way, only worthwhile work is passed onto the core team.

§  Prioritization. How do you balance bugs fixes for the last release with work on the new release? You have the dedicated evaluation team prioritize the fixes. There are four buckets: immediate fix (the rare "call the VP now" issue); urgent fix (next scheduled update); clear fix (next service pack or update); and don't fix. These buckets send clear signals to the core team about which bugs to fix at what time.

§  Unpredictability. How do you make inherently unpredictable post-release issues easy for the core team to schedule around? You make them regular events. Deploy one update per month. The urgent fixes each month are queued up by the evaluation team. The core team sets aside the necessary time each month and the fixes are designed, implemented, tested, and deployed on a predictable schedule. This is just as good for customers as it is for the core engineering team. Everyone likes predictability.

In addition, the evaluation team can create virtual images for easy debugging by the core team, improve the update experience for customers, and reflect customer needs and long-term sustainability features back into future releases.

This won't hurt a bit

See, it's not that complicated. You save on staff. You get better fixes. You catch similar issues in advance. You achieve predictability. And you ensure the core engineering team is accountable for quality and learns from its mistakes. All it costs is a relatively tiny dedicated team to manage the monthly update process by evaluating and prioritizing issues. Even that team feels valued due to their differentiated and important role and their direct engagement with solving customer problems.

Yes, sewage flows downstream and no one likes cleaning it up. However, by putting some simple processes in place, you can reduce the sewage and have those responsible mop up the mess. To me that smells like justice.

Why? Why! Why do managers make stupid decisions that cause devastating churn and tawdry results? And it's not just managers, though they are particularly proficient at promoting poor performance—architects, leads, and individual contributors flood the lives of their team with wasteful, useless, misdirected activity, leaving us even less opportunity to deliver real value. What reason is there for this farce? Simple. We are optimizing—optimizing our obsolescence.

What kind of idiot optimizes their own undoing? The ordinary kind. You do it, your friends do it, and your boss does it. It's all those good intentions that pave the way to disaster. We optimize the wrong behavior to achieve the wrong results. It's wrong and avoidable, but hey, why think when you can cause mayhem with so little effort?

You want answers?

Let me save you some trouble and reading by giving you the answer first—optimize for desired results. It sounds simple and obvious, but people pervert that goal in so many imaginative ways that I better break it down word for word.

§  Optimize—Measure how good you are now, analyze how you could be better, alter your approach, and then measure again. Microsoft is great at optimizing, but we measure what's handy rather than what matters. So, we optimize for the wrong result. You can read more about this in my column, How do you measure yourself?

§  For—Have a purpose. Optimizing for the sake of optimizing is purely self-gratification—don't do it in public. Instead, be deliberate about your purpose. Think it through. Know what you are doing. Be a professional. Wake up.

§  Desired—Focus on what you want, not what you don't. This is a common trap. People optimize around the problem instead of the solution. Bureaucracies and slow software are built upon this misdirection. They focus on controlling people or code to prevent the wrong behavior. Their focus should be on making the right behavior fast and easy, and then catching the exceptions.

§  Results—Never optimize a step or algorithm in isolation. Instead, optimize the end result you seek. We have all experienced the impact of local versus global optimization. It kills our efficiency and innovation; it's killing our planet. Yet over and over again, people can't see beyond the problem at hand to consider the outcome they're truly after.

Can you recognize when you are de-optimized? Let's run through some examples and check.

I think I can handle this

How do you handle run-time errors in your code? How fast does your code run when no errors arise? Is it a smooth or bumpy ride for error-free operation? The fastest, simplest path through your code should be the 80% case, not the 20% case. However, that doesn't mean you shortchange error handling; you just don't optimize around it. Trust your code will run error-free, making it run fast. Verify it was error-free, ensuring the right result. Trust but verify.

Likewise, how do you handle people and process errors? Do you check their every move? Do you have people do it your way, jump through hoops, and fill-out redundant forms to ensure they aren't cheating? Or do you trust people to do the right thing—clearing the desired path of obstructions, and later verify that work was done properly? Trust but verify.

My altruistic readers, including managers, might claim that they do trust their coworkers. Really? How did you react the last time something went wrong? Did you quickly fix the root cause and move on, or did you start an inquisition randomizing your team for days or weeks? Do you micromanage or do you delegate? Do you specify every step or do you specify the result? Trust is hard. Luckily, you're being paid.

Déjà vu

How decoupled and cohesive is your code? Are the classes, functions, and functionality all intertwined and unmanageable, or are they independently testable and separable, each piece having its own purpose and task to perform?

Well-architected and layered code is far easier to test, maintain, and enhance. However, it doesn't perform quite as well as tightly coupled code. It's a tradeoff. If you optimize purely for speed, you eventually get un-maintainable spaghetti code. If you optimize purely for architecture, you can't be competitive in performance. How do you strike the right balance?

Most teams don't strike a balance between architecture and performance—they ride a rollercoaster:

1.      The team starts with a nice architecture. It works great and everyone feels good.

2.      They optimize it for performance. Now it works better—the original team clearly wasn't as sophisticated.

3.      The code is unmanageable, it can't be enhanced, and performance has hit boundaries, so the team painfully refactors the code. Now it's manageable again and everyone is happy—the prior team clearly were neophytes.

4.      The performance isn't competitive, so the team optimizes again for performance. Now it's competitive again—the prior team clearly had lost its way.

5.      Now the code is unmanageable, so return to step 3.

There's another variation—the code is so twisted that team can't fathom refactoring. Their product cycle keeps getting longer and the code keeps getting slower, requiring more memory and processing speed. That's a popular variation.

The right approach is to optimize for desired results—performant code that's easy to maintain and enhance. Instead of just measuring the speed (easy), you measure the speed and the code complexity. You seek the optimal balance of both. If you're really sophisticated, you'll also measure team health and customer satisfaction indicators, seeking a balance of all four. Wow, that's almost like running a business.

The beat of a different drummer

Let's try one more subtle example—product team structure. It's a war zone out there between traditional product development and the upstart Agile adherents. Who's right? Who cares! Never optimize around a step in isolation—optimize for desired results.

The desired result is delivering the most customer value in the shortest time. Remember, customer value is not measured by feature count; it's measured by delivering delightful end-to-end scenarios with high quality.

So how do you deliver high value quickly? You apply the Theory of Constraints (TOC). TOC says that the fastest way a project can accomplish anything is constrained by the slowest step. Say your user experience, content publishing, and operations teams are shared and can scale to your needs; your PM team can spec an average of four features in a month; your development team can code two features in a month; and your test team can validate three features in a month. There's not much point in your PM team going full speed, is there?

Yet managers will push the PM team to keep writing specs the dev team can't process—optimizing locally instead of globally. Adding people to speed up the dev team doesn't work either (note The Mythical Man-Month and the economy)—again, the focus is too narrow.

The right solution is to pace the PM and test teams to the dev team. Put in buffers to account for variability between features, but never have the PM and test teams outpace the dev team. This TOC strategy is called Drum-Buffer-Rope. Because it's hard to precisely predict the dev team's pace, you constrain the size of buffers, avoiding too much work in the dev team's queue should the situation change.

This is why Feature Crews work so well. You're optimizing for the desired result—working scenarios. In Feature Crews, an approach from Office, PM, dev, and test team members tie themselves to one piece of a scenario at a time till it's completely tested and integrated. They can't get ahead of each other. Versions of Scrum and Extreme Programming work the same way. It's not the combined teams that are essential (though communication is easier); it's the pacing of work together that optimizes the delivery of complete, high-quality customer value.

Don't panic

It's so easy to get caught up in the immediate and optimize around the issues directly in front of your face, instead of the ones you actually care about. People do it all the time—I guess we're programmed instinctively that way. That's a perfectly good reason to optimize the wrong behavior for the wrong results, but it’s a poor excuse.

You should know better, and if you don't, you have no right to draw a paycheck. Consider the result you desire to achieve; think it through; measure a balance of factors; and optimize as a whole. It's not that difficult. We attempt it every day as we balance our lives. The key is to be deliberate rather than juggle; to be planned rather than panicked. You can do it if you simply keep your sights on the finish line.

Is innovation the act of creating something new (as the dictionary claims) or is it building upon the work of others? To me this is a fundamental question that Microsoft as a company and as a culture has gotten horribly wrong. We deal with the consequences every day. It shakes our self-esteem and cripples our ability to innovate.

The right answer is that innovation is enhancing the work of others—nothing is new. Consider what people claim to be innovative—the iPhone, hybrid cars, and Facebook. Are those new? Are you kidding? Not only would those advances have been impossible without the foundational technologies behind them, but almost every aspect of those innovations had been created earlier in some form. The innovation was putting old technologies together into compelling new experiences and successfully marketing them to a broad audience.

This isn't an indictment of innovation or great advances like Facebook. Quite the opposite—this is a call to acknowledge that innovation is enhancing the work of others. Microsoft culture, in an act of misguided self-mutilation, has connected being innovative with creating something "new" on your own. This has directly led to three crippling consequences:

§  We doubt our innovative spirit as a company.

§  We create disconnected new features instead of compelling new experiences.

§  We reject work we didn't create ourselves as a company, team, or individual, which impairs our productivity, our intelligence, and, ironically, our ability to innovate (an advanced tragic case of Not Invented Here, also known as NIH).

Why the long face?

We doubt our innovative spirit as a company because much of our work goes into enhancing existing products and services, which in turn were built upon ideas from a wide variety of sources. Microsoft haters out there claim this proves Microsoft isn't innovative, and we accept that guilt. It sickens me.

Is Apple not innovative because it “borrowed” from Xerox PARC (Palo Alto Research Center)? Is Google not innovative because it enhanced Yahoo? Is Toyota not innovative because it built upon Henry Ford's ideas of the automobile and lean manufacturing? That's both stupid and insulting.

Being innovative is all about enhancing the work of others. We enhance our own great work and build upon the great ideas we find around the world in all fields. Yet our self-esteem is shaken, even while every new release is filled with innovation.

Perhaps the problem is that our innovation isn't as compelling as the iPhone. Why?

Compelling, and rich

We are so eager to promote our own individual new, small ideas that we fail to create compelling new experiences. You can't market a plethora of disconnected features to a broad audience, so our individual innovations are left unnoticed.

That's beginning to change as groups realize that customer value comes from designing compelling end-to-end experiences. More and more Microsoft products and services have a consistent feel and a complete story. It started with Hardware and Office, crystallized in Xbox 360, and is making its way into Windows—Windows Vista, Windows Server, Windows Live, and Windows Mobile. We don't always get it all right, nor do any of our competitors, but each new release shows progress.

Tragically, our company's culture of individual innovation works against creating compelling end-to-end experiences. You can't assemble seamless scenarios without selflessly supporting the salient story. You have less unbridled innovation from the individual engineer, but the result is greater innovation from the customer's perspective.

As any artist knows, constraints often lead to tremendous creativity. However, individual engineers must first let go of their own preconceptions and embrace the constraints of the larger composition before they can innovate in concert like a symphony instead of a soloist.

Our greatest successes both monetarily and in emotional connection came from working together over several years to get the experience right for customers. Some of the ideas may have come overnight, but as Thomas Edison said, "Genius is 1 percent inspiration and 99 percent perspiration." Those innovative engineers from groups like Office, Developer Tools, and SQL Server, with the patience to iterate, focus on the customer, and bring full experiences together, are the ones who receive the greatest rewards. Yet Microsoft engineers still cling to the notion that working independently from nothing is the secret to success.

I'll do it myself!

Unfortunately, Microsoft internal mythos is all about the individual hero rewriting the kernel memory protection scheme over a sleepless weekend. We value independence, passion, and boldness over compliance and conformity. I love that about Microsoft, but it needs an update.

The problem is that independence, passion, and boldness somehow became "do it yourself." If you didn't invent it, it's not what you need. If your team didn't invent it, it will only cause trouble. If Microsoft didn't invent it, it's dubious at best. That NIH worldview is not only wrong and shortsighted, but it slows our success and paralyzes our progress.

You can be independent, passionate, and bold while responsibly meeting requirements and enhancing a desired customer experience. You can be independent, passionate, and bold from the advantageous platform of others' foundational work.

Conversely, if you choose to go it alone you suffer greatly:

§  Your productivity drops.

§  Your intelligence drops.

§  Your innovation drops.

Maybe I could turn this thing into my advantage

When you reject work that was NIH, you lose the advantage of time, effort, bug-fixes, and knowledge that went into that work. Would you write an operating system from scratch to optimize a small application? Of course not. By using an existing operating system you save incredible effort.

The same is true of using existing libraries, services, tools, and methods. The key to gaining advantage instead of hardship from using the work of others is ensuring the things you depend upon are stable. Take a dependency on version n-1, be careful about using the latest and greatest. That goes for libraries, tools, and even techniques. Grief is avoidable.

I'm good enough, I'm smart enough

Rejecting work NIH also leads to brain decay. At Microsoft, we hire the brightest engineers available. Are they still the brightest five years later? Well, that depends. Did they learn something new every year, or did they rebuild the same thing over and over again?

If you have to reinvent a build system every time you switch teams, are you smarter? If you have to reinvent a collection class, a threading framework, or a test harness every time you switch teams, are you smarter? If you have to reinvent project management techniques, spec templates, and bug field definitions every time you switch teams, are you smarter?

No, you’re not smarter. You are getting dumber. Congratulations. You don't get smarter by repeating yourself or by repeating others and their mistakes. You get smarter by building upon the knowledge of those before you.

Those who do not learn from history

Ignoring what others have learned and gained before you, both good and bad, puts you at an innovation disadvantage. As I already mentioned, there's the productivity disadvantage—going it alone is never as fast as building upon the progress of others. However, there's also a significant contextual disadvantage.

If you ignore past work, you're liable to make the same mistakes that people made in the past—avoidable mistakes that cost you time. Sure, you might want to revisit those challenges. Perhaps there's an opportunity for a breakthrough. But do so knowingly, not like a dolt.

If you ignore past work you're also liable to unknowingly duplicate what others have already achieved. You'll feel smart and proud right up to the point that you look like an idiot. Meanwhile, you've lost time and accomplished nothing new. NIH results in nihilism. Fitting, I suppose.

If not me, who? And if not now, when?

Microsoft is filled with smart, passionate, and innovative people. We are an innovative company, in spite of ourselves at times. We need to stop thinking of innovation as an individual effort that appears miraculously from the void, and start thinking of innovation as it really is—a culmination of effort focused on thrilling the customer.

Where do your individual ideas fit? They fit within the context of the larger desired customer experience, and build upon the learning we've gained as a group. The result is ground-breaking innovation that expands the state-of-art and excites the hearts and minds of our customers.

So when you're going to work on a new team, project, or great new idea, understand how it makes the customer experience we've set out more compelling; and learn what others have discovered about your work. Note that I said "what others have discovered" not "if others have discovered." There is nothing new. Someone out there has tried something similar and will have work or knowledge you can use. You can ignore that and be slower, dumber, and less accomplished. Or you can take it to the next level and create real innovation.

People are always looking for that amazing breakthrough technology or process that solves all their problems—enhances their love life, trims their waist, and improves the productivity of their development team. That's why process manias like Agile and Six Sigma are so enticing. Just splat the Scrum tag on your development team and "bam!"—your team is suddenly ten times as prolific.

It would be hilarious if it weren't true in the minds of the narcissistic ninnies with an adiction to magic methodologies. I'm happy that people are willing to try something new. I just wish they wouldn't confuse new with necessary, or minor fixes with major problems.

The truth is that there's nothing magic about producing great software with high quality on time. (And no, software services aren't different in this regard.) Human beings have had large teams building complex engineering marvels for thousands of years. There is no magic. There is solid design and disciplined execution. Period.

Back to basics

I worked at Boeing for five years before coming to Microsoft. The areospace industry uses the terms nominals and tolerances to refer to acceptable values and tight refinements. A nominal might be a support beam being placed two feet from a heating duct. A tolerance would be that location give or take 1.5 inches.

There were always inexperienced engineers who concerned themselves with tolerances. They were constantly looking for a clever technique to give them an extra fraction of an inch of tolerance, as if that mattered. Real engineers who designed and built the airplanes knew these tolerance tinkerers were misguided, out-of-touch, naïve fools who missed the point entirely.

The key to building an airplane isn't tinkering the tolerances—it's nailing the nominals. It's not, "Does the steel stringer bend within 0.058 inches of the specified skin?" It's, "Does the steel stringer pass right through a cooling vent?" Stop worrying about the fancy details and get the basics right, ding-a-ling!

I want to be a cowboy

The software industry suffers greatly from dingbats tinkering tolerances instead of nailing nominals. You can see it in most software, regardless of how or where it ships. The focus of the engineering was clearly on all kinds of fancy little features instead of getting the software to work right for basic cases.

Maybe the basic cases don't appeal to engineers. Maybe engineers can only relate to tinkerers, not to folks who want solutions that are simple and reliable. However, I don't think so.

I think nailing the nominals requires solid design and disciplined execution, both of which require software engineers to put the project and the customer ahead of their personal interests. Unfortunately, that's not what cowboy coders crave, which is why cowboy coders should either grow up or move out.

But it's so simple

Still here? Still believe the customer and project should come first? Still realize that true customer value only comes when you act as a team, putting the customer at the center instead of yourselves? Good, let's talk about the nominals of software.

As I said, the nominals revolve around solid design and disciplined execution. But that's true for any significant engineering venture. What does it mean for software? I'll keep it simple.

Solid design means:

§  Understanding what the customer is trying to accomplish with your software (product planning, value proposition/vision).

§  Thinking through the end-to-end experience, including pitfalls (experience design, scenarios).

§  Decomposing that experience into distinct engineerable pieces (engineering design, architecture).

Disciplined execution means:

§  Ordering work based on priority and dependency.

§  Creating a schedule based on data from past projects.

§  Establishing and upholding completion criteria for every piece.

In the simplest of terms, the nominals are "think before you act" and "define done."

Is it done?

I've written about experience and engineering design many times before (particularly, in "The other side of quality (chapter 6)"), so this time let's drill down on defining done. For a distinct engineerable piece of the architecture, be that a feature, component, API, or Web service, what should the completion criteria be?

You could set bug limits or test code coverage, but those evaluate intermediate results. What do you care about at the end? What should the final result of an engineerable piece of the architecture be? Well, it should fully satisfy its part of the end-to-end scenario, including performance targets and avoiding pitfalls, and it should be secure and reliable.

What causes software to fail scenarios and reliablity targets? Smart dedicated people, at Microsoft and around the globe, have been studying this problem for decades. The net result, which you can read in every study ever done, is that problems stem from failing to complete only five practices—five practices that should define done:

§  Documented design. Think before you act and capture it so that peers can do a …

§  Design review, which catches up to half the errors before you write a line of code which you then …

§  Code review, that, if you use inspections, can catch more than 70% of errors, though some errors will only be caught by …

§  Code analysis, like lint or PREfast that catch patterns humans can miss, but doesn't entirely replace …

§  Unit and component testing, which catches the remaining issues, including stress, performance, and edge cases, as well as end-to-end scenario checks.

Without defining done, tasks never finish (sound familiar?). Even worse, cowboy coders can claim to be finished with no accountability for quality work. Ever wonder why great developers end up cleaning up after poor developers? It's because poor developers claim completion for crud. Without defining done, who else can keep their crappy code from customers?

It's not that complicated

Notice how none of the five practices that define done are terribly exotic or revolutionary? They are basic. Everyone knows you should do them. Yet engineers will fall over themselves to become certified Scrum Masters, create cool new tools, or learn the latest technology before they'll actually run design reviews or write comprehensive unit and component tests.

Nail the nominals before tinkering the tolerances. It's not magic. It's not complicated. It's not hard. It takes discipline. It takes putting the customer and the project before yourself. It takes being a professional instead of a chump. Think you can handle it?

When I'm discussing challenges with fellow engineers, the first topic that comes up isn't estimation—it's career and people challenges. That's why those issues are so rampant in these rants. However, "How do you generate task estimates?" is always among the top non-moaning-about-your-manager-or-mates topics. After all, estimation is predicting the future. There are so many unknowns and unforeseen issues that it's impossible to provide the accurate estimates demented despots demand. Isn't it? It must be. Right?

Wrong. Estimation is among the most trivial tasks an engineer has to perform on a regular basis. Get over yourself, it is. It's so easy that there are dozens of seemingly different methods that all give you remarkably accurate predictions of completion time. All those methods come down to one simple concept—how long it took last time is how long it will take this time. Nothing could be easier.

Yeah, you've got to understand the work well enough to compare it to previous work, but that isn't too tough either. No, the real challenge isn't task estimation; the real challenge is accepting the estimate. Estimation is easy; acceptance is hard.

No one would accept the program

Let's pretend for a moment that you actually keep track of how long it takes you in calendar days to perform various tasks. (You do—the information is right there in your e-mail dates.) Let's further imagine that you provided those previous times as estimates for doing similar tasks today (you'd be quite accurate). What would the reaction be from your project leads and managers? My guess: "Oh come on, you've got to be kidding me!"

This is fun, so let's take it a step further. Let's say you told your project leads and managers that your estimates were based on hard dates collected from your previous project. What reasons would they give for not believing this hard data? Here's the big three:

§  Last time was different.

§  You get faster the second time.

§  Weird stuff happened last time.

Let's break down these feeble fallacies one at time.

It's a different kind of flying altogether

The first excuse your manager or project lead will have to reject your hard schedule data from the previous project is that the previous project was different. Things have changed. Perhaps the build system and tools have changed, the design change request process changed, the requirements changed, management changed, or perhaps the moon is in a different position relative to Saturn this time.

Out of all those excuses, only two have a small chance of affecting your estimates—the tool and process changes. Every other factor is superfluous with little or no impact to cycle time.

Even the tool and process changes would have to be extreme to noticeably affect the accuracy of your estimates. Tool changes would have to cut end-to-end build times by a factor of 5. Process changes would have to reduce the time of weekly activities by days. Otherwise, the impact is just noise in the estimate.

Look, the more the world changes, the more it stays the same. Deal with it.

I'm getting better

The second excuse to reject your hard schedule data from the previous project is that you get better the second time around. The funny thing is that you do get better the second time around. The problem is that you're not doing the same project (hopefully). The only things that are the same are the tools, process, project scope, and the general task of software engineering.

You should already be well versed in the general task of software engineering, so getting better at the details of the previous project has no impact on the estimates for the next project. Of course, if you're fresh out of school then the second project will take less time than the first.

If you did change tools and processes, your performance should actually be worse because it will be your first time using them. That's okay if the changes are small or the benefits are big. Just don't kid yourself about the impact.

You do want to compare the current project to a prior project with similar scope. The better the match, the more accurate the estimate. The big differences between estimation techniques are how they produce matches.

Oh no, not again

The final excuses your manager or project lead will have to reject your hard schedule data from the previous project are all the "weird" things that happened last time. There was that unexpected security patch, the feature that was far more complex than anticipated, the reorganization and associated project reset, not to mention the snowstorm, and that earthquake, yeah, the earthquake. There's no way you should count the earthquake!

You count the frigging earthquake. There's always a surprise patch, feature, reorganization, and natural disaster waiting for you over the course of a project. Always. Random events happen, but their impact on the schedule isn't as unpredictable. Thanks to Lyapunov's central limit theorem, their overall impact averages out. However long it took last time is likely to be nearly the same this time. That is, as long as you don't pretend this time will be different.

Same old wine

Okay, we've proven your project leads and managers are in denial. As a result, they force you to make ridiculous estimates you don't believe, only to blame you later for missing them. We've shown that accurate estimates are almost trivial to make. The big question remains, "How can you turn your trivial and accurate estimates into ones your project leads and managers will believe?"

That's where task hours are so handy. Instead of making your estimates in calendar days, you make them in task hours—the number of hours it would take if there were no earthquakes, e-mail, or bathroom breaks. Without those distractions, your estimates look far smaller and more reasonable, even though they're no different.

Task hour estimates are slightly harder to make because you don't have the data lying around in your inbox. However, you can estimate task hours quickly, easily, and accurately with a simple technique like planning poker (or it's more accurate and sophisticated sibling, Wideband Delphi).

In planning poker, three or more engineers each estimate the same task privately around a table. They all reveal their estimates simultaneously so no one exerts undue influence. If the estimates match, you're done. If they differ, the high and low estimators explain themselves, the group discusses their thinking, and then the process repeats until the estimates agree. The process also surfaces assumptions before they become a problem.

Once you have believable estimates in task hours, the argument isn't about how long the tasks will take. It's about how many hours you spend on-task in a week. Even after subtracting vacation, training, and big group meetings, most teams spend less than half of working hours on task. The rest of their time is in meetings, answering e-mail, lunch breaks, and so on. If your project leads and managers don't believe it, simply have the team spend two weeks tracking their hours. The numbers don't lie.

Your results may vary

As I mentioned earlier, there's a certain amount of randomness or "variance" that asserts itself over the course of a project. It averages out, but any one estimate has a chance of being off by some standard deviation. That deviation is a percentage; it scales with the size of the estimate. Thus, a two-day estimate will be accurate give or take a few hours, a two-week estimate might be off by a couple of days (in either direction), and a three-month estimate could be off by a couple of weeks.

As long as you avoid being overly optimistic, the randomness will even out. By the end of the project, your project deviation will be about the same as the deviation from any individual task. If you are overly optimistic ("It can't take this long next time!") your deviations will keep adding up, not averaging out.

The point is that there's little point in estimating a two-day task to the minute or a three-month task to the day. You just need order-of-magnitude estimates, like I talked about in my very first column seven year ago, "Dev schedules, flying pigs, and other fantasies" (chapter 1).

I want to believe

How would you estimate? Focus with your peers on understanding the tasks at hand and their order of magnitude using a technique like planning poker or Wideband Delphi (poker for well-understood tasks, Delphi for others). That's the easy part.

What truly matters in the end is believing and accepting your estimates, then scheduling accordingly. As I've written about before, over-committing is foolish. What's worse is that over-commitment can lead to "Marching to death" (chapter 1). As I said then, death marches are a strong indicator of weak, cowardly, deceitful, and irresponsible management.

Scheduling trouble is diabolical, but completely avoidable. When you prioritize your work properly, putting what's first first, you reduce pressure on your schedule. When you use your estimates to drive realistic commitments, you can deliver reliably to your customers and partners, build trust, and enhance your group's and our company's reputation. Deriving good estimates is easy. Trusting them and yourself is the challenge.