Stephen Cohen's Thoughts on Agile and Project Recovery

It's just software ...

It's just software ... really, it's just software.  It's not as important as your family, or your health.  It (generally) doesn't kill people or change lives.  That kind of thing is left to real people.  I will concede that those real people may be using software to help them kill people or change lives but the software alone is neither good nor evil.  It does no harm, and in the specific case of software under construction, it can wait a day, a week, or a month. 

In this era of fast small deliverables,  short timelines, and rapidly changing goals it's easy to forget it's just software.  As an industry we are facing decreased demand due to macro-economic pressures, and increasing expectations as our users first become accustomed to, then jaded by, the latest eye candy and features all of which tend to drive us to work as though our lives depended on working.  Today’s reality is quite the contrary.  Our lives may depend on our not working.

We are nearing epidemic H1N1 virus levels.  Entire organizations will close as the percentage of sick employees climbs upward.  So whether you are home with a sick child or just feeling ill yourself, do everyone a favor.  Stay home and just keep telling yourself ... It's just software.

Agile versus Formal modeling

In response to a recent discussion over at Yahoo on the requirements Engineering list, a question was posed as to why “just enough” was ever okay when creating a model.  After all incomplete models or improper use of a modeling standard has to be bad… right?  I believe there is another perspective on modeling to consider.  That is, WHY a model is being created.  I see 3 reasons for modeling;

• Modeling to communicate,
• Modeling to understand, and
• Modeling for reference.

In reverse order, Modeling for reference is the traditional goal.  It is intended to read by someone (usually at some future point in time) new to, or somehow removed from the original development effort.  The model needs to neither mislead nor have important gaps and must comply with the standards for that model type.  After all the standards allow a reader to understand the model without the original context.

Modeling to understand is a more requirements gathering effort.  The creator is the likely consumer and it is intended to be an aid, not a deliverable.  Simple and well understood things need not be included instead the models tend toward fragments being analyzed instead of depictions of the whole.  Think of them as quickly taken notes and relevant doodles done in your own handwriting using symbols likely to be meaningful only to yourself.  As such models for understanding can be, but are not required to be good historical notes about how or why a decision was made.  They are not intended to be good tools for later reference.

Last, modeling to communicate is done to share ideas or restate an understanding for validation.  They are semi-structured whiteboard-like drawings which, once done communicating their point go quickly out-of-date.  They tend to use symbols understood by the parties involved which may vary wildly from the official standards of the model type.  It is far less important to be right than to be understood.  However, should the model persist too long, or should someone from outside of the original discussion get their hands on it, clarity becomes the first causality.

A risk-reward scale for evaluating project practices

So you’re seeking to improve things. You tried to implement a “best practice” or twelve … how did that work for you? I am convinced that problems addressed by a well known practice will not always result in a well known and positive result, and it seems I am not alone.  In a survey of software development professionals conducted last year by William Money and me, 96% agreed or strongly agreed that there are different objective effects of various practices under varying situations.   (Complete survey results can be seen at http://www.projectpracticeportfolio.com/ )

In practice this means the actual value of a best practice isn’t the advertised return but a possible return from within a range of returns. If we take the example from my last post; adding a new tool to address testing quality and velocity.   It is easy to see that when a practice is done well it can provide significant improvement in overall product quality.  On a reward scale of 1 to 10, I’ll call that an 8.  When it is done poorly, it could increases cost or inject delays into the schedule.  On the risk scale, this time from -1 to –10, I put that at –6.   Now we have a range of potential that is 14 points wide going from negative 6 to positive 8. The actual value is highly dependent on the specific implementation within the context of the team and the mixed effect the combination of practices being put in place.

Before we get into the weeds around how we can use these values I want to propose a simple, consistent way to describe the points along the range.

If any of you know me, you know I'm not actually a fan of standards.  In fact I have been known to violently oppose the idea of compliance where integration will do.   I will however concede to the point that nearly every attempt at integration requires agreed upon terms if for no other reason than to limit semantic arguments.  Here then, is my proposal for a complete Risk / Reward scale for evaluating practices;

RISKS
----------------------------------
0 no impact
-1 negligible impact, easily resolved
-2 likely to create additional tasks
-3 scope of issue expands
-4 degrades team morale / communication
-5 requires notable response
-6 increases cost or delays schedule
-7 injects current and future issues
-8 degrades solution quality
-9 critical feature failure
-10 prevents delivery

REWARD
----------------------------------
0 no impact
1 improve poor performance to nominal
2 reduces isolated level of effort
3 broadly improves things
4 improves team morale / communication
5 measurable improvement
6 reduces cost or schedule
7 resolves current and future issues
8 significant improvement in overall quality
9 ensures feature delivery
10 ensures total delivery

In the next post I'll show how we can group practices and by using this scale remove much (but not all) of the guess work from selecting, and monitoring, corrective practices.

There are no right answers.

All troubled projects share one and only one characteristic.  Things have gone wrong.  Terribly wrong.  Other than that, everything is, and I contend must be, unique.  Let's face facts here; projects are the accumulated work of people.  No amount of consistent process can ensure individuals with unique personalities, under unique conditions, will do the same things, the same way nor will they have the same results.

Consider the common practice of buying new tools to improve the team’s velocity thru Automation.  New tools are frequently added to a project to automate repetitive, labor intense tasks like regression testing.  Some testing tools can reduce days of potentially error prone human testing to hours of precise, repeatable tests.  Seems simple enough. Tool based automated testing is good and will improve both product quality and team velocity.  Unfortunately this is not always true.

Troubled projects seem to have a very high likelihood of poor test practices. While many may actually be suffering from a complete lack of test all together, all too often they are mired in the "solution" tool based automation has become.  These teams didn't have experience with test before bringing a tool to the party.  The tools positive value is overcome by the negative costs and risks of un-skilled or under-skilled staff.

What seemed an obviously right answer to the testing problem is now a problem of its own. Faced with diminishing returns, the troubled project might;

A- Train the current staff on the tool
B- Hire staff already skilled with the tool
C- Abandon the tool altogether

Good leaders intuit these options and generally acknowledge their related risks;

A- Retains the current staffs’ subject matter knowledge but will cost the team some extended period of reduced velocity,
B- Will remove the tool based impacts on quality and schedule but will both slow overall velocity as the new hires become familiar with the subject matter.  There are potentially negative impacts to the team.
C- is the head-in-the-sand approach . . . and no good can come from that unless it is followed by applying another practice to address the original issue.

None of these are simply the right answer.  The unique combination of people, problem, and constraints creates an environment where we need to rethink every assumption.  More importantly, relying on good instincts to guide problem resolution moves us further from the predictability that process and practice based methodologies should provide us.   

In my next post I will propose a scale for defining the range of practice effectiveness.

What’s your projects risk tolerance?

Risk tolerance is defined on investopedia as the degree of uncertainty that an investor can handle in regard to a negative change in the value of his or her portfolio. Sounds pretty good to me for project portfolios too.  In our case, negative change includes concrete things like expenditures that fail to return expected value and more abstract things like automation in the development process that fails to create consistent, reoccurring increases in project velocity.  Some negative changes are more subtle but at least as damaging; staff failing to demonstrate expected skills, significant ambiguity in scope, and external dependencies that fail to meet their promised level of performance, just to name a few.

More clinically, risk can be defined as is standard deviation, a statistical measure of dispersion around a central tendency.  The the central tendency equates to what we generally refer to as the Planned Value portion of the Earned Value calculation.  The dispersion is represented by cost and schedule variance across a given period of time.  In our case that would be the entire duration of activities from good idea to delivered.

To do this, all you need is;
1- identify the type of project you have and
2- determine the historical trend for similar development efforts. 

Of course, this is incredibly hard given the industries terrible record keeping, our complete lack of broadly agreed on taxonomy (what is a large complex project really?), and the project stories that do exist out there are largely optimistic fables with limited basis in fact .  Even when we have reliable records our methodologies and practices have evolved so quickly that it may be impossible to find the necessary trend information to answer the question with any statistic validity.

This leaves you with two less-than-optimal but viable methods to establish a baseline.  The first, takes the total time available to you and divides it by the number of features you believe you need to produce.  We will call this "pure linear optimism".  It assumes everything works, works well, and works the first time.  The second option is look deep into your Magic 8-ball and layout a plan that reflects your personal experience, knowledge of the team, the tools, and the customer.  This has long been referred to as a scientific-wild-ass-guess or SWAG.  In the absence of real historic data both the pure linear and SWAG models are equally likely to be correct.

With your recently determined planned value baseline in hand lets figure out your risk tolerance.    Our goal is to decide, for either the entirety or at least what remains of the project timeline, where on the following scale does the project fall;

It seems simple enough but it’s not.  The more pressure you feel to improve and deliver faster, the more attractive high risk, high reward practices seem.  Alas too many high risk decisions actually increases the likelihood of incurring one or more of the associated high losses.  Conversely. it seems obvious that staying with as many low risk practices as possible should reduce overall risk and increase the likelihood of success.  But don’t forget that too many low risk choices actually increase the risk of too little return.

While there is no single model to determine risk tolerance every project here are 4 important things to factor into your risk tolerance identification process;

• Experience with the team, the tools, the technology, and the problem space must be significant factors in determining just how much risk you can take.  If you are short on experience with two or more of these areas you already have created a high risk project and should seek predominately low and medium risk remedies.
• Must meet goals such as quick release to production, more features than ever tried before, or a solution that is significantly more performant than has been achieved in the past, create constraints that limit where and when you can choose higher and in many cases lower risks.
• Your Risk capital reserves - what you can afford to lose and still deliver – in time and budget.  The more you have in reserves the more open to take risks you be.  The more Risk Capital on hand the more you can gamble on high risks yielding high returns … until a high risk gamble fails and your reserve is consumed.
• Risk pressures in the form of immovable and dangerous constraints can drive you to accept risk without alternatives.  Being the first to scale a technology beyond the vendors stated limits, or inheriting a team without the required skills may induce project performance issues that paradoxically can’t be addressed without increasing the risk further.

There is no ‘right’ answer to what your risk tolerance should be.  Without exception, your tolerance is what it needs to be.  The only question is will you determine the level of risk your project can tolerate or will you be surprised by it sometime in the future.

Introducing Project Practice Portfolios

In a recent paper, Bridge Methods: Using a Balanced Project Practice Portfolio to Integrate Agile and Formal Process Methodologies, my co-author and I dug into how nearly every methodology, be it formal or agile, allows if not requires customization to the set practices used throughout the software development lifecycle (SDLC).  We call the collection of practices a portfolio of project practices much like a financial portfolios.  Like financial instruments, you are investing in each practice with hopes of a positive return.  Additionally, each practice carries with it some amount of risk and past performance does not guarantee future performance.

More interesting to me is that, like investment portfolios, through the careful combination of practices you can both increase return and reduce project risk.  In the world of finance this is well documented as the efficient frontier.   In it’s simplest terms; carefully adding the right amount of high risk, high return stocks to typically low risk, low return bonds creates a “balanced” portfolio of investments.

Applying the same principles to a portfolio of project practices

we can add carefully selected high risk, high return practices such as organizational change to historically low risk, low return practices such as training to create a balanced portfolio of project practices. 

This is a critical element of project recovery.  When an underperforming practice is causing the project to falter we can use an off-setting practice to address the problem.  It’s just that simple … well, ok.  It’s just the underlying concept of a more complicated process.  But I will go into the application of this as it applies to your projects level of acceptable risk, using classes of practices to select off-setting practices and just how we work with the vast web of possible practices in my next few blogs.

3 Problems that prevent delivery

Here are a few random problems that prevents delivery of a solution to users that they can use.  Remember, if it fails to function as intended, fails to function at all, or fails to return a reliably correct result it’s not done.  You have problems.

1. Fixing the wrong problem.  An organization that has been responsible for the original creation and maintenance of a legacy system is so certain it understands the the needs of the users that it fails to engage the users and actual subject matter experts when creating the next generation.

2. Making the difficult even more difficult.  Consider an ancient system of decision making where each person was presented a list of items to choose from on a piece of paper.  Each person would then put a mark on next to their choice.  The papers were collected, marks counted and the the choice with the most marks was selected.  Fast forward to today's electronic voting systems.  The user interfaces changes year to year.  The system is put through a double blind audit, individual choices are displayed, printed, and stored.  Each to be verified by the voter, the system, and the auditors.  Real time numbers are held back to ensure completeness and every election seems to elicit cries of foul and tampering… that’s better isn’t it?  These digital systems are being systematically dumped for the old school paper systems.

3. Too much information. Banking is, by most accounts, an industry concerned with security.  It worries about accuracy and knows that integrity of the system is second only to the need to service the depositors.  While trying to decide just how to to present a banking customer their account balances a committee was formed to ensure every part or the organization was represented.  They created a “tiger team” to go do research, elicit opinions from academia, poll customers, and review the competition.  The result was a 2 hour presentation of finding to the committee with 6 possible solutions each clearly marked with their pro’s and con’s.  Consultants were hired to build prototypes which would run on all known platforms, support multiple languages, and be able to handle several different back end systems.  When I meet these fellows they were nearly 2 years into the project and by my estimate, tens of millions of dollars spent, and no product in the the field.  You can decide for yourself if that is a problem.

There are many, many more … what problems have you seen?

5 things that make a problem worth fixing

At it’s heart, recovery is about fixing problems.  Before we get into individual and combination problems let me lay my definition on the table for your consideration and comment. 

A problem is only a problem if it causes the team to underperform.  Underperformance is determined by plotting the teams current performance at any moment in time against a projected ideal performance baseline.  Something like this.

Given that a problem worth fixing should drive the team off the ideal line.  Here then are the top 5 reasons a problem is worth fixing;

#1. A problem is worth fixing if it prevents delivery of a solution to users that they can use.  If it fails to function as intended, fails to function at all, or fails to return a reliably correct result it’s not done.  You have problems.

#2. A problem is worth fixing if it increases risk beyond the tolerance of the project.  Much like buying stock, when a team undertakes an activity it has to balance risk against reward.  The corollary being the bigger the risk the bigger the problem when things go wrong.  Unfortunately it is common for a troubled project to indulging in high risk gambles hoping for the big score to buy back all their prior losses.

#3. A problem is worth fixing if it increases cost beyond the budget of the project.  Ever have a someone tell you to use a free tool that is the perfect solution for your problem only to find out you need to higher consultants, pay under expressed fees, and burn that most valuable asset... time.  Throwing money at a problem is one way, but understanding exactly how much you can throw before you are simply exchanging one problem for another … priceless.

#4. A problem is worth fixing if it reduces project velocity below the speed necessary to deliver, in the time remaining, the agreed on scope, at the agreed on time.  No matter how much easier it would be to deal with, velocity is not constant throughout the lifecycle of a project.  Moving from start to finish in a time-box requires constant and precise adjustments to velocity. 

#5. A problem is worth fixing if it if everyone knows the answer but no one is doing anything about it.  All too often the team knows something that, left unchecked, will kill the project.  Sometimes it is a staffing issue, sometimes it is building the wrong although clearly stated scope, and sometimes its defects yet to be addressed.  No matter what the specifics are, if more than a couple team members know the issue is laying around unresolved it is clearly worth fixing.

Implement EXPILCIT Governance

While speaking at the Dr. Dobbs' Architect and Design World conference last week Scott Ambler said "Like it or not you are governed!"  Genius in its directness and simplicity and extremely relevant for recovery projects.

Many a challenged project struggles against the unstated and frequently invisible reins of governance.  Traditional definitions of governance speak to "… the leadership and organizational structures and processes that ensure that the organisation’s IT sustains and extends the organisation’s strategies and objectives.  Recovery needs to take a more simplified, pragmatic approach.  Governance equals constraints.  The projects budget, the schedule, the network, the hardware, other applications, legacy systems, current staff, tools, and oversight are all governance.  These are the concrete manifestations of the organizations decisions ... be they intentional or coincidental.  They are real and the recovery is defined by them.

The trick is to expose the governance.  Bring it into the open for discussion and ... here is the important part ... change.  I know governance comes down from the business to IT.  I know it represents what the business wants, needs, and desires.  And I know the specific implementation of those wants, needs, and desires are negotiable.  Everything is negotiable. 

Just for fun, take a few minutes and write down all the rules and constraints from your current project.  Here are a few to get you going;

• There's no more money for people or equipment
• We must have a delivery before the next <fill in the blank> meeting
• We have committed to this tool and simply can't replace it.
• The new solution must improve <pick a metric> 200%
• The new solution must reduce operational costs X% year over year

No matter what the business states as constraints they need to be paired with some truths associated with software development.  I like to refer to these as the physics that controls the process.  Like gravity, it's not just important ... it's the law;

• A team has a maximum product development velocity.  No amount of "managing" will increase their output.
• Build completely and Test completely.  If you don't do it now you will do it later.
• The right tool in the wrong hands is no better than the wrong tool in the right hands although both are marginally better than the wrong tools in the wrong hands.
• All projects have some amount of pain to be endured.  You can accept it in small amounts throughout the project or you can put it off until the end game.  It's a bit like conservation of energy, project pain can not be created or destroyed, it can only be changed from one form to another or transferred from one body to another, but the total amount of pain remains constant (the same).
• The right thing, as in do the right thing, isn't malleable.  If you think you can fix doing the wrong thing now later in the lifecycle you will be punished by the lifecycle.
• Achieving specific business goals cost real money.  Finding a cheap solution with massive return on investment is just like hitting the lottery but less likely.

Having frank, open, and frequent discussions about the governance and goals related to a project creates an opportunity for all involved to redefine success in such a way as to make it possible.  The resulting explicit governance need not lower the business goals, but you will need to help all involved understand what they can REALISTICALLY do within the agreed upon limits of governance. 

Project Recovery MUST beat the odds

Predictions of failure rule the industry.  Any number of organizations and people look at trends and willingly, no gleefully, declare why projects will fail and, in general, I agree.  Lets face it, most projects aren't paid on delivery.  Nope, it's by the hour ... seems like a conflict of interest to me.  Once a project delivers the money stops ... so why not let it go for a while?  Why not let the customer request things that will stretch the timeline.  After all it would only be doing what they asked.  Of course that's not fair.  Many very well intentioned project teams strive to deliver even though they are hourly.  They understand the basic truth... for a team that delivers, there is a nearly endless supply of work. 

But back to those predictions of failure.  Most analysis are the result (and I know I am over simplifying here) of linear math.  You know, if it takes 2 people 2 weeks to do X then the same people will take 4 weeks to do 2X.  Simple, logical, and generally true.  It's the "generally" part that bothers me.  The goal of a good recovery effort is to beat the bell curve.  To become the statistical anomaly that, against historical trends and linear predictions, delivers.  A few examples;

• Continuous evaluation of project practices allows the team to leverage what is working and remove what isn't.
• Merciless scope management focuses the team and the customer on what is needed first, defering other requirements for a future release.
• Utilizing the best talent reduces day to day management, allows for the team to self-manage, and leaves traditional managers the freedom to continuously seek enabling changes.

Best practices, it seem, are not always the best practices for your team, at this moment, on this project.  Process, particularly  prescriptive process, are all about statistics.  If 80% of successful projects have a declared methodology and process then one should have a declared methodology and process.  It only makes sense right?  What about the other 20%?  IT seems that without a declared methodology and process 20% also succeed.  Doing the well known right thing doesn't guarantee success it merely (or significantly) increases your odds.

Done with forethought and clarity, choosing practices that fit your specific needs but are contrary to, or completely outside of, "industry best practices" can be the very difference between successful recovery and becoming another part of the failure statistics.   "Best practices" are, by definition, for the middle 80% of the bell curve.  Recovery projects live and die out on the 5% fringe where historical statistics focus on why the failures failed, not how to succeed.

We can make the choices others can't

Me: Why would you do that?

Them: You just don't understand

Me: Help me understand, what could have possibly driven you away from the <fill in obviously right thing> and to do <fill in obviously wrong thing>

Them: Well, it's complicated.  There are other groups, and schedules, and my boss said, and ...

Me: Let's not do <the obviously wrong thing>.  Tell them I made you do it.  I am happy to take the blame.

Them: ... but were different ... we can't...

Me: Doing the right thing is, by definition, the right thing to do. Just do it.

This type of thing plays out time an time again during recovery projects.  But I'm a very lucky man.  I live in a very simple world.  Decisions fall neatly into two camps;  A decision helps deliver a high quality solution to users on-time and within budgets or it prohibits deliver a high quality solution to users on-time and within budgets.  Sure there are shades of gray but any given decision will be to the right or left of center. 

As a consultant, an outsider, I get the luxury of making choices for the sole purpose of delivering.  I am guilt free and go happily about my day without the fear, uncertainty, and doubt that can cloud the judgement, or at least reset prioritizes in such a way as to elicit bad behavior, of the permanent staff. 

Recovery projects are a seemingly endless series of triage decisions.  Choosing the poor over the terrible and the merely horrible over the unforgivable.  On the really good days you get to build things or set best practices in motion that reinforces good behavior.  On a really bad day you hope to do no harm.   In the end being blamed for a truly fine idea which is compromised into a mostly acceptable, temporary solution is a win to be celebrated.

It's not about you

Software development is a world full of individual accomplishment and competition.  It shouldn't be.  Great software comes from great teams.  No one person is responsible and no one is irreplaceable.  As the industry matures we, as individuals, must also mature.  In short, it's just not about you it's about the successful creation and delivery of a product into the hands of the users.

Developers are trained, either in a long simmering environment of formal education or in the red hot crucible of hands-on self education, that success comes to those who are faster, less-error prone, and more productive, than their peers.  Testers take failure to have found a defect that was later found by another as a personal failing.  Managers all too frequently get individual rewards for the work of their team.  Personally, I have seen far too many resumes, and heard far too often during interviews, individuals who truly believe they deserve credit for the work of others.

The problem is nearly biblical in nature; Ego, hubris, and selfishness.  The cure is equally timeless; selflessness, humility, and teamwork.  When a project is having trouble I can guarantee the team is having trouble acting like a team.  On the surface this may be breakdowns in communication or misunderstanding directions, processes, or goals.  Below the surface it is likely to be a deadly rip-tide of self doubt, loneliness, and divisiveness. 

All of this is magnified when occurring within the chaos of a project recovery effort.  I would love to say I have always found a way to bring a team together during recovery.  I haven't.  Inevitably there are those who could and hadn't had the chance, those you can't and may never, and those who are new and always will.  Fractured teams slow velocity, reduce productivity, and generally ruin a fair number of otherwise perfectly good days. 

Our job is to succeed anyway.  It is harder, slower, and more difficult.  Oh well.  My priorities are very simple; Family, Team, Users, Company... in that order.  If you think you are the most important thing you can't make the kind of decisions that are best for the team and the users.  If you aren't on the team.  If I can't help you help the team.  I have failed you and for that I am sorry.  Now you are a problem to be overcome.  It isn't personal.  And it's not about you.

So you want to do project recovery?

I am seeing a gathering wave of players in the project recovery market.  I have stumbled across a half dozen new companies specializing in it.  I have been told about a few management conferences where recovery is being discussed and have certainly been speaking at increasingly diverse conferences on the topic.  I was even asked about teaching recovery for a large education company recently.  If that's not evidence enough it seems my blog readership numbers are on the increase.

I am not too surprised. While you can argue whether the industry failure rate is down below 60% or up over 70% the statistics clearly support the concept that this is an undeserved market niche.  There are certainly a reasonable number of software practitioners who can consistently deliver but I am concerned that the difference between delivering a project and recovering a project is not clear.

Project Recovery isn't for the meek. While others are fleeing from the fire you have to walk into it.  When everyone else is predicting failure you need to guarantee success.  In the middle of chaos, with random activity all around you must constantly make (what you believe to be) the right decisions.  To recover a project you have to assess quickly, pattern match across an encyclopedic number of past experiences and derive what is more likely than not to be a unique solution for the problem at hand based solely on your experience.  Not exactly CMMi level 5 predictability but that's what you are there to do.

The pain is, by most measures extreme.  You effectively interview and establish yourself as a respected leader constantly.  You are being judged by organizations who, by the very fact that you have been called in, are not functioning well.  You will need to outperform everyone in every role; architect, developer, tester, quality assurance, project and program management, and do some portion of them all at once.  You need to smile reassuringly to the CIO and CFO while effectively driving risk in the form of change into their program.  You will be deprived of sleep spending your days on your feet (I rarely take a cube and generally spend less than an hour a day at a desk) walking around teaching, correcting, and aiding communication and most of the night doing code reviews, statistical analysis to justify funding, and presentations ... lots of presentations to both convince and justify the team your direction isn't just a random thought.

You will be isolated.  Isolated from your team, isolated from your customer, isolated from your users, and in some ways your life.  If you go native, that is to be accepted by the team and the customer as "one of them", your opinion will not carry the additional weight of an outside consultant.  You will need to both protect and fight your users.  If they believe you are one of them, you will not be able to mitigate the unreasonable so you may deliver the possible.  You will need people to do what you say because they buy into the process not because they have become your friend.  Friends may do the right thing when directed but the lessons really only stick when they are internalized.  You will live in a state of nearly constant thread switching.  You must maintain a laser focus on the person, the deliverable, or problem in front of you and only that.  Of course the laser focus can't interfere with being the sole keeper of the big picture.  Really... it's fun.

If you are really good, you will document and communicate all of this, all the time.  You will delegate effectively.  You will identify people to trust and task them.  You will work with your customer leads to improve them so that they can take on the program as you exit.  You will need additional super hero members on your team and you will bring have to fight for the right team sometimes rejecting a paying job because you know that you can't staff it.  Personally I try but I do confess I am not always that good.

When everything goes well your team delivers the solution, the end users are happy, the customer staff takes over your multiple roles, and you fade quietly away.  No reward, no recognition.  When things don't go well, you still deliver the solution, the end users are still happy, the customer staff still takes over your multiple roles, but you get to be the container for the customer’s woes.  Run out of town like a sacrificial lamb only to go to another customer and do it all over again.

I do wish all who enter this new niche market the best.  If you do, and you're good at this, and you still want to do it again.  Drop me a comment.  I would like to hear from you. 

&quot;You just don't understand ....&quot;

I find it interesting that every project, well nearly every project, I am asked to recover begins the same way.  I go on-site to watch and listen.  I see conflict and chaos masquerading as progress.  I hear discussions of challenges but no admittance of trouble.  And inevitably, when I begin to talk to the leads, someone will tell me that I just don't understand.

It's true, I don't understand, and I'm proud of that.  I believe the greatest skill one can bring to a recovery is an open mind.  I am certain I don't understand, so I ask a lot of questions.  Better yet I really hear what people say whether talking to each other or responding to me.  I have no preconceived pattern they should match, but I do look for patterns.  I don't lay out plans too quickly, although they begin to develop as soon as I arrive.  Most importantly I don't pay any attention to the titles and roles people present.  Frequently the least valued team member has the most insightful view while the person at the top of the declared hierarchy is, or has become, so removed they are nearly delusional in their belief of the current state of affairs.

The best recoveries are about learning.  I learn about the people, the team, the users, and the management as well as the problem domain.  I can but hope the team sees an opportunity to learn from me as a positive use of their time.  Learning however, requires everyone, especially me, to be willing to accept new ideas and throw out old ones, and to take discarded plans and reconsider them as truly plausible solutions.

"You just don't understand..." For me it's like a red flag to a bull.  A clear indication that something is wrong and it needs to be discovered.  No one really understands everything.  Projects are exceptionally complex things with heartbeats, feelings, interrelationships, and a will of its own.  The goal shouldn't be to understand it.  Instead the goal is to help it find its own way.  No need to fight the tide but figuring out how to harnessing its power... now your on to something.

Don't bring a well known solution to an unknown problem

Are you a SCRUM Master?  A PMI certified Program Manager?  Do you have a quick reference card for for the Microsoft Solution Framework in your wallet?  How many prescriptive methodologies do you know?   More importantly, have you been repeatedly successful with one particular methodology?

I have no research to back it up, but I believe it is human nature to fall back on what has been successful in the past as your first choice for solving your next problem. This is continuously re-enforced by reports of others success using your favorite process.  Their problems were so different from yours, proving your methodology is capable of addressing a wide array of problems... right?  I think not.

Two things prevent your favorite methodology from being right for your next recovery;

First, what are the odds that all of the complexities that make up the personality of your last project are exactly the same as the next project?  Are the user’s needs the same? Is the development team the same, with the same skills, and the same tools?  Do you have the same amount of time to succeed?   It seems statistically unlikely.

Second, what is your definition of success?  Do you think it is the same as those reporting success with your favorite methodology?  Were they adding features to an existing application where success was pleasing some of the users without diminishing the features of others?  Were they re-purposing old code?  Trying to get the features to integrate with the least effort and cost possible.  Or maybe it was an update of an in-place system where success was defined as implementing a dizzying array of cutting edge technologies.  What are the odds your next projects definition of success matches one of theirs?

What is a recovery lead to do?  If you can't use what you are most comfortable with, if you can't leverage the tools that brought in the last project.  Let's consider a few possibilities.  After reviewing the problem;

1. Begin reorganizing the project to better align with your preferred approach.
2. Consider a wide range well known methods such as Waterfall, Spiral, XP, etc... Selecting the most appropriate.
3. Bring the team together and determine the approach they are most comfortable with and use that one.
4. Breakdown the project needs and characteristics and compose an approach that will fit.

I am a fan of the option 4.  Instead of bringing a well known solution to a project, take the best of what applies without regard for its methodological roots.  For example, if the team has strong developers who have a history of trusting each other, XP's pair programming is a great fit. However the rigors of the environment may not support the planning game.  If the project has ambiguous requirements, iterative design and development can help but you may need to support the major milestones from the Spiral lifecycle.  Choosing well, based on well reasoned analysis, significantly increases your likelihood of success.