A while back I was forced to justify, in plain terms, why partners and customers require, and should adopt, an application framework like EFx or any automated guidance for that matter. After all, adopting this kind of offering (some see as an intrusion) by any software development company, large or small, and would affect and force a significant change to real developers working on real solutions.

I think we know how us folks (developers) feel about change (especially if we didn't think of it!). So, our first line of defence is typically either: sceptical curiosity, or critical resistance. Responding to the later case by addressing the symptoms of the problem is typically a losing battle because you can always be challenged on the symptoms you see or experience versus the symptoms they choose to see or experience, its all about point of view then - no good for anyone.

Fortunately, as a consultant in the field who typically get to see the worst of it all and, rarely, the best of it all, and where project failures and 'bodging it up' are everyday news, we have a fairly realistic idea of the symptoms and root causes of these issues. (Lets not forget that I too was once a young keen developer too, and have made all the classic mistakes we see and hear about still today, I'm spending the rest of my career trying to make up for the earlier pillaging of the industry). It's mostly down to individuals, but to be fair to them, it's the responsibility imposed on those individuals and what they have to work with that is the root cause. Let me explain.....

Please keep in mind I do have to generalise here to make the point. So, of course there are examples where this cycle is short lived. However as I'll show you, it's not the norm in the overall scheme of things. Expecting everyone to agree with me would be foolish. After all, our industry is built upon that pastime of differing opinions - which you could say is the grand reason how this all began. I simply lay out the truth as I see it. I don't like it either - by the way, my weapon against it is being prepared to accept it, and help shape a change to it.

You can start almost anywhere in this cycle, (naturally), so I'll start with the background noise. Each of these points is directly related to the one before, most as a consequence of the previous point - that's why its a cycle.

Technology is Advancing Rapidly - Have a look at where we were 2 years ago, to the brink we are on now. I am not going to list all the new technologies available, but to be fair, as someone who is supposed to be on top of all things in software development related at the bleeding edge, I am finding the last year so hard to catch up with, and I doubt I'll enjoy that comfortable feeling anymore in the years to come.

There's way more than just the new products emerging - there's a new software revolution afoot which adds a heap more technologies and more importantly possibilities, and if you look at the vision of that revolution - well sorry, but we are becoming more specialists than generalists in the future - so by definition, you wont know it all in the near future, or feel like you have to either - which should be a relief to most.

Software Product Development is not Advancing - (overall) is no quicker or cheaper (relatively) than it was 5-10 years ago (at least not significantly enough to matter), all that has changed really are the technologies (platforms) which we implement and the tools that we use to do it. Some may say the tools are better and we are more productive than we were, and its fair to say solutions to minor problems are easier to solve because of the public internet resources. This may be true, but you are implementing more complex systems with more features and more newer technologies than you were before.

Why is this? Because of the effect of the previous point - technology is advancing rapidly, the possibilities/advantages/promises of new technology are reaching the end customers loud-and-clear now and they are pushing to get the latest and greatest in their solutions - and fair enough right? The problem is that the tools/approaches we (solution developers) use to leverage these new technologies are either not available, or they do a poor job on the whole of helping the developers get productive with the new technology. We are still spending too much time hand crafting everything!

Of course, another thing that most software developers will never admit is that, they don't understand a new technology or how best to use it efficiently (even though this is probably the major reason solutions fail in the end).

The Majority of Software Projects Fail, Overrun or Deliver Incomplete - when was the last time you participated in a project that delivered everything the customer asked for, in the budget it was given, and on the time scale expected? If you have, I am picking that; it wasn't the time scale expected by the customer, or the budget they wanted to spend on it. Or else, your team has reached the software development nirvana, and you better never leave it for the real world. Without providing statistics on this - you can find these elsewhere - basically, pick a number between 1 and 4, and multiply by 10 to get the percentage number of projects that succeed, half that number for the number that succeed on all fronts.

Budgets Always Expand - this is a natural consequence of the previous point. Everyone, including the customer and the solution implementer don't want to openly accept or portray a 'realistic' budget for a software development project - and fair enough? If you were the customer, and you heard the truth of it - you'd never pay the money and the project would never start. If you were the solution implementer, and told the truth of it - you'd never get the work.

So, how does any project get started? Well in truth, this is a business right?, and there are any number of ways of masking the reality of the situation and planning for more resources (budget being one of them), and most are legit, understandable and easily justifiable. Its actually got too easy these days, because everyone understands the irony of software development, and it now it almost expected. I'm not going into that here; the fact is that the budget will expand, as expected, as a natural consequence of the exercise. The effect of this is that it puts enormous pressure on the project team, and the customers (diminishing) expectations and satisfaction of the solution.

Customer Requirements are Expendable - What the customer wants, and what they get are two completely different things. If you ever scoped a software development project, from the implementer's point of view, you'll be familiar with the first exercise in scoping - cutting features to save on budget. The second thing that happens is the technical people (quite deliberately technical) decide on features and requirements based upon the technology they know to implement them. This results in 'fluffy features' or 'nice to haves' hitting the floor because they are 'too hard' (lack of understanding how to implement efficiently) or 'not possible' or 'out of scope' (lack of time/budget). Finally, more commonly, user requirements are misinterpreted for an excuse to show off a particular new technology (in most cases a poorly understood technology) which leads to a brilliant technological solution to the wrong problem or to a new problem the customer never had.

Bottom line, the customer only gets half of what they originally wanted with no extra cream on top. If you are the customer in this market, and you don't know 'exactly' what you want (and I mean down to the technology level), you are not going to get it, let alone extra features/advantages you may find useful (or even expect) as a result of using a newer 'advanced technology'!

Customer Satisfaction is Declining - What? How can this be true? Well, think about it - listen to your common sense. They (customers) pay more than they wanted for a solution, which is more than likely going to fail, with half the features they wanted in the first place. They didn't get what the new technology promised (or any of its promised advantages), and it's was no cheaper (or no easier process) today than it was years ago. So, who would be satisfied with this?

Business is More Competitive - So, if you were a customer, who just invested a ton of money into a software solution that didn't deliver on its promise, with half the things it was supposed to do, for loads more than you 'wanted' to pay for it, later than you wanted it to be successful in your business. Then, I am picking you'll look for someone or some other way that can deliver that - because surely they exist out there - right? After-all, if technology is advancing, this promises that things will be easier/cheaper to do in the future, which implies there must be people (solution implementers) out there who know about this new technology and how to leverage it skilfully to make it more efficient and cheaper to build software. And you will find that there are (or at least that's the perception). Regardless, of whether there actually is, or not, is irrelevant here. The point is that in order to get ahead in this industry you have to (appear) to be ahead of the field. And this industry is still very technology based. So it's the people with the biggest 'e-penis' (perceived capability) and the smallest cost (perceived greater value) that win the deal. (More fool them I say, because touting a lower cost and you more than likely right back into the cycle again). Although, there are some very successful groups who can do this....

Technology Advances... - surprise, surprise, if there is more competition in the field between solution implementers trying to get ahead of each other and win customers because they can do it better and cheaper, then they are looking to vendors to provide them with better tools and technologies to gain that competitive edge. And how do vendors respond to that? You guessed it, we design/invent a new technology to solve that problem, and not only that, but the best vendors, make that technology address a larger problem that fills a gap in the market for customers to solve their problems better. Which ultimately leads to - customers investing in that new technology, expecting to leverage its new advantages and value which should make creating a solution quicker and easier...

So here we are, back to square one. Feel free to start back at the top again, and iterate again.

Each point feeds the other, but none help diverge or break out from the basic problem (whatever that is?). If anything, the circle tightens and spirals into convergence. There are few escapes from this spiral. Project failure is common, partial completed solutions (under-featured) are another, few succeed in breaking out of the cycle.

But none of this explains or directs us to the root of the problem; these are all natural and logical consequences of this game. So, what's causing all this in the first place? Why are software projects too expensive and time consuming to complete correctly, on time, on budget and meet or (dare I say) exceed customer expectations?

Let's answer that in the next posting....