You know what they say, "Stuff rolls downhill"
If the applications are supposed to be simple, and if Enterprise Architecture is to help find simple solutions, we still have a problem. The application requirements are driven by processes in the business, and if the business has different groups or markets or products, and each one wants a different process, then we are ill positioned to say "change your requirements for the good of the company."
Without the business having a consistent and simple approach to solving problems in a common way, we may effectively be limited to providing good networking and database management services. The real commonality will be really tough to achieve. SOA can get close, but to do this right, the hydra that is the business needs to speak with one voice.
Some say that IT is the glue, that we can be that voice. I am not convinced. We can sell the idea, but the business has to decide they want simplicity for it to happen. We can advise, but that is all.
Six Sigma encourages us to examine a situation and, through analysis, data, and good old-fashioned thought, discern a root cause. The idea is this: go after the root cause, and the data should reflect a change for the better in how we manage the problem.
So let's look at the problem of IT Systems Complexity.
I define IT Systems Complexity as the following: A condition measured by the number of applications or independently configured software modules in an organization relative to the size of the organization, complexity of its competitive space, and the amount of market change typical in its industry.
Clearly the definition attempts to describe the results of a formula involving variables, including the organization's size, how many markets it competes in, how complex those markets are, and the speed by which they change.
Simply understanding this definition can give you an insight into the things that breed IT complexity. For example, to respond to very rapid change in a competitive landscape, then you need to allow substantial flexibility to your IT organizations. Large amounts of corporate oversight would reduce that agility, but some oversight is needed to create a simple environment. So clearly, the company needs to find the right balance between IT oversight and agility, with the expectation that simplifying the portfolio will have the effect of speeding up delivery of new programs and capabilities at another step of the IT process.
To put a little more meat into the definition:
In general, I'd classify IT Systems Complexity on a scale of one to ten, with one being low complexity and ten being a nearly unmanagable amount.
So, for example, a company that smelts steel may have substantial revenue and headcount, giving it a 'large' size. It may compete in markets for manufacturing raw materials (one market) in sixty countries (where the manufacturers are) which would be medium market complexity. On the other hand, it may choose to deal in only one country, say China, thus substantially reducing their market complexity. Lastly, the amount of market change would relate to the number of different innovative features being introduced by competition into the space every year. In the steel-raw-materials industry, I'd imaging this 'change factor' to be fairly low. The customers will need the mettalurgic attributes of the raw materials to be relatively consistent to predict the quality of the manufactured parts, although the level of purity may be increasing if specific products are needed. Hopefully this organization's IT complexity doesn't exceed about a 4, depending on the number of countries they sell into.
On the other end of the spectrum, a company that creates software may have a different profile altogether. Microsoft used to be a much smaller company with the revenues it has today. Now it is a pretty big place.
Microsoft competes in products for office worker productivity, vertical market spaces like accounting, ERP, and sales force automation, database products, operating systems, collaboration products, games, computer accessories (mice and keyboards), Media devices, and a long list of others. In each marketplace, there are unique competitive forces with unique products going up against the product in question.
Now add another layer of complexity: the fact that these software packages have to be customized to different languages and social norms and sold in different countries around the world. This adds complexity in legal management, a distributed sales and service force, and corporate/tax implications that can drive a great deal of complexity into the IT landscape.
Add to the fact that, in most of these marketplaces, the number of competitors is quite substantial and the speed by which they innovate is mind-boggling. To keep track of the list of changes in innovation against the list of products that compete with any product from Microsoft would be a never-ending stream of data.
In other words, it is not purely the application count that determines if IT Systems Complexity is a problem. It is the ratio of system complexity to organizational complexity that determines if you have a problem, or just a cost of doing business.
An organization like Microsoft 'needs' a complex IT structure to handle some of this business complexity.
On the other hand, it probably doesn't need as much as it has. We have literally thousands of unique software packages in production, the vast majority of which were written in-house. We need to be able to answer the question "what number is the right number?" and even more importantly, "in what area should innovation and complexity have free reign in order to encourage competitiveness, and in what area should control and oversight take top billing in order to reduce cost and process overhead?"
The best thing to do, from a complexity standpoint, is to create this scorecard. (Inside MS, we have created a CIO scorecard using Microsoft Office Business Scorecard Manager). We need the numbers to measure success. Otherwise, it would be difficult to know if any of the changes we are making are having any effect at all on going from 'where we are' to 'where we want to be.'
Have you ever wondered where business rules come from? Who thinks these things up? We say "the business" but it is a small core of leaders, thinkers, and analysts in any self-contained unit of the business that is empowered to create business rules. This group goes by various names, but in the business teams that I work with, it is often called the "policy" group. These are the folks that consider the specific business needs of a unique slice of customers and create "programs" designed to meet their needs, increase revenue, increase loyalty, and remove disincentives for ongoing business.
This notion of policy specific to a customer segment is a key element in the ability of Microsoft to respond flexibly to the needs of customers. Microsoft does not make money by building the newest innovations every year. (As the slip in Vista ship shows, unfortunately). We make money by delivering all of the basic needs and a broad swath of the "cool stuff" in a way that makes sense for both the customer and for Microsoft. This flexibility is a major competitive practice.
One thing to consider: this flexibility is part of the culture and part of the business structure, and not an explicit business rule. That means that some IT initiatives may come along that could, if not handled well, reduce costs or increase performance but may also have the unintended consequence of reducing this precious and highly valuable flexibility.
So we go the other way. We protect the flexibility, at the cost of increased expenses and potentially reducing the performance of business systems. (I'm not talking about MS Products like SQL Server here. I'm talking about internal financial, sales, marketing, management, and fulfillment systems). This protection is automatic and part of the culture.
In a way, this translates into policy like this:
Policy maker: I think it would be cool if we put feature X in our order entry system.
IT person: We've never done that before on our order entry system. That means we would need to create a new user interface and a bunch of new rules.
Policy maker: So what? Do it.
There is no structural mechanism that responds: "No. We won't adopt this requirement because, for this one requirement, we are creating a new application. If we drop this requirement, we could leverage an existing application. This is the straw that breaks the camel's back. Stop here."
There should be. Policy and Process need to work together, hand in hand, with each willing to give and take to the other. If Policy always wins, it is easy to create rules, but after a while, the IT systems fill up with overlapping and badly connected systems. If Process always wins, then it is hard to create rules, and sales will suffer.
Somewhere in the middle is the right answer. Both Policy and Process need to work together, sometimes giving, sometimes standing their ground, always cognizent of the value of the other.
Making this happen in practice needs to be part of the organizational structure... embedded in the culture. This is why every business capability (like "accept an order") needs an owner, so that there is someone for the business policy teams to negotiate with. Someone who can give and take, compromise and stand ground.
Yin and Yang. Policy and Process.
Simplification reduces the cost of maintenance. The business probably doesn't want to think about it, and they really don't want to pay for it. However, in many situations, IT has no choice. By reducing the number of applications, reducing the number of redundant data stores, making data paths simpler and more measurable, and driving for more consistence in non-value-add processes, the company can reap real benefits in cost savings.
However, simplification requires a couple of things. It requires project teams to change processes and code. It requires support teams to manage the data shifts and all the ripple effects. It requires release management coordination to make sure that new features and simplified systems do not conflict.
Where does the money for that come from? This requires the business to pay for this work. It requires that IT have staff and resources to dedicate to it.
Think of it like aircraft maintenance. When a jet airplane is being repaired, it has to be pulled out of service... for a day or two or ten. But the airline cannot simply cancel the flight because the aircraft needs a routine tune-up. The airlines have some capacity, in terms of aircraft seats, set aside so that they can allow the aircraft to be inspected, maintained, and returned to service.
So when an airline decides how many flights per day to set up, it has to plan on a certain amount of excess capacity just to cover the number of planes needing service. It does not apologize for these costs. The cost of your airline ticket includes the cost of maintaining other aircraft.
And you don't complain. Why? Because you want YOUR aircraft to be working when you get it.
Similarly, the IT department needs to have excess capacity, in terms of project teams and hardware space, to cover the costs of simplification.
This requires real leadership to pull off, but it has to be done, especially in larger IT departments where there are many developers writing code. In these environments, the amount of overlap is substantial, so the benefits are large.
So for every five project teams that you can use in business-requested maintenance or new features (airplane flights), you need a sixth team for IT driven simplification (aircraft maintenance). That team needs project managers, product managers, developers, testers, release and code management, and support members. Full out teams. The business does NOT get to drive the efforts of these teams. IT does, as needed, with the guidance of Enterprise Architecture or the EPMO.
Without this capacity, you will never be able to solve, or fund, Simplification. It's a cost of doing business.
Last note, when I say "Simplification", I'm not talking about consolodation. Sun is spending a lot of time talking about the savings that a company can earn by moving software from multiple servers to a single Sun server (lower power, lower rack space costs). That is consolodation, and it is not specific to Sun servers. You can get the same benefits from any server consolodation.
This reduces ongoing run costs, but not the costs of maintenance. In fact, it may increase the costs of maintenance somewhat by placing more complexity on a single server and decreasing the redundancy.
In Enterprise Architecture, we have a goal: be a strategic asset to the business. That means that we need to come to the business with not only constraints ("you have to use technology X or process Y or leverage the existing system Z") but also opportunities ("you can lower costs by doing A, understand your customers better by doing B, and compete better by doing C"). That's a different mindset.
Business folks are not ever going to come to IT and say "tell us what we should do." It's just not going to happen.
So where do all the good ideas in business come from, anyway? In a word, marketing. The business leaders who percieve trends, develop strategies, and then start to propose changes are subject to the same marketing messages as anyone else. They read magazines, chat with friends, attend conferences and listen to the radio, just like everyone else.
So, one way that we can get out in front is to come up with some ideas about how we can improve our organizations and then market those ideas. Perhaps publish a newsletter where IT staff write up an article every month on an opportunity that they percieve to reduce costs, increase revenue, or better compete. Perhaps conferences would help, with your company's IT team presenting options for improvement in different "booths", with the management staff invited to browse, listen to presentations, and take home ball-point pens and conference swag.
This has to occur seperately from project work, and well in advance of a project. So if you are already working on project "Atlantis", come up with ideas about the next steps after project "Atlantis" is done (hopefully before it sinks into the sea, killing every living soul and reducing the name to a mere legend).
Projects are compromises. They represent the collective approach of "what to do today" but the ideas to market from IT are more long-reaching... "what should we be doing tomorrow."
In a purely hierarchical organization, it may seem sensical to sell the ideas only to the "right" senior management person, who then directs their staff to "make it so." That sounds interesting on paper, but I've never seen an organization where a smart and respected low-level manager couldn't walk into the room of his or her manager and present a great idea. In fact, I'd suggest that this is how most really useful things happen... with the ideas of a good business person.
You job, if you choose to accept it, is to seed that discussion with good ideas.
Of course, this requires that you understand at least the basics of what the business does. You can't be waiting for someone to come hold your hand and feed you requirements. Get out there. Understand the needs as best you can. Extrapolate. Be visionary. Look for the hidden steps, the paper processes, the e-mail workflow.
Then, market them.
One of my favorite organizational 'anti-patterns' is "winning by whining." This is otherwise known as "squeaky wheel syndrome" or "fix what hurts the loudest."
Business managers get promoted by changing things for the better. So they decide what they can change and go about trying to make it happen. That required a good project manager. Clearly, it is in the best interest of a business manager to look around at the last four successful projects. Who led them? What was their business case? Was it compelling?
More often than not, and this is not specific to Microsoft, I've seen the funded projects start with a vocal PM advocate who believed in the business case, making it compelling (or at least, hard to ignore). The louder they are, the more passionate they present, and it won't matter if the business case is any good... they will win on the basis of their ability to convince, consult, and sway.
Running counter to this is the notion of DMAIC -- one of the Six Sigma tools. DMAIC stands for Define, Measure, Analyze, Improve, Control. Five steps to process improvement. You basically decide what success looks like and create a scorecard to measure how you are doing. Then, using analysis, you can ferret out potential improvements and even, if you are lucky, make some basic predictions about the amount of improvement you should see... as expressed in the measurements.
In this scenario, the person who has succeeded by simply being vocal is threatened. They are challenged to compete against numbers, and if management is truly bought in to the DMAIC approach, then the numbers will win. These project managers, often seen as successful and trusted members of the organization, will oppose DMAIC for purely selfish reasons if it isn't positioned right. This is not the case at MS, but I've seen it in other places.
So what is a business manager to do? Look for ways to improve their numbers through DMAIC... then hire the squeeky wheel project manager to get it funded. Best of both worlds.
Look, we will never get rid of the folks who make passionate business cases. We shouldn't. How boring would that be. Instead of getting rid of anyone, enlist that wonderful energy to make the RIGHT changes happen first... the ones that measure impact in the ways that the executive cares about.
This is how we turn an anti-pattern into a pattern. We show those squeeky wheel folks that their role isn't gone... it's just that they will get to pick the best projects to deliver.
Bottom line: Harness the passion of every team member, and every team member will contribute to your success. And never, ever, put someone down for being passionate... even if their passion sometimes sounds a little squeeky.