No organization is perfect. We each can look around and say "stuff is broken here."
So, how to fix things?
First off, why fix things? After all, if I am a lowly programmer, it is not up to me to fix things, right? After all, they pay executives, don't they? Shouldn't an executive earn their salary by fixing the stuff that's broken?
First off, the executives whe are responsible may not fix things. Lots of reasons:
Know what? Most executives see the problems. They see them way too much. When I was a manager, there was a standing rule in my office: Bring Solutions. It's a pretty common rule.
So if you want to fix the broken stuff, you have to do a few key things.
First and foremost, you have to show that fixing this problem is aligned with corporate strategy. If the company wants to have four completely independent divisions, you will get nowhere by convincing an executive to merge all the purchasing decisions.
On the other hand, if the company wants to take on a new market, and they have not done so, show that your change will help them to do it. If you change doesn't align to strategy, drop it. Be the "doubter" and prove it, without a shadow of a doubt, that fixing this problem is the right thing to do and this is the right time to do it.
Once you have this, find the person you have to convince. That's difficult. Spend some time thinking about it. Is it Joe in Marketing, or Sue in Sales, or Tom in operations? Think: If I were "Joe" and I decided to fix this problem, what would I do? Am I the right guy to fix it? If you have this person targeted, think about what it will take to change their mind. Do they respond to numbers or emotions? Do they believe in instincts or principles? Do they think short term or long term? Tailor your message.
So, what's in the message?
You can fix things. If you don't, no one will.
It's independence day week in the USA, so it is time for me to declare a little independence.
There are three schools of thought around "how to build an Enterprise Service Oriented Architecture." They are:
Top down works only in some organizations. I imagine smaller IT shops (with 300 or fewer total employees) and shops with a well established heirarchy or very strong central leadership can leverage this one. (I'm conjecturing, of course, having not worked in SOA in that setting. It's an educated guess). For everyone else, it is slow and difficult to follow the Top Down path.
Bottom up is an answer to top down seen as "the wisdom of crowds" or "economics in IT." The idea being that many services can be developed without much central control, as long as they are in a directory. Everyone would look in the directory to find the services they need and adopt the best or least expensive, and we'd get SOA adoption. This one has the advantage of scale. The larger the organization, the argument goes, the better the case for bottom up.
Middle out is the newest method developed by folks who prefer a blended approach. I won't go into a lot of details here.
The point I want to make today: Top down is slow, but functional. You can end up with a SOA that is good for the enterprise. Middle out has the same advantages and is a bit quicker.
Bottom up is actively harmful to the enterprise as a whole and should be discouraged at all costs.
Of course, a lot of the literature in the SOA field says "build the catalog first," effectively supporting the bottom-up approach. And that is a mistake.
As I've said before, I am not paid to solve a particular system problem. I'm paid to look out for the needs of the enterprise, to create a "gravity" towards the center to counterbalance the natural tendency of IT to develop only for the individual needs, even when it works against the strategic goals of the enterprise.
Bottom up is a mess. Let's look at the logical conclusion of bottom-up for a minute and compare what results we get with some very common enterprise strategic goals.
Let's say our goals look like this. This is a very simple set.
On first blush, SOA supports all three. It is clear that you can get common data, loose coupling, and ready integration from SOA, so a lot of folks have cried "SOA is the answer!" and then stopped. They didn't pick the path to take, so the debate rages on: top-down, bottom-up, or middle-out.
The problem with bottom up is that you end up with systemic effects, not seen by individual contributors, that work against these same strategies.
There are three ways that bottom-up SOA can go in your organization. One, it is ignored (no one decides to participate). Two, it is wildly adopted with some folks contributing and others consuming, and Three, it is adopted only be people who have services they want to advertise, but no one consumes them out of fear of taking a dependency. (There is a common word for both One and Three: "failure".) Let's look at Two: some contributors and some adopters.
What are the folks contributing? Well if we are creating a system, but we are not paid to think about the enterprise, then the system we are creating may, but probably won't, deliver capabilities that the enterprise needs. The service will be 'local-specific'. That means that the service will serve the needs of the paying customer (a business leader) but not the enterprise as a whole.
So the services that are in the catalog, most of them anyway, are not really 'enterprise ready.' That means that when others consume them, they are using identifiers that are not enterprise-ready, or they are using data in a manner that may or may not be usable in the data warehouse.
Remember that consuming a service is only demanded by a business process. If it weren't for the need to change processes, we would never need flexibility in the first place. If we do 'bottom-up SOA' without considering the business processes, that doesn't mean they aren't there. It means we are ignoring them. It is in some folks best interest that we don't look. Consuming a service "automates and hardens" a business process, making it more difficult to change. What if we are hardening really bad process? Better, what if, in different parts of the company, we are hardening different processes?
In the past, I had a run-in with very strong-willed and opinionated leaders, both with IT and the business, who strongly defended their absolute right to "do things their own way." Sometimes that's healthy, but in the cases where it is not, it can become a shouting match. Bottom-up SOA avoids the fight because if the service is attached in the "right way" you use it, but otherwise, you write your own. Fight avoided. However, when a business leader wants to take a basic, core, fundamental process, and go their own way, that is not a fight you should avoid.
According to some fairly compelling research and literation in the field of business process automation, the most successful companies have automated their core business processes, which provides less flexibility, but gives them far greater agility when it comes to taking on new challenges. The core stuff, the stuff they do every day, is routine, standard, and well thought out. It is a stable foundation, allowing the company to stretch in a dozen different ways.
"Bottom up" is simply a mechanism to allow those strong-willed leaders to create multiple infrastructures in the company "because they want to" without any consideration for creating a common and stable foundation for the business.
So lets look at a bottom-up success story three years later. A number of infrastructures have emerged. You have one based on Division A's CRM system. You have another that feeds Division B's ERP system. You have a third in Division C to surround their mainframe Operations management system, and a fourth in Division D focused on their home-grown web-order management infrastructure.
You have four legacy systems. Integrating them will require a slow, painful, expensive infrastructure, in the middle, with huge amounts of semantics and rules needed to translate the data from one legacy infrastructure to another. That central system will be necessary to support Business Intelligence and Financial Reporting (so you cannot avoid it) and it will limit the ability of the company to respond to the needs of the marketplace.
You got SOA, but at a price.
Of our three principles above:
We will have won the battle, but lost the war.
This is not a scenario that I would wish on anyone (except perhaps my direct competition). To be honest, I think my competition is smart enough to see this for themselves.
Bottom-up SOA may work if your measurement is "get services up an running" (or it may fail). If it works, it produces a monster that is not worth producing. That is not success either. Measuring SOA success by the number of services, or encouraging the creation of services without any form of central understanding of what the services should contain, is doomed.
Avoid Bottom-up. It is a pox on your house.