Inside Architecture

Notes on Enterprise Architecture, Business Alignment, Interesting Trends, and anything else that interests me this week...

July, 2007

  • Inside Architecture

    How do I fix the broken stuff?


    No organization is perfect.  We each can look around and say "stuff is broken here."

    So, how to fix things?

    First off, why fix things?  After all, if I am a lowly programmer, it is not up to me to fix things, right?  After all, they pay executives, don't they?  Shouldn't an executive earn their salary by fixing the stuff that's broken?

    First off, the executives whe are responsible may not fix things.  Lots of reasons:

    1. They may not see the problem you see.
    2. They may consider the problem you are seeing as "benign" and therefore not worth fixing. 
    3. They may see the problem but they may see no reasonable way to fix it within the budget/people/culture of the organization.
    4. They may see the problem but they may have decided not to fix it because other problems are more strategic.
    5. They may have chosen to create the problem as a tradeoff when fixing a different problem.

    Know what?  Most executives see the problems.  They see them way too much.  When I was a manager, there was a standing rule in my office: Bring Solutions.  It's a pretty common rule.

    So if you want to fix the broken stuff, you have to do a few key things.

    First and foremost, you have to show that fixing this problem is aligned with corporate strategy.  If the company wants to have four completely independent divisions, you will get nowhere by convincing an executive to merge all the purchasing decisions. 

    On the other hand, if the company wants to take on a new market, and they have not done so, show that your change will help them to do it.  If you change doesn't align to strategy, drop it.  Be the "doubter" and prove it, without a shadow of a doubt, that fixing this problem is the right thing to do and this is the right time to do it. 

    Once you have this, find the person you have to convince.  That's difficult.  Spend some time thinking about it.  Is it Joe in Marketing, or Sue in Sales, or Tom in operations?  Think: If I were "Joe" and I decided to fix this problem, what would I do?  Am I the right guy to fix it?  If you have this person targeted, think about what it will take to change their mind.  Do they respond to numbers or emotions?  Do they believe in instincts or principles?  Do they think short term or long term?  Tailor your message.

    So, what's in the message?

    • Demonstrate that the problem exists by demonstrating the financial implications of the problem.  Everything has financial implications.  Everything.  Get your facts in line.  Get the folks who are supplying your facts to agree with your conclusions.
    • Demonstrate that the problem is worth fixing.  Find a positive spin: we will perform better, save money, compete better.  Demonstrate that the problem you are trying to fix shows up a lot in 'unprofitable companies' (no one wants to be in that category). 
    • Share success stories about companies that have fixed the problem: what they got from it, how much better life is for them, how there are so few negative impacts and so many positive ones.  Remove barriers to adoption.
    • Demonstrate how their decision to fix the problem will align with corporate strategy.  Show how it will make the company go "faster" where they already want to go.  This is very important.  Paint the future and show how terribly important that they fix the problem you see before they can get to the future.
    • Demonstrate that you have thought about the tradeoffs that may have created the problem and show how your solution doesn't violate those tradeoffs.  Tell the story of a company that maintains its focus, your company, in the face of changing business climates, by leveraging the thing you want to fix.
    • Ask for the sale: Have a plan that they can sign up to.  Make it inexpensive: even if it is a plan to build a plan.  Get your executive to agree that it is worth a shot to dig a little, see if the solution can be implemented.  Look for test cases or pilot projects that you can apply your fix to. 

    You can fix things.   If you don't, no one will.


  • Inside Architecture

    Bottom-up SOA is harmful and should be discouraged


    It's independence day week in the USA, so it is time for me to declare a little independence. 

    There are three schools of thought around "how to build an Enterprise Service Oriented Architecture."  They are:

    1. Top down - central group decides everything and the dev teams adopt them.
    2. Bottom up - central group provides a directory and dev teams make whatever services they want.  Dev teams go to the directory to find services they can use.
    3. Middle out - central group provides key elements of the interface, including numbering schemes, message exchange patterns, standard communication mechanisms, and monitoring infrastructure, and encourages the dev teams to use it to build services that can be shared.

    Top down works only in some organizations.  I imagine smaller IT shops (with 300 or fewer total employees) and shops with a well established heirarchy or very strong central leadership can leverage this one.  (I'm conjecturing, of course, having not worked in SOA in that setting.  It's an educated guess).  For everyone else, it is slow and difficult to follow the Top Down path. 

    Bottom up is an answer to top down seen as "the wisdom of crowds" or "economics in IT."  The idea being that many services can be developed without much central control, as long as they are in a directory.  Everyone would look in the directory to find the services they need and adopt the best or least expensive, and we'd get SOA adoption.  This one has the advantage of scale.  The larger the organization, the argument goes, the better the case for bottom up. 

    Middle out is the newest method developed by folks who prefer a blended approach.  I won't go into a lot of details here.

    The point I want to make today: Top down is slow, but functional.  You can end up with a SOA that is good for the enterprise.  Middle out has the same advantages and is a bit quicker. 

    Bottom up is actively harmful to the enterprise as a whole and should be discouraged at all costs.

    Of course, a lot of the literature in the SOA field says "build the catalog first," effectively supporting the bottom-up approach.  And that is a mistake.

    As I've said before, I am not paid to solve a particular system problem.  I'm paid to look out for the needs of the enterprise, to create a "gravity" towards the center to counterbalance the natural tendency of IT to develop only for the individual needs, even when it works against the strategic goals of the enterprise.

    Bottom up is a mess.  Let's look at the logical conclusion of bottom-up for a minute and compare what results we get with some very common enterprise strategic goals.

    Let's say our goals look like this.  This is a very simple set.

    1. Share a common concept of customer, order, sale, shipment, and inventory so that we can get a straight answer at the end of the day about the operational and financial health of the business.
    2. Couple loosly so that we can keep the costs of maintenance as low as possible and hopefully, over time, divert some of those expensive maintenance resources over to developing new programs. 
    3. Build your applications so that other folks can easily integrate with them so that we can add new capabilities to the IT ecosystem quickly. 

    On first blush, SOA supports all three. It is clear that you can get common data, loose coupling, and ready integration from SOA, so a lot of folks have cried "SOA is the answer!" and then stopped.  They didn't pick the path to take, so the debate rages on: top-down, bottom-up, or middle-out.

    The problem with bottom up is that you end up with systemic effects, not seen by individual contributors, that work against these same strategies.

    There are three ways that bottom-up SOA can go in your organization.  One, it is ignored (no one decides to participate).  Two, it is wildly adopted with some folks contributing and others consuming, and Three, it is adopted only be people who have services they want to advertise, but no one consumes them out of fear of taking a dependency.  (There is a common word for both One and Three: "failure".)  Let's look at Two: some contributors and some adopters.

    What are the folks contributing?  Well if we are creating a system, but we are not paid to think about the enterprise, then the system we are creating may, but probably won't, deliver capabilities that the enterprise needs.  The service will be 'local-specific'.  That means that the service will serve the needs of the paying customer (a business leader) but not the enterprise as a whole.

    So the services that are in the catalog, most of them anyway, are not really 'enterprise ready.'  That means that when others consume them, they are using identifiers that are not enterprise-ready, or they are using data in a manner that may or may not be usable in the data warehouse. 

    Remember that consuming a service is only demanded by a business process.  If it weren't for the need to change processes, we would never need flexibility in the first place.  If we do 'bottom-up SOA' without considering the business processes, that doesn't mean they aren't there.  It means we are ignoring them.  It is in some folks best interest that we don't look.  Consuming a service "automates and hardens" a business process, making it more difficult to change.  What if we are hardening really bad process?  Better, what if, in different parts of the company, we are hardening different processes? 

    In the past, I had a run-in with very strong-willed and opinionated leaders, both with IT and the business, who strongly defended their absolute right to "do things their own way."  Sometimes that's healthy, but in the cases where it is not, it can become a shouting match.  Bottom-up SOA avoids the fight because if the service is attached in the "right way" you use it, but otherwise, you write your own.  Fight avoided.  However, when a business leader wants to take a basic, core, fundamental process, and go their own way, that is not a fight you should avoid. 

    According to some fairly compelling research and literation in the field of business process automation, the most successful companies have automated their core business processes, which provides less flexibility, but gives them far greater agility when it comes to taking on new challenges.  The core stuff, the stuff they do every day, is routine, standard, and well thought out. It is a stable foundation, allowing the company to stretch in a dozen different ways. 

    "Bottom up" is simply a mechanism to allow those strong-willed leaders to create multiple infrastructures in the company "because they want to" without any consideration for creating a common and stable foundation for the business.

    So lets look at a bottom-up success story three years later.  A number of infrastructures have emerged.  You have one based on Division A's CRM system.  You have another that feeds Division B's ERP system.  You have a third in Division C to surround their mainframe Operations management system, and a fourth in Division D focused on their home-grown web-order management infrastructure.

    You have four legacy systems.  Integrating them will require a slow, painful, expensive infrastructure, in the middle, with huge amounts of semantics and rules needed to translate the data from one legacy infrastructure to another.  That central system will be necessary to support Business Intelligence and Financial Reporting (so you cannot avoid it) and it will limit the ability of the company to respond to the needs of the marketplace.

    You got SOA, but at a price. 

    Of our three principles above:

    1. We have our common understanding of customer, but it is so far away from the systems that create that understanding that it is useless for generating real value. 
    2. We have our loose coupling, within each ecosystem, but each one is at war with one another, vieing for resources and investment.  The money we saved by developing the individual infrastructures has moved to the central integration hubs and BI.  No win there. 
    3. We've got easy integration, as long as the system that needs to consume the data has the same semantic understanding of the data as the system that produces it.  Otherwise, consumer and producer cannot work together, and the cost of forcing an integration is high.

    We will have won the battle, but lost the war.

    This is not a scenario that I would wish on anyone (except perhaps my direct competition).  To be honest, I think my competition is smart enough to see this for themselves.

    Bottom-up SOA may work if your measurement is "get services up an running" (or it may fail).  If it works, it produces a monster that is not worth producing.  That is not success either.  Measuring SOA success by the number of services, or encouraging the creation of services without any form of central understanding of what the services should contain, is doomed.

    Avoid Bottom-up.  It is a pox on your house.

Page 4 of 4 (20 items) 1234