Meiko asked me yesterday "To prepare for the design reviews, do I need to brush up on UML?"
It's really not such a goofy question. In Microsoft IT, lots of things are changing. In this environment, I'm hoping to be one of the many who rally for greater emphasis on excellent design, as my prior posts demonstrate. So, I champion design reviews, and peer reviews, and peer training like study groups. And not just a review where untrained peers take a peek at your text and go "nice job," but a real evaluation by architects looking at how well your design reflects tradeoffs, with the teeth to make you change it.
Hence the question.
Now to be fair to Meiko, she's a young developer. Smart, but only about eight years out of college. She hasn't been really challenged in this way before, and she wants to be prepared. I respect that. I wanted my reply to show that respect.
"I don't want to pick nits, but in order to design something that someone else will code, you have to communicate in a language they can read. So, no, you don't prepare for a design review with UML... you prepare for design with UML"
She gave me that look like 'you know what I meant.' I did. She's right. Not fair.
"OK, you want to know what I'm going to look for when I review the design, right?" I asked.
"That would help." Arms folded now. Defenses going up. I'm missing a chance. Dang.
I grab the pen and head for a white board. "Let's say you are in my seat, and you look at a system design that looks like this..." I start to draw. Her arms are not folded now. Good.
I draw a picture of some boxes and some lines. Nothing special. A database in the corner. Everything inside the boundary. Simple example.
She smiles, "I'd say the designer was stuck in 1994. That's basically a three-tier app."
"But is it any good?" I ask. "How do you know? What objective criteria do you use to decide if this design is any better or worse than any other?"
She opened her mouth but silently closed it. We stood still for a moment. We were standing in the hallway next to a really large whiteboard that was mounted on the only wall large enough to hold it. I could hear keys clicking as one of the developers in their office was busy typing. She looked at me for a second.
"Are you saying that you have objective criteria to measure a design?" She sounded a bit incredulous. She spoke the word 'measure' slowly for emphasis. I could see her science training coming through. She had told me once that her degree was in Physics.
"Well yes, but I'm not taking credit for writing it. This stuff comes from the Software Engineering Institute. It's called the ATAM method. I'll send you a link."
"Thanks" she said. Her reply was small. I had just told her I was going to 'grade' her using a method that she didn't know, developed by academics known for creating large and unweildy things like the CMM and TSP/PSP. I had just told her that she was going to have to read boring books to be able to justify what she already does quite well. Not a good message.
"Wait. It doesn't hurt. The ATAM method simply asks you to look at your design from the standpoint of what they call 'System Quality Attributes,' what we call 'the -ities.' Availability, Scalability, Reliability, Security... stuff like that. It's just a more organized way of doing what we've been asking folks to do for a long time. You don't have to prepare by reading a thousand boring pages."
She looked relieved.
"You do have to prepare by making sure your design has considered the System Quality Attributes with respect to the requirements." I stopped on the last word and repeated it. "The requirements are what determines if this silo on the board here," I turned to point to the diagram we were ignoring, "is a good design."
"There are always many ways to design a system, many choices to make." I said. She was back with me now.
I continued. "But each time you make a choice, you trade one thing for another. You may chose to make a monolithic app, and trade adaptability for a specific model of deployment. You've been making tradeoffs all along. We're just going to ask you to describe them."
"In UML?" She chided.
"UML, English, Algebra... pick your language." I smiled. She knew what to do.
How do you break down the costs of owning your application portfolio into a components that can be calculated? I have a simple expression that I believe encapsulates the TCO of an architectural component (legacy application, SOBA, service provider, etc).
The Total Cost of Ownership of your portfolio of applications is
For Each n in Portofolio: sum( BaseCost(Environment(n)) + CpxCost*Complexity(n) + ConnCost(Environment(n))*Connections(n))
I'm proposing this formula. It would require proof that it would accurately capture the cost of owning the portfolio. That said, it makes sense, and is fairly simple to understand. The formula is based on the following ideas:
I'd love to hear opinions on this formula.
When Jacob gets going, the best thing to do is step back and watch. He stands, all five-foot-six of him, at a white board, "speaking" through the myriad of boxes, arrows, and random words scrawled across the not-quite-white surface, as though to add another layer of blue ink to the washed-out background of partially erased thoughts. Words tumble enthusiastically out of his mouth only to slide and bounce against the shiny surface of ink and dust, because his back is to me as he writes, as though the pictures are his mouth, and the sounds of his voice are the crickets of a summer eve.
"The design is easy. This problem is solved." More lines and arrows emerge. He sells his confidence as a commodity, freely traded, highly valued. I watch and listen.
"We have seen countless articles that show five systems interacting over a messaging hub or bus. They all show a diagram that looks like..." An image appears that looks not unlike a spider with square feet. At the center, a circle, with lines radiating out to rectangles about the size of a paperback book. In the center circle, Jacob writes "hub" and heads around the room nod.
"The problem your typical message-based architecture is that it is designed to be reliable when one system goes down," he continues, striking a line through one of the rectangles, "but we haven't really discussed what happens when a new system comes up."
In an single fluid motion, he has slid five feet along the room-length white board. In this conference room, with six other architects, Jacob is in his element. Standing at the far left of the white board, against the corner, is Tom. Tom is tall, slim, and a picture of calm. He stands with a whiteboard marker in his hand, limp by his side. He had started Jacob going with a seemingly simple question.
"How," Tom had asked, "do we start a subscription to a new system without interrupting the message flow to the existing ones?"
Of course, no one but Jacob had really understood the question, so he asked it again, this time using the white board.
"If I start with two systems talking," Tom said to the white board, as he drew a box, a circle, and a box, a few inches apart in a straight line. He connected them with lines. System A sends 1000 messages a day. System B subscribes.
"Now I want System C to start up." Another rectangle appears, this time directly below the circle. He quickly swiped a line from the new rectangle back to the central circle. "But system C will need a lot of data that A has, so we prepopulate C's database." This time, a dashed line from one rectangle to another.
Jacob had been clearly ready to jump up, but you could see he was waiting for Tom to finish the question. I know he meant it out of respect for Tom, who is brilliant in his own right... but you could see an expression crossing Jacob's face like that of a fourth-grader, sitting in the front row, throwing his hand in the air because he has just recognized a question from the teacher that he knows the answer to.
"But that prepopulation takes time. It always does. Now, when you turn on C, it has already missed some of the transactions sent out from A. The database is out of sync unless we turn off System A... but we can't! It isn't our system and it's mission critical." Tom finished with a slight bit of agitation. His diagram had a half-finished look about it. He had circled the bottom rectangle about eight times. There was a faint smell of white-board ink
That's when Jacob had taken over. And now, he was on to his second diagram, the spider picture freshly drawn and a new diagram emerging, this time with a row of squares and long thin rectangles both above and below them.
"Let's look at this from a different angle," Jacob said, this time addressing the group. Patricia was sitting at the end of the long wooden table, and around the back, able to see the entire board, were Phil, Ram, Meiko, and myself. Tom remained standing in the corner.
Above the top rectangle, Jacob wrote "bus" and we were back on the same page. He had simply taken the "spider" diagram and blew it out, so that the message mechanism was on top, with the applications below. But what was the parallel rectangle down below?
As though to answer my question, Jacob pointed at the bottom rectangle. "This, my friends, is the data bus. Unlike the message bus," (pen moves to the top rectangle), " which passes messages from one system to another, this little beastie," (pen back down), "collects data extracts from the applications on regular intervals for BI feeds. This is where you put SQL Integration Services."
"That still doesn't answer my question." Tom chided. Jacob waved his hands and turned back to the board. This time, he drew a cylinder next to the top message bus.
"When messages are sent from A to B, they are also copied to a data store" he said, indicating his new cylinder. "Assume the data is extracted from system A at noon."
Jacob continued, pointing at the third rectangle in the middle, that we were supposed to infer was system C. "Now, when C comes online at 4pm, it subscribes. However, the subscription request includes a request for all messages that A sent since noon, since that's the timestamp of the data extract."
"An agent picks up the message request, goes to the data store, and sends the messages to C that had already been sent by A. Like replaying a key log." Now Jacob was facing the room as he spoke. His diagram was done, or at least we thought.
"So what's the point of the data bus?" Patricia's turn. I think she knew the answer, but she was just as interested as I was to hear Jacob explain it.
"To feed the other half of that data store. The data store holds basic query data, and maybe even some real-time aggregation data, so that many of the messages coming through the message bus can be answered without actually hitting the source system. Think of it as a BI base for the SOA architecture."
At this point, Jacob drew a large arrow from the bottom rectangle to the 'data store' cylinder.
"Hell, I learned something today." Phil drawled in what was left of his Alabama accent after spending a decade in the Pacific Northwest. "That's why I like these sessions" he said, as he started to collect his things.
The meeting had been over for quite some time. It was the end of the day, and six folks had stayed behind to draw diagrams and talk shop.
I left with Phil's words still bouncing around in my head. "I learned something today."
And that makes it a good day.
In order for Service Oriented Architecture to have the impact that Architects around the world have been touting, we need to be able to take large applications and architect them down to services and consumers. The services are difficult to do well. So difficult, in fact, it is easy to forget the rest of it.
But one key aspect to this path is the communication infrastructure itself. It has to be fast. I has to be flexible. It has to be standards-based, but willing to bend. It has to be secure.
Yesterday's SOAP Web Services are not fast enough for SOBA SOBA (Service Oriented Business Architecture) implies that the business app itself is composed only of consumers and services. This is an extension of SOA in that SOA apps are full (some would say legacy) apps with their own database, while SOBA consumers are simply service consumers. No database.
This means that even simple CRUD operations have to run across the SOA interface. Preloading a user interface has to consume service data. For this to work, the infrastructure has to be fast.
The SOBA infrastructure has to have a way to put intelligent caching directly into the service consumer itself, so that the number of transactions doesn't become "chatty." It needs to spend very little time serializing and deserializing data. There cannot be a noticable overhead caused by the configuration infrastructure.
In an async message-based world, these aspects don't rise to the level of notice that we consider them a problem. Not so in SOBA.
I have high hopes for Windows Communication Framework (aka Indigo). WCF is all of the things I mentioned. It is fast. It is standards-based. It consumes it's configuration quickly.
If you are finding that WCF has empowered a true SOBA environment, drop me a line. I'm interested in swapping stories.
Ever had a good idea that could make millions for your company? Did you tell the right person? Did you know who that person was?
I cannot count the times I've heard it. "You know, if we were really smart, we'd be doing XYZ." But then it falls to the 'Inventor' to sell the idea. We all know what happened to the inventory of Velcro (nothing!). People who see a good thing and come up with a good thing are often not adept at explaining, selling, convincing. Sometimes, the idea is not good. Sometimes the benefit is unclear. But the biggest reason that good ideas don't happen? The inventor doesn't know WHO to sell to.
At the end of the day, IT Governance is about knowing who to sell to. It is about having a clear idea of what group, what council, what committee, and what individual can answer all of the important questions.
I was given a short lecture on this aspect of governance yesterday, and the light went "on." Doh! How come I didn't see this as clearly before? It's stuff I knew already. Common sense, really. But I'll be darned if I really got it.
Start with your vision. What should the world look like. Then develop principles: good practices that should lead to that world.
Now take those principles and you break out the questions that need answering. Assign responsibility for answering it. Create a forum for deciding it. Make visible the communications in and out of it. Define the rules of the game. Then publish them.
And now... measure. Make sure that the 'good things' are acutally happening. If they are not, come back to these people and ask 'why not?' You will know who to ask. (So will the executives).
Vision: I want an IT ecosystem where any system can be easily hooked up to "integrate" with any other system, so that when business changes, we are ready.
Principle: Every application shall be designed with integration in mind.
Tactical: For every system of enterprise scope, or which manipulates enterprise data, interfaces will be developed and supported for communicating data, events, and functionality.
If you can answer these questions, then the inventor of a good idea has someone to talk to. Executives have someone to go to. No one is left wondering "how did this happen?"