Killing the Command message: should we use Events or Documents?

re: Killing the Command message: should we use Events or Documents?

Evan — Wed, 08 Aug 2007 03:48:58 GMT

wow..amazing post..i'm going to have to think about this one for a while.. ;-)

re: Killing the Command message: should we use Events or Documents?

Kjell-Sverre Jerijærvi — Wed, 08 Aug 2007 08:44:15 GMT

I think your figures are a bit biased towards document messages; the event figure is almost as complicated as it gets, while the document figure is too simplified. Even your document based solution might need to ask for more data from the publisher or other services, and the invoice service can still fail.

The publisher of an event do not need to subscribe to success+failure events, it can just do what is common in workflow solutions: just start a new process (workflow) and assume that it will eventually succeed, and that there is some monitoring (BAM) and a "dead-letter queue" that is handled to pick up on errors. The publisher only needs "handle external event" if the two autonomous, asynchronous processes needs to be joined at checkpoints. Otherwise, the sale process just asks for invoice data when naturally a part of its process. Afterall, the publisher might even not need to know that an invoice gets created - as the event message never contains any defined handling operations.

Using workflow for long-running processes is a proven concept, and I like to apply this design-style to SOA systems.

I think a combination of event+document is better, it removes the need for step 2 and 3 in your figure. And if you fire-and-forget the event and relies on monitoring, step 4 and 5 are also gone, at the cost of a BAM mechanism.

re: Killing the Command message: should we use Events or Documents?

NickMalik — Wed, 08 Aug 2007 11:34:22 GMT

@ Kjell-Sverre,

Perhaps my document figure is missing a possible call-back, since I was aiming for a 'normal' configuration, but I did capture 100% of the requirements of a normal configuration.

I did handle the errors in the generic subscription because the error would have the same transaction id as the sale, so it would be picked up by the same rule.

That said, I don't think it makes sense to eradicate events. As I stated, events have their place, and so do command-response messages (within an application domain boundary). I do believe, however, that document based routing is quite powerful and still simpler than typical routing scenarios based entirely on events.

Basically, it is "big events" with "rule based filters" but that doesn't sound as catchy as "document messaging." :-)

Thanks for the feedback.

re: Killing the Command message: should we use Events or Documents?

Greg Young — Thu, 09 Aug 2007 02:56:59 GMT

I am not convinced on the document model when dealing with other situations.

Unless I am misunderstanding it, it seems like it could easily introduce odd dependencies or thoughts of doing things like "merging" of documents when dealing with parallelism. The event model with granular messages seems to be a much better fit for parallelism as its easily defined in the pi-calculus.

Could you perhaps explain parallelism in a document passing based system?

re: Killing the Command message: should we use Events or Documents?

Jack van Hoof — Thu, 09 Aug 2007 11:25:39 GMT

Splendid posting, Nick. There is however one thing that I don't grasp. In my philosophy events are at the basis of documents where documents - in an ideal situation - contain a description of the full event context, including - functional - correlation identifiers like your tx identifier. So I come to "SaleMade", "SaleApproved" and "SaleDenied" documents. Is this in contrast with your philosophy?

I agree with you saying in the reply above: Basically, it is "big events" with "rule based filters" but that doesn't sound as catchy as "document messaging."

Thats looks a lot like my posting on loosely coupled process flows: http://soa-eda.blogspot.com/2007/06/how-to-implement-loosely-coupled.html

Jack

re: Killing the Command message: should we use Events or Documents?

NickMalik — Thu, 09 Aug 2007 16:40:31 GMT

Hi Jack,

In order to do content based routing, some things have to be fairly stable. In EDI, everything has an envelope and a header. We inspect the envelope for addressing and the header for context of the message. By creating three kinds of messages for SaleMade, SaleApproved, and SaleDenied, you have put the state of the sale into the header, and that is fine to a point.

I did not do that because I want the receivers to create a single subscription on the document type (Sale) and have them place a rule with the routing system asking for the documents in the desired status. Something akin to "Subscribe( Sale, Status: SaleMade | SaleDenied)"

The key thing is that the "Routing effect" of both schemes is the same. The subscriber receives ONLY the documents they want.

Having two different documents has the advantage that you can share completely different schema as the status changes. I did not capture that effect in the described mechanism above, but it is a valuable one. On the other hand, having one document with status allows a system to receive messages that it didn't exist when it was designed simply by changing a routing rule. The schema of the inbound message is the same, and the receiving system can ignore the status and use other fields... like the remote system's invoice document number.

Architecturally, the exchange pattern is the same between your approach and the one I described. It's an interesting tradeoff. I'll have to think about what criteria I would describe for deciding between the two mechanisms. Both are strong.

Your feedback is always of very high value, Jack.

--- Nick

re: Killing the Command message: should we use Events or Documents?

NickMalik — Thu, 09 Aug 2007 16:42:07 GMT

@Greg,

Pi-calculus. Quite a challenge. Let me stew on that for a while.

--- Nick

re: Killing the Command message: should we use Events or Documents?

Jack van Hoof — Thu, 09 Aug 2007 17:35:40 GMT

Thanks, Nick. Good stuff to think over...

Jack

re: Killing the Command message: should we use Events or Documents?

Rishi — Sat, 11 Aug 2007 23:50:05 GMT

Hi Nick,

Great article, keep it coming.

Alright firstly the business event that says "create an invoice" is "bill the customer" and not "we made a sale". Invoicing can be on order or pre/with/post delivery and either automated or semi-automated or manual depending on the defined business processes and the "we made a sale" is a different but related business event.

Secondly, in the event driven example, I suggest that we should make the "invoiced" and "sale denied" an enumerated status of the sale - and the mechanics in play would be that both parties shown would subscribe to the change of status events, rather than each individual event. Therefore, any change of the status would be or should be treated as a business event. Some EDA purist, would perhaps disagree with such assimilation but to me it brings flexibility and simplicity – just as you highlighted for the document based example. In fact, in one of the system I am designing the end-user can introduce their own enumerated statu(e)s, which can be changed by any part of the related business process/workflow and in turn trigger other processes/changes.

Also, I tend to think that you really have over simplified the document based approach. For example, doesn’t document based approaches bring concurrency issues? And also document based approach pushes you towards a linear execution plan, which is in contrast to event based archetypes that excel in state-machine type interactions (and therefore more dynamic and flexible business flows).

Lastly, I see that you have put a bus type middlewear for both document and event based approaches – how much does that weigh in the bigger picture of the architecture? And the underlying implementation you recommend is pub/sub or asynchronous consumption to kill the command message?

re: Killing the Command message: should we use Events or Documents?

NickMalik — Sun, 12 Aug 2007 00:58:01 GMT

@Rishi,

Wow. I think I'll be blogging for a week to answer your questions.

re: Killing the Command message: should we use Events or Documents?

Rishi — Sun, 12 Aug 2007 01:28:07 GMT

Like I said keep it comming..

Moving BizTalk to SOA

Connecting Systems the Microsoft Way — Sun, 12 Aug 2007 17:30:32 GMT

Over a number of years I have been considering, cajoling, driving, arguing and mostly thinking about...

re: Killing the Command message: should we use Events or Documents?

Udi Dahan — Thu, 16 Aug 2007 15:14:05 GMT

Just on the issue of having multiple kinds of events, using a hierarchical-topic based infrastructure, you could have systems subscribe to the top-level "sale" topic, or to the specific sub-topics: "sale made", "sale approved", "sale denied".

Would that change your thoughts about the event-based solution?

Interesting article about SOA.

Patrick Wellink — Wed, 24 Oct 2007 10:17:16 GMT

Have a read of this article : Killing the Command message: should we use Events or Documents ? The second

Is EDA the way to go?

NSilverBullet — Fri, 31 Oct 2008 04:11:19 GMT

I have been reading a lot about Event Driven Architecture (EDA) in relation t