Choosing Technology: Web Services vs. Binary Protocols
Computers are a panacea for those with short attention spans. There’s always a new technology or trend to follow, old things passing out of fashion, new things coming into fashion. Few fields can compete with computing in terms of sheer concept turnover. If you tune out for a few months and then come back, everything has changed.
This creates a tendency in people to jump onto the bandwagon as soon as a new technology comes out, even if their old technology was working perfectly well. One of two things happens: either enough people jump on board and the technology succeeds, or it doesn’t make critical mass and fails, and those who sought to gain advantage find themselves with less time and a bunch of wasted knowledge and experience.
A few examples come to mind immediately: C#/CLR vs. C++, and Web Services vs. more traditional network protocol models. In both cases the new approaches add a lot of value in many cases. Unfortunately there is a push to embrace the new and eschew the old, which makes great sense in some cases, and none whatsoever in others.
Take the case of Web Services vs. more traditional protocol development, such as binary protocols.
Web Services aren’t a new idea, but have been gathering steam as a protocol choice. There are a number of compelling reasons to adopt web services. They rely upon a plethora of standards to, in the ideal case, ensure compatibility between applications written by a variety of vendors, turning the world into a playground for interoperability and modular inter-site development.
You don’t get something for nothing, though. Generic interoperability and extensibility requires a strong set of intuitive (and not-so-intuitive) standards that people follow, such as WS-Discovery, WS-Authorization, SOAP, etc. All of these technologies take time and effort to master. But once mastered, should the become the exclusive solution for networking problems?
There’s no such thing as a universal tool. Some days you need a wrench, some days you need a can opener: even a really nifty wrench is going to make a mess of opening a can. Sure, it gets the job done, but you wind up with tuna sprayed all over the room.
Web services protocols are great for interoperability, extensibility, and value-added processing of packets. On the other hand, the resulting protocols can be complex and difficult to get right, depending upon the application, especially if you’re rolling your own SOAP-based protocol. The relevant standards contain hundreds – perhaps thousands - of pages of mind-numbing details and many, many layers of abstraction. If you’re not relying on wizards and development tools to create your web service, you’re in for a real struggle. And, at the end of it, if you want to retain the vaunted benefits of web services – interoperability, extensibility - you wind up with an extremely verbose protocol. This verbosity is the antithesis of a binary protocol.
Binary protocols have been around for ages. They focus on compact representations of data and packets. With a binary protocol you can easily optimize for the minimum number of bytes transmitted and processed. Instead of having an expression <NumberOfElements>50</NumberOfElements> in your packet, you put a single byte – or a word if you’re feeling generous – which is understood by protocol implementations to specify the number of elements for that packet.
Here’s a real-world example for you. While working on the Peer Name Resolution Protocol, a couple of us did packet-size analysis comparing a reasonable web service encoding versus a dedicated binary protocol. We found that in certain cases a packet which could be described in less than 50 bytes in a binary protocol took over 1280 bytes in the XML format, exceeding the IPv6 MTU. Since we were planning on using UDP over IPv6, this was a real problem. After a few weeks of internal debate, we chose a binary protocol. I think it was the right decision for a number of reasons: PNRP is a single-purpose protocol. It doesn’t need value-added processing, and it can’t really benefit from most of the strengths of web protocols. However, it definitely needs to be efficient to have any hope of working with low-bandwidth hosts. If we’d hopped on the web services bandwagon – as we were strongly encouraged to – we would have wound up with a very different, less efficient protocol, and would have gotten almost nothing for that cost: we simply didn’t need the benefits of web services.
And that’s the moral of today’s entry. When you approach a design task, think carefully about the options you have. You need to build in future proofing and be aware of the trends and technology directions. At the same time, you should do a cost-benefit analysis for your technology choice. In the case of networking, if you’re implementing a protocol you hope will be used by millions, make sure you pick the right technology to meet your needs.
