Welcome to The Metaverse

After reading through the first whitepaper titled "Performance of ASP.NET Web Services, Enterprise Services, and .NET Remoting" I can't help but feel like there are a couple of additional tests that might make this much more meaningful to developers making decisions about distributed object technologies. I would also like to pose some questions about the potential impacts some other tests might have on these results. I understand the objectives of the test but I wonder if we aren't asking the wrong questions in some cases.

My first question is: What is the reason for distributing code in this manner if not for interoperability anyway? I guess there are some scenarios where multiple .Net clients need the same reusable component where this might come in handy but how often is deploying the code to multiple locations really that big of a problem? Is the belief that the application logic could in fact scale differently than the UI? Is there potentially different security requirements for the middle tier that would require physical separation?

We have looked at all the specific technologies used for distributing your code physically but there is no attention paid to the optimal case which is keeping your execution in the same process/app domain. There are some specific examples that show how the work at the server side is quite small and the transport/serialization time becomes a large part of the expense. My question would be: Why take that expense at all? The scenario you present is one where a simple read operation is performed, would that ever really make sense to distribute? I will again state that I realize that was not the purpose of the whitepaper but I can't help but want to know exactly what is the price I pay to use even the fastest of these distributed technologies. I guess I should stop being lazy and just extend the test you already started for me :)

My second question is related to how much impact load would have on these numbers. I know you state early in the whitepaper that this is not an exhaustive load test but I find myself extremely curious at what point ASMX begins to require multiple servers in a load balanced environment. I know this is not an exhaustive whitepaper on capacity planning but I'm hoping others with more experience in this field can help with some direction. In your opening you say that "multiple clients do not change significantly the absolute or relative performance of the technologies being tested". Can I take that to mean this is a straight forward mathematical algorithm that'll give me my service capacity under concurrent request load. So for example if you say ASMX can handle 63 cps and we have 500 clients simultaneously making those requests on the server is it 63 / 500 = .126 cps. It sure doesn't seem correct so I'm really curious how to move forward doing this analysis (which by the way I'm trying to do in my job today). I'm not asking you to change your test in any way, I would just like some good resources for how this is done in practice out in the real world. I'm sure you don't go far in this topic before IIS tuning becomes a portion of the discussion too.

Lastly I see no mention of using some of the Begin/End async functionality built into ASMX. It was my understanding that this would greatly increase the amount of incoming requests IIS could handle. Is that true or did I misinterpret those materials.

In summary I really found this whitepaper a great read but I would caution those that look at it and on the surface use it as a justification for the "web services everywhere" approach to designing an application. There are still other viable solutions that don't involve distributing your application logic at all. I found some of the results very intriguing too, how about that the remoting SOAP formatter takes longer than ASP.NET web services. I would have expected them to be much closer.

I'll end with this question, when do we get to see how WCF performs in a similar type of testing? I'd love to hear how well the new xml serializer handles text base communications over http and of course I would assume the .Net to .Net transports are smoking fast. Another interesting statistic will be how much faster or slower a service will run when it is hosted within and then outside of of IIS.

Thank you Rich and Ingo, this was a great read.

Hi Rich,

Good work - this is a nice set of papers.

A useful 'build' on this work would be to explore down the relative perf of the ASMX WS track, i.e. a set that compares the following would be interesting to me:

- Bare ASMX (baseline, as in the tests already)
- ASMX WSE 3.0 username/token Basic Sec Profile
- ASMX WSE 3.0 kerberos/intergrated
- ASMX WSE 3.0 x509 client cert
- Indigo set for comparison?

We've done our own relative tests and been surprised by the results. What do you think, any chance of a follow-on from these papers?

Thanks,
David

very nice work.

I've been having some great conversations with my SOA blog buddies and I wanted to share. 1. 10 Things