Summer of Benchmarks: .NET vs. IBM WebSphere 7 Performance and Price Debate and Discussion
This is turning out to be the summer of benchmarks. Check out http://www.wholoveswindows.com/websphere for the latest update to the benchmark discussion, including the latest IBM and Microsoft responses. I spent the late spring finalizing and evangelizing a study, which I’ve previously discussed on my blog, that compares performance and price of Windows Server 2008/.NET Framework 3.5 systems to IBM Power6/WebSphere 7 systems. I scrutinized the results, took tips from the community and my colleagues to ensure accuracy of the tests and answered a lot of questions on the methodology. The bottom line is, I stand behind the testing methodology and the results, and I encourage customers to read the study and conduct the tests themselves. I’d also love to continue to get comments and feedback on your experiences, should you decide to take the plunge.
Showing customers how to save money and time with technology is something I care about a great deal. I also want to make sure customers can get equally solid real-world results, using the sample benchmark applications with published source code as learning guides. Full disclosure of such benchmarking and testing is extremely important, as it allows customers and competing vendors to fully analyze the results, and even replicate the testing on their own such that fully informed choices can be made. Without this level of disclosure, customer should question the credibility of their results, especially considering our specific technical counter-points included in this paper.
Turns out, I’m not quite done testing. IBM recently shared their own benchmarking results in an effort to challenge my original benchmarking results published in April 2009. Once I obtained a copy of IBM report detailing their results, I was excited to dive in and explore the findings. You can find my response to IBM’s challenge, as well as my original benchmarking results at our new website, http://www.wholoveswindows.com/websphere. The bottom line, I stand my previously published results and continue to invite IBM to meet us in an independent lab to perform the same tests of the .NET StockTrader and WSTest benchmark workloads and pricing analysis of the middle tier application servers tested in our benchmark report. In addition, we invite the IBM competitive response team to our lab in Redmond, for discussion and testing in their presence and under their review.
So, what’s next for my summer of benchmarks? It’s hard to say. I can tell you that I *hope* it includes a joint Microsoft/IBM effort to have results on the respective platforms validated by a third party!
