Welcome to MSDN Blogs Sign in | Join | Help

Rotor benchmarks

To date there has been no standard set of benchmarks that do something meaningful, run on Rotor and execute long enough to enable performance measurements.  Standard benchmarks are very important for publishing research results because they make easier to understand what applications are being presented in a given paper.  Ben Zorn from MSR has just published two benchmarks.  Hopefully, this is just a start and either Ben or others will put together more applications, so that we have a benchmark suite of 5-10 applications that can be used by all researchers looking into .NET.

 The two benchmarks available today are lcsbench and ahcbench.  Lcsbench is a front end of C# compiler written in C#.  Ahcbench is an implementation of Adaptive Huffman Compression.  Both benchmarks include three inputs of various sizes that achieve different execution times. 

Do you know of other nicely packaged benchmarks for .NET?  Do they run on Rotor?  Let me know!

Published Monday, July 19, 2004 6:21 PM by michaljc

Comments

# re: Rotor benchmarks

Do you read Werner Vogels' "All Things Distributed" blog? It is at http://weblogs.cs.cornell.edu/AllThingsDistributed/ and he has mentioned doing rotor benchmarks for high performance computing. He apparently worked on a port of a java library that does the same thing, available at http://cli-grande.sscli.net/. I don't know if this is what you are looking for, but may be interesting.
Tuesday, July 20, 2004 3:26 PM by Brian Broom

# re: Rotor benchmarks

A good point! I forgot to mention these benchmarks. They are very valuable and I encourage everyone who is interested in scientific computing to look at them. What I really meant to say is that I am not aware of any benchmarks that represent typical workloads.
Tuesday, July 20, 2004 5:15 PM by Michal Cierniak

# re: Rotor benchmarks

The adaptive huffman compression incurs in too much boxing, so it accidentally has become a GC test, rather than a computational test.

I sent comments to Ben
Wednesday, July 21, 2004 8:56 AM by Miguel de Icaza

# re: Rotor benchmarks

Are you saying that the use of boxing in ahcbench is a symptom of a poor implementation of this app? Since I haven't seen the benchmark source code, I will not defend it specifically but in general if boxing is a natural way to express what an app does, we should allow such an app as a benchmark. There's nothing wrong in having a GC-intensive app as a benchmark. Benchmarks should test all parts of a CLI implementation that matter to realistic apps. In my mind GC is relevant to the performance of a CLI implementation.
Wednesday, July 21, 2004 7:04 PM by Michal Cierniak

# CLR and JVM performance compared

Tuesday, October 19, 2004 8:05 PM by Michal Cierniak
Anonymous comments are disabled
 
Page view tracker