Shawn Hargreaves Blog
Dear Reader, I have a question for you...
I am frequently amazed when smart developers ask questions about performance problems, but then it turns out they haven't so much as tried a profiling tool on it.
I'm curious to understand why this is, and if there is anything I could write that might help people use these tools more effectively.
Are you a smart developer who has a problematic relationship with profilers? If so, what is holding you back?
I'd imagine that it's a combination of all 3. For example, I have visual studio team system, so I know I can use the built-in one. But if I didn't have it, I'd have to spend some time poking around to find one since I've never used another one aside from CLR profiler, which kind of only helps with GC issues (correct me if I'm wrong please :-) ).
I think the other main issue is knowing how to interpret the results. Honestly, even I struggle with this from time to time ... I tend to do it so infrequently that I have to reacquaint myself with how to read the results. It would be good to have a guide on what to look at in different circumstances (similar to your post about how to figure out if you're cpu or gpu bound).
Ok, I use nprof frequently because it's simple:) But I've never dived into the details of CLR profiler... simply because it's not so easy to interpret result at the first sight.
I've never pursued profilers as I haven't stumbled across any information about them. When I search for them I typically find marketing buzz that I tend not to trust, rather than honest developer reviews denoting strengths and weaknesses.
So, I guess mark me down for the category "Do you not know what tools exist?".
Also: "Do you not know what they are capable of?" might apply as well.
Yes, I have used profilers. Unfortunately, on VIsta 64 I cannot get the CLRProfiler to run my game. It launches the .exe but the .exe then crashes immediately. However, when I double click on the .exe and run it stand alone, it runs as expected. Not sure what's up with that.
Without clr profiler and Ants i wouldnt have a DBP entry right now. Profilers for teh win
I was under the impression that:
* Profilers didn't run on XNA on the 360 (our target)
* Profilers were either expensive or hard to use
* And I don't even know what tools exist because of these (perceived?) obscacles
I have used them, but infrequently. Lack of prominence is probably the number one reason. Microsoft has a profiler available in Visual Studio, as pointed out by JoelMartinez, but it is only available in VSTS versions. If Microsoft were to push that capability down the chain to Pro (and perhaps even Std) and increase the visibility, they would probably find more people would use it. Throw in some MSBuild/TeamBuild tasks and we could even set up automatic profiling as a part of the test runs in the build process to track performance impact of changes. So write your superiors and tell them the average developer needs access to the profiler across the Visual Studio version stack.
I use CLR Profiler and PIX when I need to.
When testing on the Xbox, I use the... what's that name?... "XNA Framework Remote Performance Monitor for Xbox 360"
I don't use profilers because I've never really gotten to point where I couldn't figure out the bottleneck myself. I find it more fun to analyze code myself and try to logically work through the trouble spots.
It doesn't always work but it keeps me entertained. In those cases it's never super critical, so I just don't get around to it.
In general, I haven't used profilers since the days of Borland's Turbo Profiler tool. That was an awesome tool; you simply fired it up, pointed it to the debug build of your project, and it ran the application while injecting the proper code to time every function call.
I have never had another profiler that was so easy to find and use. In the early days of my work with Microsoft Visual C++, I looked for a good profiler, but everything I found wanted custom code to install its hooks. For many years, my work tended to be input-bound and not processor-bound, so I eventually lost my need for a profiler and just stopped looking.
I think what we really need in this industry is a solid enterprise-ready profiler with zero setup, like TP, which comes preinstalled in the IDE at all levels - so the developer can stumble across it and "discover" how useful it is.
PIX + CLR Profiler.
(one that I'd love to use is nVidia's PerfHud 6 but I don't want to mess with the way D3D devices are created)
Ultrahead: it's extremaly easy: http://forums.xna.com/forums/t/9204.aspx
I've been pointing these tools out to users in #xna on efnet, and converted a few. I think what some people really need is the good tutorial that shows them how easy these tools are to use.
A lot of new xna developers are also new to programming, and their skills might fall short of the concepts within these tool sets.
As for the smart and experienced developers... they have no excuse, and it's probably related to the fact many experience developers also recode many classes that are already standard in .NET. Something about being stuck in their old ways and "not invented here" ;-)
Only the clr. I would love to give Nprof a whirl but I'm too @#* vs ignorant to get it to compile where it will run in Vista 64.
- Do you not know what tools exist?
Many people even didn't/don't know unit testing (but now VS provides them, so some should notice). So I wouldn't expect they to know about profiling. Eclipse gives a hint with "Profile as..." but since it doesn't work for all projects, nobody uses it.
- Are their interfaces too confusing?
Some of them are but all of them bring their own ugly IDE. I would be so happy to just click "Profile" in VS with timeinfo directly next to the line-numbers, but ok - even profiling using Eclipse/Java does not work that way.
- Did you try one, but then couldn't figure out how to interpret the resulting data?
nProf website does a great example in introducing itself: Architecture blah, but "No documentation is available yet."
- Other stuff
nProf (the tool i liked most) just crashes on my Vista 64bit machine - best free tool, but useless. CLR Profile has similar problems.
Aside from these problems I would really like to find a good ScreenCast/Tutorial about profiling, especially for how-to-find-* like thread-sync-probs, or graphics-related gpu/cpu bound issues etc.