I had a number of great conversations at TechEd 2008 IT Professionals down in Orlando. I mentioned in a previous posting that how you evaluate sessions at conferences has a fairly significant impact on where the take content in future conferences. To be completely transparent, I figured I'd do my conference recap right here, so you can see how I use the data to decide what to do in future conferences and get a sense for what I mean (and maybe even believe that I don't say this stuff to shill for higher scores!).
Rather than looking at raw score numbers, I prefer to make them more meaningful by taking my position ranking relative to peers. Was I the #1 speaker in a question? Was I the #last speaker? That is the more salient data for me. Let's take a look at the raw data:
Q1 Overall satisfaction with the session
Q2 Usefulness of the information presented
Q3 Presenter's knowledge of the subject
Q4 Presenter's presentation skills
Q5 Effectiveness of the presentation
Q9 Room setup, audio visual and logistics
Also, here are a couple of verbatims I ended up using in my analysis:
"...I felt I could use the slides to cover all of the areas I needed to address for application compatibility testing."
"...After Chris' presentation and some extended discussion I am confident we can remediate many of our application compatibility issues in very short order."
"...Would love to see some Hands On Labs - ones that purposely have me 'stumble around' - even if it is 'directed stumbling around' The command line debugger stuff was great! I only wish it was a little slower or had step-by-step in the slides so I could practice at home."
I also had many discussions with folks at the conference. I didn't tape record them so I don't want to mislead you by using quotes, but the essence was that people were interested in having a better understanding of how to debug applications that the tools don't catch, and that UAC and running as a standard user are still not getting a lot of love.
Given all of that, here is my analysis:
I make a room look good. Want to make an inanimate object look better? Put me in it. I normally don't look at this one at all, but with a #1 score, how could I resist? I had to taunt Mark Russinovich (whose scores I am constantly chasing) with this one. "You're number 1 almost everywhere, except one attribute... <dramatic pause> ... room setup baby..." (I did get a high 5 out of that one.)
Don't kill off my project management session. Seriously, I was thinking about this. At MMS, it had very middle-of-the-pack scores. You don't pay your hard earned money to get mediocrity, so I was thinking of punting on this session. But I wanted to give it one more chance. This time around, instead of essentially apologizing for having a non-technical session, I gave the topic the love it deserved, and I think that came through. So, my plan now is to continue down this road adding more details, action steps, and materials you can use to better implement these best practices. I'm even talking to Joshua Hoffman about how we may do some work in this space in Technet Magazine. The end result is still "you have to make your own plan based on this guidance" so you don't walk away immediately doing something - could I help get you further down this road by doing more work myself? Let's give that a try.
Deep technical is the big winner. Maybe this is just me and my own passion coming through, but my 2nd shims presentation went super deep. I mean, I had a console debugger on the screen! And this one came out with an average score of 8.55 / 9.00 and the coveted #1 room setup score (I dare you to take that back from me, Mark!). So I'm planning to do a session just on debugging - I'll be trying it out at TechReady, our internal Microsoft technical conference, get the bugs out of it (or most of them at least) and then perhaps give it a go at TechEd EMEA.
What we're presenting is still useful. The scores here are just excellent, and that tells me that what we're talking about is still helping you do your job. I'm always keeping an eye on this one, and I was delighted with these scores.
I don't know enough. Now, this question is a bit vague, but it's my lowest score all around, so how do I interpret that? I'm going to guess that it doesn't refer to depth, since I'm the guy who writes the shim documentation, which can only leave breadth. Alternately, it could be that I didn't do a good enough job leaving you feeling smarter, and having actionable steps to project that knowledge. Note the verbatim that things were going too fast. In my personal opinion, a slide deck makes for a horrible document. But perhaps I could add better notes (I don't think I have any today.) And I could point you for the step by step on how to do the demos yourself. Want to replicate how I'm debugging viewing shims? Well, I wrote it out here: http://blogs.msdn.com/cjacks/archive/2007/07/05/windows-vista-shim-internals-basics-how-shims-work-to-address-compatibility-issues-and-what-are-the-ramifications.aspx. What I should consider doing is pointing out that it's there.
It's good to balance between doing and convincing. I took a guess that people wanted to know both about how to do the work as well as how to sell people on it. Nearly 1/4 of my 2nd Shims talk is about explaining what they are to people you need to convince to let you use them. The feedback seems to indicate that this was time well spent, and I should continue to help you both do the work and talk to the business about the work.
Make sure we have UAC covered the way customers use it. I'm not usually too far into this space, because Aaron Margosis just does a much better job of it. But one thing I'm noticing from customers (both from site visits and conferences) is that we've given far too much love to Protected Admin users, and not nearly enough to Standard Users. And many IT Pros running as protected admin aren't thinking of the impact for target standard users. Maybe we can get together. A couple years back we had a hit session on LUA Tools, and since then we've been struggling with a reprise. Most likely because our collaborations have been him doing the work and me commenting. Perhaps we can think of some content where, once again, we can split up the work and have another head to head. Whatever we do, we need to give more love to the Standard User. I even sat in on a session designed to talk about going standard user, and it had consent / cred prompts featured, even though in reality most customers set up standard users to auto-deny elevations. Let's fix that.
Seriously, that room looked awesome. Thanks to those of you who shared it with me. I think I got to #1 for the room because the most powerful and attractive people were all in my session! See you next time around!