SilverLite : media

Getting to the Media FastPath

seemar — Sat, 08 Nov 2008 05:15:00 GMT

Earlier this week, us in the Silverlight perf and media teams reviewed Ben Waggoner's tutorial on how to build fast media players, check it out here:

http://on10.net/blogs/benwagg/Building-high-performance-Silverlight-Media-Players/

Tis a great how to guide.

(now I'm really going home. Seema)

EnableFramerateCounter, Setting your Framerate correctly

seemar — Sat, 08 Nov 2008 04:39:00 GMT

Debug tool
I posted about EnableFramerateCounter before -- it is a simple little debugging setting that you can use when trying to figure out:

Which of these designs hurts my framerate more?
If I add this module in, does it hurt my framerate?

To figure these out, you can:

Set the plugin's desired framerate at 1000 (silverlightObject.settings.MaxFrameRate OR Silverlight.CreateObjectEx({properties:{framerate:'framerate'}}))
Turn on the EnableFramerateCounter, which should show up in the status bar (IE/Firefox --> View --> Status Bar).

Desired Framerates
Make sure to reset the framerate to the suggested framerates for your app: Media apps @60fps or RIA/Control apps @20fps before launching your app to the public.
When trying to figure out the desired framerate of an animation or control-heavy app, use real people (not developers/designers) to gauge whether increasing the fps above 20fps makes sense. If you set your framerate at an unnecessarily high value, it will throttle the computer's cycles -- and could result in a jerkier effect than if you just had a lower framerate. So where 60fps might be slick and easy for your devbox, it could be a pain for your customer's machine -- so only use what you need.

The desired framerate of plugin hosting media is always 60fps -- this gives the media pipeline the most number of ticks to figure out the optimal timing of presenting a frame. Silverlight will decode at the framerate of the encoded video (this usually is <32fps), and we present the frames in sync with the audio. To summarize roughly, if the platform doesn't receive a tick in sync with the audio, then the platform will decode the frame, and then just throw it away. If you don't understand this, just trust me and use framerate=60.

Technically, you could set the framerate dynamically -- I haven't tried this, but it might be a useful thing to try out if your app has both media and heavy controls/animations.

Debugging Settings

So here are the debugging settings that I use (not all at once, of course)

Sys.Silverlight.createObjectEx({

     source: "xaml/Scene.xaml",

     parentElement: document.getElementById("SilverlightControlHost"),

     id: "SilverlightControl",

     properties: {

          width: "500", height: "500",

         isWindowless: false,

          background: "FF000000",

          framerate: "10000"      },

     events: { onLoad:onLoadHandler} });

function onLoadHandler() {

     agControl = document.getElementById("SilverlightControl");

     /* To see the framerate displayed in the browser status bar */

     agControl.settings.EnableFrameRateCounter = true;

     /* To see the redraw regions*/

     agControl.settings.EnableRedrawRegions= true;

}

I accidentally dumped the sugar canister into my tea, time to go home.
Seema

PDC: Deep Dive into building an optimized graphics-intensive SL app

seemar — Mon, 03 Nov 2008 22:57:00 GMT

Thank you everyone for attending. For those who could not make it, you can check out my talk here on channel9, it was PC06:

In this deep dive session we pull the covers off of the layout, rendering, and media pipelines to discuss how things work and how to optimize your application. This session covers how to set goals with your designer and developer teams, what happens behind the scenes with control visuals, the underlying mechanisms of our media stack, and how to profile your application.

My deck is here, my halo video player is here. The instructions to use xperf are available at this post.

Please fill out an eval after watching. I was honored to do a talk at PDC and I do read and appreciate all feedback.

Cheers, S

XPerf: A CPU Sampler for Silverlight

seemar — Thu, 09 Oct 2008 07:59:00 GMT

For those of you who are a) building graphics-intensive applications or b) trying to debug your performance, I would like to introduce xperf and xperfview. These are two profiling tools which can be used to analyze the performance of any Microsoft technology, including Silverlight.

These tools have existed internally for quite some time, and I'm excited to see that they have been released. I use xperf as my first step for profiling any app -- it's a simple CPU sampler and gives you a peek to why on earth something is taking up so many CPU cycles. XPerf plugs into the Event Tracing for Windows library, and listens to all the internally embedded events that all Windows products trigger. With the publicly available Silverlight symbols, you can see how these events line back to what Silverlight is doing under the hood, while your app is running.

Note: with xperf, managed code (your app code) will show up as samples on the methods that your app code calls. XPerf does not have the capabilities to show CPU cycles in managed code.

With XPerf, one can find answers to questions like:

Is my app asking Silverlight to constantly spin on CPU cycles.
Whether one UI layout or design is more expensive than the other.
Whether the time is spent in drawing (agcore.dll), the plug-in’s interactions with (npctrl.dll), or in compilation/JIT (coreclr.dll)
How much is stretching/blending/rotating that video going to cost in terms of CPU cycles? How about if you encode it differently?

Please note that the tools only work on Vista and Windows Server 2k3. For profiling on a mac, see my previous post on how to use Shark from the Apple CHUD tools.

The first-time instructions are as follows:

1. Install XPerf (formerly xperfinfo) and xperfview (formerly xperf) as available here: http://msdn.microsoft.com/en-us/library/cc305187.aspx

[10/9-9am I updated the above link: download link http://www.microsoft.com/whdc/system/sysperf/perftools.mspx]

Close everything that is not user-specific (close VS, but keep your virus checker, etc).

2. Startup your sample

I personally start up my sample without profiling because the startup sampling is very different from that of the runtime.

3. Open an Administrator-level command prompt. Set your symbol path in the cmd shell:
set _NT_SYMBOL_PATH= srv*C:\symbols*http://msdl.microsoft.com/downloads/symbols

This will set your symbol path to point to the Microsoft symbol server. The symbol server hosts debugging symbols for most all Microsoft products.

[update: cd into C:\Program Files\Microsoft Windows Performance Toolkit]

4. xperf -on base

Run the app for maybe 10 seconds for this initial run. We want to understand what method is taking up the most cycles. At the end of this, you will see a graph of CPU usage over time – so you can use various methods to see where time is going: repeatedly execute a user action (expand/collapse, scroll, etc), let the media play for a while, etc. If you are trying to compare several different options, you can execute different actions at widely-spaced intervals, and compare the graphs for each.

5. xperf –d myprofile.etl

Step #4 will both stop the profile and start the writing of the event trace log (etl) file. This can take a while, depending on the complexity of your application, how many other apps were running, and how long the profile extended.

6. xperfview myprofile.etl

We see now see multiple graphs. We are most interested in that of the CPU cycles, which will look something like this:

In my sample, I have a dual-core machine so you see two lines, one for each CPU. This graph is of a media file playing, which should have an almost steady hit for the CPU. I can drag and select any section of the graph and take a closer look:

7. TraceàLoad Symbols

8. Select the area of the CPU graph that you want to see

Right-click and select Summary Table

9. Accept the EULA for using symbols, expand IExplore.exe (or firefox.exe)

At this point, you’ll see how the time breaks down between the different processes and modules running within windows. Agcore.dll, npctrl.dll, and coreclr.dll are all Silverlight.

10. If the symbols server has correctly been set, you’ll see that you can expand and see the breakdown of time in each method call within agcore.dll:

The way to read this graph is that IE is taking up 40% of the CPU on average, during the selected sample. Within that, 32% of the time is spent in Silverlight core (graphics/decoding/property engine, etc are all represented in agcore.dll), and then of that time, 29% (9.24/32.33) of the drawing time is spent in one method call.

Note: with xperf, managed code (your app code) will show up as samples on the methods that your app code calls. XPerf does not have the capabilities to look at app code.

Now what? Well, with just the above think about how you can answer the questions at the top:

Is my app asking Silverlight to constantly spin on CPU cycles.
Do I see activity when the app does not seem to be doing anything? Where does the summary table say that time is going? Perhaps turn on EnableRedrawRegions and see if there is a draw being forced.
Whether one UI layout or design is more expensive than the other.
When I scroll now, is my average CPU higher or lower than before?
Whether the time is spent in drawing (agcore.dll), the plug-in’s interactions with (npctrl.dll), or in compilation/JIT (coreclr.dll)
How much is stretching/blending/rotating that video going to cost in terms of CPU cycles? How about if you encode it differently?
Are there new methods that you see in the stack, now that you have stretched the media? Blended it? What is the average time for CPU cycles compared to your baseline? I know that it is cheaper for me to embed an icon overlay directly into the video, but what does it really cost in terms of CPU time?

Note: The tools only work on Vista and Windows Server 2k3

For later usage, your usage pattern is quicker:

Set your symbol path in an Admin-level cmd shell:
set _NT_SYMBOL_PATH= srv*C:\symbols*http://msdl.microsoft.com/downloads/symbols
xperf -on base
xperf –d myprofile.etl
xperfview myprofile.etl

The platform code and the CPU samples will change version to version, so please keep that in mind when profiling between upgrades.

See anything interesting? Drop me a note!

Have fun!

.seema.

Perf Debugging Tips: EnableRedrawRegions & a performance bug in VideoBrush

seemar — Mon, 08 Oct 2007 04:50:00 GMT

Was chatting with Andy Beaulieu at Remix Boston, and he was commenting that it seems that Silverlight only draws when needed -- it is true, we try to not waste your CPU cycles. For Perf debugging, a way to tell when you are causing a redraw is to turn on the control's EnableRedrawRegions property.

agControl.settings.EnableRedrawRegions = true;

With this feature on, when a section of the plugin causes a draw, that section will draw in a different color. This setting is not for those susceptible to seizures =P

Bug found: With this setting, I investigated a performance issue with VideoBrush. In our 1.0 Silverlight bits, VideoBrush always requests a redraw in the next frame. This is a bug. Unfortunately, if the control framerate is set to the default (60 fps), then any VideoBrush will be redrawn 60 times in a second. In actuality, VideoBrush should draw at the framerate that its media is getting refreshed (eg. 30fps, 15fps, or 0fps if paused), and should not take up so many CPU cycles. We have a fix for this bug in 1.1. If you are blocked by this bug, please let me know and I can pass the info along to our servicing team.

[update] By the way, this bug was fixed in a servicing pack in dec 2007.

A workaround is to set the framerate on your Silverlight control to be the same as the framerate of your video. There are visual artifacts to this workaround, but it is slight

Sys.Silverlight.createObjectEx({

source: "xaml/Scene.xaml",

parentElement: document.getElementById("SilverlightControlHost"),

id: "SilverlightControl",

properties: {

width: "500",

height: "500",

background: "black",

framerate: "30" //only as much as needed