WPF3D Team Blog

What's New in Graphics for 4.0 Beta 2

2009-10-21T01:26:24Z

Last beta I mentioned we were saving the best for Beta 2 so thank you for waiting :)

New Features

The PixelShader class now accepts Pixel Shader 3.0 shaders when your hardware supports them. Important: There is no software rendering for PS 3.0. For example, you won't see PS 3.0 Effects over Remote Desktop, when printing, in RenderTargetBitmap, etc...
You can now tell us to cache the render output of an element tree in memory using what we've been calling the "cached composition" feature. The APIs for this feature are UIElement.CacheMode and BitmapCacheBrush. Think of this as hardware RenderTargetBitmap but without the BitmapSource APIs. Cached composition is more efficient than VisualBrush and allows you to specify a scale to render at if you know you'll be zooming. We'll have to write about this more later.
Brand new text rendering and layout. One of the biggest complaints about WPF has been small text "clarity" so we added "Display" mode text rendering which snaps to the pixel grid and looks just like the GDI rendering the rest of Windows uses. Rendering of East Asian fonts that have embedded bitmaps has also improved. All of the details are over at the WPF Text blog. You should be able to notice a big difference over Beta 1 in Visual Studio 2010.
My team didn't do this, but it helps us out quite a bit. WPF layout can position elements on sub-pixel boundaries which causes blurry bitmaps or seams. There have been specific workarounds for these things but no general fix. Now you can set layout rounding on the element tree root and walk away.

Some Notable Bug Fixes

We've fixed most of our D3D9D.dll issues. If you're on XP SP3 you'll still hit a problem that's out of our control. To workaround that or any other issue, you can copy d3d9d.dll to the exe folder, rename it to d3d9.dll, and check the "break on error" box in the control panel as usual but do not enable d3d9d.dll globally. Or you can use the checked D3D9.dll as mentioned in the release notes I just linked to.
Fixed all known D3DImage device lost and timing bugs. If you find more, let us know ASAP.
ColorContext should finally work in all scenarios! It can still throw exceptions when the embedded ColorContext is bad. I believe we shipped Beta 2 with some backwards compat exception issues that have been fixed for final release.
WriteableBitmap was not respecting dirty rect size. If your dirty region is small compared to the size of the entire WriteableBitmap, you will see CPU improvement.
Minor 3D performance improvements. We reduced the amount of DrawPrimitive() calls for large indexed meshes and slightly improved CPU usage for large Model3D counts.
We reduced the amount of GC.Collect(2) calls caused by bitmap allocations
The render thread no longer dies when you create "32 / number of monitors" WPF AppDomains.

A Notable Bug

Remember how I said we disabled hardware rendering if you didn't have PS 2.0? Well, that wasn't true. We thought we did but there was a bug and it's in Beta 2 as well. It'll be true for the final release.

It's also worth mentioning that .NET Framework 4.0 setup size has shrunk dramatically. The full x86 installer is down to ~38MB and the new "client" installer down to ~31MB. Full 3.5 SP1 is 231MB!

-- Jordan

WPF3D Lighting and Shading

2009-06-30T04:54:00Z

We use the standard fixed-function Blinn-Phong model. You can read up on all of the equations here at MSDN.

If you have a Tier 2 card, we actually do our lighting in a vertex shader. If you don’t have a Tier 2 card, we do the lighting on the CPU. Why not just use D3D9’s fixed-function APIs you might ask? We wanted to be able to blend a texture (SpecularMaterial.Brush) only with the specular component and it turns out the D3D9 fixed-function APIs can’t do that.

An important thing to know is all Material Brushes are treated as textures even though they might not be implemented as such (e.g. SolidColorBrush). This means if the amount of light hitting a vertex is more than 1.0, it will be capped at 1.0 before being blended with the texture. This is an unfortunate behavior of the fixed function pipeline. I discussed this more in the color knobs post.

There is one thing that may be unique to out lighting model. Since we are a retained system, we factor in a Light’s transform to all of its properties. We approximate the scale being applied to the light by taking the cube root of the absolute value of the determinant of the upper 3x3 part of the Light’s transform. For a uniform scale this will be the actual scale but for non-uniform it’s good enough (the real solution is too costly for something uncommon). Internally, this number is factored into the range and attenuation values so things make sense. For example, if the Light undergoes a uniform scale of 2.0, we’ll multiple its range by 2.0.

-- Jordan

What’s New in Graphics for 4.0 Beta 1

2009-06-25T03:48:00Z

.NET 4.0 Beta 1 was released a few weeks ago and there’s a lot to it that you can read about elsewhere so I thought I would just stick to what’s changed for WPF graphics. These are the biggest things of note:

RenderOptions.ClearTypeHint: If WPF renders text into a potentially transparent surface (e.g. a layered window like a menu or popup) we use grayscale anti-aliasing instead of ClearType because if the transparent surface is blended with another transparent surface, the ClearType will get messed up. Now with ClearTypeHint you can tell us to use ClearType and we’ll trust you that things aren’t transparent.
The BitmapEffect classes are now no-ops. They are still there so your apps will compile but don’t expect them to do anything.
The default RenderOptions.BitmapScalingMode (Unspecified) is now Linear instead of Fant. If you still want Fant, you can re-enable it.
Pixel Shader 2.0 is now required for hardware acceleration. Yes, if your card was Tier 1 but did not have PS 2.0 it is now Tier 0.
The memory leak that was easy to hit with software 3D has been fixed. I already mentioned this.
A common VisualBrush.Visual disconnect crash has been fixed. It’s in the same KB article as the memory leak.

That’s not much but that’s because we’ve been saving things for Beta 2. Stay tuned!

-- Jordan

Transparent DiffuseMaterials and Depth Sorting

2009-06-19T04:28:00Z

I hinted at this a long time ago and then forgot to follow up, whoops!

As the old post says, DiffuseMaterial writes to the depth buffer. This means if you draw one diffuse model and then draw another diffuse model behind it, the card knows not to draw the second one on top of the first because the first is closer. The depth buffer only knows about position and has no concept of transparency so if the first model is transparent, you won’t see the second model behind it because the depth buffer rejected it as being behind the first.

What does this mean? This means you have to sort your transparent DiffuseMaterials from back to front. If your scene is dynamic, you’ll have to do this any time the relationship between the transparent models and the camera changes. Peter gives an example of how to do this here.

-- Jordan

P.S. If you have a lot of overlapping models (a.k.a overdraw) you can sort your opaque models from front to back. This will prevent all of the obstructed models from being rasterized and will improve performance if rasterization is the bottleneck. If you have both opaque and transparent models, first draw the opaque ones then the transparent ones.

3.5 SP1 Software 3D Leak Fix Available

2009-06-09T01:34:00Z

In 3.5 SP1 it was really easy to leak memory when doing software 3D rendering involving a VisualBrush or DrawingBrush. We got this complaint many times online and in person. Unfortunately, there is no work around. It’s described in “Issue 2” from this new Knowledge Base article. The fix should also be in 4.0 Beta 1.

-- Jordan

3D Hit Testing

2009-05-19T04:22:00Z

How to do 3D hit testing has come up a bit recently in the forums but essentially it isn’t any different than 2D hit testing which is described on MSDN here. You can either start with a 2D point on the Viewport3D or a 3D point on a Visual3D.

Starting at the Viewport3D level is straightforward. Suppose you have a mouse down handler and you want it to hit test into 3D:

public void MouseHitTest(object sender, MouseButtonEventArgs args)
{
    Point mousePos = args.GetPosition(viewport3d);
    PointHitTestParameters hitParams = new PointHitTestParameters(mousePos);
    VisualTreeHelper.HitTest(viewport3d, null, ResultCallback, hitParams);
}

Visual3D is slightly trickier because a point is no longer sufficient in 3D space. You also need to specify a direction. An origin point plus a direction is known as a ray. It will look something like this:

    RayHitTestParameters hitParams = 
        new RayHitTestParameters(
            new Point3D(0, 0, 0),
            new Vector3D(1, 0, 0)
            );
    VisualTreeHelper.HitTest(visual3d, null, ResultCallback, hitParams);

The only difference is the HitTestParameters.

The second argument to HitTest is a filter which allows you to control which parts of the tree get tested. Skipping part of the tree is a perf win of course, but generally you’ll want to hit test everything and this is covered well in the MSDN article so I won’t do it here.

The third argument, the result callback, is where the action happens. We’ve hit something and we’ve decided to tell you about it. It could be 3D or it could be 2D. It’s up to you to decide what you’re looking for and when/if you want to stop.

public HitTestResultBehavior ResultCallback(HitTestResult result)
{
    // Did we hit 3D?
    RayHitTestResult rayResult = result as RayHitTestResult; 
    if (rayResult != null) 
    { 
        // Did we hit a MeshGeometry3D?
        RayMeshGeometry3DHitTestResult rayMeshResult = 
            rayResult as RayMeshGeometry3DHitTestResult;
                                                                                          
        if (rayMeshResult != null) 
        { 
            // Yes we did!
        } 
    } 
    
    return HitTestResultBehavior.Continue; 
}

RayMeshGeometry3DHitTestResult has MeshHit, PointHit, and DistanceToRayOrigin properties which tell you pretty much everything you want to know. It also has the triangle indices and weights (a.k.a. barycentric coordinates) so you can do things like figure out the texture coordinate of the point hit.

Performance Tip: As you mouse over 3D we do hit testing behind your back because there could be interactive 2D on the 3D or a 2D layer underneath the 3D (empty Viewport3D space is considered transparent for hit testing purposes). If you know you don’t need either of those, disable hit testing on the Viewport3D by setting its IsHitTestVisible property to false. This is an old tip but a good one :)

Sneaky Performance Tip: Can’t disable hit testing and it’s still eating up a bunch of CPU? Override HitTestCore() and, while the mouse is moving, only really issue the hit test every two or three (or more!) requests. This will introduce lag of course but that may be acceptable to your application.

-- Jordan

Transforming Bounds

2009-05-14T01:25:00Z

Many haven’t realized this, but we added the ability to transform between 2D and 3D Visuals back in 3.5. This is handy if you need to draw 2D content around your 3D object or if you want to know the 2D position of a 3D point without doing a hit test. The methods are:

(Note that Visual and Visual3D already had TransformToFoo methods to move up and down their own trees.)

General transforms have a Transform method that operates on points and a TransformBounds method operates on rects for convenience. General transforms aren’t guaranteed to succeed so you need to be careful with them.

I wrote a little 3D XAML scene that animates a torus in a Canvas and then added this code to place a blue rectangle around the torus:

void CompositionTarget_Rendering(object sender, EventArgs e)
{
    GeneralTransform3DTo2D gt = visual3d.TransformToAncestor(viewport3d);
    // If null, the transform isn't possible at all
    if (gt != null)
    {
        Rect bounds = gt.TransformBounds(VisualTreeHelper.GetDescendantBounds(visual3d));
        // If empty, visual3d's specific bounds couldn't be transformed
        if (!bounds.IsEmpty)
        {
            rect.Width = bounds.Width;
            rect.Height = bounds.Height;
            Canvas.SetLeft(rect, bounds.Left);
            Canvas.SetTop(rect, bounds.Top);
        }
    }
}

And the result looks like:

The bounds you’ll get won’t necessarily be very tight, but they will always be inclusive.

You’ll notice that there isn’t a Visual.TransformToDescendant(Visual3D descendant) and that’s because when going from 2D to 3D you get an infinite number of points along a ray. This is when you need to resort to hit testing and pick the specific point you want.

-- Jordan

D3DImage and Software Rendering

2009-01-14T05:17:00Z

If the WPF render thread is doing software rendering, D3DImage will not show up. I don't think we specifically called this out in the documentation so it can come as a surprise when you first encounter it. Off the top of my head, here are times when the render thread is in software:

RenderCapability.Tier == 0
You set HwndTarget.RenderMode = SoftwareOnly
A video card driver before November 2004 (date subject to change in the future)
Remote desktop
The window size is larger than the max texture size of the video card (rare)

All of these cases should be covered by simply checking #1 except for #2. You might be surprised by #4, but in 3.5 SP1 we had to change how remoting works.

You're probably thinking that I left out RenderTargetBitmap and printing since they both use software rendering, but they happen on the UI thread! D3DImage will show up because they call CopyBackBuffer().

-- Jordan

P.S. Don't forget to read the perf notes when using D3DImage

.NET 3.5 SP1 Graphics @ Channel 9

2008-05-14T02:50:00Z

Sorry for the lack of updates, but we've been pretty busy. On what, you may ask? Our PM, David Teitlebaum, just did a video on Channel 9 showing off the new features. He starts by covering interactive 2D on 3D and improved layered window support, both of which were in 3.5, and he finishes with the customizable ImageEffects, the much improved WriteableBitmap, BitmapScalingMode.NearestNeighbor, and D3DImage.

-- Jordan

Blender to XAML Exporter Updated

2008-01-28T22:56:00Z

Robert Hogue has updated the Python script with a bunch of new features. See this forum post for the instructions, tutorials, and demos!

-- Jordan

.NET 3.5 has been released!

2007-11-21T01:42:00Z

Yesterday, .NET Framework 3.5 and Visual Studio 2008 went live on MSDN. You can see what's new in both here and download them here.

Since this is a graphics blog, here are the graphics-specific changes of note in 3.5:

New Graphics Features

UIElement3D
Interactive 2D on 3D: Viewport2DVisual3D
Transformation services on Visual3D
BitmapSource.DecodeFailed event
HwndTarget.RenderMode to enable software rendering per window
BitmapImage.UriCachePolicy and BitmapDecoder.Create(..., RequestCachePolicy) to control web request caching

Notable Graphics Performance Improvements

Less animation jitter
RenderTargetBitmap memory leaks plugged
Big layered window improvements when combined with this on Vista or this on XP
MeshGeometry3D hit testing up to 50% faster in common scenarios

Notable Graphics Bug Fixes

Numerous layered window problems resolved

Most of these changes will appear in 3.0 SP1 as well.

-- Jordan

Augmented Reality with WPF3D

2007-10-16T02:57:00Z

Augmented Reality is the process of taking real world data, typically video, and enhancing it with computer graphics. Casey used WPF3D along with an AR toolkit and DirectShow to get some great results. Check out the sweet video!

-- Jordan

P.S. We finally got around to putting some links on the side.

Blender Exporter on CodePlex - Looking for volunteers...

2007-09-30T22:59:00Z

In July I promised to track down the Blender exporter that was lost during the GotDotNet phase out. The last version of the export script is now hosted on CodePlex (here). I am embarrassed that it has taken me this long to do this, and even more so that the script is not up to date for the latest version of Blender.

This has forced me to admit that it's time to give someone else the opportunity to take ownership of this project. If you're interested, contact me.

- Daniel

Petzold.Media3D

2007-09-08T07:16:00Z

Charles Petzold has posted his WPF3D library on the web. It includes sphere, cube, cylinder, torus, line, and teapot mesh generation.

Buying his book 3D Programming for Windows grants you royalty-free use of the library so be sure to check it out!

-- Jordan

Cel Shading

2007-09-08T07:02:00Z

Charles Petzold has been experimenting with cel shading on his blog at the request of Chris Cavanagh (whom has updated his 3D physics XBAP btw). Though we do use shaders internally, WPF3D's API is fixed function so you have to dig out the ol' fixed function playbook to achieve fancier effects. The plays usually boil down to abusing textures :)

First, make a small, one dimensional texture containing the colors you want. Here are a couple of examples I've zoomed in on and highlighted the pixels to make things clearer. Why I'm using grayscale will make sense in a minute:

Next, we need a function that maps to [0, 1] and takes into account the light position relative to a vertex. How about the cosine of the angle to the light? Turns out we can compute that fairly quickly because it's equal to the dot product of the normal and the vector to the light divided by their lengths.

    Int32Collection idxs = mesh.TriangleIndices;
    Point3DCollection pts = mesh.Positions;
    Vector3DCollection nrms = mesh.Normals;
    PointCollection txs = mesh.TextureCoordinates;
    mesh.TextureCoordinates = null;
    for (int i = 0, count = idxs.Count; i < count; ++i)
    { 
        int idx = idxs[i]; 
        Vector3D toLight = lightPos - pts[idx]; 
        toLight.Normalize(); 

        // The normals are pre-normalized, no need to do it again  
        double dp = Vector3D.DotProduct(toLight, nrms[idx]); 
        
        // A negative dot product means the vertex is facing away  
        // from the light so let's set that to the darkest color  
        if (dp < 0) 
        { 
            dp = 0; 
        } 
        
        txs[idx] = new Point(dp, 0); 
    } 
    mesh.TextureCoordinates = txs;

Naturally, you're going to need to redo this calculation any time the relationship between the light and the mesh changes so it needs to be fast. This is why I go through all of that collection nonsense (read this).

So why did I use a grayscale texture? If I baked the color in, I'd have to make a new texture any time I wanted to change the color. Instead, I can use the color knobs to set the color and use the grayscale texture like a mask. In these photos, I have a single AmbientLight with a DiffuseMaterial with AmbientColor="green" and Brush equal to the one dimensional texture. I don't actually have a real point light in the scene which helps performance.

Being that it's per-vertex, more vertices means a higher quality image and you're going to see some popping when it's animated. The teapot doesn't have too many vertices so the spout and black line on the body don't look great. On the highly tesselated torus, those issues are gone but it's still hard to get a clean, crisp edge because of the linear interpolation of the texture.

-- Jordan

P.S. If you use a texture with two colors and instead dot the normal with the direction to the camera, you'll get silhouette edges but they could look really bad because of the per-vertex nature again.