WPF3D Team Blog : 3D

What's New in Graphics for 4.0 Beta 2

wpf3d — Wed, 21 Oct 2009 01:26:24 GMT

Last beta I mentioned we were saving the best for Beta 2 so thank you for waiting :)

New Features

The PixelShader class now accepts Pixel Shader 3.0 shaders when your hardware supports them. Important: There is no software rendering for PS 3.0. For example, you won't see PS 3.0 Effects over Remote Desktop, when printing, in RenderTargetBitmap, etc...
You can now tell us to cache the render output of an element tree in memory using what we've been calling the "cached composition" feature. The APIs for this feature are UIElement.CacheMode and BitmapCacheBrush. Think of this as hardware RenderTargetBitmap but without the BitmapSource APIs. Cached composition is more efficient than VisualBrush and allows you to specify a scale to render at if you know you'll be zooming. We'll have to write about this more later.
Brand new text rendering and layout. One of the biggest complaints about WPF has been small text "clarity" so we added "Display" mode text rendering which snaps to the pixel grid and looks just like the GDI rendering the rest of Windows uses. Rendering of East Asian fonts that have embedded bitmaps has also improved. All of the details are over at the WPF Text blog. You should be able to notice a big difference over Beta 1 in Visual Studio 2010.
My team didn't do this, but it helps us out quite a bit. WPF layout can position elements on sub-pixel boundaries which causes blurry bitmaps or seams. There have been specific workarounds for these things but no general fix. Now you can set layout rounding on the element tree root and walk away.

Some Notable Bug Fixes

We've fixed most of our D3D9D.dll issues. If you're on XP SP3 you'll still hit a problem that's out of our control. To workaround that or any other issue, you can copy d3d9d.dll to the exe folder, rename it to d3d9.dll, and check the "break on error" box in the control panel as usual but do not enable d3d9d.dll globally. Or you can use the checked D3D9.dll as mentioned in the release notes I just linked to.
Fixed all known D3DImage device lost and timing bugs. If you find more, let us know ASAP.
ColorContext should finally work in all scenarios! It can still throw exceptions when the embedded ColorContext is bad. I believe we shipped Beta 2 with some backwards compat exception issues that have been fixed for final release.
WriteableBitmap was not respecting dirty rect size. If your dirty region is small compared to the size of the entire WriteableBitmap, you will see CPU improvement.
Minor 3D performance improvements. We reduced the amount of DrawPrimitive() calls for large indexed meshes and slightly improved CPU usage for large Model3D counts.
We reduced the amount of GC.Collect(2) calls caused by bitmap allocations
The render thread no longer dies when you create "32 / number of monitors" WPF AppDomains.

A Notable Bug

Remember how I said we disabled hardware rendering if you didn't have PS 2.0? Well, that wasn't true. We thought we did but there was a bug and it's in Beta 2 as well. It'll be true for the final release.

It's also worth mentioning that .NET Framework 4.0 setup size has shrunk dramatically. The full x86 installer is down to ~38MB and the new "client" installer down to ~31MB. Full 3.5 SP1 is 231MB!

-- Jordan

WPF3D Lighting and Shading

wpf3d — Tue, 30 Jun 2009 04:54:00 GMT

We use the standard fixed-function Blinn-Phong model. You can read up on all of the equations here at MSDN.

If you have a Tier 2 card, we actually do our lighting in a vertex shader. If you don’t have a Tier 2 card, we do the lighting on the CPU. Why not just use D3D9’s fixed-function APIs you might ask? We wanted to be able to blend a texture (SpecularMaterial.Brush) only with the specular component and it turns out the D3D9 fixed-function APIs can’t do that.

An important thing to know is all Material Brushes are treated as textures even though they might not be implemented as such (e.g. SolidColorBrush). This means if the amount of light hitting a vertex is more than 1.0, it will be capped at 1.0 before being blended with the texture. This is an unfortunate behavior of the fixed function pipeline. I discussed this more in the color knobs post.

There is one thing that may be unique to out lighting model. Since we are a retained system, we factor in a Light’s transform to all of its properties. We approximate the scale being applied to the light by taking the cube root of the absolute value of the determinant of the upper 3x3 part of the Light’s transform. For a uniform scale this will be the actual scale but for non-uniform it’s good enough (the real solution is too costly for something uncommon). Internally, this number is factored into the range and attenuation values so things make sense. For example, if the Light undergoes a uniform scale of 2.0, we’ll multiple its range by 2.0.

-- Jordan

Transparent DiffuseMaterials and Depth Sorting

wpf3d — Fri, 19 Jun 2009 04:28:00 GMT

I hinted at this a long time ago and then forgot to follow up, whoops!

As the old post says, DiffuseMaterial writes to the depth buffer. This means if you draw one diffuse model and then draw another diffuse model behind it, the card knows not to draw the second one on top of the first because the first is closer. The depth buffer only knows about position and has no concept of transparency so if the first model is transparent, you won’t see the second model behind it because the depth buffer rejected it as being behind the first.

What does this mean? This means you have to sort your transparent DiffuseMaterials from back to front. If your scene is dynamic, you’ll have to do this any time the relationship between the transparent models and the camera changes. Peter gives an example of how to do this here.

-- Jordan

P.S. If you have a lot of overlapping models (a.k.a overdraw) you can sort your opaque models from front to back. This will prevent all of the obstructed models from being rasterized and will improve performance if rasterization is the bottleneck. If you have both opaque and transparent models, first draw the opaque ones then the transparent ones.

3.5 SP1 Software 3D Leak Fix Available

wpf3d — Tue, 09 Jun 2009 01:34:00 GMT

In 3.5 SP1 it was really easy to leak memory when doing software 3D rendering involving a VisualBrush or DrawingBrush. We got this complaint many times online and in person. Unfortunately, there is no work around. It’s described in “Issue 2” from this new Knowledge Base article. The fix should also be in 4.0 Beta 1.

-- Jordan

3D Hit Testing

wpf3d — Tue, 19 May 2009 04:22:00 GMT

How to do 3D hit testing has come up a bit recently in the forums but essentially it isn’t any different than 2D hit testing which is described on MSDN here. You can either start with a 2D point on the Viewport3D or a 3D point on a Visual3D.

Starting at the Viewport3D level is straightforward. Suppose you have a mouse down handler and you want it to hit test into 3D:

public void MouseHitTest(object sender, MouseButtonEventArgs args)
{
    Point mousePos = args.GetPosition(viewport3d);
    PointHitTestParameters hitParams = new PointHitTestParameters(mousePos);
    VisualTreeHelper.HitTest(viewport3d, null, ResultCallback, hitParams);
}

Visual3D is slightly trickier because a point is no longer sufficient in 3D space. You also need to specify a direction. An origin point plus a direction is known as a ray. It will look something like this:

    RayHitTestParameters hitParams = 
        new RayHitTestParameters(
            new Point3D(0, 0, 0),
            new Vector3D(1, 0, 0)
            );
    VisualTreeHelper.HitTest(visual3d, null, ResultCallback, hitParams);

The only difference is the HitTestParameters.

The second argument to HitTest is a filter which allows you to control which parts of the tree get tested. Skipping part of the tree is a perf win of course, but generally you’ll want to hit test everything and this is covered well in the MSDN article so I won’t do it here.

The third argument, the result callback, is where the action happens. We’ve hit something and we’ve decided to tell you about it. It could be 3D or it could be 2D. It’s up to you to decide what you’re looking for and when/if you want to stop.

public HitTestResultBehavior ResultCallback(HitTestResult result)
{
    // Did we hit 3D?
    RayHitTestResult rayResult = result as RayHitTestResult; 
    if (rayResult != null) 
    { 
        // Did we hit a MeshGeometry3D?
        RayMeshGeometry3DHitTestResult rayMeshResult = 
            rayResult as RayMeshGeometry3DHitTestResult;
                                                                                          
        if (rayMeshResult != null) 
        { 
            // Yes we did!
        } 
    } 
    
    return HitTestResultBehavior.Continue; 
}

RayMeshGeometry3DHitTestResult has MeshHit, PointHit, and DistanceToRayOrigin properties which tell you pretty much everything you want to know. It also has the triangle indices and weights (a.k.a. barycentric coordinates) so you can do things like figure out the texture coordinate of the point hit.

Performance Tip: As you mouse over 3D we do hit testing behind your back because there could be interactive 2D on the 3D or a 2D layer underneath the 3D (empty Viewport3D space is considered transparent for hit testing purposes). If you know you don’t need either of those, disable hit testing on the Viewport3D by setting its IsHitTestVisible property to false. This is an old tip but a good one :)

Sneaky Performance Tip: Can’t disable hit testing and it’s still eating up a bunch of CPU? Override HitTestCore() and, while the mouse is moving, only really issue the hit test every two or three (or more!) requests. This will introduce lag of course but that may be acceptable to your application.

-- Jordan

Transforming Bounds

wpf3d — Thu, 14 May 2009 01:25:00 GMT

Many haven’t realized this, but we added the ability to transform between 2D and 3D Visuals back in 3.5. This is handy if you need to draw 2D content around your 3D object or if you want to know the 2D position of a 3D point without doing a hit test. The methods are:

(Note that Visual and Visual3D already had TransformToFoo methods to move up and down their own trees.)

General transforms have a Transform method that operates on points and a TransformBounds method operates on rects for convenience. General transforms aren’t guaranteed to succeed so you need to be careful with them.

I wrote a little 3D XAML scene that animates a torus in a Canvas and then added this code to place a blue rectangle around the torus:

void CompositionTarget_Rendering(object sender, EventArgs e)
{
    GeneralTransform3DTo2D gt = visual3d.TransformToAncestor(viewport3d);
    // If null, the transform isn't possible at all
    if (gt != null)
    {
        Rect bounds = gt.TransformBounds(VisualTreeHelper.GetDescendantBounds(visual3d));
        // If empty, visual3d's specific bounds couldn't be transformed
        if (!bounds.IsEmpty)
        {
            rect.Width = bounds.Width;
            rect.Height = bounds.Height;
            Canvas.SetLeft(rect, bounds.Left);
            Canvas.SetTop(rect, bounds.Top);
        }
    }
}

And the result looks like:

The bounds you’ll get won’t necessarily be very tight, but they will always be inclusive.

You’ll notice that there isn’t a Visual.TransformToDescendant(Visual3D descendant) and that’s because when going from 2D to 3D you get an infinite number of points along a ray. This is when you need to resort to hit testing and pick the specific point you want.

-- Jordan

D3DImage and Software Rendering

wpf3d — Wed, 14 Jan 2009 05:17:00 GMT

If the WPF render thread is doing software rendering, D3DImage will not show up. I don't think we specifically called this out in the documentation so it can come as a surprise when you first encounter it. Off the top of my head, here are times when the render thread is in software:

RenderCapability.Tier == 0
You set HwndTarget.RenderMode = SoftwareOnly
A video card driver before November 2004 (date subject to change in the future)
Remote desktop
The window size is larger than the max texture size of the video card (rare)

All of these cases should be covered by simply checking #1 except for #2. You might be surprised by #4, but in 3.5 SP1 we had to change how remoting works.

You're probably thinking that I left out RenderTargetBitmap and printing since they both use software rendering, but they happen on the UI thread! D3DImage will show up because they call CopyBackBuffer().

-- Jordan

P.S. Don't forget to read the perf notes when using D3DImage

.NET 3.5 SP1 Graphics @ Channel 9

wpf3d — Wed, 14 May 2008 02:50:00 GMT

Sorry for the lack of updates, but we've been pretty busy. On what, you may ask? Our PM, David Teitlebaum, just did a video on Channel 9 showing off the new features. He starts by covering interactive 2D on 3D and improved layered window support, both of which were in 3.5, and he finishes with the customizable ImageEffects, the much improved WriteableBitmap, BitmapScalingMode.NearestNeighbor, and D3DImage.

-- Jordan

Blender to XAML Exporter Updated

wpf3d — Mon, 28 Jan 2008 22:56:00 GMT

Robert Hogue has updated the Python script with a bunch of new features. See this forum post for the instructions, tutorials, and demos!

-- Jordan

.NET 3.5 has been released!

wpf3d — Wed, 21 Nov 2007 01:42:00 GMT

Yesterday, .NET Framework 3.5 and Visual Studio 2008 went live on MSDN. You can see what's new in both here and download them here.

Since this is a graphics blog, here are the graphics-specific changes of note in 3.5:

New Graphics Features

UIElement3D
Interactive 2D on 3D: Viewport2DVisual3D
Transformation services on Visual3D
BitmapSource.DecodeFailed event
HwndTarget.RenderMode to enable software rendering per window
BitmapImage.UriCachePolicy and BitmapDecoder.Create(..., RequestCachePolicy) to control web request caching

Notable Graphics Performance Improvements

Less animation jitter
RenderTargetBitmap memory leaks plugged
Big layered window improvements when combined with this on Vista or this on XP
MeshGeometry3D hit testing up to 50% faster in common scenarios

Notable Graphics Bug Fixes

Numerous layered window problems resolved

Most of these changes will appear in 3.0 SP1 as well.

-- Jordan

Augmented Reality with WPF3D

wpf3d — Tue, 16 Oct 2007 02:57:00 GMT

Augmented Reality is the process of taking real world data, typically video, and enhancing it with computer graphics. Casey used WPF3D along with an AR toolkit and DirectShow to get some great results. Check out the sweet video!

-- Jordan

P.S. We finally got around to putting some links on the side.

Blender Exporter on CodePlex - Looking for volunteers...

wpf3d — Sun, 30 Sep 2007 22:59:00 GMT

In July I promised to track down the Blender exporter that was lost during the GotDotNet phase out. The last version of the export script is now hosted on CodePlex (here). I am embarrassed that it has taken me this long to do this, and even more so that the script is not up to date for the latest version of Blender.

This has forced me to admit that it's time to give someone else the opportunity to take ownership of this project. If you're interested, contact me.

- Daniel

Petzold.Media3D

wpf3d — Sat, 08 Sep 2007 07:16:00 GMT

Charles Petzold has posted his WPF3D library on the web. It includes sphere, cube, cylinder, torus, line, and teapot mesh generation.

Buying his book 3D Programming for Windows grants you royalty-free use of the library so be sure to check it out!

-- Jordan

Cel Shading

wpf3d — Sat, 08 Sep 2007 07:02:00 GMT

Charles Petzold has been experimenting with cel shading on his blog at the request of Chris Cavanagh (whom has updated his 3D physics XBAP btw). Though we do use shaders internally, WPF3D's API is fixed function so you have to dig out the ol' fixed function playbook to achieve fancier effects. The plays usually boil down to abusing textures :)

First, make a small, one dimensional texture containing the colors you want. Here are a couple of examples I've zoomed in on and highlighted the pixels to make things clearer. Why I'm using grayscale will make sense in a minute:

Next, we need a function that maps to [0, 1] and takes into account the light position relative to a vertex. How about the cosine of the angle to the light? Turns out we can compute that fairly quickly because it's equal to the dot product of the normal and the vector to the light divided by their lengths.

    Int32Collection idxs = mesh.TriangleIndices;
    Point3DCollection pts = mesh.Positions;
    Vector3DCollection nrms = mesh.Normals;
    PointCollection txs = mesh.TextureCoordinates;
    mesh.TextureCoordinates = null;
    for (int i = 0, count = idxs.Count; i < count; ++i)
    { 
        int idx = idxs[i]; 
        Vector3D toLight = lightPos - pts[idx]; 
        toLight.Normalize(); 

        // The normals are pre-normalized, no need to do it again  
        double dp = Vector3D.DotProduct(toLight, nrms[idx]); 
        
        // A negative dot product means the vertex is facing away  
        // from the light so let's set that to the darkest color  
        if (dp < 0) 
        { 
            dp = 0; 
        } 
        
        txs[idx] = new Point(dp, 0); 
    } 
    mesh.TextureCoordinates = txs;

Naturally, you're going to need to redo this calculation any time the relationship between the light and the mesh changes so it needs to be fast. This is why I go through all of that collection nonsense (read this).

So why did I use a grayscale texture? If I baked the color in, I'd have to make a new texture any time I wanted to change the color. Instead, I can use the color knobs to set the color and use the grayscale texture like a mask. In these photos, I have a single AmbientLight with a DiffuseMaterial with AmbientColor="green" and Brush equal to the one dimensional texture. I don't actually have a real point light in the scene which helps performance.

Being that it's per-vertex, more vertices means a higher quality image and you're going to see some popping when it's animated. The teapot doesn't have too many vertices so the spout and black line on the body don't look great. On the highly tesselated torus, those issues are gone but it's still hard to get a clean, crisp edge because of the linear interpolation of the texture.

-- Jordan

P.S. If you use a texture with two colors and instead dot the normal with the direction to the camera, you'll get silhouette edges but they could look really bad because of the per-vertex nature again.

Subclassing UIElement3D

wpf3d — Thu, 06 Sep 2007 04:20:00 GMT

Subclassing from UIElement3D to create your own elements that respond to input, focus and eventing is simple to do in 3.5. In this example we'll create a Sphere class which derives from UIElement3D and will show off some new features in the process.

Deriving from UIElement3D

The first step is to derive from UIElement3D:

    public class Sphere : UIElement3D
    {

Even though UIElement3D is declared abstract, there aren't any abstract methods you need to override. The above will compile - it just won't do anything interesting.

Rendering a Sphere

Although the above lets us derive from UIElement3D, we still don't have anything rendering to look like a sphere. Before going in to how to render the sphere, first we're going to want to create a couple of dependency properties to control how the sphere looks. These will be PhiDiv and ThetaDiv, to represent the number of slices to make in the horizontal and vertical directions respectively, as well as Radius, to represent the radius of the sphere. The code for these is below:

        /// <summary>
        /// The number of vertical slices to make on the sphere
        /// </summary>
        public static readonly DependencyProperty ThetaDivProperty =
            DependencyProperty.Register("ThetaDiv",
                                        typeof(int),
                                        typeof(Sphere),
                                        new PropertyMetadata(15, ThetaDivPropertyChanged));

        private static void ThetaDivPropertyChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
        {
            Sphere s = (Sphere)d;
            s.InvalidateModel();
        }

        public int ThetaDiv
        {
            get
            {
                return (int)GetValue(ThetaDivProperty);
            }

            set
            {
                SetValue(ThetaDivProperty, value);
            }
        }

        /// <summary>
        /// The number of horizontal slices to make on the sphere
        /// </summary>
        public static readonly DependencyProperty PhiDivProperty =
            DependencyProperty.Register("PhiDiv",
                                        typeof(int),
                                        typeof(Sphere),
                                        new PropertyMetadata(15, PhiDivPropertyChanged));

        private static void PhiDivPropertyChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
        {
            Sphere s = (Sphere)d;
            s.InvalidateModel();
        }

        public int PhiDiv
        {
            get
            {
                return (int)GetValue(PhiDivProperty);
            }

            set
            {
                SetValue(PhiDivProperty, value);
            }
        }

        /// <summary>
        /// The radius of the sphere
        /// </summary>
        public static readonly DependencyProperty RadiusProperty =
            DependencyProperty.Register("Radius",
                                        typeof(double),
                                        typeof(Sphere),
                                        new PropertyMetadata(1.0, RadiusPropertyChanged));

        private static void RadiusPropertyChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
        {
            Sphere s = (Sphere)d;
            s.InvalidateModel();
        }

        public double Radius
        {
            get
            {
                return (double)GetValue(RadiusProperty);
            }

            set
            {
                SetValue(RadiusProperty, value);
            }
        }

InvalidateModel, OnUpdateModel and Visual3DModel

The above should look like standard C# WPF code to create a few dependency properties, but the one new method that should look unfamiliar is the InvalidateModel call made whenever any of the above dependency properties changes. InvalidateModel is similar to the InvalidateVisual method that exists for the 2D UIElement. Calling InvalidateModel indicates that the 3D model representing the UIElement3D has changed and should be updated.

In response to InvalidateModel, OnUpdateModel will be called to update the 3D model that represents the object. In this way, you can make multiple changes to properties that affect the visual appearance of the UIElement3D and only make one final change to the model, rather than having to regenerate it each time a change is made. For instance, say changes are made to ThetaDiv, PhiDiv, and Radius above. Rather than having to regenerate the model each time, InvalidateModel can be called each time a property changes, and then all of the changes can be dealt with when OnUpdateModel is called. Of course, the model can also be changed in other places, but this provides one standard option to make the change.

The last thing that hasn't been discussed is how to actually change the model. In 3.5, Visual3D now exposes a protected CLR property Visual3DModel which represents the 3D model for the object. This is the 3D equivalent to the render data of a 2D Visual. To set what the visual representation for the Visual3D is then, it's necessary to set this property. For instance, ModelUIElement3D's Model property takes care of setting this. There is one subtle point about setting this property. Just like render data, setting this property won't set up the links necessary for things such as data bindings to work. To make this happen, you'll also want to have a dependency property for the model itself.

The code for InvalidateModel is shown below, as well as the Model dependency property:

        protected override void OnUpdateModel()
        {
            GeometryModel3D model = new GeometryModel3D();

            model.Geometry = Tessellate(ThetaDiv, PhiDiv, Radius);
            model.Material = new DiffuseMaterial(Brushes.Blue);

            Model = model;
        }

        /// <summary>
        /// The Model property for the sphere
        /// </summary>
        private static readonly DependencyProperty ModelProperty =
            DependencyProperty.Register("Model",
                                        typeof(Model3D),
                                        typeof(Sphere),
                                        new PropertyMetadata(ModelPropertyChanged));

        private static void ModelPropertyChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
        {
            Sphere s = (Sphere)d;
            s.Visual3DModel = (Model3D)e.NewValue;
        }

        private Model3D Model
        {
            get
            {
                return (Model3D)GetValue(ModelProperty);
            }

            set
            {
                SetValue(ModelProperty, value);
            }
        }

Attached to this post you'll find a simple example app which has the full code to the above Sphere class.