Welcome to MSDN Blogs Sign in | Join | Help

WPF3D Lighting and Shading

We use the standard fixed-function Blinn-Phong model. You can read up on all of the equations here at MSDN.

If you have a Tier 2 card, we actually do our lighting in a vertex shader. If you don’t have a Tier 2 card, we do the lighting on the CPU. Why not just use D3D9’s fixed-function APIs you might ask? We wanted to be able to blend a texture (SpecularMaterial.Brush) only with the specular component and it turns out the D3D9 fixed-function APIs can’t do that.

An important thing to know is all Material Brushes are treated as textures even though they might not be implemented as such (e.g. SolidColorBrush). This means if the amount of light hitting a vertex is more than 1.0, it will be capped at 1.0 before being blended with the texture. This is an unfortunate behavior of the fixed function pipeline. I discussed this more in the color knobs post.

There is one thing that may be unique to out lighting model. Since we are a retained system, we factor in a Light’s transform to all of its properties. We approximate the scale being applied to the light by taking the cube root of the absolute value of the determinant of the upper 3x3 part of the Light’s transform. For a uniform scale this will be the actual scale but for non-uniform it’s good enough (the real solution is too costly for something uncommon). Internally, this number is factored into the range and attenuation values so things make sense. For example, if the Light undergoes a uniform scale of 2.0, we’ll multiple its range by 2.0.

-- Jordan

Posted by wpf3d | 0 Comments
Filed under: , , , ,

What’s New in Graphics for 4.0 Beta 1

.NET 4.0 Beta 1 was released a few weeks ago and there’s a lot to it that you can read about elsewhere so I thought I would just stick to what’s changed for WPF graphics. These are the biggest things of note:

  • RenderOptions.ClearTypeHint: If WPF renders text into a potentially transparent surface (e.g. a layered window like a menu or popup) we use grayscale anti-aliasing instead of ClearType because if the transparent surface is blended with another transparent surface, the ClearType will get messed up. Now with ClearTypeHint you can tell us to use ClearType and we’ll trust you that things aren’t transparent.
  • The BitmapEffect classes are now no-ops. They are still there so your apps will compile but don’t expect them to do anything.
  • The default RenderOptions.BitmapScalingMode (Unspecified) is now Linear instead of Fant. If you still want Fant, you can re-enable it.
  • Pixel Shader 2.0 is now required for hardware acceleration. Yes, if your card was Tier 1 but did not have PS 2.0 it is now Tier 0.
  • The memory leak that was easy to hit with software 3D has been fixed. I already mentioned this.
  • A common VisualBrush.Visual disconnect crash has been fixed. It’s in the same KB article as the memory leak.

That’s not much but that’s because we’ve been saving things for Beta 2. Stay tuned!

-- Jordan

Posted by wpf3d | 9 Comments
Filed under: ,

Transparent DiffuseMaterials and Depth Sorting

I hinted at this a long time ago and then forgot to follow up, whoops!

As the old post says, DiffuseMaterial writes to the depth buffer. This means if you draw one diffuse model and then draw another diffuse model behind it, the card knows not to draw the second one on top of the first because the first is closer. The depth buffer only knows about position and has no concept of transparency so if the first model is transparent, you won’t see the second model behind it because the depth buffer rejected it as being behind the first.

What does this mean? This means you have to sort your transparent DiffuseMaterials from back to front. If your scene is dynamic, you’ll have to do this any time the relationship between the transparent models and the camera changes. Peter gives an example of how to do this here.

-- Jordan

P.S. If you have a lot of overlapping models (a.k.a overdraw) you can sort your opaque models from front to back. This will prevent all of the obstructed models from being rasterized and will improve performance if rasterization is the bottleneck. If you have both opaque and transparent models, first draw the opaque ones then the transparent ones.

Posted by wpf3d | 0 Comments
Filed under: , , ,

3.5 SP1 Software 3D Leak Fix Available

In 3.5 SP1 it was really easy to leak memory when doing software 3D rendering involving a VisualBrush or DrawingBrush. We got this complaint many times online and in person. Unfortunately, there is no work around. It’s described in “Issue 2” from this new Knowledge Base article. The fix should also be in 4.0 Beta 1.

-- Jordan

Posted by wpf3d | 4 Comments

3D Hit Testing

How to do 3D hit testing has come up a bit recently in the forums but essentially it isn’t any different than 2D hit testing which is described on MSDN here. You can either start with a 2D point on the Viewport3D or a 3D point on a Visual3D.

Starting at the Viewport3D level is straightforward. Suppose you have a mouse down handler and you want it to hit test into 3D:

public void MouseHitTest(object sender, MouseButtonEventArgs args)
{
    Point mousePos = args.GetPosition(viewport3d);
    PointHitTestParameters hitParams = new PointHitTestParameters(mousePos);
    VisualTreeHelper.HitTest(viewport3d, null, ResultCallback, hitParams);
}

Visual3D is slightly trickier because a point is no longer sufficient in 3D space. You also need to specify a direction. An origin point plus a direction is known as a ray. It will look something like this:

    RayHitTestParameters hitParams = 
        new RayHitTestParameters(
            new Point3D(0, 0, 0),
            new Vector3D(1, 0, 0)
            );
    VisualTreeHelper.HitTest(visual3d, null, ResultCallback, hitParams);

The only difference is the HitTestParameters.

The second argument to HitTest is a filter which allows you to control which parts of the tree get tested. Skipping part of the tree is a perf win of course, but generally you’ll want to hit test everything and this is covered well in the MSDN article so I won’t do it here.

The third argument, the result callback, is where the action happens. We’ve hit something and we’ve decided to tell you about it. It could be 3D or it could be 2D. It’s up to you to decide what you’re looking for and when/if you want to stop.

public HitTestResultBehavior ResultCallback(HitTestResult result)
{
    // Did we hit 3D?
    RayHitTestResult rayResult = result as RayHitTestResult; 
    if (rayResult != null) 
    { 
        // Did we hit a MeshGeometry3D?
        RayMeshGeometry3DHitTestResult rayMeshResult = 
            rayResult as RayMeshGeometry3DHitTestResult;
                                                                                          
        if (rayMeshResult != null) 
        { 
            // Yes we did!
        } 
    } 
    
    return HitTestResultBehavior.Continue; 
}

RayMeshGeometry3DHitTestResult has MeshHit, PointHit, and DistanceToRayOrigin properties which tell you pretty much everything you want to know. It also has the triangle indices and weights (a.k.a. barycentric coordinates) so you can do things like figure out the texture coordinate of the point hit.

Performance Tip: As you mouse over 3D we do hit testing behind your back because there could be interactive 2D on the 3D or a 2D layer underneath the 3D (empty Viewport3D space is considered transparent for hit testing purposes). If you know you don’t need either of those, disable hit testing on the Viewport3D by setting its IsHitTestVisible property to false. This is an old tip but a good one :)

Sneaky Performance Tip: Can’t disable hit testing and it’s still eating up a bunch of CPU? Override HitTestCore() and, while the mouse is moving, only really issue the hit test every two or three (or more!) requests. This will introduce lag of course but that may be acceptable to your application.

-- Jordan

Posted by wpf3d | 2 Comments
Filed under: , , ,

Transforming Bounds

Many haven’t realized this, but we added the ability to transform between 2D and 3D Visuals back in 3.5. This is handy if you need to draw 2D content around your 3D object or if you want to know the 2D position of a 3D point without doing a hit test. The methods are:

(Note that Visual and Visual3D already had TransformToFoo methods to move up and down their own trees.)

General transforms have a Transform method that operates on points and a TransformBounds method operates on rects for convenience. General transforms aren’t guaranteed to succeed so you need to be careful with them.

I wrote a little 3D XAML scene that animates a torus in a Canvas and then added this code to place a blue rectangle around the torus:

void CompositionTarget_Rendering(object sender, EventArgs e)
{
    GeneralTransform3DTo2D gt = visual3d.TransformToAncestor(viewport3d);
    // If null, the transform isn't possible at all
    if (gt != null)
    {
        Rect bounds = gt.TransformBounds(VisualTreeHelper.GetDescendantBounds(visual3d));
        // If empty, visual3d's specific bounds couldn't be transformed
        if (!bounds.IsEmpty)
        {
            rect.Width = bounds.Width;
            rect.Height = bounds.Height;
            Canvas.SetLeft(rect, bounds.Left);
            Canvas.SetTop(rect, bounds.Top);
        }
    }
}

And the result looks like:

torusBounds

The bounds you’ll get won’t necessarily be very tight, but they will always be inclusive.

You’ll notice that there isn’t a Visual.TransformToDescendant(Visual3D descendant) and that’s because when going from 2D to 3D you get an infinite number of points along a ray. This is when you need to resort to hit testing and pick the specific point you want.

-- Jordan

Posted by wpf3d | 1 Comments
Filed under: , ,

D3DImage and Software Rendering

If the WPF render thread is doing software rendering, D3DImage will not show up. I don't think we specifically called this out in the documentation so it can come as a surprise when you first encounter it. Off the top of my head, here are times when the render thread is in software:

  1. RenderCapability.Tier == 0
  2. You set HwndTarget.RenderMode = SoftwareOnly
  3. A video card driver before November 2004 (date subject to change in the future)
  4. Remote desktop
  5. The window size is larger than the max texture size of the video card (rare)

All of these cases should be covered by simply checking #1 except for #2. You might be surprised by #4, but in 3.5 SP1 we had to change how remoting works.

You're probably thinking that I left out RenderTargetBitmap and printing since they both use software rendering, but they happen on the UI thread! D3DImage will show up because they call CopyBackBuffer().

-- Jordan

P.S. Don't forget to read the perf notes when using D3DImage

Posted by wpf3d | 2 Comments
Filed under: , , ,

.NET 3.5 SP1 Graphics @ Channel 9

Sorry for the lack of updates, but we've been pretty busy. On what, you may ask? Our PM, David Teitlebaum, just did a video on Channel 9 showing off the new features. He starts by covering interactive 2D on 3D and improved layered window support, both of which were in 3.5, and he finishes with the customizable ImageEffects, the much improved WriteableBitmap, BitmapScalingMode.NearestNeighbor, and D3DImage.

-- Jordan

Posted by wpf3d | 4 Comments
Filed under: , , ,

Blender to XAML Exporter Updated

Robert Hogue has updated the Python script with a bunch of new features. See this forum post for the instructions, tutorials, and demos!

-- Jordan

Posted by wpf3d | 3 Comments
Filed under: , ,

.NET 3.5 has been released!

Yesterday, .NET Framework 3.5 and Visual Studio 2008 went live on MSDN. You can see what's new in both here and download them here.

Since this is a graphics blog, here are the graphics-specific changes of note in 3.5:

New Graphics Features
  • UIElement3D
  • Interactive 2D on 3D: Viewport2DVisual3D
  • Transformation services on Visual3D
  • BitmapSource.DecodeFailed event
  • HwndTarget.RenderMode to enable software rendering per window
  • BitmapImage.UriCachePolicy and BitmapDecoder.Create(..., RequestCachePolicy) to control web request caching

Notable Graphics Performance Improvements

  • Less animation jitter
  • RenderTargetBitmap memory leaks plugged
  • Big layered window improvements when combined with this on Vista or this on XP
  • MeshGeometry3D hit testing up to 50% faster in common scenarios

Notable Graphics Bug Fixes

  • Numerous layered window problems resolved

Most of these changes will appear in 3.0 SP1 as well. 

-- Jordan

Posted by wpf3d | 0 Comments
Filed under: , ,

Augmented Reality with WPF3D

Augmented Reality is the process of taking real world data, typically video, and enhancing it with computer graphics. Casey used WPF3D along with an AR toolkit and DirectShow to get some great results. Check out the sweet video!

-- Jordan

P.S. We finally got around to putting some links on the side.

Posted by wpf3d | 3 Comments
Filed under: , ,

Blender Exporter on CodePlex - Looking for volunteers...

In July I promised to track down the Blender exporter that was lost during the GotDotNet phase out.  The last version of the export script is now hosted on CodePlex (here).  I am embarrassed that it has taken me this long to do this, and even more so that the script is not up to date for the latest version of Blender.

This has forced me to admit that it's time to give someone else the opportunity to take ownership of this project.  If you're interested, contact me.

- Daniel

Posted by wpf3d | 1 Comments
Filed under: , ,

Petzold.Media3D

Charles Petzold has posted his WPF3D library on the web. It includes sphere, cube, cylinder, torus, line, and teapot mesh generation.

Buying his book 3D Programming for Windows grants you royalty-free use of the library so be sure to check it out!

-- Jordan

Posted by wpf3d | 3 Comments
Filed under: , , , ,

Cel Shading

Charles Petzold has been experimenting with cel shading on his blog at the request of Chris Cavanagh (whom has updated his 3D physics XBAP btw). Though we do use shaders internally, WPF3D's API is fixed function so you have to dig out the ol' fixed function playbook to achieve fancier effects. The plays usually boil down to abusing textures :)

First, make a small, one dimensional texture containing the colors you want. Here are a couple of examples I've zoomed in on and highlighted the pixels to make things clearer. Why I'm using grayscale will make sense in a minute:

big_strip
big_strip2

Next, we need a function that maps to [0, 1] and takes into account the light position relative to a vertex. How about the cosine of the angle to the light? Turns out we can compute that fairly quickly because it's equal to the dot product of the normal and the vector to the light divided by their lengths.

    Int32Collection idxs = mesh.TriangleIndices;
    Point3DCollection pts = mesh.Positions;
    Vector3DCollection nrms = mesh.Normals;
    PointCollection txs = mesh.TextureCoordinates;
    mesh.TextureCoordinates = null;
    for (int i = 0, count = idxs.Count; i < count; ++i)
    { 
        int idx = idxs[i]; 
        Vector3D toLight = lightPos - pts[idx]; 
        toLight.Normalize(); 

        // The normals are pre-normalized, no need to do it again  
        double dp = Vector3D.DotProduct(toLight, nrms[idx]); 
        
        // A negative dot product means the vertex is facing away  
        // from the light so let's set that to the darkest color  
        if (dp < 0) 
        { 
            dp = 0; 
        } 
        
        txs[idx] = new Point(dp, 0); 
    } 
    mesh.TextureCoordinates = txs;

Naturally, you're going to need to redo this calculation any time the relationship between the light and the mesh changes so it needs to be fast. This is why I go through all of that collection nonsense (read this).

So why did I use a grayscale texture? If I baked the color in, I'd have to make a new texture any time I wanted to change the color. Instead, I can use the color knobs to set the color and use the grayscale texture like a mask. In these photos, I have a single AmbientLight with a DiffuseMaterial with AmbientColor="green" and Brush equal to the one dimensional texture. I don't actually have a real point light in the scene which helps performance.

Being that it's per-vertex, more vertices means a higher quality image and you're going to see some popping when it's animated. The teapot doesn't have too many vertices so the spout and black line on the body don't look great. On the highly tesselated torus, those issues are gone but it's still hard to get a clean, crisp edge because of the linear interpolation of the texture. 

-- Jordan

P.S. If you use a texture with two colors and instead dot the normal with the direction to the camera, you'll get silhouette edges but they could look really bad because of the per-vertex nature again.

Posted by wpf3d | 1 Comments
Filed under: , , , ,

Subclassing UIElement3D

Subclassing from UIElement3D to create your own elements that respond to input, focus and eventing is simple to do in 3.5.  In this example we'll create a Sphere class which derives from UIElement3D and will show off some new features in the process.

Deriving from UIElement3D

The first step is to derive from UIElement3D:

    public class Sphere : UIElement3D
    {
    }

Even though UIElement3D is declared abstract, there aren't any abstract methods you need to override.  The above will compile - it just won't do anything interesting. 

Rendering a Sphere

Although the above lets us derive from UIElement3D, we still don't have anything rendering to look like a sphere.  Before going in to how to render the sphere, first we're going to want to create a couple of dependency properties to control how the sphere looks.  These will be PhiDiv and ThetaDiv, to represent the number of slices to make in the horizontal and vertical directions respectively, as well as Radius, to represent the radius of the sphere.  The code for these is below:

        /// <summary>
        /// The number of vertical slices to make on the sphere
        /// </summary>
        public static readonly DependencyProperty ThetaDivProperty =
            DependencyProperty.Register("ThetaDiv",
                                        typeof(int),
                                        typeof(Sphere),
                                        new PropertyMetadata(15, ThetaDivPropertyChanged));

        private static void ThetaDivPropertyChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
        {
            Sphere s = (Sphere)d;
            s.InvalidateModel();
        }

        public int ThetaDiv
        {
            get
            {
                return (int)GetValue(ThetaDivProperty);
            }

            set
            {
                SetValue(ThetaDivProperty, value);
            }
        }

        /// <summary>
        /// The number of horizontal slices to make on the sphere
        /// </summary>
        public static readonly DependencyProperty PhiDivProperty =
            DependencyProperty.Register("PhiDiv",
                                        typeof(int),
                                        typeof(Sphere),
                                        new PropertyMetadata(15, PhiDivPropertyChanged));

        private static void PhiDivPropertyChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
        {
            Sphere s = (Sphere)d;
            s.InvalidateModel();
        }

        public int PhiDiv
        {
            get
            {
                return (int)GetValue(PhiDivProperty);
            }

            set
            {
                SetValue(PhiDivProperty, value);
            }
        }

        /// <summary>
        /// The radius of the sphere
        /// </summary>
        public static readonly DependencyProperty RadiusProperty =
            DependencyProperty.Register("Radius",
                                        typeof(double),
                                        typeof(Sphere),
                                        new PropertyMetadata(1.0, RadiusPropertyChanged));

        private static void RadiusPropertyChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
        {
            Sphere s = (Sphere)d;
            s.InvalidateModel();
        }

        public double Radius
        {
            get
            {
                return (double)GetValue(RadiusProperty);
            }

            set
            {
                SetValue(RadiusProperty, value);
            }
        }

InvalidateModel, OnUpdateModel and Visual3DModel

The above should look like standard C# WPF code to create a few dependency properties, but the one new method that should look unfamiliar is the InvalidateModel call made whenever any of the above dependency properties changes. InvalidateModel is similar to the InvalidateVisual method that exists for the 2D UIElement.  Calling InvalidateModel indicates that the 3D model representing the UIElement3D has changed and should be updated.  

In response to InvalidateModel, OnUpdateModel will be called to update the 3D model that represents the object. In this way, you can make multiple changes to properties that affect the visual appearance of the UIElement3D and only make one final change to the model, rather than having to regenerate it each time a change is made.  For instance, say changes are made to ThetaDiv, PhiDiv, and Radius above.  Rather than having to regenerate the model each time, InvalidateModel can be called each time a property changes, and then all of the changes can be dealt with when OnUpdateModel is called. Of course, the model can also be changed in other places, but this provides one standard option to make the change.

The last thing that hasn't been discussed is how to actually change the model. In 3.5, Visual3D now exposes a protected CLR property Visual3DModel which represents the 3D model for the object.  This is the 3D equivalent to the render data of a 2D Visual.  To set what the visual representation for the Visual3D is then, it's necessary to set this property.  For instance, ModelUIElement3D's Model property takes care of setting this. There is one subtle point about setting this property.  Just like render data, setting this property won't set up the links necessary for things such as data bindings to work.  To make this happen, you'll also want to have a dependency property for the model itself. 

The code for InvalidateModel is shown below, as well as the Model dependency property:

 

        protected override void OnUpdateModel()
        {
            GeometryModel3D model = new GeometryModel3D();

            model.Geometry = Tessellate(ThetaDiv, PhiDiv, Radius);
            model.Material = new DiffuseMaterial(Brushes.Blue);

            Model = model;
        }

        /// <summary>
        /// The Model property for the sphere
        /// </summary>
        private static readonly DependencyProperty ModelProperty =
            DependencyProperty.Register("Model",
                                        typeof(Model3D),
                                        typeof(Sphere),
                                        new PropertyMetadata(ModelPropertyChanged));

        private static void ModelPropertyChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
        {
            Sphere s = (Sphere)d;
            s.Visual3DModel = (Model3D)e.NewValue;
        }

        private Model3D Model
        {
            get
            {
                return (Model3D)GetValue(ModelProperty);
            }

            set
            {
                SetValue(ModelProperty, value);
            }
        }

Attached to this post you'll find a simple example app which has the full code to the above Sphere class.

Posted by wpf3d | 10 Comments
Filed under: , , , ,

Attachment(s): Shapes.zip
More Posts Next page »
 
Page view tracker