WPF3D Team Blog

.NET 3.5 SP1 Graphics @ Channel 9

Sorry for the lack of updates, but we've been pretty busy. On what, you may ask? Our PM, David Teitlebaum, just did a video on Channel 9 showing off the new features. He starts by covering interactive 2D on 3D and improved layered window support, both of which were in 3.5, and he finishes with the customizable ImageEffects, the much improved WriteableBitmap, BitmapScalingMode.NearestNeighbor, and D3DImage.

-- Jordan

Blender to XAML Exporter Updated

Robert Hogue has updated the Python script with a bunch of new features. See this forum post for the instructions, tutorials, and demos!

-- Jordan

.NET 3.5 has been released!

Yesterday, .NET Framework 3.5 and Visual Studio 2008 went live on MSDN. You can see what's new in both here and download them here.

Since this is a graphics blog, here are the graphics-specific changes of note in 3.5:

• UIElement3D
• Interactive 2D on 3D: Viewport2DVisual3D
• Transformation services on Visual3D
• BitmapSource.DecodeFailed event
• HwndTarget.RenderMode to enable software rendering per window
• BitmapImage.UriCachePolicy and BitmapDecoder.Create(..., RequestCachePolicy) to control web request caching

Notable Graphics Performance Improvements

• Less animation jitter
• RenderTargetBitmap memory leaks plugged
• Big layered window improvements when combined with this on Vista or this on XP
• MeshGeometry3D hit testing up to 50% faster in common scenarios

Notable Graphics Bug Fixes

• Numerous layered window problems resolved

Most of these changes will appear in 3.0 SP1 as well. 

-- Jordan

Augmented Reality with WPF3D

Augmented Reality is the process of taking real world data, typically video, and enhancing it with computer graphics. Casey used WPF3D along with an AR toolkit and DirectShow to get some great results. Check out the sweet video!

-- Jordan

P.S. We finally got around to putting some links on the side.

Blender Exporter on CodePlex - Looking for volunteers...

In July I promised to track down the Blender exporter that was lost during the GotDotNet phase out.  The last version of the export script is now hosted on CodePlex (here).  I am embarrassed that it has taken me this long to do this, and even more so that the script is not up to date for the latest version of Blender.

This has forced me to admit that it's time to give someone else the opportunity to take ownership of this project.  If you're interested, contact me.

- Daniel

Petzold.Media3D

Charles Petzold has posted his WPF3D library on the web. It includes sphere, cube, cylinder, torus, line, and teapot mesh generation.

Buying his book 3D Programming for Windows grants you royalty-free use of the library so be sure to check it out!

-- Jordan

Cel Shading

Charles Petzold has been experimenting with cel shading on his blog at the request of Chris Cavanagh (whom has updated his 3D physics XBAP btw). Though we do use shaders internally, WPF3D's API is fixed function so you have to dig out the ol' fixed function playbook to achieve fancier effects. The plays usually boil down to abusing textures :)

First, make a small, one dimensional texture containing the colors you want. Here are a couple of examples I've zoomed in on and highlighted the pixels to make things clearer. Why I'm using grayscale will make sense in a minute:

Next, we need a function that maps to [0, 1] and takes into account the light position relative to a vertex. How about the cosine of the angle to the light? Turns out we can compute that fairly quickly because it's equal to the dot product of the normal and the vector to the light divided by their lengths.

    Int32Collection idxs = mesh.TriangleIndices;
    Point3DCollection pts = mesh.Positions;
    Vector3DCollection nrms = mesh.Normals;
    PointCollection txs = mesh.TextureCoordinates;
    mesh.TextureCoordinates = null;
    for (int i = 0, count = idxs.Count; i < count; ++i)
    { 
        int idx = idxs[i]; 
        Vector3D toLight = lightPos - pts[idx]; 
        toLight.Normalize(); 

        // The normals are pre-normalized, no need to do it again  
        double dp = Vector3D.DotProduct(toLight, nrms[idx]); 
        
        // A negative dot product means the vertex is facing away  
        // from the light so let's set that to the darkest color  
        if (dp < 0) 
        { 
            dp = 0; 
        } 
        
        txs[idx] = new Point(dp, 0); 
    } 
    mesh.TextureCoordinates = txs;

Naturally, you're going to need to redo this calculation any time the relationship between the light and the mesh changes so it needs to be fast. This is why I go through all of that collection nonsense (read this).

So why did I use a grayscale texture? If I baked the color in, I'd have to make a new texture any time I wanted to change the color. Instead, I can use the color knobs to set the color and use the grayscale texture like a mask. In these photos, I have a single AmbientLight with a DiffuseMaterial with AmbientColor="green" and Brush equal to the one dimensional texture. I don't actually have a real point light in the scene which helps performance.

Being that it's per-vertex, more vertices means a higher quality image and you're going to see some popping when it's animated. The teapot doesn't have too many vertices so the spout and black line on the body don't look great. On the highly tesselated torus, those issues are gone but it's still hard to get a clean, crisp edge because of the linear interpolation of the texture. 

-- Jordan

P.S. If you use a texture with two colors and instead dot the normal with the direction to the camera, you'll get silhouette edges but they could look really bad because of the per-vertex nature again.

Subclassing UIElement3D

Subclassing from UIElement3D to create your own elements that respond to input, focus and eventing is simple to do in 3.5.  In this example we'll create a Sphere class which derives from UIElement3D and will show off some new features in the process.

Deriving from UIElement3D

The first step is to derive from UIElement3D:

    public class Sphere : UIElement3D
    {

    }

Even though UIElement3D is declared abstract, there aren't any abstract methods you need to override.  The above will compile - it just won't do anything interesting. 

Rendering a Sphere

Although the above lets us derive from UIElement3D, we still don't have anything rendering to look like a sphere.  Before going in to how to render the sphere, first we're going to want to create a couple of dependency properties to control how the sphere looks.  These will be PhiDiv and ThetaDiv, to represent the number of slices to make in the horizontal and vertical directions respectively, as well as Radius, to represent the radius of the sphere.  The code for these is below:

        /// <summary>
        /// The number of vertical slices to make on the sphere
        /// </summary>
        public static readonly DependencyProperty ThetaDivProperty =
            DependencyProperty.Register("ThetaDiv",
                                        typeof(int),
                                        typeof(Sphere),
                                        new PropertyMetadata(15, ThetaDivPropertyChanged));

        private static void ThetaDivPropertyChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
        {
            Sphere s = (Sphere)d;
            s.InvalidateModel();
        }

        public int ThetaDiv
        {
            get
            {
                return (int)GetValue(ThetaDivProperty);
            }

            set
            {
                SetValue(ThetaDivProperty, value);
            }
        }

        /// <summary>
        /// The number of horizontal slices to make on the sphere
        /// </summary>
        public static readonly DependencyProperty PhiDivProperty =
            DependencyProperty.Register("PhiDiv",
                                        typeof(int),
                                        typeof(Sphere),
                                        new PropertyMetadata(15, PhiDivPropertyChanged));

        private static void PhiDivPropertyChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
        {
            Sphere s = (Sphere)d;
            s.InvalidateModel();
        }

        public int PhiDiv
        {
            get
            {
                return (int)GetValue(PhiDivProperty);
            }

            set
            {
                SetValue(PhiDivProperty, value);
            }
        }

        /// <summary>
        /// The radius of the sphere
        /// </summary>
        public static readonly DependencyProperty RadiusProperty =
            DependencyProperty.Register("Radius",
                                        typeof(double),
                                        typeof(Sphere),
                                        new PropertyMetadata(1.0, RadiusPropertyChanged));

        private static void RadiusPropertyChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
        {
            Sphere s = (Sphere)d;
            s.InvalidateModel();
        }

        public double Radius
        {
            get
            {
                return (double)GetValue(RadiusProperty);
            }

            set
            {
                SetValue(RadiusProperty, value);
            }
        }

InvalidateModel, OnUpdateModel and Visual3DModel

The above should look like standard C# WPF code to create a few dependency properties, but the one new method that should look unfamiliar is the InvalidateModel call made whenever any of the above dependency properties changes. InvalidateModel is similar to the InvalidateVisual method that exists for the 2D UIElement.  Calling InvalidateModel indicates that the 3D model representing the UIElement3D has changed and should be updated.  

In response to InvalidateModel, OnUpdateModel will be called to update the 3D model that represents the object. In this way, you can make multiple changes to properties that affect the visual appearance of the UIElement3D and only make one final change to the model, rather than having to regenerate it each time a change is made.  For instance, say changes are made to ThetaDiv, PhiDiv, and Radius above.  Rather than having to regenerate the model each time, InvalidateModel can be called each time a property changes, and then all of the changes can be dealt with when OnUpdateModel is called. Of course, the model can also be changed in other places, but this provides one standard option to make the change.

The last thing that hasn't been discussed is how to actually change the model. In 3.5, Visual3D now exposes a protected CLR property Visual3DModel which represents the 3D model for the object.  This is the 3D equivalent to the render data of a 2D Visual.  To set what the visual representation for the Visual3D is then, it's necessary to set this property.  For instance, ModelUIElement3D's Model property takes care of setting this. There is one subtle point about setting this property.  Just like render data, setting this property won't set up the links necessary for things such as data bindings to work.  To make this happen, you'll also want to have a dependency property for the model itself. 

The code for InvalidateModel is shown below, as well as the Model dependency property:

        protected override void OnUpdateModel()
        {
            GeometryModel3D model = new GeometryModel3D();

            model.Geometry = Tessellate(ThetaDiv, PhiDiv, Radius);
            model.Material = new DiffuseMaterial(Brushes.Blue);

            Model = model;
        }

        /// <summary>
        /// The Model property for the sphere
        /// </summary>
        private static readonly DependencyProperty ModelProperty =
            DependencyProperty.Register("Model",
                                        typeof(Model3D),
                                        typeof(Sphere),
                                        new PropertyMetadata(ModelPropertyChanged));

        private static void ModelPropertyChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
        {
            Sphere s = (Sphere)d;
            s.Visual3DModel = (Model3D)e.NewValue;
        }

        private Model3D Model
        {
            get
            {
                return (Model3D)GetValue(ModelProperty);
            }

            set
            {
                SetValue(ModelProperty, value);
            }
        }

Attached to this post you'll find a simple example app which has the full code to the above Sphere class.

What’s new in WPF 3.5

With the release of WPF 3.5 beta 2 (download here: http://www.microsoft.com/downloads/details.aspx?FamilyId=D2F74873-C796-4E60-91C8-F0EF809B09EE&displaylang=en), we’ve added some exciting new features to WPF 3D.  At a very high level these can be grouped in to two main additions: UIElement3D and Viewport2DVisual3D.  Over the next couple of weeks we’ll be adding examples and tips and tricks to the blog on these new additions, but for now we’ll start with a quick overview of what both of these provide.

UIElement3D

In the 2D world, UIElement adds layout, input, focus and eventing on to Visual.  UIElement3D brings these same things (except no layout) to 3D.  What this means is that the standard events and means of adding event handlers that you’re used to with UIElements now applies in the 3D world with UIElement3D. 

UIElement3D itself is an abstract class that derives from Visual3D.  To make it useable out of the box without having to derive from UIElement3D yourself, we’ve provided two new classes:  ModelUIElement3D and ContainerUIElement3D. 

ContainerUIElement3D does exactly what its name says.  It is a container for other Visual3Ds.  It has one main property, Children, which is used to add and remove 3D children.  The ContainerUIElement3D doesn’t have a visual representation itself, but rather is just a collection of other 3D objects.

ModelUIElement3D has one property, Model, which is the Model3D that should be displayed to represent the UIElement3D.  It has no children itself, and in some ways you can think of ModelUIElement3D like a Shape in the 2D world.

If you’re familiar with ModelVisual3D, then ContainerUIElement3D and ModelUIElement3D should look very familiar.  The difference is we’ve gone and split the functionality of ModelVisual3D (i.e. a model and children) in to two separate classes, one with a model and the other with children.

With these then, making use of layout, focus and eventing is very easy.  For instance, say you want to create a 3D object that responds to mouse events, you can just do:

                <ModelUIElement3D  MouseDown="OnMouseDown"/>

And then in the code behind have:

      protected void OnMouseDown(object sender, MouseButtonEventArgs e)

      {

          Console.WriteLine("Hello");

      }

If you have experience adding event handlers in the 2D world, then this should look immediately familiar.  In fact it’s exactly the same, and UIElement3D is going to handle the same routed events that a UIElement deals with.  With the addition of UIElement3D you now get all the great functionality that UIElement provided to 2D, but now in the 3D world!

Viewport2DVisual3D

The second main addition is Viewport2DVisual3D which enables you to put interactive 2D on 3D in WPF.  If you’ve used the 3DTools work that was released shortly after the 3.0 release of WPF, Viewport2DVisual3D will look very familiar to you.  With 3.5 though we’ve fully integrated 2D on 3D in to WPF rather than requiring a separate DLL to have this feature.  The 2D and 3D trees are also integrated now, so if you walk up to a 2D visual on 3D and ask it for its parent, it will tell you the 3D object it is on.

Viewport2DVisual3D derives from Visual3D and has three main dependency properties: Visual, Geometry and Material.

Visual – This is the 2D Visual that will be placed on the 3D object.

Geometry – The 3D geometry for the Viewport2DVisual3D

Material – This describes the look of the 3D object.  You can use any material you want.  For the material that you want to have the Visual be placed on, you simply need to set the Viewport2DVisual3D.IsVisualHostMaterial attached property to true.

Below is a XAML example using Viewport2DVisual3D

<Viewport2DVisual3D Geometry="{StaticResource plane}">

        <Viewport2DVisual3D.Material>

          <DiffuseMaterial  Viewport2DVisual3D.IsVisualHostMaterial="true" />

        </Viewport2DVisual3D.Material>

        <Button>3.5!</Button>          

      </Viewport2DVisual3D>

The above is a very quick overview of what is added in 3.5, so expect more posts on more specific details as well as some code samples on how to use it.

-Kurt Berglund

More 3D Transitions

The WPF Feature Montage has been updated to include a custom transition control that contains four 3D transitions: page turn, curtain pull, explosion, and door open. The full source is included in the zip.

-- Jordan 

P.S. For those of you who saw us give our WPF performance talk at TechEd, we aren't allowed to post our slides here, sorry.

Latest 3D Apps

• It sounds like the Shaxam folks are improving their Lightwave to XAML exporter. Richard Godfrey has two fantastic examples up on his blog.
• Chris Cavanagh has created a little 3D physics XBAP demo and he has also posted the source.
• Japan's famous Asahiyama Zoo released an XBAP called "Mother Earth." 3D is used for the leaping intro and the rotating Earth.
• Yousef has used the MS Virtual Earth APIs to create a zoomable globe XBAP.

-- Jordan

2D -> 3D -> 2D Transitions

WPF Architect Greg Schecter has written a few really cool 3D transitions for 2D elements. We've gotten questions about transitions on the forums before and his explanation and source code should help you implement similar effects yourself.

• The demo
• The source
• The math behind the transition

-- Jordan

Bootcamp Lecture

We gave a lecture on 3D yesterday at a "WPF Bootcamp" event and I've attached the slides. Here's the outline:

• Intro to WPF3D with a simple example
• Interactive 2D on 3D
• Performance tips (based on this paper)

-- Jordan

Material Behavior

What Material you choose will impact your GeometryModel3D in three ways:

1. Lighting model
2. Depth write
3. Blend with the back buffer

The first one is straightforward since it's right in the name of the Material plus it's documented. The last two aren't obvious at all and this is probably the first you've heard of them.

Depth write and the depth buffer

If you haven't heard of the depth buffer (a.k.a. the z buffer) before, it's essentially how the card keeps track of the distance to an object from the camera per pixel. When a polygon is being rasterized at a pixel that has already been written to, do you keep the old value or take the new value? For opaque objects, you typically want the one closest to the camera to win because you won't be able to see the occluded one. This is what we tell the card to do (for you D3D programmers out there, we use D3DCMP_LESSEQUAL).

For example, say you draw a triangle with a DiffuseMaterial. If you then draw another triangle directly behind, it will be ignored in the area of overlap because the first triangle is in front of it. But if a polygon doesn't write to the depth buffer, then it's like the card doesn't know it's there. Say you draw a triangle with an EmissiveMaterial. If you then draw an overlapping triangle it doesn't matter if it's behind or in front -- the new triangle overwrites the old one because the card has no "record" of the old one. 

Can you think of why the depth buffer might cause problems with transparent objects in a dynamic scene? That's a topic for another post :)

Blend

In the equation above, S means source (the GeometryModel3D you're drawing) and D means destination (what you've drawn to the back buffer already). From the equations you can make a few observations like a fully opaque DiffuseMaterial will never let you see what was previously written or even opaque additive Materials will appear transparent because they don't block out what is already there.

Why are there different behaviors?

The behaviors came out of how we think most people use Materials. Just about every Model3D will contain a DiffuseMaterial and will want to write to the depth buffer. If you then want your Model3D to glow or shine, you throw in an EmissiveMaterial or SpecularMaterial. Since you already wrote the depth, you don't need to do it again in the emissive and specular passes. Furthermore, you don't want emissive and specular to subtract away from the diffuse layer you just put down.

Extra Tips

"AmbientMaterial" is in quotes because it doesn't exist but you can make a Material that only responds to ambient light. See my earlier color knobs post.

If you'd like one of the additive Materials to write depth, you can do so by putting them in a MaterialGroup along with a transparent black DiffuseMaterial. That way the DiffuseMaterial doesn't affect the color but does affect the depth buffer. You can put this "depth pass" DiffuseMaterial before or after the other Materials but you will get different results between the two if you have any overlapping front faces (or back faces if we're talking about BackMaterial). 

-- Jordan

Flickr Photo Browser Source Code Now Available

In the Channel9 video showing the 2D on 3D feature found in the 3DTools Codeplex project, we demoed a photo browsing application that made use of 2D on 3D and that talked to Flickr for images.  I'm happy to announce that the source code for this application is now available for download.  You can get it from here: http://www.codeplex.com/3DTools/Release/ProjectReleases.aspx?ReleaseId=2058

And in case you forgot what it looked like, here's an image of it running.

A Flickr API key isn't included with the source, so in order to actually search for photos yourself you'll need to get a Flickr API key from http://www.flickr.com/services/api/keys/apply/ and then add it to Flickr.cs

The earth image comes from http://planetpixelemporium.com/planets.html  and is copyright James Hastings-Trew.

-Kurt