Daniel Lehenbauer's Blog

It is perhaps an understatement to say it's been a while since I blogged.

As is commonly the case with Microsoft blogs, the reason for going dark is that I left the WPF team almost two years ago for an opportunity I couldn't pass by to join a systems incubation startup.

We're a small team with big ambitions, and as you might expect, my role involves the exploration of a radically different approach to the UI/Graphics platform which guarantees security, responsiveness, and leverages modern GPUs and manycore.  To deliver on this vision, we are revisiting every layer of the stack from device drivers, through rendering engines, up to application frameworks and programming/computation models.

While we’re not specifically seeking to fill headcount on our team, we are always on the lookout for luminaries with insight, experience, and who love to write code.  If you are interested in joining a deep systems incubation that believes in small, hand-picked teams of truly outstanding developers, please do contact me.

While much is demanded of everyone in this group, we also get to participate in what I personally believe is the most exciting and revolutionary work to happen in the industry since PARC.

[UPDATE: The Xaml Exporter now is hosted on CodePlex (here)] 

I checked in an updated Blender exporter to 3D Tools for Windows Presentation Foundation which adds support for double sided materials and flat shaded geometry.  I have not yet package a new release, so for the moment you’ll need to grab xaml_export.py from source control.

These features are accessible via the circled buttons on the material and mesh panels during editing.  Thanks to SEISENBERG for reporting this in the message board.

I have posted a new release of the 3D Tools for the Windows Presentation Foundation targeting the Beta 2 release of the WinFX runtime.  The big addition in this release is a community owned implementation of the ScreenSpaceLines3D class. 

ScreenSpaceLines3D is a 3D line primitive whose thickness is constant in 2D space post projection.*  This means that the lines do not become “skinnier” as they receded from a perspective camera.  ScreenSpaceLines3D are most frequently used to implement bounding boxes, wireframe mode, or visualization of normals for editors.  Occasionally they are used in 3D scenes to represent telephone wires and similar objects that are too thin to be rendered reliably under a perspective projection.

I also started a MathUtils class which contains some generally useful utility methods, such as getting the View/Projection matrices from any camera and a method which returns the 3D -> 2D transform (useful for positioning 2D content, like text labels, relative to 3D content.)  I’ve also added a wireframe mode to the ModelViewer sample to test ScreenSpaceLines3D.

The project has been refactored into just two assemblies: 3DTools.dll which contains the reusable runtime components and the ModelViewer.exe sample.  Enjoy!

* The name ScreenSpaceLines3D was a bit of a misnomer.  The lines were actually of constant thickness in the post-projective space of the Viewport3DVisual, not “device space” or “screen space” as the name implied.

Minor update: The "Extending Visual3D - Sphere, Cone, and Cylinder" sample has been updated to the December and January CTP (either will work).  Thanks to Barry Briggs for doing this and sending me the updated source.

[UPDATE: The Xaml Exporter now is hosted on CodePlex (here)] 

The 3D features in the Windows Presentation Foundation make it easier than ever to integrate 3D content into your applications.  However, unless you are satisfied with basic solid geometry you are going to need more than just the WPF.  You’re going to need 3D authoring tools to create interesting content.

Unfortunately 3D tools tend to use their own proprietary file formats with limited import/export support for other formats.  Xaml, being new to the scene (no pun intended), is not yet supported by most tools.  However, some modeling programs feature an add-in architecture that allows custom exporters to be written by 3rd parties.  The free 3D software package Blender is such a tool.

All very interesting you say, but where is this going?

I have posted a new release of the 3D Tools for the WPF which includes a Python script to export Blender scenes as Xaml.  The Xaml can then be dynamically loaded or compiled into your WPF applications.  Figure 1a shows a 3D model being edited with Blender.  Figure 1b shows the final exported Xaml as viewed with the WPF Model Viewer.

Figure 1a The Steve Ballmer model* is authored in Blender and then exported to Xaml.	Figure 1b The final model as it appears in the WPF Model Viewer.
(Click for full sized images)

Because the WPF and Blender have different goals for their rendering engines their feature sets do not match precisely.  Specifically, the WPF uses the hardware accelerated real-time rendering capabilities of your GPU.  Blender is a software based rendering engine not targeted at real-time applications.  However, the results of the export are frequently quite good.  Figure 2a is the same 3D model as rendered by Blender.  Figure 2b is the exported model viewed from a similar angle with the same lighting configuration in the WPF Model Viewer.

Figure 2a Steve Ballmer model* rendered using Blender’s internal software engine.	Figure 2b Exported model rendered using the WPF.  The blue background is not part of the scene.
(Click for full sized images)

To get the Xaml Export Script for Blender and the WPF Model Viewer download the latest version of the 3D Tools for WPF (available here).  Copy the xaml_export.py script to your Blender scripts directory.  Blender will automatically add the script to the ~File~Export menu the next time it is started.  More detailed instructions and a list of known issues are included in the Read-Me.

* Steve Ballmer model courtesy of Bonnie Lehenbauer

Ernie posted about the differences between the old LookAtPoint property and the new LookDirection property on ProjectionCamera (via KarstenJ.)  The quick summary is that LookAtPoint caused the camera to look at a fixed point in space (Figure 1a).  The new LookDirection property causes the camera to look in a fixed direction (Figure 1b).

In terms of the camera configurations that can be represented the two properties are equally expressive.  It is easy to convert between the two using the following equation:

LookDirection = LookAtPoint - Position

However, there are some pragmatic advantages to choosing LookDirection.  One is that LookDirection is similar to other properties in the system (e.g., SpotLight.Direction.)  Another is that LookAtPoint leads to a common misunderstanding that the camera can be rotated by simply updating the camera’s position.  Unless you also transform the UpDirection this will lead to (usually) unexpected flipping behavior at the poles.

It was the addition of Camera.Transform, however, that was the largest motivation to switch to LookDirection.  Being able to apply Transform3Ds to Cameras simplifies a lot of animation scenarios.  In fact, it’s currently the only way to animate a MatrixCamera.  Also, the ability to share Transform3Ds between Cameras and nodes in the Visual tree enables scenarios where the Camera follows an object in the scene.  (Incidentally, this is how the headlight feature of the ModelViewer sample works.)

The addition of Camera.Transform raised questions about LookAtPoint.  Should LookAtPoint be affected by the transform or should it continue to be defined in world space?  If LookAtPoint is in world space, as most people initially suppose, it breaks the illusion that the camera is a physical object in the scene because rotating the camera 180 degrees does not necessarily mean that the camera is looking in the opposite direction since it will continue to look in the direction of the same fixed point in world space.

On the other hand, if the camera’s transform LookAtPoint is affected by the transform then users need to convert into the camera’s coordinate system to specify the LookAtPoint.  This is no less difficult than calculating the LookDirection and more error prone.

When all these things were considered: the difficulty in reconciling LookAtPoint with Camera.Transform, the fact that the perceived simplicity of LookAtPoint is negated by the need to handle the UpDirection, and that elsewhere in the system we have consistently specified orientation with direction vectors, we decided that in the long term our customers will be happier with LookDirection.

I've packaged a new release of the 3D Tools workspace targeting the December CTP WinFX bits.  This release adds two requested features to the ModelViewer sample.  The first is the ability to light the scene.  This is convenient if you are viewing a model which does not contain lights.  There are two options for lighting:

1.  Headlight – This will add a white directional light which follows the viewing direction of the camera (imagine a flashlight duct-taped to the front of your camera.)  The headlight is now on by default.

2.  Ambient Light – This will add a white ambient light which will fully illuminate the scene.  The lack of depth cues this causes will make the scene appear “flat”, but this mode can be useful for examining a model for correctness.  The ambient light is off by default.

No lights	Headlight	Ambient Light

Changing the lighting options in the ModelViewer does not change the model (i.e., it is non-destructive.)

The other new feature is the ability to reload the currently open file to view any changes since the file was last opened.  Both new features are available via the View menu.

The release contains both binary and sources.  To run the ModelViewer, you need .NET Framework 2.0 Redist and the December CTP WinFX Runtime.  To build the solution you also need the Windows SDK and optionally VS 2005 and the VS Extensions for WinFX.

Possibly the most common 3D UI paradigm is to rotate an object with the mouse in order to view it from all angles.  This is illustrated by the images of the tiger model below: 

This UI technique is known as a "virtual trackball" and it has been a staple of interactive computer graphics since the 80s.  It has applications ranging from CAD to department store kiosks to gaming.

To help developers working with the Windows Presentation Foundation get a jump-start using 3D in their applications I have added a sample trackball implementation to the 3DTools workspace with an accompanying article which discusses the mechanics of how a trackball works.

Reading the full article is not required to successfully use the sample code in your own applications, but skimming it might be useful.  You do not need to be a member of the workspace in order to download releases which include both binaries and source (here).

If you are like me and have been holding off on installing the final version of the .NET Framework 2.0 and Visual Studio 2005 until you had Windows Presentation Foundation (formerly code-named Avalon) bits to work with then the wait is over.  The November CTP of the WPF works with the final .NET/VS bits.

Tim Sneath posted links to the downloads.
Karsten Januszewski has a summary changes in this release.

Not having to reinstall VS for each WinFX release should make it a lot easier to keep up with future CTPs.

Introduction

People new to 3D are frequently confused by the extra level of indirection introduced by texture coordinates.  3D veterans familiar with texture coordinates from other platforms are sometimes surprised that the default behavior in the Windows Presentation Foundation (formerly code-named Avalon) is a bit different than what they are accustomed to in other 3D APIs.  Below is a quick overview of what texture coordinates are and how they are specified.  The last section talks about the default behavior in the WPF and how the tradition behavior can be attained by tweaking Brush properties.

What are Texture Coordinates?

Texture coordinates specify which part of the Material shows up an which part of the MeshGeometry3D.  Developers from a 2D background are usually surprised that this is necessary.  Most 2D graphics APIs give the users drawing primitives like Ellipses, Rectangles, Paths, etc. with a predefined fill behavior that stretches the brush to the bounds of the given geometry.  This is the default in WPF as well.  Consider the following diagonal GradientBrush which is white in the upper left corner, red in the middle, and black in the lower right corner:

<LinearGradientBrush x:Key="MyGrad" StartPoint="0,0" EndPoint="1,1">
    <LinearGradientBrush.GradientStops>
        <GradientStopCollection>
            <GradientStop Color="#FFFFFF" Offset="0" />
            <GradientStop Color="#FF0000" Offset="0.5" />
            <GradientStop Color="#000000" Offset="1" />
        </GradientStopCollection>
    </LinearGradientBrush.GradientStops>
</LinearGradientBrush>

By default when a brush is used to fill a 2D geometry the brush is applied relative to the bounding box of the shape.  If applied to a Rectangle like the 100x100 square below the entire brush will be visible.  If the geometry does not entirely fill it’s bounding box parts of the brush will not be visible.  For example, the non-rectangular parallelogram below does not show the extreme white and black corners of the brush.

People working with 3D for the first time often expect that materials will be similarly stretched to fit the shape of their geometry.  Indeed, in my last post my sample 3D primitives have default texture coordinates which mimic this behavior:

However, there is no single mapping which provides a reasonable default for all 3D geometry.  Even with simple geometry like the 3 quadrics depicted above the method for generating the texture coordinates of the sphere differ from that of the cone and cylinder.  In order to make the material wrap sensibly around each primitive I had to explicitly specify a 2D texture coordinate for each 3D position.  For example, the cone primitive generates a mesh similar to the one depicted below.  Each 3D position in the mesh has a corresponding 2D texture coordinate which maps that 3D position to a 2D coordinate in the material.

Specifying Texture Coordinates

You specify texture coordinates by adding 2D Points to the MeshGeometry3D.TextureCoordinates collection.  For example, the following MeshGeometry3D maps the lower right half portion of the material to a triangle:

<MeshGeometry3D
    Positions="-1,1,0 -1,-1,0 1,-1,0"
    TextureCoordinates="0,0 0,1 1,1"
    TriangleIndices="0,1,2" />

The ith entry in the TextureCoordinates list corresponds to the ith entry in the Positions list, so texture coordinate 0,0 is associated with the vertex at position -1,1,0.  The mapping between texture coordinates and positions is depicted below:

              Texture Coordinates                                                 Rendered Triangle

Of course, this is not the only possibly mapping.  If we change the 2nd texture coordinate from 0,1 to 0,0.5 we produce the following image:

              Texture Coordinates                                                  Rendered Triangle

Notice that the area of the triangle specified by the texture coordinates on the left is stretched to fit the render triangle.

Texture coordinates are required when using a material with an ImageBrush, DrawingBrush, VisualBrush, LinearGradientBrush, or RadialGradientBrush (i.e., any Brush other than a SolidColorBrush in which case every position in the mesh maps to the same color.)

Texture Coordinates in the Windows Presentation Foundation

There are a couple of behavior differences that 3D veterans should be aware of when working with texture coordinates in the Windows Presentation Foundation.  These differences arise from the fact that 3D Materials leverage 2D Brushes as the source of their textures.  The wins from being able to take any arbitrary 2D brush and use it to paint a 3D mesh are enormous.  However brushes default to behavior which, while traditional for 2D, is sometimes surprising for 3D developers.  Fortunately brushes are highly configurable and it is easy to get the typical 3D behavior.

The first difference is that the +Y axis in brush space (and consequently texture coordinate space) points down by default.  This is consistent with 2D but different from the traditional 3D coordinate system where the +Y axis points up.  In order to get the +Y axis pointing up apply a Y scale of -1 and Y translation of +1:

<ImageBrush Transform="scale(1,-1) translate(0,1)" ImageSource="…" />

The second difference is that the brushes default to relative units.  At the beginning of this article I discussed how by default the brush maps to the relative bounding box of the 2D geometry it is being used to fill.  The same thing happens by default with the bounding box of your texture coordinates.  In the above samples I happened to use texture coordinates in the range of 0..1, but texture coordinates of “-20,-20 -20,10 10,10” would have also filled the triangle with the same part of the image.  TileBrush’s Viewport property defaults to the rect (0,0)-(1,1) so to get the standard behavior where the bounds of the texture source maps to 0..1 you just need to specify you are using absolute units:

<ImageBrush ViewportUnits="Absolute" Transform="scale(1,-1) translate(0,1)" ImageSource="…" />

These 2 settings should configure brush / texture space to be identical to what you are probably used to from other APIs.  (Incidently, all of these Brush settings have the same affect in 2D as well.)  Finally, if you need tiling you turn it on with TileMode=”Tile”.

Update: The source has been updated for the December or January CTP (either will work).

In my previous post I mentioned that one of the motivations for Visual3D was sub-classing.  Prior to the September CTP of the Windows Presentation Foundation (formerly code-named Avalon) there was no extensibility point for creating your own 3D scene nodes.  One of the many possible uses for this new extensibility point is to build a library of 3D primitives.  I put together a short sample which derives a Sphere, a Cone, and a Cylinder primitive from ModelVisual3D.  (Source available here.)

You will notice that these primitives are directly usable from Xaml:

<Viewport3D Camera=”{Camera}” >
  <my:Sphere3D Transform=”{XForm1}” Material=”{Azul}” />
  <my:Cone3D Transform=”{XForm2}” Material=”{Tulips}” />
  <my:Cylinder3D Transform=”{XForm3}” Material=”{Autumn}” />
</Viewport3D>

The primitives are centered at the origin and have a natural size of Radius=1 and Height=2 (i.e., they are bounded by (-1,-1,-1) – (1,1,1)).

This sample does not constitute a terribly complete or expressive library of 3D primitives, but it does illustrate how one could be built.  More compelling examples include deriving from Visual3Ds to add new events and behavior (like collision detection) or surfacing properties like databindable properties which dynamically generate 3D charts and graphs.

(NOTE: You might find Karsten's pragmatic perspective a useful prelude before we dive into the artictural underpinnings.)

In the September CTP release of the Windows Presentation Foundation (formerly code-named Avalon) we made 3D a first class citizen in the Visual tree with the introducion of Visual3D.  Up until this point 3D existed solely as content attached to a 2D Visual .  3D users migrating to the September CTP will encounter the Visual/Content split for the first time.  This has lead to some confusion about when nested ModelVisual3Ds should be used to group models together vs. a Model3DGroup.  Below is a short description of Visuals and Content and how they fit together.  At the end is a concrete example with guidelines on when to use a ModelVisual3D vs. a Model3DGroup.

What is a Visual?

A Visual (or Visual3D) is a low-level node in the composition tree.  For people coming from a Win32 background a Visual is frequently compared to an HWND:

• A Visual has a graphical representation.
• A Visual keeps track of its dirty regions.
• If dirty, a Visual automatically re-renders each frame.
• Visuals can be composed into trees.
• A Visual can only be parented once.  (i.e., you can not share/reuse Visuals)
• Visuals can be positioned relative to their parent via a Transform.
• You can hit test a Visual.

There are however some notable differences between Visuals and HWNDs.  Probably the most notable is that WPF is a retained graphics architecture.  Visuals do not call back to user code to repaint when invalidated.  Instead you specify a Visual’s content (i.e., a description of what it should render).  If the Visual is dirtied for any reason it automatically repaints the dirty portion from the content specification on the next frame.  A sample Visual tree is pictured below:

Most anything that renders in the WPF derives from Visual.  For example, Controls derive from FrameworkElement which derives from UIElement which derives from Visual.

What is Content?

Content is a description of what a Visual should render.  There are a few different APIs for specifying 2D content.  Currently in 3D the only way to specify content is via a graph of Model3Ds attached to a ModelVisual3D.  Like Visuals, you can build graphs of content using DrawingGroup (2D) or Model3DGroup (3D).  You can also position content via a Transform.  However, despite these superficial similarities there are important differences between Visuals and content:

• Visuals are optimized to be scene nodes.  Content are serialized rendering instructions for scene nodes.
• Visuals are the base class for Shapes and Controls.  Content classes are sealed.
• Visuals are single-use.  Content can be shared/reused between Visuals.
• Visuals support services like hit testing.  Content are serialized rendering instructions and nothing else.

Where Do Visuals End and Content Begin?

The distinction between Visuals and content is usually clear.  The one area which can be a bit subjective is whether a set of drawing primitives should be grouped together in content or by a tree of Visuals.  To give a concrete example, lets say you have a spaceship mesh that you need to appear three places on screen.  Should you use a single Visual that renders 3 spaceships?

Or 3 Visuals which render one spaceship each?

The answer depends.  If a cluster of 3 spaceships are flying together in tight formation then treating them as a single model may be reasonable, especially if you want several of these clusters cruising around the screen.  Or if you have a piece of clip-art featuring three tiny animated spaceships dog fighting around the 3D logo of your BSG fan site you might want to treat the clip-art, spaceships and all, as a single piece of content.

More typically you will want to each spaceship to be a separate node in the Visual tree.  Although you could use DrawingGroup and Model3DGroup to compose everything in your scene (assuming you do not need any Visual services like hit testing), this was not really the intent of DrawingGroup and Model3DGroup.  The purpose of DrawingGroup and Model3DGroup is to group together the pieces of a single shareable/reusable drawing or model.

I have discovered an effective way to make sure you follow up on your promises: arrange to have them captured on video.  I had not been able to find the 15 minutes to try Rob’s suggestion for inserting a resource into a collection since November, but somehow tonight was different.  I actually heard from the community before I received the internal email notifying me that the segment had been posted.  :)

Here is the updated source which no longer uses the WindowsLoaded event.  It’s now 100% Xaml.  The trick I was missing was the <x:Include x:Content="{ball}" /> which is a Xaml element that evaluates to a reference to the resource named “ball”.  This allows me to take the same animated bouncing ball model and insert it into two Viewport3Ds to efficiently create the mirroring effect in Xaml which I was previously falling back to C# in the code-behind to do.

But there are some differences (mostly in the code):

• It's running on Windows XP (note the Luna theme with blue color scheme - blue is very important!)
• The source comes in the form of a VS.NET solution.
• Window1.xaml's code behind is almost empty:
  • The ball is now animated entirely using the platform animation features instead of a Timer.
  • The ball's geometry is shared via <Application.Resources> in MyApp.xaml instead of being parsed from a string at runtime.

If you want to have some fun with the animation, try moving the animated RotateTransform3D to the end of the Transform3DCollection to get an effect that looks less like physics and more like poltergeist activity:

(If you're not sure what is going on, this might help.)