Daniel Lehenbauer's Blog

My not-so new opportunity...

Daniel Lehenbauer — Wed, 13 May 2009 16:13:00 GMT

It is perhaps an understatement to say it's been a while since I blogged.

As is commonly the case with Microsoft blogs, the reason for going dark is that I left the WPF team almost two years ago for an opportunity I couldn't pass by to join a systems incubation startup.

We're a small team with big ambitions, and as you might expect, my role involves the exploration of a radically different approach to the UI/Graphics platform which guarantees security, responsiveness, and leverages modern GPUs and manycore. To deliver on this vision, we are revisiting every layer of the stack from device drivers, through rendering engines, up to application frameworks and programming/computation models.

While we’re not specifically seeking to fill headcount on our team, we are always on the lookout for luminaries with insight, experience, and who love to write code. If you are interested in joining a deep systems incubation that believes in small, hand-picked teams of truly outstanding developers, please do contact me.

While much is demanded of everyone in this group, we also get to participate in what I personally believe is the most exciting and revolutionary work to happen in the industry since PARC.

New Blender Exporter Features: Double Sided & Solid (Flat) Shading

Daniel Lehenbauer — Fri, 23 Jun 2006 07:33:00 GMT

[UPDATE: The Xaml Exporter now is hosted on CodePlex (here)]

I checked in an updated Blender exporter to 3D Tools for Windows Presentation Foundation which adds support for double sided materials and flat shaded geometry. I have not yet package a new release, so for the moment you’ll need to grab xaml_export.py from source control.

These features are accessible via the circled buttons on the material and mesh panels during editing. Thanks to SEISENBERG for reporting this in the message board.

Looking for ScreenSpaceLines3D? -- 3D Tools v0.4 released

Daniel Lehenbauer — Tue, 30 May 2006 01:08:13 GMT

I have posted a new release of the 3D Tools for the Windows Presentation Foundation targeting the Beta 2 release of the WinFX runtime. The big addition in this release is a community owned implementation of the ScreenSpaceLines3D class.

ScreenSpaceLines3D is a 3D line primitive whose thickness is constant in 2D space post projection.* This means that the lines do not become “skinnier” as they receded from a perspective camera. ScreenSpaceLines3D are most frequently used to implement bounding boxes, wireframe mode, or visualization of normals for editors. Occasionally they are used in 3D scenes to represent telephone wires and similar objects that are too thin to be rendered reliably under a perspective projection.

I also started a MathUtils class which contains some generally useful utility methods, such as getting the View/Projection matrices from any camera and a method which returns the 3D -> 2D transform (useful for positioning 2D content, like text labels, relative to 3D content.) I’ve also added a wireframe mode to the ModelViewer sample to test ScreenSpaceLines3D.

The project has been refactored into just two assemblies: 3DTools.dll which contains the reusable runtime components and the ModelViewer.exe sample. Enjoy!

* The name ScreenSpaceLines3D was a bit of a misnomer. The lines were actually of constant thickness in the post-projective space of the Viewport3DVisual, not “device space” or “screen space” as the name implied.

Sphere, Cone, and Cylinder Sample Updated

Daniel Lehenbauer — Sun, 12 Feb 2006 07:23:00 GMT

Minor update: The "Extending Visual3D - Sphere, Cone, and Cylinder" sample has been updated to the December and January CTP (either will work). Thanks to Barry Briggs for doing this and sending me the updated source.

Xaml Export Script for Blender

Daniel Lehenbauer — Sun, 15 Jan 2006 07:13:00 GMT

[UPDATE: The Xaml Exporter now is hosted on CodePlex (here)]

The 3D features in the Windows Presentation Foundation make it easier than ever to integrate 3D content into your applications. However, unless you are satisfied with basic solid geometry you are going to need more than just the WPF. You’re going to need 3D authoring tools to create interesting content.

Unfortunately 3D tools tend to use their own proprietary file formats with limited import/export support for other formats. Xaml, being new to the scene (no pun intended), is not yet supported by most tools. However, some modeling programs feature an add-in architecture that allows custom exporters to be written by 3rd parties. The free 3D software package Blender is such a tool.

All very interesting you say, but where is this going?

I have posted a new release of the 3D Tools for the WPF which includes a Python script to export Blender scenes as Xaml. The Xaml can then be dynamically loaded or compiled into your WPF applications. Figure 1a shows a 3D model being edited with Blender. Figure 1b shows the final exported Xaml as viewed with the WPF Model Viewer.


Figure 1a The Steve Ballmer model is authored in Blender and then exported to Xaml.*	Figure 1b The final model as it appears in the WPF Model Viewer.
(Click for full sized images)

Because the WPF and Blender have different goals for their rendering engines their feature sets do not match precisely. Specifically, the WPF uses the hardware accelerated real-time rendering capabilities of your GPU. Blender is a software based rendering engine not targeted at real-time applications. However, the results of the export are frequently quite good. Figure 2a is the same 3D model as rendered by Blender. Figure 2b is the exported model viewed from a similar angle with the same lighting configuration in the WPF Model Viewer.


Figure 2a Steve Ballmer model rendered using Blender’s internal software engine.*	Figure 2b Exported model rendered using the WPF. The blue background is not part of the scene.
(Click for full sized images)

To get the Xaml Export Script for Blender and the WPF Model Viewer download the latest version of the 3D Tools for WPF (available here). Copy the xaml_export.py script to your Blender scripts directory. Blender will automatically add the script to the ~File~Export menu the next time it is started. More detailed instructions and a list of known issues are included in the Read-Me.

* Steve Ballmer model courtesy of Bonnie Lehenbauer

Why did we change LookAtPoint to LookDirection?

Daniel Lehenbauer — Wed, 04 Jan 2006 09:43:44 GMT

Ernie posted about the differences between the old LookAtPoint property and the new LookDirection property on ProjectionCamera (via KarstenJ.) The quick summary is that LookAtPoint caused the camera to look at a fixed point in space (Figure 1a). The new LookDirection property causes the camera to look in a fixed direction (Figure 1b).

Figure 1a

When the camera’s position moves from point A to point B it is automatically re-orientated so that the LookAtPoint remains centered in its view.

Figure 1b

When the camera’s position moves from point A to point B it continues to face in the direction specified by the LookDirection.

In terms of the camera configurations that can be represented the two properties are equally expressive. It is easy to convert between the two using the following equation:

LookDirection = LookAtPoint - Position

However, there are some pragmatic advantages to choosing LookDirection. One is that LookDirection is similar to other properties in the system (e.g., SpotLight.Direction.) Another is that LookAtPoint leads to a common misunderstanding that the camera can be rotated by simply updating the camera’s position. Unless you also transform the UpDirection this will lead to (usually) unexpected flipping behavior at the poles.

It was the addition of Camera.Transform, however, that was the largest motivation to switch to LookDirection. Being able to apply Transform3Ds to Cameras simplifies a lot of animation scenarios. In fact, it’s currently the only way to animate a MatrixCamera. Also, the ability to share Transform3Ds between Cameras and nodes in the Visual tree enables scenarios where the Camera follows an object in the scene. (Incidentally, this is how the headlight feature of the ModelViewer sample works.)

The addition of Camera.Transform raised questions about LookAtPoint. Should LookAtPoint be affected by the transform or should it continue to be defined in world space? If LookAtPoint is in world space, as most people initially suppose, it breaks the illusion that the camera is a physical object in the scene because rotating the camera 180 degrees does not necessarily mean that the camera is looking in the opposite direction since it will continue to look in the direction of the same fixed point in world space.

On the other hand, if the camera’s transform LookAtPoint is affected by the transform then users need to convert into the camera’s coordinate system to specify the LookAtPoint. This is no less difficult than calculating the LookDirection and more error prone.

When all these things were considered: the difficulty in reconciling LookAtPoint with Camera.Transform, the fact that the perceived simplicity of LookAtPoint is negated by the need to handle the UpDirection, and that elsewhere in the system we have consistently specified orientation with direction vectors, we decided that in the long term our customers will be happier with LookDirection.

3D Tools Workspace updated to December CTP bits

Daniel Lehenbauer — Tue, 20 Dec 2005 22:58:00 GMT

I've packaged a new release of the 3D Tools workspace targeting the December CTP WinFX bits. This release adds two requested features to the ModelViewer sample. The first is the ability to light the scene. This is convenient if you are viewing a model which does not contain lights. There are two options for lighting:

1. Headlight – This will add a white directional light which follows the viewing direction of the camera (imagine a flashlight duct-taped to the front of your camera.) The headlight is now on by default.

2. Ambient Light – This will add a white ambient light which will fully illuminate the scene. The lack of depth cues this causes will make the scene appear “flat”, but this mode can be useful for examining a model for correctness. The ambient light is off by default.


No lights	Headlight	Ambient Light

Changing the lighting options in the ModelViewer does not change the model (i.e., it is non-destructive.)

The other new feature is the ability to reload the currently open file to view any changes since the file was last opened. Both new features are available via the View menu.

The release contains both binary and sources. To run the ModelViewer, you need .NET Framework 2.0 Redist and the December CTP WinFX Runtime. To build the solution you also need the Windows SDK and optionally VS 2005 and the VS Extensions for WinFX.

Rotating the Camera with the Mouse

Daniel Lehenbauer — Fri, 16 Dec 2005 05:29:00 GMT

Possibly the most common 3D UI paradigm is to rotate an object with the mouse in order to view it from all angles. This is illustrated by the images of the tiger model below:

Figure 1a

Tiger model in its original configuration

Figure 1b

Tiger model after the mouse has been dragged to the left and slightly down.

This UI technique is known as a "virtual trackball" and it has been a staple of interactive computer graphics since the 80s. It has applications ranging from CAD to department store kiosks to gaming.

To help developers working with the Windows Presentation Foundation get a jump-start using 3D in their applications I have added a sample trackball implementation to the 3DTools workspace with an accompanying article which discusses the mechanics of how a trackball works.

Reading the full article is not required to successfully use the sample code in your own applications, but skimming it might be useful. You do not need to be a member of the workspace in order to download releases which include both binaries and source (here).

WinFX November CTP Released!

Daniel Lehenbauer — Wed, 30 Nov 2005 21:54:00 GMT

If you are like me and have been holding off on installing the final version of the .NET Framework 2.0 and Visual Studio 2005 until you had Windows Presentation Foundation (formerly code-named Avalon) bits to work with then the wait is over. The November CTP of the WPF works with the final .NET/VS bits.

Tim Sneath posted links to the downloads.
Karsten Januszewski has a summary changes in this release.

Not having to reinstall VS for each WinFX release should make it a lot easier to keep up with future CTPs.

3D for the Rest of Us: Texture Coordinates

Daniel Lehenbauer — Mon, 07 Nov 2005 02:05:00 GMT

Introduction

People new to 3D are frequently confused by the extra level of indirection introduced by texture coordinates. 3D veterans familiar with texture coordinates from other platforms are sometimes surprised that the default behavior in the Windows Presentation Foundation (formerly code-named Avalon) is a bit different than what they are accustomed to in other 3D APIs. Below is a quick overview of what texture coordinates are and how they are specified. The last section talks about the default behavior in the WPF and how the tradition behavior can be attained by tweaking Brush properties.

What are Texture Coordinates?

Texture coordinates specify which part of the Material shows up an which part of the MeshGeometry3D. Developers from a 2D background are usually surprised that this is necessary. Most 2D graphics APIs give the users drawing primitives like Ellipses, Rectangles, Paths, etc. with a predefined fill behavior that stretches the brush to the bounds of the given geometry. This is the default in WPF as well. Consider the following diagonal GradientBrush which is white in the upper left corner, red in the middle, and black in the lower right corner:

<LinearGradientBrush x:Key="MyGrad" StartPoint="0,0" EndPoint="1,1">
    <LinearGradientBrush.GradientStops>
      <GradientStopCollection>
            <GradientStop Color="#FFFFFF" Offset="0" />
            <GradientStop Color="#FF0000" Offset="0.5" />
            <GradientStop Color="#000000" Offset="1" />
        </GradientStopCollection>
</LinearGradientBrush.GradientStops>
</LinearGradientBrush>

By default when a brush is used to fill a 2D geometry the brush is applied relative to the bounding box of the shape. If applied to a Rectangle like the 100x100 square below the entire brush will be visible. If the geometry does not entirely fill it’s bounding box parts of the brush will not be visible. For example, the non-rectangular parallelogram below does not show the extreme white and black corners of the brush.

People working with 3D for the first time often expect that materials will be similarly stretched to fit the shape of their geometry. Indeed, in my last post my sample 3D primitives have default texture coordinates which mimic this behavior:

However, there is no single mapping which provides a reasonable default for all 3D geometry. Even with simple geometry like the 3 quadrics depicted above the method for generating the texture coordinates of the sphere differ from that of the cone and cylinder. In order to make the material wrap sensibly around each primitive I had to explicitly specify a 2D texture coordinate for each 3D position. For example, the cone primitive generates a mesh similar to the one depicted below. Each 3D position in the mesh has a corresponding 2D texture coordinate which maps that 3D position to a 2D coordinate in the material.

Specifying Texture Coordinates

You specify texture coordinates by adding 2D Points to the MeshGeometry3D.TextureCoordinates collection. For example, the following MeshGeometry3D maps the lower right half portion of the material to a triangle:

<MeshGeometry3D
    Positions="-1,1,0 -1,-1,0 1,-1,0"
    TextureCoordinates="0,0 0,1 1,1"
    TriangleIndices="0,1,2" />

The ith entry in the TextureCoordinates list corresponds to the ith entry in the Positions list, so texture coordinate 0,0 is associated with the vertex at position -1,1,0. The mapping between texture coordinates and positions is depicted below:

Texture Coordinates Rendered Triangle

Of course, this is not the only possibly mapping. If we change the 2nd texture coordinate from 0,1 to 0,0.5 we produce the following image:

Texture Coordinates Rendered Triangle

Notice that the area of the triangle specified by the texture coordinates on the left is stretched to fit the render triangle.

Texture coordinates are required when using a material with an ImageBrush, DrawingBrush, VisualBrush, LinearGradientBrush, or RadialGradientBrush (i.e., any Brush other than a SolidColorBrush in which case every position in the mesh maps to the same color.)

Texture Coordinates in the Windows Presentation Foundation

There are a couple of behavior differences that 3D veterans should be aware of when working with texture coordinates in the Windows Presentation Foundation. These differences arise from the fact that 3D Materials leverage 2D Brushes as the source of their textures. The wins from being able to take any arbitrary 2D brush and use it to paint a 3D mesh are enormous. However brushes default to behavior which, while traditional for 2D, is sometimes surprising for 3D developers. Fortunately brushes are highly configurable and it is easy to get the typical 3D behavior.

The first difference is that the +Y axis in brush space (and consequently texture coordinate space) points down by default. This is consistent with 2D but different from the traditional 3D coordinate system where the +Y axis points up. In order to get the +Y axis pointing up apply a Y scale of -1 and Y translation of +1:

<ImageBrush Transform="scale(1,-1) translate(0,1)" ImageSource="…" />

The second difference is that the brushes default to relative units. At the beginning of this article I discussed how by default the brush maps to the relative bounding box of the 2D geometry it is being used to fill. The same thing happens by default with the bounding box of your texture coordinates. In the above samples I happened to use texture coordinates in the range of 0..1, but texture coordinates of “-20,-20 -20,10 10,10” would have also filled the triangle with the same part of the image. TileBrush’s Viewport property defaults to the rect (0,0)-(1,1) so to get the standard behavior where the bounds of the texture source maps to 0..1 you just need to specify you are using absolute units:

<ImageBrush ViewportUnits="Absolute" Transform="scale(1,-1) translate(0,1)" ImageSource="…" />

These 2 settings should configure brush / texture space to be identical to what you are probably used to from other APIs. (Incidently, all of these Brush settings have the same affect in 2D as well.) Finally, if you need tiling you turn it on with TileMode=”Tile”.

Extending Visual3D - Sphere, Cone, and Cylinder

Daniel Lehenbauer — Sun, 16 Oct 2005 20:38:00 GMT

Update: The source has been updated for the December or January CTP (either will work).

In my previous post I mentioned that one of the motivations for Visual3D was sub-classing. Prior to the September CTP of the Windows Presentation Foundation (formerly code-named Avalon) there was no extensibility point for creating your own 3D scene nodes. One of the many possible uses for this new extensibility point is to build a library of 3D primitives. I put together a short sample which derives a Sphere, a Cone, and a Cylinder primitive from ModelVisual3D. (Source available here.)

You will notice that these primitives are directly usable from Xaml:

<Viewport3D Camera=”{Camera}” >
<my:Sphere3D Transform=”{XForm1}” Material=”{Azul}” />
<my:Cone3D Transform=”{XForm2}” Material=”{Tulips}” />
<my:Cylinder3D Transform=”{XForm3}” Material=”{Autumn}” />
</Viewport3D>

The primitives are centered at the origin and have a natural size of Radius=1 and Height=2 (i.e., they are bounded by (-1,-1,-1) – (1,1,1)).

This sample does not constitute a terribly complete or expressive library of 3D primitives, but it does illustrate how one could be built. More compelling examples include deriving from Visual3Ds to add new events and behavior (like collision detection) or surfacing properties like databindable properties which dynamically generate 3D charts and graphs.

ModelVisual3D vs. Model3DGroup -- When to use which?

Daniel Lehenbauer — Mon, 10 Oct 2005 01:44:00 GMT

(NOTE: You might find Karsten's pragmatic perspective a useful prelude before we dive into the artictural underpinnings.)

In the September CTP release of the Windows Presentation Foundation (formerly code-named Avalon) we made 3D a first class citizen in the Visual tree with the introducion of Visual3D. Up until this point 3D existed solely as content attached to a 2D Visual . 3D users migrating to the September CTP will encounter the Visual/Content split for the first time. This has lead to some confusion about when nested ModelVisual3Ds should be used to group models together vs. a Model3DGroup. Below is a short description of Visuals and Content and how they fit together. At the end is a concrete example with guidelines on when to use a ModelVisual3D vs. a Model3DGroup.

What is a Visual?

A Visual (or Visual3D) is a low-level node in the composition tree. For people coming from a Win32 background a Visual is frequently compared to an HWND:

A Visual has a graphical representation.
A Visual keeps track of its dirty regions.
If dirty, a Visual automatically re-renders each frame.
Visuals can be composed into trees.
A Visual can only be parented once. (i.e., you can not share/reuse Visuals)
Visuals can be positioned relative to their parent via a Transform.
You can hit test a Visual.

There are however some notable differences between Visuals and HWNDs. Probably the most notable is that WPF is a retained graphics architecture. Visuals do not call back to user code to repaint when invalidated. Instead you specify a Visual’s content (i.e., a description of what it should render). If the Visual is dirtied for any reason it automatically repaints the dirty portion from the content specification on the next frame. A sample Visual tree is pictured below:

Most anything that renders in the WPF derives from Visual. For example, Controls derive from FrameworkElement which derives from UIElement which derives from Visual.

What is Content?

Content is a description of what a Visual should render. There are a few different APIs for specifying 2D content. Currently in 3D the only way to specify content is via a graph of Model3Ds attached to a ModelVisual3D. Like Visuals, you can build graphs of content using DrawingGroup (2D) or Model3DGroup (3D). You can also position content via a Transform. However, despite these superficial similarities there are important differences between Visuals and content:

Visuals are optimized to be scene nodes. Content are serialized rendering instructions for scene nodes.
Visuals are the base class for Shapes and Controls. Content classes are sealed.
Visuals are single-use. Content can be shared/reused between Visuals.
Visuals support services like hit testing. Content are serialized rendering instructions and nothing else.

Where Do Visuals End and Content Begin?

The distinction between Visuals and content is usually clear. The one area which can be a bit subjective is whether a set of drawing primitives should be grouped together in content or by a tree of Visuals. To give a concrete example, lets say you have a spaceship mesh that you need to appear three places on screen. Should you use a single Visual that renders 3 spaceships?

Or 3 Visuals which render one spaceship each?

The answer depends. If a cluster of 3 spaceships are flying together in tight formation then treating them as a single model may be reasonable, especially if you want several of these clusters cruising around the screen. Or if you have a piece of clip-art featuring three tiny animated spaceships dog fighting around the 3D logo of your BSG fan site you might want to treat the clip-art, spaceships and all, as a single piece of content.

More typically you will want to each spaceship to be a separate node in the Visual tree. Although you could use DrawingGroup and Model3DGroup to compose everything in your scene (assuming you do not need any Visual services like hit testing), this was not really the intent of DrawingGroup and Model3DGroup. The purpose of DrawingGroup and Model3DGroup is to group together the pieces of a single shareable/reusable drawing or model.

Mantis 3D Space Shooter

Daniel Lehenbauer — Tue, 04 Oct 2005 08:36:00 GMT

When we decided to integrate 3D into the Windows Presentation Foundation (Avalon) the scenarios we were targeting for v1 were:

3D effects, like a “Next Page” transition that looks like the turning of a page in a book
Data visualization (charts, graphs, etc.)
Catalogs where you can rotate items in 3D
3D icons and vector graphics styled on to 2D controls (Buttons, ListViewItems, etc.)
A mall kiosk which shows an interactive floor plan in 3D

…and related scenarios which add a bit of sizzle to traditional applications excluding games. However, while gaming was not a targeted scenario for v1 there is some coincidental overlap which makes WPF a decent option for certain types of games. (Mostly games where you are clicking with the mouse instead of madly staffing. :)

One example is Mantis, a polished looking 3D space shooter that Mitch Walker has been building using the Windows Presentation Foundation (Avalon) and Windows Communication Foundation (Indigo). Follow the links to his blog for a description of the project, future directions, and a video snippet of actual game play. Here are a couple of screenshots from Mitch’s blog to entice you:

Minor Boing Update for Channel 9 Interview...

Daniel Lehenbauer — Wed, 05 Jan 2005 06:10:00 GMT

I have discovered an effective way to make sure you follow up on your promises: arrange to have them captured on video. I had not been able to find the 15 minutes to try Rob’s suggestion for inserting a resource into a collection since November, but somehow tonight was different. I actually heard from the community before I received the internal email notifying me that the segment had been posted. :)

Here is the updated source which no longer uses the WindowsLoaded event. It’s now 100% Xaml. The trick I was missing was the <x:Include x:Content="{ball}" /> which is a Xaml element that evaluates to a reference to the resource named “ball”. This allows me to take the same animated bouncing ball model and insert it into two Viewport3Ds to efficiently create the mirroring effect in Xaml which I was previously falling back to C# in the code-behind to do.

Bouncing Back...

Daniel Lehenbauer — Tue, 23 Nov 2004 08:39:00 GMT

Trying to ease back into the community after a short hiatus, I ported the Boing demo that I blogged about back in June to the Avalon CTP build. You can get the updated source here. Just load the solution in to the VS.NET Whidbey Beta and press F5. If you are using Visual C# Express, see RRelyea's notes (via ChrisAn).

You will probably notice that the demo hasn't changed much since WinHEC:

But there are some differences (mostly in the code):

It's running on Windows XP (note the Luna theme with blue color scheme - blue is very important!)
The source comes in the form of a VS.NET solution.
Window1.xaml's code behind is almost empty:
- The ball is now animated entirely using the platform animation features instead of a Timer.
- The ball's geometry is shared via <Application.Resources> in MyApp.xaml instead of being parsed from a string at runtime.

If you want to have some fun with the animation, try moving the animated RotateTransform3D to the end of the Transform3DCollection to get an effect that looks less like physics and more like poltergeist activity:

(If you're not sure what is going on, this might help.)

Avalon CTP released -- Yes it runs on XP!

Daniel Lehenbauer — Mon, 22 Nov 2004 01:54:00 GMT

Like many many others I am also excited about the Avalon CTP release for a number of reasons:

It runs on Windows XP! (I no longer need to boot into a separate partition to write samples for my (recently neglected) blog ;)
We have the beginnings of tool support! The WinFX SDK installs templates for VS.NET for Avalon projects.
The have been numerous advancements across Avalon since WinHEC.

Up until this point only the very brave, installing Longhorn on a spare box and working entirely from the command line, have been able to provide us with feedback. I am hopeful that the CTP release, while still a bit unpolished and clearly not final, will be accessible enough that we will begin to hear more from the community.

How do you get it? It's available for download for MSDN subscribers (including Academic Alliance).

The Donut Primitive, V2

Daniel Lehenbauer — Sun, 05 Sep 2004 09:05:00 GMT

I had mentioned previously that I was dissatisfied with the 1^st post in the 3D for the Rest of Us series because I felt that the post did not leave the reader with anything they could immediately go out and try. I was much happier with the 2^nd post, even more so now that I have seen The Donut Primitive, V2. Apologies for the delay of the 3^rd post. I was in and out of the office sporadically over August.

3D for the Rest of Us, Part 2: Transforms

Daniel Lehenbauer — Sun, 08 Aug 2004 22:44:00 GMT

Introduction

This is the second in a series of posts intended to fill in the missing pieces for a developer familiar with a 2D graphics API who wants to dabble in 3D with the Avalon platform To be honest, I really was not particularily happy with the first article. I wanted this series to be a very pragmatic introduction to 3D graphics and my last article quickly veered into into the abstract with its discussion about model space vs. world space. I aim to correct that this time around. This time we will be exploring transforms and using them to construct the 3D axis model we saw in the previous article. Along the way we will need to acquaint ourselves with several classes in the System.Windows.Media3D namespace.

Model3D

Model3D is the abstract base class for any object which participates as a node in the 3D scene graph. From 2D you are probably familiar with building UI by creating a tree of UIElements (or controls, or HWNDs, etc. depending on the terminology used by your favorite platform). Each element probably surfaced properties like Top, Left, Width, and Height, etc. to position and size the child control relative to its parent. Model3D is similar to the element base class from your favorite 2D platform. You create a 3D scene by building a graph of objects which all derive from Model3D. Model3D has a Transform property which can be used to position, size, and orient the Model3D relative to its parent.

The comparison with UIElement is not entirely accurate, however. UIElements are heavier weight objects which support layout, events, are extensible, etc. Model3Ds are light weight sealed graphics primitives.

Model3DCollection

The Model3DCollection is a concrete subclass of Model3D. The purpose of a Model3DCollection is to group a set of Model3Ds together so they can be transform (moved, rotated, scaled, etc.) together. Continuing the UIElement analogy, a Model3DCollection would be like a container element (or panel, canvas, etc.) You add Model3Ds to a Model3DCollection via the Children property.

Example:

Model3DCollection group = new Model3DCollection();
group.Children.Add(myModel1);
group.Children.Add(myModel2);
group.Transform = new TranslateTransform3D(...); // Move entire group

Transform3D

We talked about transforms at a high level in the first article. Transform3D is the abstract base class for our 3D transform classes.

TranslateTransform3D

Applying a TranslateTransform3D to a Model3D effectively moves all of the points in the Model3D in the direction of the supplied offset vector. For example, an offset vector of <0,1.6,1> would move the point (4,4,4) to (4,5.6,5). (I say effectively because as we discussed last time we are not modifying the value of our point. The point is still (4,4,4) in model space. What we have done is change our local frame of reference so that (4,4,4) in model space is (4,5.6,5) in world space.)

Example:

myModel.Transform = new TranslateTransform3D(new Vector3D(0, 1.6, 1));

ScaleTransform3D

ScaleTransform3D effectively scales the model by the given scale vector about the given center point. Applying a uniform scale (a scale where the scale vector has the same value for X, Y, and Z) proportionally changes the size of the object. For example, a scale of <0.5,0.5,0.5> would make the object 1/2 of its original size while a scale of <2,2,2> would double the model's size. (A scale of <1,1,1> leaves the object the same size). Applying a non-uniform scale (one where the values of X, Y, Z differ) "stretches" or "squishes" the model. For example, <1,2,1> would leave the model the same width and thickness, but make it twice as tall.

By default, scaling causes vertices to expand or contract about the origin (0,0,0) which may not always be the desired behavior. Consider a line segment with points at (2,2,2) and (4,4,4). Applying a uniform scale of <0.1, 0.1, 0.1> you quite possibly want to resize the line segment "in place". That is, since the original center of the segment was at (3,3,3) you expect the new segment to be (2.9,2.9,2,9) - (3.1,3.1,3.1). In actuality what happens is that each component of the end points is multiplied by 0.1 and you get the segment (0.2, 0.2, 0.2) - (0.4, 0.4, 0.4) which is of the expected length, but the position has translated towards the origin. It is common practice to author models with the center about the origin and translate them into place to avoid this issue, but in cases where this is not convenient the ScaleTransform3D offers a ScaleCenter property which allows you to specify the point in 3-space you would like the scale to expand/contract about. If you choose this point to be the perceived center of your model your model will scale "in place". (In actuality what is happening when you define the ScaleCenter is that model space is scaled and then translated to put the center point back in it's original position in world space.)

Example:

myModel.Transform = new ScaleTransform3D(new Vector3D(2, 2, 2));

RotateTransform3D

RotateTransform3D effectively rotates a model the specified number of degrees about the specified axis. The figures below illustrate rotation about the three cardinal axes.

As we mentioned previously, Avalon 3D is a right handed platform which means that if curl your fingers and point the thumb of your right hand in the direction of the axis your are rotating about the direction of the curl of your fingers is the direction of rotation with a positive value for angle (counter-clockwise). Similar to ScaleTransform3D, RotateTransform3D exposes a Center property which allows you to control the center of rotation:

Unlike translation and positive scale transforms which move the origin and change the scale of the coordinate system but leave the cardinal axes pointing in the original direction, rotation changes the direction of the axes in model space. (Aside: a negative scalar will invert the axis which effectively mirrors the model.)

Example:

myModel.Transform = new RotateTransform3D(new Vector3D(0, 1, 0), 45);

Transform3DCollection

Frequently you want to apply more than one transform to a model (e.g. rotate it and then translate it). A Transform3DCollection allows you to do this. Transforms added to the Transform3DCollection's Children property are applied to the model in first to last order. Ordering is important because translating and then rotating does not yield the same result as rotating and then translating:

This assumes rotation about the origin. Remember that using the Center property you could have rotated about the point (1,0,0) in figure 5b which would have then rotated the square "in place" and yielded the same end result as Figure 5a. Because geometry is typically modeled about the origin, first sizing the model (scale), then orientating it (rotate), and then moving it to the desired location (translate) commonly results in the desired outcome. There are scenarios where you do want different orderings (e.g., you want to pivot the translated model as shown in 5b). In my experience I find that it usually is more natural to add more hierarchy to your scene (in this case meaning a parent Model3DCollection which rotates the translated child), but there are exceptions of course.

Example:

Transform3DCollection xforms = new Transform3DCollection();
xforms.Children.Add(new ScaleTransform3D(...));
xforms.Children.Add(new RotateTransform3D(...));
xforms.Children.Add(new TranslateTransform3D(...));
myModel.Transform = xforms;

Building the Axes Model

We are now to the "hands-on" portion of this article. In part 1 I used a model of the three cardinal axes in my illustrations for which I conspicuously did not provide source.

This time you get the source, at least most of it. Building the scene graph for the axes model has been removed and left as an exercise for the reader. You will find a comment in the OnLoaded handler in Window1.xaml.cs indicating where this should be done. The code contains a cylinder and cone models as well as models of the letters used to label the axes. (All converted with IanG's x-to-xaml converter). Your task is to use Model3DCollections with the various Transform3Ds we have been discussing to construct the scene graph for the axes shown above. (Download the source here.)

Step 1: Building a reusable arrow

The first step will be to build a reusable arrow model from the provided Cone and Cylinder models:

Both models fit approximately into the unit cube (a cube 1.0 units tall, wide, and deep) and are centered about the origin. What you will want to do is apply a non-uniform scale to the cylinder to keep it 1.0 unit tall, but make it "skinnier". You will then want to perform a uniform scale to the cone to make it smaller and translate it to above the cylinder. Finally, you will want to group both of these primitives together into a Model3DCollection to create an arrow Model3D that you can re-use in your scene. This part of the scene graph will look roughly like this:

With some experimentation you should be able to create an arrow which looks approximately like this:

Step 2: Composing arrows into the axes

The next step is to take our newly created arrow model, orientate and translate it into position, and then draw it for each of the cardinal axes. This raises the question how do we get our arrow to draw 3 times. One solution would be to use the Copy() method to clone the arrow so we have three copies which can then be grouped under a single Model3DCollection:

This works, however each of those arrows models contains an identical copy of the same 3D geometry. A lighter weight solution is to take advantage of multiparenting of Model3Ds to draw the same arrow three times, but adding new transforms each time to position it differently.

Using multiparenting when possible conserves memory and is considered a best practice, but either approach is fine for a small model like our axes. Whichever method you choose, the result you are looking for at the end of step 2 is:

Step 3: Adding the axis labels

The finishing touch is to reapply the same techniques to add the letter models to the scene to label the axes. As an added challenge my 3D modeler exported these models such that by default the infinitly thin edge is facing the camera. The models pictured below have been rotated 90 degrees about the Y-axis to make them visible:

You may want to add a rotation to your top Model3DCollection to view the finished result from a few angles to verify that your Z-axis is as expected:

Conclusion

At this point you have now built your first scene graph. Along the way he have learned about grouping Model3Ds with a Model3DCollection and we have covered the 3 basic Transform3Ds and learned about combining them with a Transform3DCollection. Next time we will look at the ViewPort3D and Camera classes which will get you well on your way to building your first 3D application "from scratch".

Interpreted Python and WindowsForms experiment

Daniel Lehenbauer — Sun, 01 Aug 2004 03:47:00 GMT

I have been shopping around for a while now for a dynamic language to use when prototyping pet projects at home. Everything I read about Python seemed perfect except, of course, the lack of a .NET compiler. This week Jim Hugunin of Jython fame finally dropped the first binary build of IronPython (Python for .NET).

According to the readme.htm you can build regular IL assemblies (.exes and .dlls) with IronPython, but what I have been playing with is the interpreted python console.

Something I was pleasantly surprised to find was how well the design goals of WindowsForms mesh with the needs of interpreted languages. I think there are two reasons for this:

· We have tried to avoid invalid states (e.g., for the most part you can set properties in any order, you can call methods at any time, etc.) This is part of the Managed API Guidelines and is just common sense when building a predictable platform.

· The second is that WindowsForms supports imperative style programming. When working in a compiled language I tend to follow a pattern of subclassing Form and having all of my child controls encapsulated. When working with the python interpreter I found it easier to write in an imperative style where I instantiate a vanilla Form (i.e., not a MyDialog subclass) and start adding controls to it as follows:

>>> from System.Windows.Forms import *

>>> f = Form()

>>> b = Button()

>>> def OnClick(sender, args):

... MessageBox.Show("Clicked");

...

>>> b.Click += OnClick
>>> f.Controls.Add(b)

>>> Application.Run(f)

Which as it turns out, WindowsForms was intentionally engineered to support.

Why was WindowsForms engineer to support this you ask? It is because WindowsForms had an interpreter as a customer since day 1: the Visual Studio designer. When you are designing a dialog the designer does not create an instance of your derived class. It creates an instance of the base class (which is essentially a new Form() unless you are designing an inherited Form) and then interprets the code statements in the InitializeComponents() method to populate the form with buttons, checkboxes, etc. (BrianPe goes into the details here).

I have run into a snag, however, while experimenting with WindowsForms. When the Form is closed it is automatically disposed which means you can not tweak it and then Application.Run(f) a second time because WindowsForms does not allow resurrection of a Disposed() class. This kills iterative development which is pretty much the whole goal of an interpreted language. I haven't played with it long enough to know if there is a work around (I suspect with some foresight one could sink the Closing event to Hide() rather than Close() because I seem to recall that the Closing event was cancelable).

When I get a chance I would like to recompile IronPython with the CLR build from Longhorn 4074 and see how Avalon fairs in the same scenario. We have an intrinsic advantage in that so far we have avoided needing the Dispose() pattern for our controls.

Oh, and if you miss the news, Jim Hugunin is starting his first day as a Microsoft employee on Monday.

3D for the Rest of Us, Part 1: 3D Coordinate System

Daniel Lehenbauer — Tue, 20 Jul 2004 02:05:00 GMT

Introduction

A hallway conversation which Chris Anderson made me realize that when people ask me about 3D support in Avalon I have a bad habit of telling them everything they do not need to know. After all, it is all the details you do not have to worry about that consume the waking hours of our 3D team (and it is what gives the Avalon 3D platform its value.) However, despite our efforts to abstract the hardware, to encapsulate the mathematics, and to keep the 2D and 3D APIs symmetrical, there are some concepts which are inherently unique to 3D such as cameras and lighting.

This is the first in a series of posts intended to fill in the missing pieces for a developer familiar with a 2D graphics API who wants to dabble in 3D with the Avalon platform. The topics I initially intend to cover are:

The 3D coordinate system
Transforms - (Part 2 now online)
The Camera
Defining 3D Geometry
Lighting
Materials

...but I am happy to adjust according to feedback.

The 3D Coordinate System

Most people are aware that 3D adds an extra dimension, Z, perpendicular to the X and Y they are familiar with from 2D. The 3D coordinate system is frequently explained to 2D developers as follows:

The origin is now in the center of the screen instead of the upper left.
The positive X axis still points right, but Y now points up instead of down.
There is a new axis, Z, that points out of the screen (assuming a right handed coordinate system like Avalon).

This is expressed pictorially in figures 1 and 2 below:

The issue with this description is that while we can choose a camera orientation consistent with the above criteria (Figure 3) we can also choose vantage points like Figures 4 or even turn the camera upside down like Figure 5.


LookAtPoint=(0,0,0) Position=(0,0,5) Up=(0,1,0) Figure 3	LookAtPoint=(0,0,0) Position=(3,3,-3) Up=(0,1,0) Figure 4	LookAtPoint=(1,1,0) Position=(3,3,-3) Up=(0,-1,0) Figure 5

The global frame of reference (aka World Space)

From Figures 3-5 hopefully the reader begins to get a sense that there exists a space created by the 3 perpendicular axes. This space is commonly referred to as world space. World space extends infinitely in both the positive and negative directions for all three axes. In the above screenshots (Figures 3-5), the three axes always meet at (0,0,0) in world space (even though the location on the screen changes). In world space, the model of the arrow labeled X extends along the positive X-axis <1,0,0>, the Y arrow along <0,1,0>, and the Z along <0,0,1>. Again this is true in all three screenshots (Figures 3-5) even though the directions appear to change based on the camera orientation.

The point is, we can position our camera anywhere in world space and point it in any direction to generate images, but moving the camera does not change where things exist in world space. World space is the stationary global frame of reference. Since we are not focusing on positioning the camera this time around we will choose to adopt an orientation like Figure 6 (which was used to make the screenshot in Figure 3) for the remainder of this post.

Local Frames of Reference (aka Model space)

The fact that I stated that world space is the global frame of reference implies that there are other frames of reference. Each model object has a Transform property which we can use to move, size, orientate, etc. the model. When we do this, we create a local frame of reference. These local frames of references are commonly referred to as model spaces. The concept of model spaces also exists in 2D, although we do not call them that. Consider a canvas which contains a red rectangle defined to as (0,0)-(100,100) relative to the canvas:

Now consider Figure 8 which shows the canvas in relation to the screen. Here we can see that while the rectangle exists in (0,0) - (100,100) relative to the canvas' local frame of reference, it may actually occupy (50,50)-(150,150) on the screen.

In 3D, it is common practice to author your geometry centered about the origin like the teapot shown in Figure 9.

Figure 9

And then use the three basic transforms (translate, scale, and rotate) to position, size and orientate the model to where you want it in world space. The Avalon 3D object model exposes these transforms as the TranslateTransform3D, ScaleTransform3D and RotateTransform3D classes. Like the camera, we will touch on how to configure these transform classes in a future post. The point we want to make this time is that just like in Figure 8 where when the canvas is moved the rectangle is still defined as (0,0)-(100,100), when you apply a transform to a model 3D you are transforming the space in which the model is defined rather than the positions of the points in the geometry. This local frame of reference is called model space.


translate Figure 10	translate & scale Figure 11	translate, scale, and rotate Figure 12

Figures 10 through 12 show the effects of the basic transforms when applied to our teapot model. A second axis has been added to show the effect this has on the model space of the teapot. In Figure 10 we can see that when we add a translation of <0.5, 0,5, 0.5> this moved the teapot up, right, and towards the viewer. This also created a model space where (0,0,0) in the model space is equal to (0.5, 0.5, 0.5) in the global world space.

Similarly, Figure 11 illustrates that adding a uniform scale of 0.5 not only scales the teapot, but also scales the local model space in which the teapot resides. Moving 1 unit in this model space moves 0.5 units in global space. Finally Figure 12 shows that applying a rotation changes the orientation of the teapot and at the same time changes the orientation of model space. Moving along the Z axis in figure 12, for example, causes a change in all of X, Y, and Z in world space.

The importance of model space will became more clear next time when we investigate the transforms and the Model3DCollection class which allows us to build hierarchies of model spaces that allow us to do things like take the cone and cylinder which make one of the arrows in the axes we have been seeing in the screenshots and treat them as one "arrow" model even though they are separate primitives.

Continue: Part 2 now online

3D Graph Demo Posted

Daniel Lehenbauer — Fri, 02 Jul 2004 01:36:00 GMT

Coincidently, the same day that Kerry Hammil's talk which featured the 3D Blood Sugar Tracker demo aired Joe Beda managed to get the source hosted at MSDN.

Avalon 3D Video Segment

Daniel Lehenbauer — Thu, 01 Jul 2004 04:07:00 GMT

Kerry Hammil, a program manager with the Avalon team, discusses the goals and features of 3D support in Avalon and walks through writing your first Avalon 3D application in this 30 minute video segment (56K, 150K or 300K). You will want to be sure to stay until the end for the look at future Avalon 3D features and the 3D demos that Nathan Dunlap and others members of our demo team have put together.

Avalon 3D Hit Testing Demo

Daniel Lehenbauer — Mon, 28 Jun 2004 01:29:00 GMT

One area where the Avalon platform blurs the line between 2D and 3D is with our hit testing services. It's easy to discover that we support traditional ray/mesh hit testing via the Model3D.HitTest(...) API, but rarely does the user want to initiate hit testing in world 3-space (this is especially true for the 3D scenarios targeted by Avalon). Consider this simple demo (binary, source) which displays a rotating teapot which changes color when the user clicks on it. (Demo works on Longhorn build 4074).

Mouse input arrives to us a MouseButtonEventArgs. Using e.GetPosition(myViewPort) we get a 2D point relative to the bounds of our ViewPort3D. How does one go from a 2D point like (327, 216) to knowing which part of the teapot is under the mouse pointer? Further more, we do not just want to know if the teapot was hit, we want to know exactly where (in model space) the teapot was hit and in the case of multiple intersections we want distance information so we can sort our results. (And just to be clear, this is is not bounding box testing. Clicking in the "hole" of the teapot's handle should not register as a hit.)

Don't Do This

It turns out that this is a platform feature of Avalon so what I am about to describe is exactly what you do not need to know, but ignoring this for the moment the first step would be to use the bounds of the ViewPort3D to normalize the 2D point to the projection plane in post-projective homogeneous space (figure 1).

Of course, in 3-space we are not interested in hit testing a single point. We want to know everything that is under the mouse pointer so we need to project a ray through the point in the viewing direction (figure 2). In post-projective homogeneous space this is along the Z axis. Next we need to transform our ray back into world space. The implementation varies by camera, but is equivalent to transforming by the inverse of the projection transform and view transforms.

This gives us a ray in world space which is exactly the ray you need to provide to Model3D.HitTest(...) at the root of your model hierarchy which will then walk the Model3D hierachy and report which Model3Ds were intersected by the ray.

What You Want To Do

As I mentioned earlier, integrated 2D/3D is a platform feature of Avalon. If you call the 2D hit testing service on IVisual:

PointHitTestResult result2D = visual.HitTest(point);

and if a ViewPort3D is hit during the visual tree walk the point is automatically projected into the ray for the ViewPort3D's bounds and camera and hit testing continues seamlessly into 3D (figure 3).

If a Model3D is hit during the 3D portion of the hit test the PointHitTestResult object returned by IVisual.HitTest(...) can be cast into a Model3DHitTestResult from which you can find out which Model3D was hit and the details of the intersection.

IanG's First Look at 3-D Support in Avalon

Daniel Lehenbauer — Sun, 20 Jun 2004 01:17:00 GMT

MSDN just published an introductory article on 3D support in Avalon by fellow blogger Ian Griffiths of DevelopMentor. The article is liberal with screenshots and provides a good mapping between the high level concepts and the Avalon object model. It's a great way to get yourself oriented to the platfrom and includes a link to Ian's .x file to xaml convesion tool I mentioned earlier.

Longhorn on Virtual PC

Daniel Lehenbauer — Tue, 15 Jun 2004 01:54:00 GMT

I was asked if Longhorn runs on Virtual PC. I haven't tried build 4074 first hand, but according to Stephen McGibbons it is “a breeze” to setup, performance is acceptable, and he even shows it running 3D in his screenshot.